Markov random field (MRF) models are powerful tools to model image characteristics accurately and have been successfully applied to a large number of image processing applications. This paper investigates the problem of fusion of remote sensing images, e.g., multispectral image fusion, based on MRF models and incorporates the contextual constraints via MRF models into the fusion model. Fusion algorithms under the maximum a posteriori criterion are developed to search for solutions. Our algorithm is applicable to both multiscale decomposition (MD)-based image fusion and non-MD-based image fusion. Experimental results are provided to demonstrate the improvement of fusion performance by our algorithms.