Frontiers of Earth Science

, Volume 12, Issue 4, pp 779–790 | Cite as

Multi-sensor image registration by combining local self-similarity matching and mutual information

  • Xiaoping Liu
  • Shuli Chen
  • Li ZhuoEmail author
  • Jun Li
  • Kangning Huang
Research Article


Automatic multi-sensor image registration is a challenging task in remote sensing. Conventional image registration algorithms may not be applicable when common underlying visual features are not distinct. In this paper, we propose a novel image registration approach that integrates local self-similarity (LSS) and mutual information (MI) for multi-sensor images with rigid/nonrigid radiometric and geometric distortions. LSS is a well-performing descriptor that captures common, local internal layout features for multi-sensor images, whereas MI focuses on global intensity relationships. First, potential control points are identified by using the Harris algorithm and screened based on the self-similarity of their local surrounding internal layouts. Second, a Bayesian probabilistic model for matching the ensemble of the LSS features is introduced. Third, a particle swarm optimization (PSO) algorithm is adopted to optimize the point and region correspondences for maximum self-similarity and MI and, ultimately, a robust mapping function. The proposed approach is compared with several conventional image registration algorithms that are based on the sum of squared differences (SSD), scale-invariant feature transforms (SIFT), and speeded-up robust features (SURF) through the experimental registration of pairs of Landsat TM, SPOT, and RADARSAT SAR images. The results demonstrate that the proposed approach is efficient and accurate.


automatic registration multi-sensor images local self-similarity mutual information particle swarm optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The work was supported by the National Natural Science Foundation of China (Grant No. 41371499) and the Natural Science Foundation of Guangdong Province (No. 2015A030313505).


  1. Abdel Sayed S, Ionescu D, Goodenough D (1995). Matching and registration method for remote sensing images. In: Proceedings of Geoscience and Remote Sensing Symposium, 2, 1029–1031Google Scholar
  2. Arévalo V, González J (2008). Improving piecewise linear registration of high-resolution satellite images through mesh optimization. IEEE Trans Geosci Remote Sens, 46(11): 3792–3803CrossRefGoogle Scholar
  3. Atousa T (2011). Local self-similarity as a dense stereo correspondence measure for thermal-visible video registration. In: Proceedings of the 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,Washington, DC, USAGoogle Scholar
  4. Bay H, Ess A, Tuytelaars T, Van Gool L (2008). Speeded-up robust features (SURF). Comput Vis Image Underst, 110(3): 346–359CrossRefGoogle Scholar
  5. Belongie S, Malik J, Puzicha J (2002). Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell, 24(4): 509–522CrossRefGoogle Scholar
  6. Bentoutou Y, Taleb N (2005a). A 3-D space‒time motion detection for an invariant approach image registration approach in digital subtraction angiography. Comput Vis Image Underst, 97(1): 30–50CrossRefGoogle Scholar
  7. Bentoutou Y, Taleb N (2005b). Automatic extraction of control points for digital subtraction angiography image enhancement. IEEE Trans Nucl Sci, 52(1): 238–246CrossRefGoogle Scholar
  8. Bentoutou Y, Taleb N, Chikr El Mezouar M, Taleb M, Jetto L (2002). An invariant approach for image registration in digital subtraction angiography. Pattern Recognit, 35(12): 2853–2865CrossRefGoogle Scholar
  9. Boiman O, Irani M (2007). Detecting irregularities in images and in video. Int J Comput Vis, 74(1): 17–31CrossRefGoogle Scholar
  10. Borzi A, Bisceglie M D, Galdi C, Giangregorio G (2009). Robust registration of satellite images with local distortions. In: Proceedings of 2009 IEEE International Geoscience and Remote Sensing Symposium, 3: III-251–III-254CrossRefGoogle Scholar
  11. Bouchiha R, Besbes K (2013). Automatic remote-sensing image registration using SURF. International Journal of Computer Theory and Engineering, 5(1): 88–92CrossRefGoogle Scholar
  12. Brook A, Ben-Dor E (2011). Automatic registration of airborne and spaceborne image topology map matching with SURF processor algorithm. Remote Sens, 3(1): 65–82CrossRefGoogle Scholar
  13. Chen H M, Arora M K, Varshney P K (2003a). Mutual informationbased image registration for remote sensing data. Int J Remote Sens, 24(18): 3701–3706CrossRefGoogle Scholar
  14. Chen H M, Varshney P K, Arora M K (2003b). Performance of mutual information similarity measure for registration of multitemporal remote sensing images. IEEE Trans Geosci Remote Sens, 41(11): 2445–2454CrossRefGoogle Scholar
  15. Clerc M, Kennedy J (2002). The particle swarm—Explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput, 6(1): 58–73CrossRefGoogle Scholar
  16. Cole-Rhodes A A, Eastman R D (2011). Gradient descent approaches to image registration. In: Moigne J L, Netanyahu N S, Eastman R D, eds. Image Registration for Remote Sensing. Cambridge: Cambridge University, 265–276CrossRefGoogle Scholar
  17. Cole-Rhodes A, Johnson K L, Moigne J L, Zavorin I (2003). Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Transactions on Image, 12(12): 1495–1511CrossRefGoogle Scholar
  18. Cole-Rhodes A, Johnson K, Le Moigne J (2012). Multiresolution registration of remote sensing images using stochastic gradient. In: Szu H H, Buss J R, eds. Wavelet and Independent Component Analysis Applications IX. SPIE Proceedings Vol. 4738Google Scholar
  19. Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P, Marchal G (1995). Automated multimodality image registration based on information theory. Inf Process Med Imaging, 3: 263–274Google Scholar
  20. Farah I R, Boulila W, Ettabaâ K S, Solaiman B, Ahmed M B (2008). Interpretation of multisensor remote sensing images: multiapproach fusion of uncertain information. IEEE Trans Geosci Remote Sens, 46 (12): 4142–4152CrossRefGoogle Scholar
  21. Goshtasby A, Stockman G C, Page C V (1986). A region-based approach to digital image registration with subpixel accuracy. IEEE Trans Geosci Remote Sens, GE-24(3): 390–399Google Scholar
  22. Greenfeld J S (2002). Matching GPS Observation to Location on a Digital Map. In: Proceedings of the 81st Annual Meeting of the Transportation Research Board, (3): 13Google Scholar
  23. Harris C, Stephens M (1988). A combined corner and edge detector. In: Proceedings of Fourth Alvey Vision Conference,147–151CrossRefGoogle Scholar
  24. Hoyer P O (2004). Non-negative matrix factorization with sparseness constraints. J Mach Learn Res, 5: 1457–1469Google Scholar
  25. Jiao W (2012). Free Viewpoint Action Recognition based on Selfsimilarities. In: Proceedings of the 11th International Conference on Signal Processing (ICSP), 2, 1131–1134Google Scholar
  26. Ken C (2009). Efficient Retrieval of Deformable Shape Classes using Local Self-Similarities. In: Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops, 264–271Google Scholar
  27. Kennedy J, Eberhart R C (1995). Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, 4, 1942–1948CrossRefGoogle Scholar
  28. Kennedy J, Eberhart R C (2001). Swarm Intelligence. San Francisco: Morgan Kaufmann PublisherGoogle Scholar
  29. Kim J, Fessler J A (2004). Intensity-based image registration using robust correlation coefficients. IEEE Trans Med Imaging, 23(11): 1430–1444CrossRefGoogle Scholar
  30. Klein L A (2004). Sensor and Data Fusion: A Tool for Information Assessment and Decision Making. Bellingham: SPIE Press, 8–10Google Scholar
  31. Lee H K, Kim T C (2012). Local self-similarity based backprojection for image upscaling. In: Proceedings of 2012 IEEE International Symposium on Circuits and Systems (ISCAS), 1215–1218CrossRefGoogle Scholar
  32. Li H, Manjunath B S, Mitra S K (1995). A contour-based approach to multisensor image registration. IEEE Trans Image Process, 4(3): 320–334CrossRefGoogle Scholar
  33. Liang J, Liu X, Huang K, Li X, Wang D, Wang X (2014). Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric. IEEE Trans Geosci Remote Sens, 52(1): 603–615CrossRefGoogle Scholar
  34. Liu S, Du X Y, Zhang J H (2009). Structure extracting and matching based on similarity-pictorial structure model for microscopic images. In: Proceedings of International Conference on Artificial Intelligence, 3: 181–185Google Scholar
  35. Lowe D G (2004). Distinctive image features from scale-invariant key points. Int J Comput Vis, 60(2): 91–110CrossRefGoogle Scholar
  36. Meskine F, Mezouar M C E, Taleb N (2010). A rigid image registration based on the non subsampled contourlet transform and genetic algorithms. Sensors (Basel), 10(9): 8553–8571CrossRefGoogle Scholar
  37. Messerschmidt L, Engelbrecht A P (2004). Learning to play games using a PSO-based competitive learning approach. IEEE Trans Evol Comput, 8(3): 280–288CrossRefGoogle Scholar
  38. Pratt W K (1974). Correlation techniques of image registration. IEEE Trans Aerosp Electron Syst, AES-10(3): 353–358Google Scholar
  39. Ricardo G (2012). Landmark localisation in brain MR images using feature point descriptors based on 3D local self-similarities. In: Proceedings of the 9th IEEE International Symposium on Biomedical Imaging, 1535–1538Google Scholar
  40. Richards J A, Jia X (2006). Remote Sensing Digital Image Analysis (4th ed). Berlin: Springer-Verlag, 56–58Google Scholar
  41. Sedaghat A, Ebadi H (2015). Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching. ISPRS J Photogramm Remote Sens, 108: 62–71CrossRefGoogle Scholar
  42. Shechtman E, Irani M (2007). Matching local self-similarities across images and videos. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1–8Google Scholar
  43. Suri S, Reinartz P (2010). Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. IEEE Trans Geosci Remote Sens, 48(2): 939–949CrossRefGoogle Scholar
  44. Taleb N, Bentoutou Y, Deforges O, Taleb A (2001). A 3-D space-time motion evaluation for image registration in digital subtraction angiography. Comput Med Imaging Graph, 25(3): 223–233CrossRefGoogle Scholar
  45. Viola P, Wells W M III (1997). Alignment by maximization of mutual information. Int J Comput Vis, 24(2): 137–154CrossRefGoogle Scholar
  46. Wachowiak M P, Smolikova R, Zheng Y, Zurada J M, Elmaghraby A S (2004). An approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Trans Evol Comput, 8 (3): 289–301Google Scholar
  47. Wolberg G, Zokai S (2000). Robust image registration using log-polar transform. In: Proceedings of IEEE International Conference on Image Processing, 1: 493–496Google Scholar
  48. Wong A, Clausi D A (2007). ARRSI: automatic registration of remote sensing images. IEEE Trans Geosci Remote Sens, 45(5): 1483–1493CrossRefGoogle Scholar
  49. Yang H, Hou X (2012). Local self-similarity based texture classification. In: Proceedings of the 5th International Congress on Image and Signal Processing (CISP), 795–799Google Scholar
  50. Yi Z, Chen Z, Yang X (2008). Multi-spectral remote image registration based on SIFT. Electron Lett, 44(2): 107–108CrossRefGoogle Scholar
  51. Zhang H G, Bai X, Zheng H X, Zhao H J, Zhou J, Cheng J, Lu H (2013). Hierarchical remote sensing image analysis via graph laplacian energy. IEEE Geosci Remote Sens Lett, 10(2): 396–400CrossRefGoogle Scholar
  52. Zheng H (2011). A novel approach for satellite image classification using local self-similarity. In: Proceedings of Geoscience and Remote Sensing Symposium (IGARSS), 2011 IEEE International, 2888‒2891Google Scholar
  53. Zitová B, Flusser J (2003). Image registration methods: a survey. Image Vis Comput, 21(11): 977–1000CrossRefGoogle Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Xiaoping Liu
    • 1
  • Shuli Chen
    • 1
  • Li Zhuo
    • 1
    Email author
  • Jun Li
    • 1
  • Kangning Huang
    • 2
  1. 1.Guangdong Provincial Key Laboratory of Urbanization and Geo-simulation, School of Geography and PlanningSun Yat-Sen UniversityGuangzhouChina
  2. 2.School of Forestry and Environmental StudiesYale UniversityNew HavenUSA

Personalised recommendations