Skip to main content
SpringerLink
Log in

Multi-sensor image registration by combining local self-similarity matching and mutual information

  • Research Article
  • Published:
Frontiers of Earth Science Aims and scope Submit manuscript

Abstract

Automatic multi-sensor image registration is a challenging task in remote sensing. Conventional image registration algorithms may not be applicable when common underlying visual features are not distinct. In this paper, we propose a novel image registration approach that integrates local self-similarity (LSS) and mutual information (MI) for multi-sensor images with rigid/nonrigid radiometric and geometric distortions. LSS is a well-performing descriptor that captures common, local internal layout features for multi-sensor images, whereas MI focuses on global intensity relationships. First, potential control points are identified by using the Harris algorithm and screened based on the self-similarity of their local surrounding internal layouts. Second, a Bayesian probabilistic model for matching the ensemble of the LSS features is introduced. Third, a particle swarm optimization (PSO) algorithm is adopted to optimize the point and region correspondences for maximum self-similarity and MI and, ultimately, a robust mapping function. The proposed approach is compared with several conventional image registration algorithms that are based on the sum of squared differences (SSD), scale-invariant feature transforms (SIFT), and speeded-up robust features (SURF) through the experimental registration of pairs of Landsat TM, SPOT, and RADARSAT SAR images. The results demonstrate that the proposed approach is efficient and accurate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abdel Sayed S, Ionescu D, Goodenough D (1995). Matching and registration method for remote sensing images. In: Proceedings of Geoscience and Remote Sensing Symposium, 2, 1029–1031

    Google Scholar 

  • Arévalo V, González J (2008). Improving piecewise linear registration of high-resolution satellite images through mesh optimization. IEEE Trans Geosci Remote Sens, 46(11): 3792–3803

    Article  Google Scholar 

  • Atousa T (2011). Local self-similarity as a dense stereo correspondence measure for thermal-visible video registration. In: Proceedings of the 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society,Washington, DC, USA

    Google Scholar 

  • Bay H, Ess A, Tuytelaars T, Van Gool L (2008). Speeded-up robust features (SURF). Comput Vis Image Underst, 110(3): 346–359

    Article  Google Scholar 

  • Belongie S, Malik J, Puzicha J (2002). Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell, 24(4): 509–522

    Article  Google Scholar 

  • Bentoutou Y, Taleb N (2005a). A 3-D space‒time motion detection for an invariant approach image registration approach in digital subtraction angiography. Comput Vis Image Underst, 97(1): 30–50

    Article  Google Scholar 

  • Bentoutou Y, Taleb N (2005b). Automatic extraction of control points for digital subtraction angiography image enhancement. IEEE Trans Nucl Sci, 52(1): 238–246

    Article  Google Scholar 

  • Bentoutou Y, Taleb N, Chikr El Mezouar M, Taleb M, Jetto L (2002). An invariant approach for image registration in digital subtraction angiography. Pattern Recognit, 35(12): 2853–2865

    Article  Google Scholar 

  • Boiman O, Irani M (2007). Detecting irregularities in images and in video. Int J Comput Vis, 74(1): 17–31

    Article  Google Scholar 

  • Borzi A, Bisceglie M D, Galdi C, Giangregorio G (2009). Robust registration of satellite images with local distortions. In: Proceedings of 2009 IEEE International Geoscience and Remote Sensing Symposium, 3: III-251–III-254

    Book  Google Scholar 

  • Bouchiha R, Besbes K (2013). Automatic remote-sensing image registration using SURF. International Journal of Computer Theory and Engineering, 5(1): 88–92

    Article  Google Scholar 

  • Brook A, Ben-Dor E (2011). Automatic registration of airborne and spaceborne image topology map matching with SURF processor algorithm. Remote Sens, 3(1): 65–82

    Article  Google Scholar 

  • Chen H M, Arora M K, Varshney P K (2003a). Mutual informationbased image registration for remote sensing data. Int J Remote Sens, 24(18): 3701–3706

    Article  Google Scholar 

  • Chen H M, Varshney P K, Arora M K (2003b). Performance of mutual information similarity measure for registration of multitemporal remote sensing images. IEEE Trans Geosci Remote Sens, 41(11): 2445–2454

    Article  Google Scholar 

  • Clerc M, Kennedy J (2002). The particle swarm—Explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput, 6(1): 58–73

    Article  Google Scholar 

  • Cole-Rhodes A A, Eastman R D (2011). Gradient descent approaches to image registration. In: Moigne J L, Netanyahu N S, Eastman R D, eds. Image Registration for Remote Sensing. Cambridge: Cambridge University, 265–276

    Chapter  Google Scholar 

  • Cole-Rhodes A, Johnson K L, Moigne J L, Zavorin I (2003). Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient. IEEE Transactions on Image, 12(12): 1495–1511

    Article  Google Scholar 

  • Cole-Rhodes A, Johnson K, Le Moigne J (2012). Multiresolution registration of remote sensing images using stochastic gradient. In: Szu H H, Buss J R, eds. Wavelet and Independent Component Analysis Applications IX. SPIE Proceedings Vol. 4738

  • Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P, Marchal G (1995). Automated multimodality image registration based on information theory. Inf Process Med Imaging, 3: 263–274

    Google Scholar 

  • Farah I R, Boulila W, Ettabaâ K S, Solaiman B, Ahmed M B (2008). Interpretation of multisensor remote sensing images: multiapproach fusion of uncertain information. IEEE Trans Geosci Remote Sens, 46 (12): 4142–4152

    Article  Google Scholar 

  • Goshtasby A, Stockman G C, Page C V (1986). A region-based approach to digital image registration with subpixel accuracy. IEEE Trans Geosci Remote Sens, GE-24(3): 390–399

    Google Scholar 

  • Greenfeld J S (2002). Matching GPS Observation to Location on a Digital Map. In: Proceedings of the 81st Annual Meeting of the Transportation Research Board, (3): 13

    Google Scholar 

  • Harris C, Stephens M (1988). A combined corner and edge detector. In: Proceedings of Fourth Alvey Vision Conference,147–151

    Book  Google Scholar 

  • Hoyer P O (2004). Non-negative matrix factorization with sparseness constraints. J Mach Learn Res, 5: 1457–1469

    Google Scholar 

  • Jiao W (2012). Free Viewpoint Action Recognition based on Selfsimilarities. In: Proceedings of the 11th International Conference on Signal Processing (ICSP), 2, 1131–1134

    Google Scholar 

  • Ken C (2009). Efficient Retrieval of Deformable Shape Classes using Local Self-Similarities. In: Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops, 264–271

    Google Scholar 

  • Kennedy J, Eberhart R C (1995). Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, 4, 1942–1948

    Chapter  Google Scholar 

  • Kennedy J, Eberhart R C (2001). Swarm Intelligence. San Francisco: Morgan Kaufmann Publisher

    Google Scholar 

  • Kim J, Fessler J A (2004). Intensity-based image registration using robust correlation coefficients. IEEE Trans Med Imaging, 23(11): 1430–1444

    Article  Google Scholar 

  • Klein L A (2004). Sensor and Data Fusion: A Tool for Information Assessment and Decision Making. Bellingham: SPIE Press, 8–10

    Google Scholar 

  • Lee H K, Kim T C (2012). Local self-similarity based backprojection for image upscaling. In: Proceedings of 2012 IEEE International Symposium on Circuits and Systems (ISCAS), 1215–1218

    Chapter  Google Scholar 

  • Li H, Manjunath B S, Mitra S K (1995). A contour-based approach to multisensor image registration. IEEE Trans Image Process, 4(3): 320–334

    Article  Google Scholar 

  • Liang J, Liu X, Huang K, Li X, Wang D, Wang X (2014). Automatic registration of multisensor images using an integrated spatial and mutual information (SMI) metric. IEEE Trans Geosci Remote Sens, 52(1): 603–615

    Article  Google Scholar 

  • Liu S, Du X Y, Zhang J H (2009). Structure extracting and matching based on similarity-pictorial structure model for microscopic images. In: Proceedings of International Conference on Artificial Intelligence, 3: 181–185

    Google Scholar 

  • Lowe D G (2004). Distinctive image features from scale-invariant key points. Int J Comput Vis, 60(2): 91–110

    Article  Google Scholar 

  • Meskine F, Mezouar M C E, Taleb N (2010). A rigid image registration based on the non subsampled contourlet transform and genetic algorithms. Sensors (Basel), 10(9): 8553–8571

    Article  Google Scholar 

  • Messerschmidt L, Engelbrecht A P (2004). Learning to play games using a PSO-based competitive learning approach. IEEE Trans Evol Comput, 8(3): 280–288

    Article  Google Scholar 

  • Pratt W K (1974). Correlation techniques of image registration. IEEE Trans Aerosp Electron Syst, AES-10(3): 353–358

    Google Scholar 

  • Ricardo G (2012). Landmark localisation in brain MR images using feature point descriptors based on 3D local self-similarities. In: Proceedings of the 9th IEEE International Symposium on Biomedical Imaging, 1535–1538

    Google Scholar 

  • Richards J A, Jia X (2006). Remote Sensing Digital Image Analysis (4th ed). Berlin: Springer-Verlag, 56–58

    Google Scholar 

  • Sedaghat A, Ebadi H (2015). Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching. ISPRS J Photogramm Remote Sens, 108: 62–71

    Article  Google Scholar 

  • Shechtman E, Irani M (2007). Matching local self-similarities across images and videos. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1–8

    Google Scholar 

  • Suri S, Reinartz P (2010). Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas. IEEE Trans Geosci Remote Sens, 48(2): 939–949

    Article  Google Scholar 

  • Taleb N, Bentoutou Y, Deforges O, Taleb A (2001). A 3-D space-time motion evaluation for image registration in digital subtraction angiography. Comput Med Imaging Graph, 25(3): 223–233

    Article  Google Scholar 

  • Viola P, Wells W M III (1997). Alignment by maximization of mutual information. Int J Comput Vis, 24(2): 137–154

    Article  Google Scholar 

  • Wachowiak M P, Smolikova R, Zheng Y, Zurada J M, Elmaghraby A S (2004). An approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Trans Evol Comput, 8 (3): 289–301

    Google Scholar 

  • Wolberg G, Zokai S (2000). Robust image registration using log-polar transform. In: Proceedings of IEEE International Conference on Image Processing, 1: 493–496

    Google Scholar 

  • Wong A, Clausi D A (2007). ARRSI: automatic registration of remote sensing images. IEEE Trans Geosci Remote Sens, 45(5): 1483–1493

    Article  Google Scholar 

  • Yang H, Hou X (2012). Local self-similarity based texture classification. In: Proceedings of the 5th International Congress on Image and Signal Processing (CISP), 795–799

    Google Scholar 

  • Yi Z, Chen Z, Yang X (2008). Multi-spectral remote image registration based on SIFT. Electron Lett, 44(2): 107–108

    Article  Google Scholar 

  • Zhang H G, Bai X, Zheng H X, Zhao H J, Zhou J, Cheng J, Lu H (2013). Hierarchical remote sensing image analysis via graph laplacian energy. IEEE Geosci Remote Sens Lett, 10(2): 396–400

    Article  Google Scholar 

  • Zheng H (2011). A novel approach for satellite image classification using local self-similarity. In: Proceedings of Geoscience and Remote Sensing Symposium (IGARSS), 2011 IEEE International, 2888‒2891

    Google Scholar 

  • Zitová B, Flusser J (2003). Image registration methods: a survey. Image Vis Comput, 21(11): 977–1000

    Article  Google Scholar 

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation of China (Grant No. 41371499) and the Natural Science Foundation of Guangdong Province (No. 2015A030313505).

Author information

Authors and Affiliations

  1. Guangdong Provincial Key Laboratory of Urbanization and Geo-simulation, School of Geography and Planning, Sun Yat-Sen University, Guangzhou, 510275, China

    Xiaoping Liu, Shuli Chen, Li Zhuo & Jun Li

  2. School of Forestry and Environmental Studies, Yale University, New Haven, CT, 06511, USA

    Kangning Huang

Authors
  1. Xiaoping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuli Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Zhuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kangning Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Zhuo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Chen, S., Zhuo, L. et al. Multi-sensor image registration by combining local self-similarity matching and mutual information. Front. Earth Sci. 12, 779–790 (2018). https://doi.org/10.1007/s11707-018-0717-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11707-018-0717-9

Keywords

Search

Navigation