Journal of Real-Time Image Processing

, Volume 15, Issue 3, pp 523–536 | Cite as

Robust feature matching via Gaussian field criterion for remote sensing image registration

  • Qing Ma
  • Xu Du
  • Jiahao Wang
  • Yong Ma
  • Jiayi MaEmail author
Special Issue Paper


Feature matching, which refers to establishing reliable feature correspondences between two images of the same scene, is a critical prerequisite in a wide range of remote sensing tasks including environment monitoring, multispectral image fusion, image mosaic, change detection, map updating. In this paper, we propose a method for robust feature matching and apply it to the problem of remote sensing image registration. We start by creating a set of putative feature matches which can contain a number of unknown false matches, and then focus on mismatch removal. This is formulated as a robust regression problem, and we customize a robust estimator, namely the Gaussian field criterion, to solve it. The robust criterion can handle both linear and nonlinear image transformations. In the linear case, we use a general homography to model the transformation, while in the nonlinear case, the non-rigid functions located in a reproducing kernel Hilbert space are considered, and a regularization term is added to the objective function to ensure its well-posedness. Moreover, we apply a sparse approximation to the non-rigid transformation and reduce the computational complexity from cubic to linear. Extensive experiments on various natural and remote sensing images show the effectiveness of our approach, which is able to yield superior results compared to other state-of-the-art methods.


Feature matching Image registration Remote sensing Gaussian field Robust estimation 



This work was supported by the National Natural Science Foundation of China under Grant Nos. 61773295 and 61503288.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest


  1. 1.
    Jiang, J., Chen, C., Ma, J., Wang, Z., Wang, Z., Hu, R.: Srlsp: A face image super-resolution algorithm using smooth regression with local structure prior. IEEE Trans. Multimed. 19(1), 27–40 (2017)CrossRefGoogle Scholar
  2. 2.
    Ma, J., Chen, C., Li, C., Huang, J.: Infrared and visible image fusion via gradient transfer and total variation minimization. Inf. Fusion 31, 100–109 (2016)CrossRefGoogle Scholar
  3. 3.
    Li, Y., Huang, X., Liu, H.: Unsupervised deep feature learning for urban village detection from high-resolution remote sensing images. Photogramm. Eng. Remote Sens. 83(8), 567–579 (2017)CrossRefGoogle Scholar
  4. 4.
    Li, Y., Tao, C., Tan, Y., Shang, K., Tian, J.: Unsupervised multilayer feature learning for satellite image scene classification. IEEE Geosci. Remote Sens. Lett. 13(2), 157–161 (2016)CrossRefGoogle Scholar
  5. 5.
    Ma, J., Ma, Y., Li, C.: Infrared and visible image fusion methods and applications: a survey. Inf. Fusion. 45, 153–178 (2018)CrossRefGoogle Scholar
  6. 6.
    Liu, T., Liu, H., Chen, Z., Lesgold, A.M.: Fast blind instrument function estimation method for industrial infrared spectrometers. IEEE Trans. Ind. Inf. (2018). CrossRefGoogle Scholar
  7. 7.
    Liu, Y., Chen, X., Peng, H., Wang, Z.: Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36, 191–207 (2017)CrossRefGoogle Scholar
  8. 8.
    Gao, C., Wang, L., Xiao, Y., Zhao, Q., Meng, D.: Infrared small-dim target detection based on Markov random field guided noise modeling. Pattern Recognit. 76, 463–475 (2018)CrossRefGoogle Scholar
  9. 9.
    Liu, Y., Chen, X., Wang, Z., Wang, Z.J., Ward, R.K., Wang, X.: Deep learning for pixel-level image fusion: recent advances and future prospects. Inf. Fusion 42, 158–173 (2018)CrossRefGoogle Scholar
  10. 10.
    Ma, J., Zhou, H., Zhao, J., Gao, Y., Jiang, J., Tian, J.: Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans. Geosci. Remote Sens. 53(12), 6469–6481 (2015)CrossRefGoogle Scholar
  11. 11.
    Wong, A., Clausi, D.A.: ARRSI: automatic registration of remote-sensing images. IEEE Trans. Geosci. Remote Sens. 45(5), 1483–1493 (2007)CrossRefGoogle Scholar
  12. 12.
    Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)CrossRefGoogle Scholar
  13. 13.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Ma, J., Zhao, J., Tian, J., Yuille, A.L., Tu, Z.: Robust point matching via vector field consensus. IEEE Trans. Image Process. 23(4), 1706–1721 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Jiang, J., Hu, R., Wang, Z., Han, Z., Ma, J.: Facial image hallucination through coupled-layer neighbor embedding. IEEE Trans. Circuits Syst. Video Technol. 26(9), 1674–1684 (2016)CrossRefGoogle Scholar
  16. 16.
    Jiang, J., Ma, J., Chen, C., Jiang, X., Wang, Z.: Noise robust face image super-resolution through smooth sparse representation. IEEE Trans. Cybern. 47(11), 3991–4002 (2017)CrossRefGoogle Scholar
  17. 17.
    Maier, J., Humenberger, M., Murschitz, M., Zendel, O., Vincze, M.: Guided matching based on statistical optical flow for fast and robust correspondence analysis. In: Proceedings of European Conference on Computer Vision. pp. 101–117 (2016)Google Scholar
  18. 18.
    Gao, Y., Ma, J., Yuille, A.L.: Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. IEEE Trans. Image Process. 26(5), 2545–2560 (2017)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Liu, Y., De Dominicis, L., Wei, B., Chen, L., Martin, R.R.: Regularization based iterative point match weighting for accurate rigid transformation estimation. IEEE Trans. Vis. Comput. Graph. 21(9), 1058–1071 (2015)CrossRefGoogle Scholar
  20. 20.
    Yang, K., Pan, A., Yang, Y., Zhang, S., Ong, S.H., Tang, H.: Remote sensing image registration using multiple image features. Remote Sens. 9(6), 581 (2017)CrossRefGoogle Scholar
  21. 21.
    Guo, X., Li, Y., Ling, H.: Lime: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2017)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Ma, J., Jiang, J., Liu, C., Li, Y.: Feature guided Gaussian mixture model with semi-supervised em and local geometric constraint for retinal image registration. Inf. Sci. 417, 128–142 (2017)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Wang, G., Wang, Z., Chen, Y., Zhao, W.: Robust point matching method for multimodal retinal image registration. Biomed. Signal Process. Control 19, 68–76 (2015)CrossRefGoogle Scholar
  24. 24.
    Li, J., Hu, Q., Ai, M.: Robust feature matching for remote sensing image registration based on \(l_q\)-estimator. IEEE Geosci. Remote Sens. Lett. 13(12), 1989–1993 (2016)Google Scholar
  25. 25.
    Li, J., Hu, Q., Ai, M., Zhong, R.: Robust feature matching via support-line voting and affine-invariant ratios. ISPRS J. Photogramm. Remote Sens. 132, 61–76 (2017)CrossRefGoogle Scholar
  26. 26.
    Li, Y., Zhang, Y., Huang, X., Zhu, H., Ma, J.: Large-scale remote sensing image retrieval by deep hashing neural networks. IEEE Trans. Geosci. Remote Sens. 52(2), 950–965 (2018)CrossRefGoogle Scholar
  27. 27.
    Shi, X., Jiang, J.: Automatic registration method for optical remote sensing images with large background variations using line segments. Remote Sens. 8(5), 426 (2016)CrossRefGoogle Scholar
  28. 28.
    Wei, Z., Han, Y., Li, M., Yang, K., Yang, Y., Luo, Y., Ong, S.-H.: A small UAV based multi-temporal image registration for dynamic agricultural terrace monitoring. Remote Sens. 9(9), 904 (2017)CrossRefGoogle Scholar
  29. 29.
    Gonzalez, R.C., Wintz P.: Digital image processing. New York, NY, USA: Addison-Wesley (1987)Google Scholar
  30. 30.
    Reddy, B.S., Chatterji, B.N.: An FFT-based technique for translation, rotation, and scale-invariant image registration. IEEE Trans. Image Process. 5(8), 1266–1271 (1996)CrossRefGoogle Scholar
  31. 31.
    Rangarajan, A., Chui, H., Duncan, J.S.: Rigid point feature registration using mutual information. Med. Image Anal. 3(4), 425–440 (1999)CrossRefGoogle Scholar
  32. 32.
    Le Moigne, J., Campbell, W.J., Cromp, R.F.: An automated parallel image registration technique based on the correlation of wavelet features. IEEE Trans. Geosci. Remote Sens. 40(8), 1849–1864 (2002)CrossRefGoogle Scholar
  33. 33.
    Chen, Q.-S., Defrise, M., Deconinck, F.: Symmetric phase-only matched filtering of Fourier–Mellin transforms for image registration and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 16(12), 1156–1168 (1994)CrossRefGoogle Scholar
  34. 34.
    Uss, M.L., Vozel, B., Lukin, V.V., Chehdi, K.: Multimodal remote sensing image registration with accuracy estimation at local and global scales. IEEE Trans. Geosci. Remote Sens. 54(11), 6587–6605 (2016)CrossRefGoogle Scholar
  35. 35.
    Guo, X., Cao, X.: Good match exploration using triangle constraint. Pattern Recognit. Lett. 33(7), 872–881 (2012)CrossRefGoogle Scholar
  36. 36.
    Pele, O., Werman, M.: A linear time histogram metric for improved SIFT matching. In: Proceedings of European Conference on Computer Vision. pp. 495–508 (2008)Google Scholar
  37. 37.
    Li, Q., Wang, G., Liu, J., Chen, S.: Robust scale-invariant feature matching for remote sensing image registration. IEEE Geosci. Remote Sens. Lett. 6(2), 287–291 (2009)CrossRefGoogle Scholar
  38. 38.
    Hu, Y.-T., Lin, Y.-Y., Chen, H.-Y., Hsu, K.-J., Chen, B.-Y.: Matching images with multiple descriptors: an unsupervised approach for locally adaptive descriptor selection. IEEE Trans. Image Process. 24(12), 5995–6010 (2015)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Wang, C., Wang, L., Liu, L.: Progressive mode-seeking on graphs for sparse feature matching. In: Proceedings of European Conference on Computer Vision. pp. 788–802 (2014)Google Scholar
  40. 40.
    Cho, M., Lee, K.M.: Progressive graph matching: making a move of graphs via probabilistic voting. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 398–405, (2012)Google Scholar
  41. 41.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with application to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Torr, P.H.S., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Understand. 78(1), 138–156 (2000)CrossRefGoogle Scholar
  43. 43.
    Chum, O., Matas, J.: Matching with PROSAC—progressive sample consensus. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition. pp. 220–226 (2005)Google Scholar
  44. 44.
    Li, X., Hu, Z.: Rejecting mismatches by correspondence function. Int. J. Comput. Vis. 89(1), 1–17 (2010)CrossRefGoogle Scholar
  45. 45.
    Liu, H., Yan, S.: Common visual pattern discovery via spatially coherent correspondence. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition. pp. 1609–1616 (2010)Google Scholar
  46. 46.
    Yao, G., Cui, J., Deng, K., Zhang, L.: Robust Harris corner matching based on the quasi-homography transform and self-adaptive window for wide-baseline stereo images. IEEE Trans. Geosci. Remote Sens. 56(1), 559–574 (2018)CrossRefGoogle Scholar
  47. 47.
    Yu, Z., Zhou, H., Li, C.: Fast non-rigid image feature matching for agricultural UAV via probabilistic inference with regularization techniques. Comput. Electron. Agric. 143, 79–89 (2017)CrossRefGoogle Scholar
  48. 48.
    Ma, J., Zhao, J., Jiang, J., Zhou, H.: Non-rigid point set registration with robust transformation estimation under manifold regularization. In: Proceedings on AAAI Conference of Artificial Intelligence. pp. 4218–4224 (2017)Google Scholar
  49. 49.
    Boughorbel, F., Koschan, A., Abidi, B., Abidi, M.: Gaussian fields: a new criterion for 3D rigid registration. Pattern Recognit. 37(7), 1567–1571 (2004)CrossRefGoogle Scholar
  50. 50.
    Ma, J., Zhao, J., Ma, Y., Tian, J.: Non-rigid visible and infrared face registration via regularized Gaussian fields criterion. Pattern Recognit. 48(3), 772–784 (2015)CrossRefGoogle Scholar
  51. 51.
    Wang, G., Zhou, Q., Chen, Y.: Robust non-rigid point set registration using spatially constrained Gaussian fields. IEEE Trans. Image Process. 26(4), 1759–1769 (2017)MathSciNetCrossRefGoogle Scholar
  52. 52.
    Wang, G., Chen, Y., Zheng, X.: Gaussian field consensus: a robust nonparametric matching method for outlier rejection. Pattern Recognit. 74, 305–316 (2018)CrossRefGoogle Scholar
  53. 53.
    Greengard, L., Strain, J.: The fast Gauss transform. SIAM J. Sci. Stat. Comput. 12(1), 79–94 (1991)MathSciNetzbMATHCrossRefGoogle Scholar
  54. 54.
    Yuille, A.L.: Generalized deformable models, statistical physics, and matching problems. Neural Comput. 2(1), 1–24 (1990)MathSciNetCrossRefGoogle Scholar
  55. 55.
    Micchelli, C.A., Pontil, M.: On learning vector-valued functions. Neural Comput. 17(1), 177–204 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Ma, J., Zhao, J., Tian, J., Bai, X., Tu, Z.: Regularized vector field learning with sparse approximation for mismatch removal. Pattern Recognit. 46(12), 3519–3532 (2013)zbMATHCrossRefGoogle Scholar
  57. 57.
    Liu, H., Liu, S., Huang, T., Zhang, Z., Hu, Y., Zhang, T.: Infrared spectrum blind deconvolution algorithm via learned dictionaries and sparse representation. Appl. Opt. 55(10), 2813–2818 (2016)CrossRefGoogle Scholar
  58. 58.
    Ma, J., Zhao, J., Guo, H., Jiang, J., Zhou, H., Gao, Y.: Locality preserving matching. In: Proceedings of International Joint Conference on Artificial Intelligence. pp. 4492–4498 (2017)Google Scholar
  59. 59.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., van Gool, L.: A comparison of affine region detectors. Int. J. Comput. Vis. 65(1), 43–72 (2005)CrossRefGoogle Scholar
  60. 60.
    Vedaldi, A., Fulkerson, B.: VLFeat—an open and portable library of computer vision algorithms. In: Proceedings of the ACM International Conference on Multimedia. pp. 1469–1472 (2010)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.National Engineering Research Center for E-LearningCentral China Normal UniversityWuhanChina
  2. 2.Electronic Information SchoolWuhan UniversityWuhanChina

Personalised recommendations