Color and depth image registration algorithm based on multi-vector-fields constraints

  • Xiaolin Li
  • Daoqing Li
  • Li Peng
  • Huabing ZhouEmail author
  • Deng Chen
  • Yanduo Zhang
  • Liang Xie


Image registration, which aim to establish a reliable feature relationship between images, is a critical problem in the field of image processing. In order to enhance the accuracy of color and depth image registration, this paper proposes an novel image registration algorithm based on multi-vector-fields constraints. We first initialize the edge information features of color and depth images, and establish putative correspondences based on edge information. Consider the correlation between the images, establish the functional relationships of the multi-vector-fields constraints based on the relationships. In the reproducing nuclear Hilbert space (RKHS), this constraint is added to the probability model, and the model parameters are optimized using the EM algorithm. Finally, the probability of corresponding edge points of the image is obtained. In order to further improve registration accuracy, this paper will change the input from one pair to two pairs and let the feature transformation relationship between images be iteratively evaluated using the parameter model. Taking publicly available RGB-D images as experimental subjects, results show that for single object image registration, the algorithm image registration accuracy in this paper is improved by about 5% compared with SC, ICP, and CPD algorithms. In addition, artificial noise was used to test the proposed algorithm’s anti-noise ability, results show that the proposed algorithm has superior anti-noise ability relative to SC, ICP and CPD algorithms.


EM algorithm Image registration Depth image Multi-vector-fields constraints 



  1. 1.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. PAMI 24(24):509–522CrossRefGoogle Scholar
  2. 2.
    Chum O, Matas J (2005) Matching with PROSAC – progressive sample consensus. Proc IEEE Conf Comput Vis Pattern Recog 1:220–226Google Scholar
  3. 3.
    Essmaeel K, Gallo L, Damiani E, Pietro G, Dipanda A (2015) Comparison of methods for filtering for kinect depth data. Multimed Tools Appl 74(17):7331–7354CrossRefGoogle Scholar
  4. 4.
    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with application to image analysis and automated cartography. Commun ACM 24(6):381–395MathSciNetCrossRefGoogle Scholar
  5. 5.
    Hill DL, Batchelor PG, Holden M, Hawkes DJ (2001) Medical image registration. Phys Med Biol 46(3):R1–45CrossRefGoogle Scholar
  6. 6.
    Jackson BP, Goshtasby AA (2010) Registering aerial video images using the projective constraint [J]. IEEE Trans Image Process 19(3):795–804MathSciNetCrossRefGoogle Scholar
  7. 7.
    Jean-Louis F, Van DDA (2000) The PX-EM algorithm for fast stable fitting of Henderson’s mixed model[J]. Gen Select Evol 32(2):143–163CrossRefGoogle Scholar
  8. 8.
    Jing P, Su Y, Nie L, et al (2018) A framework of joint low-rank and sparse regression for image memorability prediction[J]. IEEE Trans Circ Syst Vid Technol PP (99):1–1Google Scholar
  9. 9.
    Jing P, Su Y, Nie L, et al (2018) Low-rank multi-view embedding learning for micro-video popularity prediction[J]. IEEE Trans Knowl Data Eng PP(99):1519–1532CrossRefGoogle Scholar
  10. 10.
    Jorge-Peas A, Bov H, Sanen K et al (2017) 3D full-field quantification of cell-induced large deformations in fibrillar biomaterials by combining non-rigid image registration with label-free second harmonic generation[J]. Biomaterials 136:86–97CrossRefGoogle Scholar
  11. 11.
    Lowe DG (2004) Distinctive image features from scale invariant keypoints[J]. Int J Comput Vis 60(2):91–110MathSciNetCrossRefGoogle Scholar
  12. 12.
    Ma J, Zhao J, Tian J, et al (2013) Regularized vector field learning with sparse approximation for mismatch removal. Pattern Recognit 46(12):3519–3532CrossRefGoogle Scholar
  13. 13.
    Ma J, Zhao J, Tian J, Yuille AL, Tu Z (2014) Robust point matching via vector field consensus. IEEE Trans Image Process 23(4):1706–1721MathSciNetCrossRefGoogle Scholar
  14. 14.
    Ma J, Zhao J, Ma Y, Tian J (2015) Non-rigid visible and infrared face registration via regularized Gaussian fields criterion. Pattern Recogn 48(3):772–784CrossRefGoogle Scholar
  15. 15.
    Ma J, Qiu W, Zhao J, et al (2015) Robust L2E estimation of transformation for non-rigid registration. IEEE Trans Signal Process 63(5):1115–1129MathSciNetCrossRefGoogle Scholar
  16. 16.
    Ma J, Zhou H, Zhao J, et al (2015) Robust feature matching for remote sensing image registration via locally linear transforming[J]. IEEE Trans Geosci Remote Sens 53(12):6469–6481CrossRefGoogle Scholar
  17. 17.
    Ma J, Zhao J, Yuille AL (2016) Non-rigid point set registration by preserving global and local structures. IEEE Trans Image Process 25(1):53–64MathSciNetCrossRefGoogle Scholar
  18. 18.
    Ma J, Chen C, Li C, et al (2016) Infrared and visible image fusion via gradient transfer and total variation minimization. Inf Fusion 31:100–109CrossRefGoogle Scholar
  19. 19.
    Ma J, Jiang J, Liu C, Li Y (2017) Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration. Inf Sci 417:128–142MathSciNetCrossRefGoogle Scholar
  20. 20.
    Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178CrossRefGoogle Scholar
  21. 21.
    Ma J, Yu W, Liang P, et al (2019) FusionGAN: a generative adversarial network for infrared and visible image fusion. Inf Fusion 48:11–26CrossRefGoogle Scholar
  22. 22.
    Torr PHS, Zisserman A (2000) MLESAC: a new robust estimator with application to estimating image geometry. Comput Vis Image Under-stand 78(1):138–156CrossRefGoogle Scholar
  23. 23.
    Xie L, Shen J, Han J et al (2017) Dynamic multi-view hashing for online image retrieval[C]. In: Twenty-sixth international joint conference on artificial intelligence, pp 3133–3139Google Scholar
  24. 24.
    Zhou H, Ma J, Yang C, et al (2016) Nonrigid feature matching for remote sensing images via probabilistic inference with global and local regularizations[J]. IEEE Geosc Rem Sens Lett 13(3):374–378Google Scholar
  25. 25.
    Zhou H, Kuang Y, Yu Z, Ren S, Zhang Y, Lu T, Ma J (2018) Image deformation with vector-field interpolation based on MRLS-TPS. IEEE Access 7(1)Google Scholar
  26. 26.
    Zhu L, Shen J, Xie L, et al (2017) Unsupervised visual hashing with semantic assistant for content-based image retrieval[J]. IEEE Trans Knowl Data Eng 29(2):472–486CrossRefGoogle Scholar
  27. 27.
    Zitová B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Hubei Key Laboratory of Intelligent RobotWuhan Institute of TechnologyWuhanChina
  2. 2.Department of MathematicsWuhan University of TechonologyWuhanChina

Personalised recommendations