Skip to main content
SpringerLink
Log in

Cross-spectral registration of natural images with SIPCFE

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

A Publisher Correction to this article was published on 27 February 2020

This article has been updated

Abstract

Image registration is a viable task in the field of computer vision with many applications. When images are captured under different spectrum conditions, a challenge is imposed on the task of registration. Researchers carefully handcraft a local module insensitive to illumination changes across cross-spectral image pairs to tackle this challenge. We, in this paper, develop an optimized feature-based approach Single Instance Phase Congruency Feature Extractor (SIPCFE) to tackle the problem of natural cross-spectral image registration. SIPCFE uses the phase information of an image pair to quickly identify and describe reliable keypoints that are insensitive to illumination. It then employs a sequence of outlier removal processes to find the matching feature points accurately and the Direct Linear Transformation to estimate the geometric transformation to align the image pair. We extensively study the proposed approach for every module in the system to give more insights into the challenges. We benchmark our proposed method and other state-of-the-art feature-based methods developed for cross-spectral imagery on three datasets with various settings and image contents. The comprehensive analysis of cross-spectral registration results of natural images demonstrates that SIPCFE achieves up to 47.24%, 14.29%, and 12.45% accuracy improvement on the first, second, and third dataset, respectively, over the second best registration method in the benchmark.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Change history

  • 27 February 2020

    The articles listed below were published in Issue January 2020, Issue 1, instead of Issue February 2020, Issues 1–2.

References

  1. Aguilera, C., Barrera, F., Lumbreras, F., Sappa, A.D., Toledo, R.: Multispectral image feature points. Sensors 12(9), 12661–12672 (2012)

    Article  Google Scholar 

  2. Aguilera, C., Sappa, A.D., Toledo, R.: LGHD: A feature descriptor for matching across non-linear intensity variations. In: Proceedings of IEEE IEEE International Conference on Image Process (ICIP), pp. 178–181. IEEE (2015)

  3. Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  5. Biswas, M., Om, H.: A new soft-thresholding image denoising method. Procedia Technol. 6, 10–15 (2012)

    Article  Google Scholar 

  6. Brown, M., Süsstrunk, S.: Multi-spectral SIFT for scene category recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 177–184. IEEE (2011)

  7. Chen, M., Carass, A., Jog, A., Lee, J., Roy, S., Prince, J.L.: Cross contrast multi-channel image registration using image synthesis for MR brain images. Med. Image Anal. 36, 2–14 (2017)

    Article  Google Scholar 

  8. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886–893. IEEE (2005)

  9. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 39, 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  10. Donoho, D.L., Johnstone, J.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  11. Faraji, M.R., Qi, X.: Face recognition under illumination variations based on eight local directional patterns. IET Biom. 4(1), 10–17 (2015)

    Article  Google Scholar 

  12. Firmenichy, D., Brown, M., Süsstrunk, S.: Multispectral interest points for RGB-NIR image registration. In: Proceedings of the IEEE International Conference on Image Processing, pp. 181–184. IEEE (2011)

  13. Han, J., Pauwels, E.J., Zeeuw, P.D.: Visible and infrared image registration in man-made environments employing hybrid visual features. Pattern Recognit. Lett. 34(1), 42–51 (2013)

    Article  Google Scholar 

  14. Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, vol. 15, pp. 10–5244. Manchester, UK (1988)

  15. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  16. Heinrich, M.P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F.V., Brady, M., Schnabel, J.A.: MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Med. Image Anal. 16(7), 1423–1435 (2012)

    Article  Google Scholar 

  17. Hrkać, T., Kalafatić, Z., Krapac, J.: Infrared-visual image registration based on corners and hausdorff distance. In: Image Analysis, pp. 383–392 (2007)

  18. Ikeda, K., Ino, F., Hagihara, K.: Efficient acceleration of mutual information computation for nonrigid registration using CUDA. IEEE J. Biomed. Health Inf. 18(3), 956–968 (2014)

    Article  Google Scholar 

  19. Kim, S., Min, D., Ham, B., Ryu, S., Do, M.N., Sohn, K.: DASC: dense adaptive self-correlation descriptor for multi-modal and multi-spectral correspondence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2103–2112 (2015)

  20. Kovesi, P.: Image features from phase congruency. Videre J. Comput. Vis. Res. 1(3), 1–26 (1999)

    Google Scholar 

  21. Kovesi, P.: Phase congruency detects corners and edges. In: The Australian Pattern Recognition Society Conference, DICTA (2003)

  22. Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: Binary robust invariant scalable keypoints. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2548–2555. IEEE (2011)

  23. Loeckx, D., Slagmolen, P., Maes, F., Vandermeulen, D., Suetens, P.: Nonrigid image registration using conditional mutual information. IEEE Trans. Med. Imaging 29(1), 19–29 (2010)

    Article  Google Scholar 

  24. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  25. Ma, J., Zhao, J., Tian, J., Yuille, A.L., Tu, Z.: Robust point matching via vector field consensus. IEEE Trans. Image Proc. 23(4), 1706–1721 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  26. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16(2), 187–198 (1997)

    Article  Google Scholar 

  27. Mellor, M., Brady, M.: Phase mutual information as a similarity measure for registration. Med. Image Anal. 9(4), 330–343 (2005)

    Article  Google Scholar 

  28. Morrone, M.C., Ross, J., Burr, D.C., Owens, R.: Mach bands are phase dependent. Nature 324(6094), 250–253 (1986)

    Article  Google Scholar 

  29. Mouats, T., Aouf, N., Sappa, A.D., Aguilera, C., Toledo, R.: Multispectral stereo odometry. IEEE Trans. Intell. Transp. Syst. 16(3), 1210–1224 (2015)

    Article  Google Scholar 

  30. Pang, G., Neumann, U.: The Gixel array descriptor (GAD) for multimodal image matching. In: WACV, pp. 497–504 (2013)

  31. Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Image registration by maximization of combined mutual information and gradient information. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 452–461. Springer (2000)

  32. Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003)

    Article  Google Scholar 

  33. Qin, Y., Cao, Z., Zhuo, W., Yu, Z.: Robust key point descriptor for multi-spectral image matching. J. Syst. Eng. Electron. 25(4), 681–687 (2014)

    Article  Google Scholar 

  34. Rivaz, H., Karimaghaloo, Z., Collins, D.L.: Self-similarity weighted mutual information: a new nonrigid image registration metric. Med. Image Anal. 18(2), 343–358 (2014)

    Article  Google Scholar 

  35. Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: Proceedings of the 10th IEEE International Conference on Computer Vision, vol. 2, pp. 1508–1515. IEEE (2005)

  36. Rosten, E., Porter, R., Drummond, T.: Faster and better: a machine learning approach to corner detection. IEEE Trans. Patt. Anal. Mach. Intell. 32(1), 105–119 (2010)

    Article  Google Scholar 

  37. Rueckert, D., Clarkson, M.J., Hill, D.L.G., Hawkes, D.J.: Non-rigid registration using higher-order mutual information. Proc. SPIE Med. Image Image Process 3979, 439–447 (2000)

    Google Scholar 

  38. Sendur, L., Selesnick, I.W.: Bivariate shrinkage with local variance estimation. IEEE Signal Proc. Lett. 9(12), 438–441 (2002)

    Article  Google Scholar 

  39. Shi, J., Tomasi, C.: Good features to track. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 593–600. IEEE (1994)

  40. Smith, S.M., Brady, J.M.: SUSAN—a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)

    Article  Google Scholar 

  41. Studholme, C., Drapaca, C., Iordanova, B., Cardenas, V.: Deformation-based mapping of volume change from serial brain MRI in the presence of local tissue contrast change. IEEE Trans. Med. Imaging 25(5), 626–639 (2006)

    Article  Google Scholar 

  42. Tikhonov, A.N., Arsenin, V.Y., John, F.: Solutions of Ill-Posed Problems, vol. 14. Winston, Washington (1977)

    MATH  Google Scholar 

  43. Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997)

    Article  Google Scholar 

  44. Wachinger, C., Navab, N.: Entropy and Laplacian images: structural representations for multi-modal registration. Med. Image Anal. 16(1), 1–17 (2012)

    Article  Google Scholar 

  45. Weiss, Y., Adelson, E.H.: Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision. MIT AI Lab Tech, Rep (1998)

  46. Yang, X., Kwitt, R., Niethammer, M.: Quicksilver: fast predictive image registration—a deep learning approach. NeuroImage 158, 378–396 (2017)

    Article  Google Scholar 

  47. Zhao, C., Zhao, H., Lv, J., Sun, S., Li, B.: Multimodal image matching based on multimodality robust line segment descriptor. Neurocomputing 177, 290–303 (2016)

    Article  Google Scholar 

  48. Zhao, D., Yang, Y., Ji, Z., Hu, X.: Rapid multimodality registration based on MM-SURF. Neurocomputing 131, 87–97 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Utah State University, 4205 Old Main Hill, Logan, UT, 84322-4205, USA

    Amir Hossein Farzaneh & Xiaojun Qi

Authors
  1. Amir Hossein Farzaneh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaojun Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Hossein Farzaneh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farzaneh, A.H., Qi, X. Cross-spectral registration of natural images with SIPCFE. Machine Vision and Applications 31, 10 (2020). https://doi.org/10.1007/s00138-020-01057-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01057-6

Keywords

Search

Navigation