Abstract
Since the visible and infrared images have different imaging mechanisms, the difficulty of image registration has greatly increased. The grayscale difference between visible and infrared images is very disadvantageous for extracting feature points in homogenous region, but they both retain the obvious contour edge in the scene. After using the morphological gradient method, the grayscale edge of visible and infrared images can be obtained and their similarity is greatly improved, and their difference may be seen as the difference in brightness or grayscale. Therefore, we proposed a novel algorithm to realise real-time adaptive registration of visible and infrared images using morphological gradient and C_SIFT. Firstly, the morphological gradient method is used to extract the rough edges of visible and infrared images for aligning their visual features as a single similar type. Secondly, the C_SIFT feature detection operator is used to detect and extract feature points from the extracted edges. The C_SIFT uses the centroid method to describe the main direction of feature points, makes rotation invariance feasible. Finally, to verify the effectiveness of the proposed algorithm, we carried out a series of experiments in eight various scenarios. The experimental results show that the proposed algorithm has achieved good experimental results. The registration of visible and infrared images can be completed quickly by the proposed algorithm, and the registration accuracy is satisfactory.
Similar content being viewed by others
References
Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24, 325–376 (1992)
Li, H., Manjunath, B.S., Mitra, S.K.: A contour-based approach to multisensor image registration. IEEE Trans. Image Process. A Publ. IEEE Signal Process. Soc. 4, 320–334 (1995)
Li, H.H., Zhou, Y.T.: Automatic visual/IR image registration. Opt. Eng. 35, 391–400 (1996)
Maes, F., Collignon, A.: Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16, 187–198 (1997)
Studholme, C., Hill, D.L.G., Hawkes, D.J.: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recogn. 32, 71–86 (1999)
Irani, M., Anandan, P.: Robust multi-sensor image alignment. In: International conference on computer vision, pp. 959–966 (1998)
Coiras, E., Santamaria, J., Miravet, C.: Segment-based registration technique for visual-infrared images. Opt. Eng. 39, 202–207 (2000)
Li, T., Chen, Z., Wang, R.: Fuzzy feature matching between infrared image and optical image. Proceedings of the SPIE, vol. 4552, pp 259–264 (2001)
Gonzalez, C.I., Castro, J.R., Martinez, G.E., Melin, P., Castillo, O.: A new approach based on generalized type-2 fuzzy logic for edge detection. In: IFSA World Congress and Nafips Meeting, pp. 424–429 (2013)
Barrenechea, E., Bustince, H., Baets, B.D., Lopez-Molina, C.: Construction of interval-valued fuzzy relations with application to the generation of fuzzy edge images. IEEE Trans. Fuzzy Syst. 19, 819–830 (2011)
Colerhodes, A., Zavorin, I., Moigne, J.L.: Mutual information as a similarity measure for remote sensing image registration. Proc. SPIE 4383, 51–61 (2001)
Kern, J.P., Pattichis, M.S.: Robust multispectral image registration using mutual-information models. IEEE Trans. Geosci. Remote Sens. 45, 1494–1505 (2007)
Lee, J.H., Yong, S.K., Lee, D., Kang, D.G., Ra, J.B.: Robust CCD and IR image registration using gradient-based statistical information. IEEE Signal Process. Lett. 17, 347–350 (2010)
Wang, L., Niu, Z., Wu, C., Xie, R., Huang, H.: A robust multisource image automatic registration system based on the SIFT descriptor. Int. J. Remote Sens. 33, 3850–3869 (2012)
Keller, Y., Averbuch, A.: Multisensor image registration via implicit similarity. IEEE Trans. Pattern Anal. Mach. Intell. 28, 794–801 (2006)
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Image registration by maximization of combined mutual information and gradient information. IEEE Trans. Med. Imaging 19, 809–814 (2002)
Bilodeau, G.A., Torabi, A., Morin, F.: Visible and infrared image registration using trajectories and composite foreground images. Image Vis. Comput. 29, 41–50 (2011)
Wu, F., Wang, B., Yi, X., Li, M., Hao, J., Qin, H., Zhou, H.: Visible and infrared image registration based on visual salient features. J. Electron. Imaging 24, 053017 (2015)
Yao, J., Goh, K.L.: A refined algorithm for multisensor image registration based on pixel migration. IEEE Trans. Image Process. A Publ. IEEE Signal Process. Soc. 15, 1839–1847 (2006)
Yang, B., Qiuze, Y.U., Liu, Y., Tian, J.: Image matching based on new weighted normalized cross correlation. J. Project. Rockets Missiles Guidance 28, 199–202 (2008)
Tian, T., Mei, X., Yu, Y., Zhang, C., Zhang, X.: Automatic visible and infrared face registration based on silhouette matching and robust transformation estimation. Infrared Phys. Technol. 69, 145–154 (2015)
Yang, M.Y., Qiang, Y., Rosenhahn, B.: A global-to-local framework for infrared and visible image sequence registration. In: 2015 IEEE Winter Conference on Applications of Computer Vision, pp. 381–388. IEEE, Waikoloa, 5–9 January 2015
Dou, J., Qin, Q., Tu, Z., Peng, X., Li, Y.: Infrared and visible image registration based on SIFT and sparse representation. In: Control and decision conference, pp. 5420–5424 (2016)
Liu, G., Zhou, H., Liang, X.G., Wang, M.J.: Image registration algorithm for infrared and visible light based on non-subsampled contourlet transform. Comput. Sci. 43, 313–316 (2016)
Tang, C., Tian, G.Y., Chen, X., Wu, J., Li, K., Meng, H., Tang, C., Tian, G.Y., Chen, X., Wu, J.: Infrared and visible images registration with adaptable local-global feature integration for rail inspection. Infrared Phys. Technol. 87, 31–39 (2017)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 91–110 (2004)
Butt, M.A.: Mathematical morphology and its applications to image and signal processing. Kluwer Academic Publishers, New York (2000)
Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. 207, 187–217 (1980)
Witkin, A.P.: Scale-space filtering. In: Proceeding of International Joint Conference Artificial Intelligence Karlsruhe, pp. 1019–1022 (1983)
Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: European Conference on Computer Vision, pp. 778–792 (2010)
Yi, X., Wang, B., Fang, Y., Liu, S.: Registration of infrared and visible images based on the correlation of the edges. In: International Congress on Image and Signal Processing, pp. 990–994 (2014)
Acknowledgements
This work was supported in part by the Nation Natural Science Foundation of China (Grant no. 61602064) and the Key Project of Sichuan Provincial Department of Education (18ZA0100) and the Research Project of Sichuan Science and Technology Department (2017HH0088, 2018JY0146, 2019YFH0187) and the Young Scholar Leadership Fund of CUIT (J201709).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zeng, Q., Adu, J., Liu, J. et al. Real-time adaptive visible and infrared image registration based on morphological gradient and C_SIFT. J Real-Time Image Proc 17, 1103–1115 (2020). https://doi.org/10.1007/s11554-019-00858-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-019-00858-x