Optical Review

, Volume 24, Issue 3, pp 370–382 | Cite as

A fusion method for infrared–visible image and infrared-polarization image based on multi-scale center-surround top-hat transform

  • Pan Zhu
  • Zhanhua Huang
Regular Paper


This paper presents a fusion method for infrared–visible image and infrared-polarization image based on multi-scale center-surround top-hat transform which can effectively extract the feature information and detail information of source images. Firstly, the multi-scale bright (dark) feature regions of source images at different scale levels are respectively extracted by multi-scale center-surround top-hat transform. Secondly, the bright (dark) feature regions at different scale levels are refined for eliminating the redundancies by spatial scale. Thirdly, the refined bright (dark) feature regions from different scales are combined into the fused bright (dark) feature regions through adding. Then, a base image is calculated by performing dilation and erosion on the source images with the largest scale outer structure element. Finally, the fusion image is obtained by importing the fused bright and dark features into the base image with a reasonable strategy. Experimental results indicate that the proposed fusion method can obtain state-of-the-art performance in both aspects of objective assessment and subjective visual quality.


Infrared–visible image fusion Infrared-polarization image fusion Multi-scale top-hat transform Bright and dark feature extraction 



We would like to thank the editors and the reviewers for their careful work and invaluable suggestions for helping us to improve this paper. We are also grateful to the websites and for providing the experiment images. This work is supported by the National Natural Science Foundation of China (Grant Nos: 61275009, 61475113).


  1. 1.
    Kong, W., Yang, L.: Technique for image fusion between gray-scale visual light and infrared images based on NSST and improved RF. Optik 124, 6423–6431 (2013)ADSCrossRefGoogle Scholar
  2. 2.
    Jamal, S., Karim, F.: Infrared and visible image fusion using fuzzy logic and population-based optimization. Appl. Soft. Comput. 12, 1041–1054 (2012)CrossRefGoogle Scholar
  3. 3.
    Scott, T.J., Goldstein, D.L., Chenault, D.B., Shaw, J.A.: Review of passive imaging polarimetry for remote sensing applications. Appl. Opt. 45, 5453–5469 (2006)ADSCrossRefGoogle Scholar
  4. 4.
    Burt, P., Adelson, E.: The laplacian pyramid as compact image code. IEEE Trans. Commun. 31, 532–540 (1983)CrossRefGoogle Scholar
  5. 5.
    Lu, H., Zhang, L., Serikawa, S.: Maximum local energy: an effective approach for multisensory image fusion in beyond wavelet transform domain. Comput. Math. Appl. 64, 996–1003 (2012)CrossRefMATHGoogle Scholar
  6. 6.
    Wang, H., Yang, Q., Li, R.: Tunable-Q contourlet-based multi-sensor image fusion. Signal Process. 93, 1879–1891 (2013)CrossRefGoogle Scholar
  7. 7.
    Li, T.J., Wang, Y.Y.: Biological image fusion using a NSCT based variable-weight method. Inf. Fusion. 12, 85–92 (2011)CrossRefGoogle Scholar
  8. 8.
    Zhang, Q., Guo, B.L.: Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process. 89, 1334–1346 (2009)ADSCrossRefMATHGoogle Scholar
  9. 9.
    Li, S., Yang, B., Hu, J.: Performance comparison of different multi-resolution transforms for image fusion. Inf. Fusion. 12, 74–84 (2011)CrossRefGoogle Scholar
  10. 10.
    Mikula, K., Preusser, T., Rumpf, M.: Morphological image sequence processing. Comput. Vis. Sci. 6, 197–209 (2004)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Restaino, R., Vivone, G., Mura, M.D., Chanussot, J.: Fusion of multispectral and panchromatic images based on morphological operators. IEEE Trans. Image Process. 25, 2882–2895 (2016)ADSMathSciNetCrossRefGoogle Scholar
  12. 12.
    Jiang, Y., Wang, M.: Image fusion with morphological component analysis. Inf. Fusion. 18, 107–118 (2014)CrossRefGoogle Scholar
  13. 13.
    Li, Y., Yong, B., Wu, H., An, R., Xu, H.: An improved top-hat filter with sloped brim for extracting ground points from airborne lidar point clouds. Remote Sens. 6, 12885–12908 (2014)ADSCrossRefGoogle Scholar
  14. 14.
    Liao, M., Zhao, Y.Q., Wang, X.H., Dai, P.S.: Retinal vessel enhancement based on multi-scale top-hat transformation and histogram fitting stretching. Opt. Laser Technol. 58, 56–62 (2014)ADSCrossRefGoogle Scholar
  15. 15.
    Mukhopadhyay, S., Chanda, B.: A multiscale morphological approach to local contrast enhancement. Signal Process. 80, 685–696 (2000)CrossRefMATHGoogle Scholar
  16. 16.
    Tan, X.Y., Chen, M., Jiang, C.S.: The small target detection base on wavelet transform and mathematical morphology. Electron. Opt. Control. 15, 25–28 (2008)Google Scholar
  17. 17.
    Barata, T., Pina, P.: Improving classification rates by modelling the clusters of training sets in features space using mathematical morphology operators. Pattern Recognit. 4, 90–93 (2002)Google Scholar
  18. 18.
    Klingler, J.W., Vaughan, C.L., Fraker, T.D., Andrews, L.T.: Segmentation of echocardiographic images using mathematical morphology. IEEE Trans. Bio Eng. 11, 925–934 (1988)CrossRefGoogle Scholar
  19. 19.
    Mukhopadhyay, S., Chanda, B.: Fusion of 2D grayscale images using multiscale morphology. Pattern Recognit. 34, 1939–1949 (2001)CrossRefMATHGoogle Scholar
  20. 20.
    Li, Y.F., Feng, X.Y., Xu, M.W.: Infrared and visible image features enhancement and fusion using multi-scale top-hat decomposition. Infrared Laser Eng. 41(10), 2825–2832 (2012)Google Scholar
  21. 21.
    Bai, X., Gu, S., Zhou, F., Xue, B.: Weighted image fusion based on multi-scale top-hat transform: algorithms and a comparison study. Optik 124, 1660–1668 (2013)ADSCrossRefGoogle Scholar
  22. 22.
    Bai, X., Zhang, Y.: Detail preserved fusion of infrared and visual images by using opening and closing based toggle operator. Opt. Laser Technol. 63, 105–113 (2014)ADSCrossRefGoogle Scholar
  23. 23.
    Bai, X., Zhou, F.: Analysis of new top-hat transformation and the application for infrared dim small target detection. Pattern Recognit. 43, 2145–2156 (2010)CrossRefMATHGoogle Scholar
  24. 24.
    Bai, X., Zhou, F., Xue, B.: Infrared image enhancement through contrast enhancement by using multi scale new top-hat transform. Infrared Phys. Technol. 54, 61–69 (2011)ADSCrossRefGoogle Scholar
  25. 25.
    Bai, X., Zhou, F., Xue, B.: Fusion of infrared and visual images through region extraction by using multi scale center-surround top-hat transform. Opt. Express 9, 8444–8457 (2011)ADSCrossRefGoogle Scholar
  26. 26.
    Piella, G.: A general framework for multiresolution image fusion: from pixels to regions. Inf. Fusion. 4(4), 259–280 (2003)CrossRefGoogle Scholar
  27. 27.
    Yang, F., Wei, H.: Fusion of infrared polarization and intensity images using support value transform and fuzzy combination rules. Infrared Phys. Technol. 60, 235–243 (2013)ADSCrossRefGoogle Scholar

Copyright information

© The Optical Society of Japan 2017

Authors and Affiliations

  1. 1.Key Laboratory of Opto-electronic Information TechnologyMinistry of Education, Tianjin UniversityTianjinChina

Personalised recommendations