Skip to main content
SpringerLink
Log in

Image fusion using dual tree discrete wavelet transform and weights optimization

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Image fusion is a useful context in image processing. It goals to produce more informative image using multi-image data with different sensors. In this study, an effective approach in discrete wavelet transform domain for infrared and visible image fusion is proposed. In fact, important parts of thermal images along with details of visual image must be considered in fused images. Therefore, dual tree discrete wavelet transform is used to extract both subjects based on an optimization process. The optimization considers parts of input images with maximum entropy and minimum mean square error in fused image in comparison with both input images. Experimental results on a standard database demonstrate that proposed method can achieve a superior performance compared with other fusion methods in both subjective and objective assessments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Convolutional Neural Network.

  2. Generative Adversarial Network.

References

  1. Zhou, Z., Dong, M., Xie, X., Gao, Z.: Fusion of infrared and visible images for night-vision context enhancement. Appl. Opt. 55, 6480–6490 (2016)

    Article  Google Scholar 

  2. Ma, J., Jiang, J., Liu, C., Li, Y.: Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration. Inf. Sci. 417, 128–142 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  3. Liu, C., Ma, J., Ma, Y., Huang, J.: Retinal image registration via feature-guided Gaussian mixture model. JOSA A 33, 1267–1276 (2016)

    Article  Google Scholar 

  4. Bhatnagar, G., Wu, Q.J., Liu, Z.: A new contrast based multimodal medical image fusion framework. Neurocomputing 157, 143–152 (2015)

    Article  Google Scholar 

  5. Li, H., Manjunath, B., Mitra, S.K.: Multisensor image fusion using the wavelet transform. Graph. Models Image Process. 57, 235–245 (1995)

    Article  Google Scholar 

  6. Wei, Z., Han, Y., Li, M., Yang, K., Yang, Y., Luo, Y., et al.: A small UAV based multi-temporal image registration for dynamic agricultural terrace monitoring. Remote Sens. 9, 904 (2017)

    Article  Google Scholar 

  7. Yang, K., Pan, A., Yang, Y., Zhang, S., Ong, S.H., Tang, H.: Remote sensing image registration using multiple image features. Remote Sens. 9, 581 (2017)

    Article  Google Scholar 

  8. Dong, L., Yang, Q., Wu, H., Xiao, H., Xu, M.: High quality multi-spectral and panchromatic image fusion technologies based on Curvelet transform. Neurocomputing 159, 268–274 (2015)

    Article  Google Scholar 

  9. Chen, C., Li, Y., Liu, W., Huang, J.: Image fusion with local spectral consistency and dynamic gradient sparsity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2760–2765. (2014)

  10. Zhou, C., Zhao, J., Pan, Z., Hong, Q., Huang, L.: Fusion of visible and infrared images based on IHS transformation and regional variance matching degree. In IOP Conference Series: Earth and Environmental Science, p. 012021. (2019)

  11. Gao, Y., Ma, J., Yuille, A.L.: Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. IEEE Trans. Image Process. 26, 2545–2560 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  12. Kong, S.G., Heo, J., Boughorbel, F., Zheng, Y., Abidi, B.R., Koschan, A., et al.: Multiscale fusion of visible and thermal IR images for illumination-invariant face recognition. Int. J. Comput. Vis. 71, 215–233 (2007)

    Article  Google Scholar 

  13. Toet, A., Ijspeert, J.K., Waxman, A.M.: Aguilar, fusion of visible and thermal imagery improves situational awareness. Displays 18, 85–95 (1997)

    Article  Google Scholar 

  14. Ghaneizad, M., Kavehvash, Z., Aghajan, H.: Human detection in occluded scenes through optically inspired multi-camera image fusion. JOSA A 34, 856–869 (2017)

    Article  Google Scholar 

  15. Yang, C., Ma, J., Qi, S., Tian, J., Zheng, S., Tian, X.: Directional support value of Gaussian transformation for infrared small target detection. Appl. Opt. 54, 2255–2265 (2015)

    Article  Google Scholar 

  16. Ma, J., Zhao, J., Ma, Y., Tian, J.: Non-rigid visible and infrared face registration via regularized Gaussian fields criterion. Pattern Recogn. 48, 772–784 (2015)

    Article  Google Scholar 

  17. Wang, N., Ma, Y., Zhan, K.: Spiking cortical model for multifocus image fusion. Neurocomputing 130, 44–51 (2014)

    Article  Google Scholar 

  18. Meng, F., Guo, B., Song, M., Zhang, X.: Image fusion with saliency map and interest points. Neurocomputing 177, 1–8 (2016)

    Article  Google Scholar 

  19. Li, Y., Tao, C., Tan, Y., Shang, K., Tian, J.: Unsupervised multilayer feature learning for satellite image scene classification. IEEE Geosci. Remote Sens. Lett. 13, 157–161 (2016)

    Article  Google Scholar 

  20. Ma, J., Zhao, J., Jiang, J., Zhou, H., Guo, X.: Locality preserving matching. Int. J. Comput. Vis. 1–20 (2017)

  21. Yang, Y., Ong, S.H., Foong, K.W.C.: A robust global and local mixture distance based non-rigid point set registration. Pattern Recogn. 48, 156–173 (2015)

    Article  Google Scholar 

  22. Ma, J., Zhao, J., Tian, J., Bai, X., Tu, Z.: Regularized vector field learning with sparse approximation for mismatch removal. Pattern Recogn. 46, 3519–3532 (2013)

    Article  MATH  Google Scholar 

  23. Burt, P., Adelson, E.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31, 532–540 (1983)

    Article  Google Scholar 

  24. Toet, A.: Image fusion by a ratio of low-pass pyramid. Pattern Recogn. Lett. 9, 245–253 (1989)

    Article  MATH  Google Scholar 

  25. Toet, A.: A morphological pyramidal image decomposition. Pattern Recogn. Lett. 9, 255–261 (1989)

    Article  MATH  Google Scholar 

  26. Nencini, F., Garzelli, A., Baronti, S., Alparone, L.: Remote sensing image fusion using the curvelet transform. Inf. Fusion 8, 143–156 (2007)

    Article  Google Scholar 

  27. Zhenfeng, S., Jun, L., Qimin, C.: Fusion of infrared and visible images based on focus measure operators in the curvelet domain. Appl. Opt. 51, 1910–1921 (2012)

    Article  Google Scholar 

  28. Adu, J., Gan, J., Wang, Y., Huang, J.: Image fusion based on nonsubsampled contourlet transform for infrared and visible light image. Infrared Phys. Technol. 61, 94–100 (2013)

    Article  Google Scholar 

  29. Zhang, Q., Maldague, X.: An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing. Infrared Phys. Technol. 74, 11–20 (2016)

    Article  Google Scholar 

  30. Huang, Y., Bi, D., Wu, D.: Infrared and visible image fusion based on different constraints in the non-subsampled shearlet transform domain. Sensors 18, 1169 (2018)

    Article  Google Scholar 

  31. El-Khamy, S.E., Hadhoud, M.M., Dessouky, M.I., Salam, B.M., El-Samie, F.E.A.: Blind multichannel reconstruction of high-resolution images using wavelet fusion. Appl. Opt. 44, 7349–7356 (2005)

    Article  Google Scholar 

  32. Zhou, Y., Gao, K., Dou, Z., Hua, Z., Wang, H.: Target-aware fusion of infrared and visible images. IEEE Access 6, 79039–79049 (2018)

    Article  Google Scholar 

  33. Ma, J., Zhou, Z., Wang, B., Zong, H.: Infrared and visible image fusion based on visual saliency map and weighted least square optimization. Infrared Phys. Technol. 82, 8–17 (2017)

    Article  Google Scholar 

  34. Hou, R., Nie, R., Zhou, D., Cao, J., Liu, D.: Infrared and visible images fusion using visual saliency and optimized spiking cortical model in non-subsampled shearlet transform domain. Multimed. Tools Appl. 78, 28609–28632 (2019)

    Article  Google Scholar 

  35. Jin, X., Jiang, Q., Yao, S., Zhou, D., Nie, R., Lee, S.-J., et al.: Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain. Infrared Phys. Technol. 88, 1–12 (2018)

    Article  Google Scholar 

  36. Heijmans, H.J., Goutsias, J.: Nonlinear multiresolution signal decomposition schemes. II. Morphological wavelets. IEEE Trans. Image Process. 9, 1897–1913 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  37. Toet, A., Van Ruyven, L.J., Valeton, J.M.: "Merging thermal and visual images by a contrast pyramid. Opt. Eng. 28, 287789 (1989)

    Article  Google Scholar 

  38. Kingsbury, N.: Image processing with complex wavelets. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 357, 2543–2560 (1999)

    Article  MATH  Google Scholar 

  39. Sun, C., Zhang, C., Xiong, N.: Infrared and visible image fusion techniques based on deep learning: a review. Electronics 9, 2162 (2020)

    Article  Google Scholar 

  40. Hou, R., Zhou, D., Nie, R., Liu, D., Xiong, L., Guo, Y., et al.: VIF-Net: an unsupervised framework for infrared and visible image fusion. IEEE Trans. Comput. Imaging 6, 640–651 (2020)

    Article  Google Scholar 

  41. Li, H., Wu, X.-J., Durrani, T.S.: Infrared and visible image fusion with ResNet and zero-phase component analysis. Infrared Phys. Technol. 102, 103039 (2019)

    Article  Google Scholar 

  42. Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM J. Optim. 9, 112–147 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  43. http://figshare.com/articles/TNO_Image_Fusion_Dataset/1008029

  44. AbouRayan, M.: Real-time image fusion processing for astronomical images. (2016)

  45. Sadjadi, F.: Comparative image fusion analysais. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05)-workshops, pp. 8–8. (2005)

  46. Leung, L.W., King, B., Vohora, V.: Comparison of image data fusion techniques using entropy and INI. In: 22nd Asian Conference on Remote Sensing, p. 9. (200)

  47. Chandana, M., Amutha, S., Kumar, N.: A hybrid multi-focus medical image fusion based on wavelet transform. Int. J. Res. Rev. Comput. Sci. 2, 948 (2011)

    Google Scholar 

  48. Thung, K.H., Raveendran, P.: A survey of image quality measures. In: 2009 international conference for technical postgraduates (TECHPOS), pp. 1–4. (2009)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Damghan University, Damghan, Iran

    Javad Abbasi Aghamaleki

  2. Department of Electrical Engineering, Sari Branch, Islamic Azad University, Sari, Iran

    Alireza Ghorbani

Authors
  1. Javad Abbasi Aghamaleki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alireza Ghorbani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alireza Ghorbani.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

All images used in this paper are provided by database [43] and they are in turn as follows:

  1. 1.

    Duine,

  2. 2.

    Soldier in trench

  3. 3.

    Bunker

  4. 4.

    Bench

  5. 5.

    Nato camp

  6. 6.

    Lake

  7. 7.

    Two men in front of the house

  8. 8.

    Soldier behind smoke

  9. 9.

    Man in doorway

  10. 10.

    Soldier behind smoke 2

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aghamaleki, J.A., Ghorbani, A. Image fusion using dual tree discrete wavelet transform and weights optimization. Vis Comput 39, 1181–1191 (2023). https://doi.org/10.1007/s00371-021-02396-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02396-9

Keywords

Search

Navigation