Skip to main content
SpringerLink
Log in

QRCP Decomposition-Based Hybrid Approach for Fusion of Visible and Infrared Images

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Image fusion integrates several images from different modalities into a single image which contains more spatial and spectral resolution. Artifacts, smoothing and ringing are major issues in convolutional neural networks, edge-preserving filters and transform-based image fusion methods. Orthogonal rectangular with column pivoting (QRCP) matrix factorization-based hybrid approach is proposed in this work to overcome the above issues and to improve the fusion of visible and infrared images. QRCP decomposition is an accurate matrix decomposition that separates base layer and detail layer from source images. Discrete cosine transform and local spatial frequency concept are employed to fuse the base layers. Weight maps are utilized to transfer information into detail layers that are obtained directly from base layers. The obtained fused image is a linear combination of final base layer and final detail layer. The proposed method outperforms the existing methods in terms of performance measures such as entropy, spatial frequency, mutual information, normalized weight performance index, mean and standard deviation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Source images, b, e base layers, c, f detail layers

Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability Statement

The raw/processed data required to reproduce these findings cannot be shared at this time as the data also form part of an ongoing study.

References

  1. D.P. Bavirisetti, R. Dhuli, Two-scale image fusion of visible and infrared images using saliency detection. Infrared Phys. Technol. 76, 52–64 (2016)

    Google Scholar 

  2. D.P. Bavirisetti, R. Dhuli, Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen–Loeve transform. IEEE Sens. J. 16(1), 203–209 (2015)

    Google Scholar 

  3. P. Burt, E. Adelson, The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983). https://doi.org/10.1109/TCOM.1983.1095851

    Article  Google Scholar 

  4. P.J. Burt, R.J. Kolczynski, Enhanced image capture through fusion, in 1993 (4th) International Conference on Computer Vision (IEEE (1993), pp. 173–182

  5. D. Cai, X. He, J. Han, T.S. Huang, Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1548–1560 (2010)

    Google Scholar 

  6. E. Candes, L. Demanet, D. Donoho, L. Ying, Fast discrete curvelet transforms. Multiscale Model. Simul. 5, 861–899 (2006)

    MathSciNet  MATH  Google Scholar 

  7. S. Chakroborty, G. Saha, Feature selection using singular value decomposition and QR factorization with column pivoting for text-independent speaker identification. Speech Commun. 52(9), 693–709 (2010)

    Google Scholar 

  8. A.L. Da Cunha, J. Zhou, M.N. Do, The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans. Image Process. 15(10), 3089–3101 (2006)

    Google Scholar 

  9. M.N. Do, M. Vetterli, The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 14(12), 2091–2106 (2005)

    Google Scholar 

  10. V.N. Gangapure, S. Banerjee, A.S. Chowdhury, Steerable local frequency based multispectral multifocus image fusion. Inf. Fusion 23, 99–115 (2015)

    Google Scholar 

  11. P. Ghamisi, R. Gloaguen, P. Atkinson, J. Benediktsson, B. Rasti, N. Yokoya, Q. Wang, B. Hofle, L. Bruzzone, F. Bovolo, M. Chi, K. Anders, Multisource and multitemporal data fusion in remote sensing: a comprehensive review of the state of the art. IEEE Geosci. Remote Sens. Mag. 7, 6–39 (2019)

    Google Scholar 

  12. C. Hu, X. Lu, Y. Du, An adaptive method based on QRCP decomposition for single sample problem, in Proceedings of the 33rd Chinese Control Conference (IEEE, 2014), pp. 4826–4830

  13. C.H. Hu, X.B. Lu, P. Liu, X.Y. Jing, D. Yue, Single sample face recognition under varying illumination via QRCP decomposition. IEEE Trans. Image Process. 28(5), 2624–2638 (2018)

    MathSciNet  Google Scholar 

  14. L.C. Jiao, S. Tan, F. Liu, Ridgelet theory: from ridgelet transform to curvelet. Chin. J. Eng. Math. 22(5), 761–773 (2005)

    MathSciNet  MATH  Google Scholar 

  15. X. Jin, R. Nie, D. Zhou, Q. Wang, K. He, Multifocus color image fusion based on NSST and PCNN. J. Sens. 206, 1–12 (2016)

    Google Scholar 

  16. X. Jin, Q. Jiang, S. Yao, D. Zhou, R. Nie, J. Hai, K. He, A survey of infrared and visual image fusion methods. Infrared Phys. Technol. 85, 478–501 (2017)

    Google Scholar 

  17. X. Jin, Q. Jiang, S. Yao, D. Zhou, R. Nie, S.J. Lee, K. He, Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain. Infrared Phys. Technol. 88, 1–12 (2018)

    Google Scholar 

  18. P.P. Kanjilal, G. Saha, T.J. Koickal, On robust nonlinear modeling of a complex process with large number of inputs using m-QRcp factorization and C/sub p/statistic. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 29(1), 1–12 (1999)

    Google Scholar 

  19. W. Kong, L. Zhang, Y. Lei, Novel fusion method for visible light and infrared images based on NSST–SF–PCNN. Infrared Phys. Technol. 65, 103–112 (2014)

    Google Scholar 

  20. H. Li, B. Manjunath, S. Mitra, Multisensor image fusion using the wavelet transform. Graph. Models Image Process. 57, 235–245 (1995)

    Google Scholar 

  21. S. Li, X. Kang, L. Fang, J. Hu, H. Yin, Pixel-level image fusion: a survey of the state of the art. Inf. Fusion 33, 100–112 (2017)

    Google Scholar 

  22. S. Li, X. Kang, J. Hu, Image fusion with guided filtering. IEEE Trans. Image Process. 22(7), 2864–2875 (2013)

    Google Scholar 

  23. H. Li, X.J. Wu, Infrared and visible image fusion using latent low-rank representation. arXiv:1804.08992 (2018)

  24. X. Li, L. Wang, Q. Cheng, P. Wu, W. Gan, L. Fang, Cloud removal in remote sensing images using nonnegative matrix factorization and error correction. ISPRS J. Photogramm. Remote. Sens. 148, 103–113 (2019)

    Google Scholar 

  25. W.Q. Lim, Nonseparable shearlet transform. IEEE Trans. Image Process. 22(5), 2056–2065 (2013)

    MathSciNet  MATH  Google Scholar 

  26. Y. Liu, X. Chen, J. Cheng, H. Peng, Z. Wang, Infrared and visible image fusion with convolutional neural networks. Int. J. Wavel. Multiresolut. Inf. Process. 16(03), 1850018 (2018)

    MathSciNet  MATH  Google Scholar 

  27. J. Ma, Y. Zhou, Infrared and visible image fusion via gradientlet filter. Comput. Vis. Image Underst. 197–198, 103016 (2020)

    Google Scholar 

  28. J. Ma, Y. Ma, C. Li, Infrared and visible image fusion methods and applications: a survey. Inf. Fusion 45, 153–178 (2019)

    Google Scholar 

  29. J. Ma, Z. Zhou, B. Wang, H. Zong, Infrared and visible image fusion based on visual saliency map and weighted least square optimization. Infrared Phys. Technol. 82, 8–17 (2017)

    Google Scholar 

  30. S. Maqsood, U. Javed, Multi-modal medical image fusion based on two-scale image decomposition and sparse representation. Biomed. Signal Process. Control 57, 101810 (2020)

    Google Scholar 

  31. S. Maqsood, U. Javed, M. Riaz, M. Muzammil, F. Muhammad, S. Kim, Multiscale image matting based multi-focus image fusion technique. Electronics 9, 472 (2020)

    Google Scholar 

  32. V. Naidu, J. Raol, Pixel-level image fusion using wavelets and principal component analysis. Def. Sci. J. 58, 338–352 (2008)

    Google Scholar 

  33. C. Pohl, J. Van Genderen, Review article multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19, 823–854 (1998)

    Google Scholar 

  34. E.P. Simoncelli, W.T. Freeman, The steerable pyramid: a flexible architecture for multi-scale derivative computation, in Proceedings., International Conference on Image Processing vol. 3. (IEEE, 1995), pp. 444–447

  35. S. Singh, R.S. Anand, Multimodal medical image sensor fusion model using sparse K-SVD dictionary learning in nonsubsampled shearlet domain. IEEE Trans. Instrum. Meas. 69(2), 593–607 (2019)

    Google Scholar 

  36. J. Swoger, P. Verveer, K. Greger, J. Huisken, E. Stelzer, Multi-view image fusion improves resolution in three-dimensional microscopy. Opt. Express 15, 8029 (2007)

    Google Scholar 

  37. W. Tan, H. Zhou, J. Song, H. Li, Y. Yu, J. Du, Infrared and visible image perceptive fusion through multi-level Gaussian curvature filtering image decomposition. Appl. Opt. 58(12), 3064–3073 (2019)

    Google Scholar 

  38. A. Ufade, M. Kawade, Comparison of spatial domain and transformation domain image fusion technique for restoration of blur images. Int. J. Electron. Commun. Soft Comput. Sci. Eng. IJECSCSE 2, 93 (2013)

    Google Scholar 

  39. Z. Wang, D. Ziou, C. Armenakis, D. Li, Q. Li, A comparative analysis of image fusion methods. IEEE Trans. Geosci. Remote Sens. 43(6), 1391–1402 (2005)

    Google Scholar 

  40. X. Wang, J. Yin, K. Zhang, S. Li, J. Yan, Infrared weak-small targets fusion based on latent low-rank representation and DWT. IEEE Access 7, 112681–112692 (2019)

    Google Scholar 

  41. Q. Xiao-Bo, Y. Jing-Wen, X.I. Hong-Zhi, Z. Zi-Qian, Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Autom. Sin. 34(12), 1508–1514 (2008)

    Google Scholar 

  42. L. Xuejun, W. Minghui, Research of multi-focus image fusion algorithm based on sparse representation and orthogonal matching pursuit. in Chinese Conference on Image and Graphics Technologies (Springer, Berlin, Heidelberg, 2014), pp. 57–66

  43. C.A. Xydeas, V. Petrovic, Objective image fusion performance measure. Electron. Lett. 36(4), 308–309 (2000)

    Google Scholar 

  44. S. Yin, Y. Zhang, Singular value decomposition-based anisotropic diffusion for fusion of infrared and visible images. Int. J. Image Data Fusion 10(2), 146–163 (2019)

    Google Scholar 

  45. S. Yu, X. Chen, Infrared and visible image fusion based on a latent low-rank representation nested with multiscale geometric transform. IEEE Access 8, 110214–110226 (2020)

    Google Scholar 

  46. B. Zhang, X. Lu, H. Pei, Y. Zhao, A fusion algorithm for infrared and visible images based on saliency analysis and non-subsampled Shearlet transform. Infrared Phys. Technol. 73, 286–297 (2015)

    Google Scholar 

  47. Q. Zhang, X. Maldague, An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing. Infrared Phys. Technol. 74, 11–20 (2016)

    Google Scholar 

  48. T.Y. Zhang, Q. Zhou, H.J. Feng, Z.H. Xu, Q. Li, Y.T. Chen, Fusion of infrared and visible light images based on nonsubsampled shearlet transform. in International Symposium on Photoelectronic Detection and Imaging 2013: Infrared Imaging and Applications, vol. 8907 (International Society for Optics and Photonics 2013), p. 89071H

  49. Y. Zhang, L. Chen, J. Jia, Z. Zhao, Multi-focus image fusion based on non-negative matrix factorization and difference images. Signal Process. 105, 84–97 (2014)

    Google Scholar 

  50. J. Zhao, S. Qu, A better algorithm for fusion of infrared and visible image based on curvelet transform and adaptive pulse coupled neural networks (PCNN). J. Northwest. Polytech. Univ. 29(6), 849–853 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Vidya Jyothi Institute of Technology, Hyderabad, 500075, India

    C. Rajakumar

  2. Anurag Group of Institutions, Hyderabad, 500088, India

    S. Satheeskumaran

Authors
  1. C. Rajakumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Satheeskumaran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Rajakumar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rajakumar, C., Satheeskumaran, S. QRCP Decomposition-Based Hybrid Approach for Fusion of Visible and Infrared Images. Circuits Syst Signal Process 40, 6146–6172 (2021). https://doi.org/10.1007/s00034-021-01757-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-021-01757-y

Keywords

Search

Navigation