Skip to main content
SpringerLink
Log in

Multi-exposure image fusion using structural weights and visual saliency map

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multi Exposure Image Fusion (MEIF) represents a procedure for combining multiple images with various exposure levels into a single image for good visual perception. The traditional techniques often suffer from spatial inconsistency, visual distortion, noisy weights maps, losing of vivid colour information. To addresses these issues, in this article we proposed a MEIF using structural weights and a visual saliency map. Source images are decomposed into contrast, structure and intensity features to find the its detail layers. To preserve the edge information for better spatial consistent structures, base layers of source images will be generated through Rolling Guided Filter (RGF). To retain vivid colours and avoid visual distortion we used saliency maps of source images. A weight map generator compares the base layers and saliency maps in order to avoid noisy weight maps. Finally fused image will be generated through fused base and detail layers. The effectiveness of the proposed MEIF method has been evaluated both objectively and subjectively, and the results show that it is superior to a subset of already available solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  1. Ma K, Li H, Yong H, Wang Z, Meng D, Zhang L (2017) Robust multiexposure image fusion: A structural patch decomposition approach. IEEE Trans Image Process 26(5):2519–2532

    Article  MathSciNet  Google Scholar 

  2. Shen J, Zhao Y, Yan S, Li X (2014) Exposure fusion using boosting Laplacian pyramid. IEEE Trans Cybern 44(9):1579–1590

    Article  Google Scholar 

  3. Li H, Yang Z, Zhang Y, Tao D, Zhengtao Yu (2024) Single-Image HDR Reconstruction Assisted Ghost Suppression and Detail Preservation Network for Multi-Exposure HDR Imaging. IEEE Trans Comput Imaging 10:429–445

    Article  Google Scholar 

  4. Zhang Z, Wang H, Liu S, Wang X, Lei L, Zuo W (2023) Self-supervised high dynamic range imaging with multi-exposure images in dynamic scenes, in arXiv preprint arXiv:2310.01840

  5. Liu Z, Wang Y, Zeng B, Liu S (2022) Ghost-free High Dynamic Range Imaging with Context-Aware Transformer, in European Conference on computer vision 2022, ECCV 2022

  6. Chen H, Ren Y, Cao J, Liu W, Liu K (2019) Multi-exposure fusion for welding region based on multi-scale transform and hybrid weight. Int J Adv Manuf Technol 101:105–117

    Article  Google Scholar 

  7. Yuan L, Wenbo Wu, Dong S, He Q, Zhang F (2023) A High Dynamic Range Image Fusion Method Based on Dual Gain Image. Int J Image Data Fusion 14(1):15–37

    Article  Google Scholar 

  8. Krishnamoorthy S, Punithavathani S, Priya JK (2017) Extraction of well-exposed pixels for image fusion with a sub-banding technique for high dynamic range images. Int J Image Data Fusion 8(1):54–72

    Article  Google Scholar 

  9. TirumalaVasu G, Palanisamy P (2023) Gradient-based multi-focus image fusion using foreground and background pattern recognition with weighted anisotropic diffusion filter. Signal Image Video Process 17:2531–2543

    Article  Google Scholar 

  10. TirumalaVasu G, Palanisamy P (2022) Multi-focus image fusion using anisotropic diffusion filter. Soft Comput 26(24):14029–14040

    Article  Google Scholar 

  11. Yadav SKr, Sarawadekar K (2023) Effective edge-aware weighting filter-based structural patch decomposition multi-exposure image fusion for single image dehazing. Multidim Syst Signal Process 34:543–574

    Article  Google Scholar 

  12. Mertens T, Kautz J, Van Reeth F (2009) Exposure Fusion: A Simple and Practical Alternative to High Dynamic Range Photography. Comput Graph Forum 28(1):161–171

    Article  Google Scholar 

  13. Hu J, Gallo O, Pulli K, Sun X (2013) HDR Deghosting: How to Deal with Saturation?, in 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland

  14. Ahmad A, Riaz MM, Ghafoor A, Zaidi T (2016) Noise Resistant Fusion for Multi-Exposure Sensors. IEEE Sens J 16(13):5123–5124

    Article  Google Scholar 

  15. Yang Y, Cao W, Wu S, Li Z (2018) Multi-Scale Fusion of Two Large-Exposure-Ratio Images. IEEE Signal Process Lett 25(12):1885–1889

    Article  Google Scholar 

  16. Singh H, Cristobal G, Bueno G, Blanco S, Singh S, Hrisheekesha PN, Mittal N (2022) Multi-exposure microscopic image fusion-based detail enhancement algorithm. Ultramicroscopy 236:113499

    Article  Google Scholar 

  17. Ma K, Wang Z (2015) Multi-exposure image fusion: A patch-wise approach, in 2015 IEEE International Conference on Image Processing (ICIP)

  18. Huang F, Zhou D, Nie R, Yu C (2018) A Color Multi-Exposure Image Fusion Approach Using Structural Patch Decomposition. IEEE Access 6:42877–42885

    Article  Google Scholar 

  19. Li H, Ma K, Yong H, Zhang L (2020) Fast multi-scale structural patch decomposition for multi-exposure image fusion. IEEE Trans Image Process 29:5805–5816. https://doi.org/10.1109/TIP.2020.2987133

  20. Li H, Chan TN, Qi X, Xie W (2021) Detail-preserving multi-exposure fusion with edge-preserving structural patch decomposition. IEEE Trans Circuits Syst Video Technol 31(11):4293–4304. https://doi.org/10.1109/TCSVT.2021.3053405

  21. Jian L, Yang X, Zhou Z, Zhou K, Liu K (2018) Multi-scale image fusion through rolling guidance filter. Futur Gener Comput Syst 83:310–325

    Article  Google Scholar 

  22. Zhang Q, Shen X, Xu L, Jia J (2014) Rolling Guidance Filter, in Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science

  23. Liu Y, Zhiyong Wu, Han X, Sun Q, Zhao J, Liu J (2022) Infrared and visible image fusion based on visual saliency map and image contrast enhancement. Sensors 22(17):6390

    Article  Google Scholar 

  24. Liu Yi, Zhang D, Zhang Q, Han J (2022) Part-object relational visual saliency. IEEE Trans Pattern Anal Mach Intell 44(7):3688–3704

    Google Scholar 

  25. Yang Y, Zhang Y, Huang S, Zuo Y, Sun J (2021) Infrared and visible image fusion using visual saliency sparse representation and detail injection model. IEEE Trans Instrum Meas 70:1–15

    Article  Google Scholar 

  26. Zhang Q, Liu Yi, Blum RS, Han J, Tao D (2018) Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review. Information Fusion 40:57–75

    Article  Google Scholar 

  27. Romaniak P, Janowski L, Leszczuk M, Papir Z (2011) A no reference metric for the quality assessment of videos affected by exposure distortion, in 2011 IEEE International Conference on Multimedia and Expo, Barcelona, Spain

  28. Yang X, Lin W, Lu Z, Ong EP, Yao S (2005) Motion-compensated residue preprocessing in video coding based on just-noticeable-distortion profile. IEEE Trans Circ Syst Video Technol 15(6):742–752

    Article  Google Scholar 

  29. Liu A, Lin W, Paul M, Deng C, Zhang F (2010) Just noticeable difference for images with decomposition model for separating edge and textured regions. IEEE Trans Circuits Syst Video Technol 20(11):1648–1652

    Article  Google Scholar 

  30. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  31. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images, in Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271), Bombay, India

  32. Godtliebsen F, Spj⊘tvoll E, Marron JS (1996) A nonlinear gaussian filter applied to images with discontinuities. J Nonparametric Stat 8(1):21–43

    Article  MathSciNet  Google Scholar 

  33. Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Graph 27(3):1–10

    Article  Google Scholar 

  34. TirumalaVasu G, Palanisamy P (2023) CT and MRI multi-modal medical image fusion using weight-optimized anisotropic diffusion filtering. Soft Comput 27(13):9105–9117

    Article  Google Scholar 

  35. Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues, in Proceedings of the 14th ACM international conference on Multimedia

  36. Cheng M-M, Mitra NJ, Huang X, Torr PHS, Hu S-M (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582

    Article  Google Scholar 

  37. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection, in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA

  38. HDR Photography Gallery Samples (2016) [Online]. Available: http://www.easyhdr.com/examples

  39. Dani Lischinski HDR Webpage (2016) [Online]. Available: http://www.cs.huji.ac.il/~/hdr/pages/belgium.html

  40. Martin Cˆadík HDR Webpage (2016) [Online]. Available: http://cadik.posvete.cz/tmo

  41. MATLAB HDR Webpage (2016) [Online]. Available: http://www.mathworks.com/help/images/ref/makehdr.html

  42. Li W, Xiao X, Xiao P, Wang H, Xu F (2022) Change detection in multitemporal SAR images based on slow feature analysis combined with improving image fusion strategy. IEEE J Sel Top Appl Earth Obs Remote Sens 15:3008–3023

    Article  Google Scholar 

  43. Jindal M, Bajal E, Chakraborty A, Singh P, Diwakar M, Kumar N (2021) A novel multi-focus image fusion paradigm: A hybrid approach. Mater Today Proc 37(2):2952–2958

    Article  Google Scholar 

  44. Guo L, Cao X, Liu L (2020) Dual-tree biquaternion wavelet transform and its application to color image fusion. Signal Process 171:107513s

    Article  Google Scholar 

  45. Kong W, Miao Q, Lei Y, Ren C (2022) Guided filter random walk and improved spiking cortical model based image fusion method in NSST domain. Neurocomputing 488:509–527

    Article  Google Scholar 

  46. Zhang X, He H, Zhang J-X (2022) Multi-focus image fusion based on fractional order differentiation and closed image matting. ISA Trans 129:703–714

    Article  Google Scholar 

  47. Jia J, Sun J, Zhu Z (2021) A multi-scale patch-wise algorithm for multi-exposure image fusion. Optik 248:168120

    Article  Google Scholar 

  48. Han Y, Cai Y, Cao Y, Xu X (2013) A new image fusion performance metric based on visual information fidelity. Inf Fusion 14:127–135

    Article  Google Scholar 

  49. Wang Z, Bovik A, Sheikh H, Simoncelli E (2004) Image Quality Assessment: From Error Measurement to Structural Similarity. IEEE Trans Image Proc 13(4):600–612

    Article  Google Scholar 

Download references

Funding

This research received no specific grant from any funding agency.

Author information

Authors and Affiliations

  1. Department of Electronics & Communications Engineering, National Institute of Technology, Tiruchirapalli, Tamil Nadu, 620015, India

    G. Tirumala Vasu & P. Palanisamy

  2. Presidency University, Bangalore, 560064, India

    G. Tirumala Vasu

Authors
  1. G. Tirumala Vasu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Palanisamy
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Tirumala Vasu G – Algorithm implementation and simulation.

P. Palanisamy – Problem statement and Quality metrics.

Corresponding author

Correspondence to G. Tirumala Vasu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tirumala Vasu, G., Palanisamy, P. Multi-exposure image fusion using structural weights and visual saliency map. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-19355-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-19355-w

Keywords

Search

Navigation