Skip to main content
SpringerLink
Log in

A coarse-to-fine ghost removal scheme for HDR imaging

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Ghost removal in high dynamic range imaging is a challenging problem especially when relative camera or object motion exists. To solve the problem, an effective coarse-to-fine deghosting method combining registration and matching based on PatchMatch is proposed. Firstly, the coarse registration scheme based on Scale-Invariant Feature Transform is used to achieve the consistency of image scale space. Secondly, similarity measure of underlayer information is established by Mutual Information to realize fine registration. Thirdly, different from general distance measurement, structural similarity index measurement is employed to build the objective function to search for the best-matched patch in the fusion process. Experimental results demonstrate the algorithm can remove the ghost artifacts effectively. Furthermore, objective evaluations show that the algorithm accuracy has been improved comprehensively. Compared with the existing methods, the proposed algorithm can achieve a convincing result for dynamic senses, especially for large moving objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Niu, Y., Wu, J., Liu, W., et al.: Hdr-gan: Hdr image reconstruction from multi-exposed ldr images with large motions. IEEE Trans. Image Process. 45, 3885–3896 (2020)

    Google Scholar 

  2. Liao, X., Li, K., Zhu, X., et al.: Robust detection of image operator chain with two-stream convolutional neural network. IEEE J. Sel. Topics Signal Process. 14(5), 955–968 (2020)

    Article  Google Scholar 

  3. Lee, S., An, G. H., Kang, S.-J.: “Deep recursive hdri: Inverse tone mapping using generative adversarial networks,” In: proceedings of the European Conference on Computer Vision (ECCV), 596–611 (2018)

  4. Rana, A., Singh, P., Valenzise, G., et al.: Deep tone mapping operator for high dynamic range images. IEEE Trans. Image Process. 25, 99 (2019)

    MATH  Google Scholar 

  5. Yin, J.-L., Chen, B.-H., Peng, Y.-T.: Two exposure fusion using prior-aware generative adversarial network. IEEE Trans. Multim. 871, 1–1 (2021)

    Google Scholar 

  6. Yin, J.L., Chen, B.H., Peng, Y.T., et al.: Automatic intermediate generation with deep reinforcement learning for robust two-exposure image fusion. IEEE Trans. Neural Netw. Learn. Syst. 99, 1–10 (2021)

    Google Scholar 

  7. Ma, K., Duanmu, Z., Yeganeh, H., et al.: Multi-exposure image fusion by optimizing a structural similarity index. IEEE Trans. Comput Imag. 4(1), 60–72 (2017)

    Article  MathSciNet  Google Scholar 

  8. Liao, X., Yin, J., Chen, M., et al.: Adaptive payload distribution in multiple images steganography based on image texture features. IEEE Trans. Dependable Secure Comput. 24, 1 (2020)

    Google Scholar 

  9. Liao, X., Yu, Y., Li, B., et al.: A new payload partition strategy in color image steganography. IEEE Trans. Circuit Syst. Video Technol. 30(3), 685–696 (2020)

    Article  Google Scholar 

  10. Mertens, T., Kautz, J., Van, R.F.: Exposure fusion: a simple and practical alternative to high dynamic range photography. Comput. Graph Forum. 28(1), 161–171 (2009)

    Article  Google Scholar 

  11. Rui, S., Cheng, I., S, J., et al.: Generalized random walks for fusion of multi-exposure images. IEEE Trans. Image Process. 20(12), 3634–3646 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Lee, C., Li, Y., Monga, V.: Ghost-free high dynamic range imaging via rank minimization. IEEE Signal Process. Lett. 21(9), 1045–1049 (2014)

    Article  Google Scholar 

  13. Hu, J., Gallo, O., Pulli, K., et al.: “Hdr deghosting: How to deal with saturation?,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Portland., 1164–1170 (2013)

  14. Wang, Z., Bovik, A.C., Sheikh, H.R., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  15. Heo, Y. S., Lee, S. U., K. M., Lee, “Ghost-free high dynamic range imaging,” in Proc. Conf. on Comp. Vision., San Francisco., 486–500 (2010)

  16. Gallo, O., Troccoli, A., Hu, J., et al.: “Locally non-rigid registration for mobile hdr photography,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Boston., wos:000378887900101 (2015)

  17. Qin, X., Shen, J., Mao, X., et al.: Robust match fusion using optimization. IEEE Trans. Cybern. 45(8), 1549–1560 (2015)

    Article  Google Scholar 

  18. Sen, P., Kalantari, N.K., Yaesoubi, M., et al.: Robust patch-based hdr reconstruction of dynamic scenes. ACM Trans. Graph. 31(6), 541 (2012). https://doi.org/10.1145/23661452366222

    Article  Google Scholar 

  19. Kalantari, N.K., Ramamoorthi, R.: Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph. 36(4), 1–12 (2017)

    Article  Google Scholar 

  20. Yan, Q., Gong, D., Shi, Q., et al.: “Attention-guided network for ghost-free high dynamic range imaging,” In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Los Angeles., 1851–1760 (2019)

  21. Huang, W., Zhang, W., Pan, L., et al.: “Image fusion localization algorithm based on sift and 3d model,” In: Seventh Symposium on Novel Photoelectronic Detection Technology and Application 2020, (2021)

  22. Wang, L., Chang, C., Liu, Z., et al.: A medical image fusion method based on sift and deep convolutional neural network in the sist domain. J. Healthcare Eng. 2021, 9958017 (2021)

    Google Scholar 

  23. Pluim, J.P.W., Maintz, J., Viergever, M.A.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med Imag. 22(8), 986–1004 (2003)

    Article  Google Scholar 

  24. C. available HDR processing software, “Photomatix,” (2012)

  25. Tursun, O.T., Akyüz, A., Erdem, A., et al.: An objective deghosting quality metric for hdr images. IEEE Signal Process Lett. 35(2), 139–152 (2016)

    Google Scholar 

  26. Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)

    Article  Google Scholar 

  27. Baroncini, V., Capodiferro, L., Di, E. D.: “The polar edge coherence: A quasi blind metric for video quality assessment,” In: European Signal Processing Conference, Nice., 564–568 (2015)

  28. Yeganeh, H., Wang, Z.: Objectsive quality assessment of tone-mapped images. IEEE Trans. Image Process. 22(2), 657–667 (2012)

    Article  MATH  Google Scholar 

  29. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind image quality analyzer. IEEE Signal Process Lett. 22(3), 209–213 (2012)

    Article  Google Scholar 

  30. Xia, X., Yao, Y., Liang, J., et al.: Evaluation of focus measures for the autofocus of line scan cameras. Optik - Int. J. Light Electron Opt. 127(19), 7762–7775 (2016)

    Article  Google Scholar 

  31. Hui, Z.: An entropy-based objective evaluation method for image segmentation. Storage Retriev. Methods Appl. Multim., San Jose. 5307, 38–49 (2004)

    Google Scholar 

  32. Narwaria, M., Mantiuk, R.K., Silva, M., et al.: Hdr-vdp-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images. J. Electron Imag. 24(1), 010501 (2014)

    Article  Google Scholar 

Download references

Acknowledgements

The work described in this paper was partially supported by the National Natural Science Foundation of China (61772432, 61503309).

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Southwest University, Chongqing, 400715, China

    Shufang Xia & Songtao Guo

  2. College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China

    Shufang Xia & Zhong Qu

  3. College of Computer Science, Chongqing University, Chongqing, 400044, China

    Songtao Guo

  4. Department of Electrical & Computer Engineering, Stony Brook University, Stony Brook, NY, 11794, USA

    Yuanyuan Yang

Authors
  1. Shufang Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Songtao Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhong Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuanyuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Songtao Guo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xia, S., Guo, S., Qu, Z. et al. A coarse-to-fine ghost removal scheme for HDR imaging. Vis Comput 39, 2515–2528 (2023). https://doi.org/10.1007/s00371-022-02475-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02475-5

Keywords

Search

Navigation