Skip to main content
SpringerLink
Log in

Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Nighttime image dehazing aims to remove the effect of haze on the images captured in nighttime, which however, raises new challenges such as severe color distortion, more complex lighting conditions, and lower contrast. Instead of estimating the transmission map and atmospheric light that are difficult to be accurately acquired in nighttime, we propose a nighttime image dehazing method composed of a color cast removal and a dual path multi-scale fusion algorithm. We first propose a human visual system (HVS) inspired color correction model, which is effective for removing the color deviation on nighttime hazy images. Then, we propose to use dual path strategy that includes an underexposure and a contrast enhancement path for multi-scale fusion, where the weight maps are achieved by selecting appropriate exposed areas under Gaussian pyramids. Extensive experiments demonstrate that the visual effect of the hazy nighttime images in real-world datasets can be significantly improved by our method regarding contrast, color fidelity, and visibility. In addition, our method outperforms the state-of-the-art methods qualitatively and quantitatively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Gao Y, Su Y, Li Q M, Li H Y, Li J. Single image dehazing via self-constructing image fusion. Signal Processing, 2020, 167: 107284

    Article  Google Scholar 

  2. Li Y, You S D, Brown M S, Tan R T. Haze visibility enhancement: a survey and quantitative benchmarking. Computer Vision and Image Understanding, 2017, 165: 1–16

    Article  Google Scholar 

  3. Dai C G, Lin M X, Wu X J, Zhang D. Single hazy image restoration using robust atmospheric scattering model. Signal Processing, 2020, 166: 107257

    Article  Google Scholar 

  4. Lin Y T, Wu Y, Yan C G, Xu M L, Yang Y. Unsupervised person reidentification via cross-camera similarity exploration. IEEE Transactions on Image Processing, 2020, 29: 5481–5490

    Article  MATH  Google Scholar 

  5. Liu Q, Gao X B, He L H, Lu W. Haze removal for a single visible remote sensing image. Signal Processing, 2017, 137: 33–43

    Article  Google Scholar 

  6. Zhao D, Xu L, Yan Y H, Chen J, Duan L Y. Multi-scale optimal fusion model for single image dehazing. Signal Processing: Image Communication, 2019, 74: 253–265

    Google Scholar 

  7. Li C Y, Guo J C, Cong R M, Pang Y W, Wang B. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Transactions on Image Processing, 2016, 25(12): 5664–5677

    Article  MathSciNet  MATH  Google Scholar 

  8. Li C Y, Guo C L, Guo J C, Han P, Fu H Z, Cong R M. PDR-Net: perception-inspired single image dehazing network with refinement. IEEE Transactions on Multimedia, 2020, 22(3): 704–716

    Article  Google Scholar 

  9. Yuan F, Huang H. Image haze removal via reference retrieval and scene prior. IEEE Transactions on Image Processing, 2018, 27(9): 4395–4409

    Article  MathSciNet  MATH  Google Scholar 

  10. Liu P J, Horng S J, Lin J S, Li T R. Contrast in haze removal: configurable contrast enhancement model based on dark channel prior. IEEE Transactions on Image Processing, 2019, 28(5): 2212–2227

    Article  MathSciNet  Google Scholar 

  11. He K M, Sun J, Tang X. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353

    Article  Google Scholar 

  12. Zhu Q S, Mai J M, Shao L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015, 24(11): 3522–3533

    Article  MathSciNet  MATH  Google Scholar 

  13. Berman D, Treibitz T, Avidan S. Non-Local image dehazing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1674–1682

  14. Santra S, Chanda B. Day/night unconstrained image dehazing. In: Proceedings of the 23rd International Conference on Pattern Recognition. 2016, 1406–1411

  15. Kim G, Kwon J. Robust pixel-wise dehazing algorithm based on advanced haze-relevant features. In: Proceedings of British Machine Vision Conference. 2017, 1–12

  16. Yang A P, Liu J, Ji Z, Pan Y W. Detail-preserving single nighttime image dehazing. Journal of Electronic Imaging, 2020, 29(4): 043010

    Article  Google Scholar 

  17. Finlayson G D, Trezzi E. Shades of gray and colour constancy. In: Proceedings of the 12th Color Imaging Conference: Color Science and Engineering Systems, Technologies, Applications. 2004, 37–41

  18. Van De Weijer J, Gevers T, Gijsenij A. Edge-based color constancy. IEEE Transactions on Image Processing, 2007, 16(9): 2207–2214

    Article  MathSciNet  Google Scholar 

  19. Gao S, Yang K, Li C. Color constancy using double-opponency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(10): 1973–1985

    Article  Google Scholar 

  20. Ancuti C O, Ancuti C, Vleeschouwer C D, Bekaert P. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 2018, 27(1): 379–393

    Article  MathSciNet  MATH  Google Scholar 

  21. Galdran A. Image dehazing by artificial multiple-exposure image fusion. Signal Processing, 2018, 149: 135–147

    Article  Google Scholar 

  22. Zhang X S, Gao S B, Li C Y, Li Y J. A retina inspired model for enhancing visibility of hazy images. Frontiers in Computational Neuroscience, 2015, 9: 1–13

    Article  Google Scholar 

  23. Zhang X, Gao S, Li R, Du X, Li C, Li Y. A retinal mechanism inspired color constancy model. IEEE Transactions on Image Processing, 2016, 25(3): 1219–1232

    Article  MathSciNet  MATH  Google Scholar 

  24. Pei S C, Lee T Y. Nighttime haze removal using color transfer preprocessing and dark channel prior. In: Proceedings of the 19th IEEE International Conference on Image Processing. 2012, 957–960

  25. Zhang J, Cao Y, Wang Z. Nighttime haze removal based on a new imaging model. In: Proceedings of IEEE International Conference on Image Processing. 2014, 4557–4561

  26. Li Y, Tan R T, Brown M S. Nighttime haze removal with glow and multiple light colors. In: Proceedings of IEEE International Conference on Computer Vision. 2015, 226–234

  27. Zhang J, Cao Y, Fang S, Kang Y, Chen C W. Fast haze removal for nighttime image using maximum reflectance prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 7016–7024

  28. Yang M M, Liu J C, Li Z G. Superpixel-based single nighttime image haze removal. IEEE Transactions on Multimedia, 2018, 20(11): 3008–3018

    Article  Google Scholar 

  29. Ancuti C, Ancuti C O, Vleeschouwer C D, Bovik A C. Night-time dehazing by fusion. In: Proceedings of IEEE International Conference on Image Processing. 2016, 2256–2260

  30. Ancuti C, Ancuti C O, Vleeschouwer C D, Bovik A C. Day and nighttime dehazing by local airlight estimation. IEEE Transactions on Image Processing, 2020, 29: 6264–6275

    Article  MATH  Google Scholar 

  31. Liao Y, Su Z, Liang X, Qiu B. HDP-Net: haze density prediction network for nighttime dehazing. In: Hong R, Cheng W H, Yamasaki T, Wang M, Ngo C W, eds. Advances in Multimedia Information Processing. Springer, Cham, 2018, 469–480

    Google Scholar 

  32. Kuanar S, Rao K R, Mahapatra D, Bilas M. Night time haze and glow removal using deep dilated convolutional network. 2019, arXiv preprint arXiv: 1902.00855

  33. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z H, Shi W Z. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 4681–4690

  34. Engin D, Genc A, Ekenel H K. Cycle-Dehaze: enhanced CycleGAN for single image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018, 938–946

  35. Zhang K H, Luo W H, Zhong Y R, Ma L, Liu W, Li H D. Adversarial spatio-temporal learning for video deblurring. IEEE Transactions on Image Processing, 2019, 28(1): 291–301

    Article  MathSciNet  Google Scholar 

  36. Zhang K H, Luo W H, Zhong Y R, Ma L, Stenger B, Liu W, Li H D. Deblurring by realistic blurring. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2020, 2737–2746

  37. Reinhard E, Adhikhmin M, Gooch B, Shirley P. Color transfer between images. IEEE Computer Graphics and Applications, 2001, 21(5): 34–41

    Article  Google Scholar 

  38. Lee B B, Martin P R, Grünert U. Retinal connectivity and primate vision. Progress in Retinal and Eye Research, 2010, 29: 622–639

    Article  Google Scholar 

  39. Gao S B, Yang K F, Li C Y, Li Y J. A color constancy model with double-opponency mechanisms. In: Proceedings of IEEE International Conference on Computer Vision. 2013, 929–936

  40. Li Y N, Miao Q G, Liu R Y, Song J F, Quan Y N, Huang Y H. A multi-scale fusion scheme based on haze-relevant features for single image dehazing. Neurocomputing, 2018, 283: 73–86

    Article  Google Scholar 

  41. Zuiderveld K. Contrast Limited Adaptive Histogram Equalization. Academic Press Professional, Inc., 1994, 474–485

  42. Gijsenij A, Gevers T, Van De Weijer J. Computational color constancy: survey and experiments. IEEE Transactions on Image Processing, 2011, 20(9): 2475–2489

    Article  MathSciNet  MATH  Google Scholar 

  43. Gijsenij A, Gevers T, Van De Weijer J. Improving color constancy by photometric edge weighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(5): 918–929

    Article  Google Scholar 

  44. Yu T, Song K, Miao P, Yang G, Yang H, Chen C. Nighttime single image dehazing via pixel-wise alpha blending. IEEE Access, 2019, 7: 114619–114630

    Article  Google Scholar 

  45. Xu Y, Wen J, Fei L, Zhang Z. Review of video and image defogging algorithms and related studies on image restoration and enhancement. IEEE Access, 2016, 4: 165

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Higher Education Scientific Research Project of Ningxia (NGY2017009).

Author information

Authors and Affiliations

  1. School of Physics and Electronic-Electrical Engineering, Ningxia University, Yinchuan, 750021, China

    Bo Wang, Li Hu, Bowen Wei & Zitong Kang

  2. School of Computer Science and Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Chongyi Li

Authors
  1. Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bowen Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zitong Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chongyi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Wang.

Additional information

Bo Wang received his PhD degree from School of Electrical and Information Engineering, Tianjin University, China in 2016. He is currently a lecturer in School of Physics and Electronic-Electrical Engineering, Ningxia University, China. His research interests include image restoration and enhancement, image classification and medical image processing.

Li Hu received her BS degree in Ningxia University, China in 2018. She is currently working toward MS degree in School of Physics and Electronic-Electrical Engineering in Ningxia University, China. Her research interests include image processing and computer vision.

Bowen Wei received his BS degree in Ningxia Normal University, China in 2018. He is currently working toward MS degree in School of Physics and Electronic Electrical Engineering in Ningxia University, China. His research interests include computer vision and machine learning.

Zitong Kang received her BS degree from School of Automation and Electronic Engineering, Qingdao University of Science and Technology, China in 2019. She is currently studying in School of Physics, Electronic and Electrical Engineering, Ningxia University, China, majoring in Electronic and Communication Engineering. Her research interests include computer vision and data mining.

Chongyi Li received his PhD degree from School of Electrical and Information Engineering, Tianjin University, China in June 2018. From 2016 to 2017, he was a joint-training PhD Student with Australian National University, Australia. He was a postdoctoral fellow with Department of Computer Science, City University of Hong Kong, China. He is currently a research fellow with School of Computer Science and Engineering, Nanyang Technological University, Singapore. His current research focuses on image processing, computer vision, and deep learning, particularly in the domains of image restoration and enhancement.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, B., Hu, L., Wei, B. et al. Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy. Front. Comput. Sci. 16, 164706 (2022). https://doi.org/10.1007/s11704-021-0162-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-021-0162-x

Keywords

Search

Navigation