Skip to main content
SpringerLink
Log in

An ensemble multi-scale residual attention network (EMRA-net) for image Dehazing

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image dehazing aims to recover a clean image from a hazy image, which is a challengingly longstanding problem. In this paper, we propose an Ensemble Multi-scale Residual Attention Network (EMRA-Net) to directly generate a clean image, which include two parts: a three-scale residual attention CNN (TRA-CNN), and an ensemble attention CNN (EA-CNN). In TRA-CNN, we employ wavelet transform to obtain the downsampled images, instead of using common spatial downsampling methods, such as nearest downsampling and strided-convolution. With the help of wavelet transform, we can avoid the loss of image texture details. Moreover, in each scale-branch, Res2Net modules are connected in series to make full use of the hierarchical features from the original hazy images, and channel attention mechanism is introduced to focus channel-dimension information. Finally, an EA-CNN is proposed to fuse coarse images generated from TRA-CNN into a refined clean image. Extensive experiments on the benchmark synthetic hazy datasets and the real-world hazy dataset prove that proposed EMRA-Net is superior to previous state-of-the-art methods both in subjective visual perception and objective image quality assessment metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

Not applicable.

References

  1. Ancuti C, Ancuti C O, De Vleeschouwer C (2016) D-HAZY: a dataset to evaluate quantitatively dehazing algorithms. In IEEE international conference on image processing (ICIP), Phoenix, AZ, USA, pp. 2226–2230

  2. Berman D, Avidan S (2016) Non-local image Dehazing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1674–1682

  3. Blau Y, Mechrez R, Timofte R, Michaeli T, Zelnik-Manor L (2018) The 2018 PIRM challenge on perceptual image super-resolution. arXiv:1809.07517

  4. Cai B, Xu X, Jia K, Qing C, Tao D (2016) DehazeNet: an end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198

    Article  MathSciNet  Google Scholar 

  5. Dong Y, Liu Y, Zhang H, Chen S, Qiao Y (2020) FD-GAN: generative adversarial networks with fusion-discriminator for single image Dehazing. In AAAI, New York City, NY, USA, pp. 10729–10736

  6. Engin D, Genç A, Kemal Ekenel H (2018) Cycle-Dehaze: enhanced CycleGAN for single image Dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, Salt Lake City, UT, USA, pp. 825–833

  7. Fattal R (2014) Dehazing using color-lines. ACM Trans Graph 34(1):1–14

    Article  Google Scholar 

  8. Gao S, Cheng MM, Zhao K, Zhang XY, Yang MH, Torr PH (2019) Res2Net: a new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell 43(2):652–662

    Article  Google Scholar 

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. In Proceedings of the 27th international conference on neural information processing systems, Montreal, Quebec, Canada, pp. 2672–2680

  10. Guo T, Seyed Mousavi H, Huu Vu T, Monga V (2017) Deep wavelet prediction for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, pp. 104–113

  11. He K, Sun J, Tang X (2010) Single image haze removal using Dark Channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Google Scholar 

  12. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 7132–7141

  13. Kingma D P, Ba J (2015) Adam: A method for stochastic optimization. arXiv:1412.6980

  14. Kumar A (2019) Design of Secure Image Fusion Technique Using Cloud for privacy-preserving and copyright protection. International Journal of Cloud Applications and Computing (IJCAC) 9(3):22–36

    Article  Google Scholar 

  15. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 4681–4690

  16. Li B, Peng X, Wang Z, Xu J, Feng D (2017) AOD-net: all-in-one Dehazing network. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, pp. 4770–4778

  17. Li R, Pan J, Li Z, Tang J (2018) Single image Dehazing via conditional generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 8202–8211

  18. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2018) Benchmarking Single-Image Dehazing and Beyond. IEEE Trans Image Process 28(1):492–505

    Article  MathSciNet  Google Scholar 

  19. Li D, Deng L, Gupta BB, Wang H, Choi C (2019) A novel CNN based security guaranteed image watermarking generation scenario for smart city applications. Inf Sci 479:432–447

    Article  Google Scholar 

  20. Lin T Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 2117–2125

  21. Ma C, Yang CY, Yang X, Yang MH (2017) Learning a no-reference quality metric for single-image super-resolution. Comput Vis Image Underst 158:1–16

    Article  Google Scholar 

  22. Mei K, Jiang A, Li J, Wang M (2018) Progressive feature fusion network for realistic image Dehazing. In Asian Conference on Computer Vision, Perth, Australia, pp. 203–215

  23. Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters 20(3):209–212

    Article  Google Scholar 

  24. Nah S, Hyun Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic Deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 3883–3891

  25. Narasimhan SG, Nayar SK (2000) Chromatic framework for vision in bad weather. In proceedings of the IEEE conference on computer vision and pattern recognition, Hilton Head Island, SC, USA, 1:598-605

  26. Narasimhan SG, Nayar SK (2002) Vision and the atmosphere. Int J Comput Vis 48(3):233–254

    Article  Google Scholar 

  27. Qu Y, Chen Y, Huang J, Xie Y (2019) Enhanced Pix2pix Dehazing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, pp. 8160–8168

  28. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang MH (2016) Single image Dehazing via multi-scale convolutional neural networks. In proceedings of the European conference on computer vision, Amsterdam, the Netherlands, pp. 154-169

  29. Ren W, Ma L, Zhang J, Pan J, Cao X, Liu W, Yang M H (2018) Gated fusion network for single image Dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, pp. 3253–3261

  30. Scharstein D, Szeliski R (2003) High-accuracy stereo depth maps using structured light. In IEEE computer society conference on computer vision and pattern recognition, Madison, WI, USA, pp. I-I

  31. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from RGBD images. In Proceedings of the European Conference on Computer Vision, Firenze, Italy, pp. 746–760

  32. Tan RT (2008) Visibility in bad weather from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8

  33. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 3156–3164

  34. Wang T C, Liu M Y, Zhu J Y, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 8798–8807

  35. Wang H, Li Z, Li Y, Gupta BB, Choi C (2020) Visual saliency guided complex image retrieval. Pattern Recogn Lett 130:64–72

    Article  Google Scholar 

  36. Wu Q, Ren W, Cao X (2019) Learning interleaved Cascade of shrinkage fields for joint image Dehazing and Denoising. IEEE Trans Image Process 29:1788–1801

    Article  MathSciNet  Google Scholar 

  37. Yang H H, Fu Y (2019) Wavelet U-net and the chromatic adaptation transform for single image Dehazing. In IEEE international conference on image processing (ICIP), Taipei, Taiwan, China, pp. 2736–2740

  38. Yang D, Sun J (2018) Proximal Dehaze-net: a prior learning-based deep network for single image Dehazing. In Proceedings of the European conference on computer vision, Munich, Germany, pp. 702–717

  39. Yang X, Xu Z, Luo J (2018) Towards perceptual image Dehazing by physics-based disentanglement and adversarial training. In AAAI, New Orleans, LA, USA, pp. 7485–7492

  40. Yang H H, Yang C H H, Tsai Y C J (2020) Y-net: multi-scale feature aggregation network with wavelet structure similarity loss function for single image Dehazing. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, pp. 2628–2632

  41. Zhang H, Patel V M (2018) Densely connected pyramid Dehazing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, pp. 3194–3203

  42. Zhang Y, Ding L, Sharma G (2017) HazeRD: an outdoor scene dataset and benchmark for single image dehazing. In IEEE international conference on image processing (ICIP), Beijing, China, pp. 3205–3209

  43. Zhang X, Wang T, Qi J, Lu H, Wang G (2018) Progressive attention guided recurrent network for salient object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, pp. 714–722

  44. Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24(11):3522–3533

    Article  MathSciNet  Google Scholar 

  45. Zhu J Y, Park T, Isola P, Efros A A (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, Venice, Italy, pp. 2223–2232

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 61771223).

Code availability

Applicable.

Funding

This study was funded by National Natural Science Foundation of China (No. 61771223).

Author information

Authors and Affiliations

  1. Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai, 201306, China

    Jixiao Wang & Chaofeng Li

  2. School of Information Science and Engineering, Changzhou University, Changzhou, 213164, China

    Shoukun Xu

Authors
  1. Jixiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaofeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shoukun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chaofeng Li.

Ethics declarations

Conflicts of interest/competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Li, C. & Xu, S. An ensemble multi-scale residual attention network (EMRA-net) for image Dehazing. Multimed Tools Appl 80, 29299–29319 (2021). https://doi.org/10.1007/s11042-021-11081-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11081-x

Keywords

Search

Navigation