Skip to main content
SpringerLink
Log in

Single image defogging with a dual multiscale neural network model

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Images captured by image acquisition systems in scenes with fog or haze contain missing details, dull color, and reduced brightness. To address this problem, the dual multiscale neural network model based on the AOD theory is proposed in this paper. First, two parameters, namely transmittance and atmospheric layer coefficient, of the atmospheric scattering model are combined into a single parameter. The new neural network model proposed in this paper is then used to train this parameter. The network model proposed in this paper consists of two multiscale modules and a mapping module. In order to extract more perfect image features, this paper designs two multiscale modules for feature extraction. The convolution parameters of Multiscale Module 1 are designed to maintain the size of original images during feature extraction by adding pooling, sampling, etc. After each convolution operation, multiscale module 2 uses multiple small-sized convolution kernels for convolution, in which the concat operation is added to better connect the individual kernels, the mapping module maps the fogged images onto the extracted feature map and is able to extract more detail from the original image to obtain better defogging results after processing. Training is performed to derive a unified parameter model for image defogging, and finally, the defogged image is obtained using this parameter estimation model. The experimental results show that the model proposed this paper not only outperforms the AOD network in terms of peak signal-to-noise ratio, structural similarity, and subjective vision but also outperforms the mainstream deep learning and traditional methods in terms of image defogging; moreover, the defogged images are optimized in terms of detail, color, and brightness. In addition, ablation experiments had demonstrated that all of the structures in this paper were necessary.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

  1. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE T. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011). https://doi.org/10.1109/TPAMI.2010.168

    Article  Google Scholar 

  2. Tarel, J., Hautière, N.: Fast visibility restoration from a single color or gray level image. In: 2009 IEEE 12th Int. Conf. Comput. Vis. pp 2201–2208 (2009). https://doi.org/10.1109/ICCV.2009.5459251

  3. Sulami, M., Glatzer, I., Fattal, R., Werman, M.: Automatic recovery of the atmospheric light in hazy images. In: 2014 IEEE Int. Conf. Comput. Photogr. pp 1–11 (2014)

  4. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: DehazeNet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016). https://doi.org/10.1109/TIP.2016.2598681

    Article  MATH  MathSciNet  Google Scholar 

  5. Li, B., Zhao, J., Fu, H.: DLT-Net: deep learning transmittance network for single imagehaze removal. Signal Image Video Process. 14(6), 1245–1253 (2020). https://doi.org/10.1007/s11760-020-01665-9

    Article  Google Scholar 

  6. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: AOD-Net: All-in-One Dehazing Network, In: 2017 IEEE Int. Conf. Comput. Vis. pp 4780–4788 (2017). https://doi.org/10.1109/ICCV.2017.511

  7. Zhang, S., He, F., Ren, W., Yao, J.: Joint learning of image detail and transmission map for single image dehazing. Vis. Comput. 36, 305–316 (2020). https://doi.org/10.1007/s00371-018-1612-9

    Article  Google Scholar 

  8. Wang, A., Wang, W., Liu, J., Gu, N.: AIPNet: image-to-image single image dehazing with atmospheric illumination prior. IEEE T. Image Process. 28(1), 381–393 (2019). https://doi.org/10.1109/TIP.2018.2868567

    Article  MATH  MathSciNet  Google Scholar 

  9. Santra, S., Mondal, R., Chanda, B.: Learning a patch quality comparator for single image dehazing. IEEE T. Image Process. 27(9), 4598–4607 (2018). https://doi.org/10.1109/TIP.2018.2841198

    Article  MathSciNet  Google Scholar 

  10. Huang, Y., Wang, Y., Su, Z.: Single image dehazing via a joint deep modeling. In: 2018 25th IEEE Int. Conf. Image Process. pp 2840–2844 (2018). https://doi.org/10.1109/ICIP.2018.8451663

  11. Zhang, S., He, F., Yao, J.: Single image dehazing using deep convolution neural networks. Pac. RIM Conf. Multimed. 10735, 128–137 (2017). https://doi.org/10.1007/978-3-319-77380-3_13

    Article  Google Scholar 

  12. Chen, D., He M., Fan, Q., Liao, J., Zhang, L., Hou, D., Yuan, L., Hua, G.: Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter Conf. Appl. Comput. Vis. pp 1375–1383 (2019). https://doi.org/10.1109/WACV.2019.00151

  13. Kim, G., Ha, S., and Kwon, J.: Adaptive patch based convolutional neural network for robust dehazing. In: 2018 25th IEEE Int. Conf. Image Process. pp 2845–2849, (2018). https://doi.org/10.1109/ICIP.2018.8451252

  14. Li, C., Guo, J., Porikli, F., Fu, H., Pang, Y.: A cascaded convolutional neural network for single image dehazing. IEEE Access 6, 24877–24887 (2018). https://doi.org/10.1109/ACCESS.2018.2818882

    Article  Google Scholar 

  15. Song, Y., Li, J., Wang, X., Chen, X.: Single image dehazing using ranking convolutional neural network. IEEE T. Multimed. 20(6), 1548–1560 (2018). https://doi.org/10.1109/TMM.2017.2771472

    Article  Google Scholar 

  16. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M. H.: Single image dehazing via multi-scale convolutional neural networks. Eur. Conf. Comput. Vis. pp 154–169 (2016)

  17. You, Y., Lu, C., Wang, W., Tang, C.: Relative CNN-RNN: learning relative atmospheric visibility from images. IEEE T. Image Process. 28(1), 45–55 (2019). https://doi.org/10.1109/TIP.2018.2857219

    Article  MATH  MathSciNet  Google Scholar 

  18. Liu, K., He, L., Ma, S., Gao, S., Bi, D.: A sensor image dehazing algorithm based on feature learning. Sensors 18, 2606 (2018). https://doi.org/10.3390/s18082606

    Article  Google Scholar 

  19. Xiao, J., Zhou, J., Lei, J., Xu, C., Sui, H.: Image hazing algorithm based on generative adversarial networks. IEEE Access 8, 15883–15894 (2020). https://doi.org/10.1109/ACCESS.2019.2962784

    Article  Google Scholar 

  20. Li, R., Pan, J., Li, Z., Tang, J.: Single Image dehazing via conditional generative adversarial network. In: 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recogn. pp 8202–8211 (2018). https://doi.org/10.1109/CVPR.2018.00856

  21. Liu, W., Hou, X., Duan, J., Qiu, G.: End-to-End single image fog removal using enhanced cycle consistent adversarial networks. IEEE T. Image Process. 29, 7819–7833 (2020). https://doi.org/10.1109/TIP.2020.3007844

    Article  MATH  Google Scholar 

  22. Pan, G.: Research on image defogging, effect assessment and application, Doctoral thesis, Hunan: Central South University, (2012). https://oss.wanfangdata.com.cn/file/download/degree_Y2200026.aspx

  23. Narasimhan, S. G., Nayar, S. K.: Removing weather effects from monochrome images. In: Proc. 2001 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 2, II-II (2001), https://doi.org/10.1109/CVPR.2001.990956.

  24. Qin, X., Wang, Z., Bai, Y., Xie, X., Jia, H.: FFA-Net: feature fusion attention network for single image dehazing. Proc. AAAI Conf. Artif. Intell. 34(07), 11908–11915 (2020)

    Google Scholar 

  25. Chen, D., et al.: Gated context aggregation network for image dehazing and deraining. IEEE Winter Conf. Appl. Comput. Vision (WACV) 2019, 1375–1383 (2019). https://doi.org/10.1109/WACV.2019.00151

    Article  Google Scholar 

  26. Wu, H., Qu, Y., Lin, S., Zhou, J., Qiao, R., Zhang, Z., Xie, Y., Ma, L.: Contrastive learning for compact single image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10551–10560 (2021).

  27. Zhao, S., Zhang, L., Shen, Y., Zhou, Y.: RefineDNet: a weakly supervised refinement framework for single image dehazing. IEEE Trans. Image Process. 30, 3391–3404 (2021)

    Article  Google Scholar 

  28. Ren, W., et al.: Gated fusion network for single image dehazing. IEEE/CVF Conf. Comput. Vision Pattern Recogn. 2018, 3253–3261 (2018). https://doi.org/10.1109/CVPR.2018.00343

    Article  Google Scholar 

  29. Liu, X., Ma, Y., Shi, Z., Chen, J.: Griddehazenet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 7314–7323 (2019).

  30. Malav, R., Kim, A., Sahoo, S.R., Pandey, G.: DHSGAN: An End to End dehazing network for fog and smoke. In: conference paper [J].lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in Bioinformatics), Vol. 11365, pp. 593–608 (2019)

  31. Park, J., Han, D.K., Ko, H.: Fusion of heterogeneous adversarial networks for single image dehazing. IEEE Trans. Image Process. (2020). https://doi.org/10.1109/TIP.2020.2975986

    Article  MATH  Google Scholar 

  32. Li, Y., Liu, Y., Yan, Q., Zhang, K.: Deep dehazing network with latent ensembling architecture and adversarial learning. IEEE Trans. Image Process. 30, 1354–1368 (2021). https://doi.org/10.1109/TIP.2020.3044208

    Article  Google Scholar 

  33. Jiao, W., Jia, X., Liu, Y., Jiang, Q., Sun, Z.: Single image mixed dehazing method based on numerical iterative model and DehazeNet. PLoS One 16(7), e0254664 (2021). https://doi.org/10.1371/journal.pone.0254664

    Article  Google Scholar 

  34. Liu, J., Wang, S., Wang, X., Ju, M., Zhang, D.: A review of remote sensing image dehazing. Sensors 21(11), 3926 (2021). https://doi.org/10.3390/s21113926.PMID:34200320;PMCID:PMC8201244

    Article  Google Scholar 

  35. Chung, W.Y., Kim, S.Y., Kang, C.H.: Image dehazing using LiDAR generated grayscale depth prior. Sensors 22(3), 1199 (2022). https://doi.org/10.3390/s22031199.PMID:35161944;PMCID:PMC8839317

    Article  Google Scholar 

  36. Ngo, D., Lee, S., Ngo, T.M., Lee, G.D., Kang, B.: Visibility restoration: a systematic review and meta-analysis. Sensors 21(8), 2625 (2021). https://doi.org/10.3390/s21082625.PMID:33918021;PMCID:PMC8069147

    Article  Google Scholar 

  37. Ancuti, C.O., Ancuti, C., Timofte, R.: NH-HAZE: an image dehazing benchmark with non-homogeneous hazy and haze-free images, In: IEEE CVPR NTIRE Workshop, (2020)

  38. Ancuti, C.O., Ancuti, C., Vasluianu, F.A., Timofte, R. et al.: NTIRE 2020 Challenge on NonHomogeneous Dehazing, In: IEEE CVPR NTIRE Workshop, (2020)

Download references

Author information

Authors and Affiliations

  1. College of Communication Engineering, Jilin University, Changchun, 130012, China

    Yong Wang & Xinchao Li

Authors
  1. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinchao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Wang and Li wrote the main manuscript text and prepared all figures All authors reviewed the manuscript.

Corresponding author

Correspondence to Yong Wang.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Li, X. Single image defogging with a dual multiscale neural network model. SIViP 17, 1643–1651 (2023). https://doi.org/10.1007/s11760-022-02374-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02374-1

Keywords

Search

Navigation