Skip to main content
SpringerLink
Log in

An end-to-end multi-resolution feature fusion defogging network

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Traditional convolutional neural networks work well on single-image defogging synthetic datasets, but for real-world images with different concentrations, it leads to incomplete defogging or distortion and loss of image detail information. In this paper, the authors propose an end-to-end single-image defogging network, which adopts an encoder–decoder structure, obtains a large perceptual field of view of high pixels through a large number of pooling operations in the U-Net network, and uses operations such as hopping connections to retain most of the image feature information. To achieve superior quality haze removal, the proposed method utilizes a bilateral grid to capture high-frequency information pertaining to the image edges in low-resolution pixels. Additionally, relevant haze-related features are extracted, and a local affine model is fitted within the bilateral space. Finally, the high and low pixel data are integrated with the extracted features to generate clear and vivid images. The authors compare the algorithm qualitatively and quantitatively with several state-of-the-art algorithms and show that the algorithm achieves better defogging results in both the SOTS dataset and real-world images, retains high-frequency image details, achieves higher peak signal-to-noise ratio, and performs better defogging in haze images with different concentrations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

RESIDE datasets can be obtained from https://sites.google.com/view/reside-dehaze-datasets.

References

  1. Zhang, Z., Tao, D.: Slow feature analysis for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 436–450 (2012)

    Article  Google Scholar 

  2. Narasimhan, S.G., Nayar, S.K.: Contrast restoration of weather degraded images. IEEE Trans. Pattern Anal. Mach. Intell. 25(6), 713–724 (2003)

    Article  Google Scholar 

  3. Song, Y., Qian, H., Du, X.: StarEnhancer: learning real-time and style-aware image enhancement (2021)

  4. Chen, C., Wang, C., Liu, B., He, C., Cong, L., Wan, S.: Edge intelligence empowered vehicle detection and image segmentation for autonomous vehicles. IEEE Trans. Intell. Transport. Syst. (2023). https://doi.org/10.1109/TITS.2022.3232153

    Article  Google Scholar 

  5. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18, pp. 234–241. Springer (2015)

  6. Miao, Y., Zhao, X., Kan, J.: An end-to-end single image dehazing network based on U-net. SIViP 16(7), 1739–1746 (2022)

    Article  Google Scholar 

  7. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  8. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    Google Scholar 

  9. Berman, D., Avidan, S.: Non-local image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1674–1682 (2016)

  10. Wang, A., Wang, W., Liu, J., Gu, N.: AIPNet: image-to-image single image dehazing with atmospheric illumination prior. IEEE Trans. Image Process. 28(1), 381–393 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  11. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: Aod-net: all-in-one dehazing network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4770–4778 (2017)

  13. Zhu, H., Peng, X., Chandrasekhar, V., Li, L., Lim, J.-H.: Dehazegan: when image dehazing meets differential programming. In: IJCAI, pp. 1234–1240 (2018)

  14. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14, pp. 154–169. Springer (2016)

  15. Mei, K., Jiang, A., Li, J., Wang, M.: Progressive feature fusion network for realistic image dehazing. In: Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part I 14, pp. 203–215. Springer (2019)

  16. Liu, X., Ma, Y., Shi, Z., Chen, J.: Griddehazenet: attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7314–7323 (2019)

  17. Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., Yang, M.-H.: Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2157–2167 (2020)

  18. Qin, X., Wang, Z., Bai, Y., Xie, X., Jia, H.: FFA-Net: feature fusion attention network for single image dehazing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11908–11915 (2020)

  19. Fang, F., Li, J., Zeng, T.: Soft-edge assisted network for single image super-resolution. IEEE Trans. Image Process. 29, 4656–4668 (2020)

    Article  MATH  Google Scholar 

  20. Wang, L., Wang, T., Yang, D., Fang, X., Wan, S.: Near-infrared fusion for deep lightness enhancement. Int. J. Mach. Learn. Cybern. 14(5), 1621–1633 (2023)

    Article  Google Scholar 

  21. Duan, H., Shen, W., Min, X., Tian, Y., Jung, J.-H., Yang, X., Zhai, G.: Develop then rival: a human vision-inspired framework for superimposed image decomposition. IEEE Trans. Multim. (2022)

  22. Zhang, Y., Zhang, F., Jin, Y., Cen, Y., Voronin, V., Wan, S.: Local correlation ensemble with GCN based on attention features for cross-domain person Re-ID. ACM Trans. Multim. Comput. Commun. Appl. (2023). https://doi.org/10.1145/3542820

    Article  Google Scholar 

  23. Song, Y., He, Z., Qian, H., Du, X.: Vision transformers for single image dehazing. IEEE Trans. Image Process. 32, 1927–1941 (2023). https://doi.org/10.1109/tip.2023.3256763

    Article  Google Scholar 

  24. Xu, Q., Wang, L., Wang, Y., Sheng, W., Deng, X.: Deep bilateral learning for stereo image super-resolution. IEEE Signal Process. Lett. 28, 613–617 (2021)

    Article  Google Scholar 

  25. Chen, C., Liu, W., Lu, T.: Single image defogging via recurrent bilateral learning. In: 2022 4th International Conference on Robotics and Computer Vision (ICRCV), pp. 193–199 (2022). https://doi.org/10.1109/ICRCV55858.2022.9953235

  26. Gabiger-Rose, A., Kube, M., Weigel, R., Rose, R.: An FPGA-based fully synchronized design of a bilateral filter for real-time image denoising. IEEE Trans. Ind. Electron. 61(8), 4093–4104 (2013)

    Article  Google Scholar 

  27. Gharbi, M., Chen, J., Barron, J.T., Hasinoff, S.W., Durand, F.: Deep bilateral learning for real-time image enhancement. ACM Trans. Graph. (TOG) 36(4), 1–12 (2017)

    Article  Google Scholar 

  28. Zheng, Z., Ren, W., Cao, X., Hu, X., Wang, T., Song, F., Jia, X.: Ultra-high-definition image dehazing via multi-guided bilateral learning. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16180–16189. IEEE (2021)

  29. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14, pp. 694–711. Springer (2016)

  30. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)

  32. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)

  33. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., Wang, Z.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28(1), 492–505 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  34. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. ECCV 5(7576), 746–760 (2012)

    Google Scholar 

  35. Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, vol. 1. IEEE (2003)

  36. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)

  37. Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., Yuan, L., Hua, G.: Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375–1383. IEEE (2019)

Download references

Funding

Harbin Science and Technology Innovation Talent Project, No.2016RQXXJ055.

Author information

Authors and Affiliations

  1. The Higher Educational Key Laboratory for Measuring and Control Technology and Instrumentation of Heilongjiang Province, Harbin University of Science and Technology, 52 Xue Fu Lu, Harbin, 150080, Heilongjiang, China

    Ping Xue

  2. Department of College of Measurement and Communication Engineering, Harbin University of Science and Technology, 52 Xue Fu Lu, Harbin, 150080, Heilongjiang, China

    ShiXiong Deng

Authors
  1. Ping Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. ShiXiong Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PX was involved in writing, review and editing, research supervision and guidance. SD helped in data collection and processing, visualization of experimental results.

Corresponding author

Correspondence to Ping Xue.

Ethics declarations

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Ethical approval

This work did not require ethical approval under the research governance guidelines operating at the time of the research.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, P., Deng, S. An end-to-end multi-resolution feature fusion defogging network. SIViP 17, 4189–4197 (2023). https://doi.org/10.1007/s11760-023-02651-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02651-7

Keywords

Search

Navigation