Skip to main content
SpringerLink
Log in

Two-stage underwater image restoration based on gan and optical model

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Due to the unique characteristics of the underwater environment, the underwater images often have the problems of blurring and hazing, which affects the identification of image details. To enhance the details of the image, this paper proposes a two-stage restoration method based on the underwater optical model and generative adversarial network. Firstly, we synthesize the paired datasets using the underwater imaging optical model. Then, a two-stage deep learning method is employed to process the underwater images. In the first stage, the images are dehazed; in the second stage, the details of the image are improved. Finally, quantitative and qualitative experiments were conducted to evaluate the performance of the proposed method. The qualitative results show that compared with other state-of-the-art methods, our method can better highlight the image details and effectively improve the visual effects of the images. In the quantitative evaluation, the images restored using the method proposed in this paper achieved higher scores in each of the metrics, which proves to the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data that support the findings of this study are available on request from the corresponding author or the first author.

References

  1. Jiang, Q., Gu, Y., Li, C., Cong, R., Shao, F.: Underwater image enhancement quality evaluation: benchmark dataset and objective metric. IEEE Trans. Circuits Syst. Video Technol. 32(9), 5959–5974 (2022)

    Article  Google Scholar 

  2. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. ECCV 5(7576), 746–760 (2012)

    Google Scholar 

  3. Zhang, W., Wang, Y., Li, C.: Underwater image enhancement by attenuated color channel correction and detail preserved contrast enhancement. IEEE J. Ocean. Eng. 47(3), 718–735 (2022)

    Article  Google Scholar 

  4. Hummel, R.: Image enhancement by histogram transformation. Comput. Gr. Image Process. 6(2), 184–195 (1975)

    Article  Google Scholar 

  5. Zhou, J., Pang, L., Zhang, D., Zhang, W.: Underwater image enhancement method via multi-interval subhistogram perspective equalization. IEEE J. Ocean. Eng. (2023)

  6. Zhuang, P., Li, C., Wu, J.: Bayesian retinex underwater image enhancement. Eng. Appl. Artif. Intell. 101, 104171 (2021)

    Article  Google Scholar 

  7. Zhang, W., Dong, L., Xu, W.: Retinex-inspired color correction and detail preserved fusion for underwater image enhancement. Comput. Electron. Agric. 192, 106585 (2022)

    Article  Google Scholar 

  8. Zhang, W., Zhuang, P., Sun, H.-H., Li, G., Kwong, S., Li, C.: Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement. IEEE Trans. Image Process. 31, 3997–4010 (2022)

    Article  Google Scholar 

  9. Drews, P., Nascimento, E., Moraes, F., Botelho, S., Campos, M.: Transmission estimation in underwater single images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 825–830 (2013)

  10. Peng, Y.-T., Cosman, P.C.: Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 26(4), 1579–1594 (2017)

    Article  MathSciNet  Google Scholar 

  11. Song, W., Wang, Y., Huang, D., Liotta, A., Perra, C.: Enhancement of underwater images with statistical model of background light and optimization of transmission map. IEEE Trans. Broadcast. 66(1), 153–169 (2020)

    Article  Google Scholar 

  12. Zhuang, P., Wu, J., Porikli, F., Li, C.: Underwater image enhancement with hyper-Laplacian reflectance priors. IEEE Trans. Image Process. 31, 5442–5455 (2022)

    Article  Google Scholar 

  13. Islam, M.J., Xia, Y., Sattar, J.: Fast underwater image enhancement for improved visual perception. IEEE Robot. Autom. Lett. 5(2), 3227–3234 (2020)

    Article  Google Scholar 

  14. Li, C., Anwar, S., Hou, J., Cong, R., Guo, C., Ren, W.: Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Trans. Image Process. 30, 4985–5000 (2021)

    Article  Google Scholar 

  15. Wang, Y., Guo, J., Gao, H., Yue, H.: UIEC2-Net: CNN-based underwater image enhancement using two color space. Signal Process.: Image Commun. 96, 116250 (2021)

    Google Scholar 

  16. Zhang, D., Zhou, J., Zhang, W., Lin, Z., Yao, J., Polat, K., Alenezi, F., Alhudhaif, A.: ReX-Net: a reflectance-guided underwater image enhancement network for extreme scenarios. Expert Syst. Appl. 231, 120842 (2023)

    Article  Google Scholar 

  17. Zhou, J., Sun, J., Zhang, W., Lin, Z.: Multi-view underwater image enhancement method via embedded fusion mechanism. Eng. Appl. Artif. Intell. 121, 105946 (2023)

    Article  Google Scholar 

  18. Zhou, J., Zhang, D., Zhang, W.: Cross-view enhancement network for underwater images. Eng. Appl. Artif. Intell. 121, 105952 (2023)

    Article  Google Scholar 

  19. Anwar, S., Li, C.: Diving deeper into underwater image enhancement: a survey. Signal Process.: Image Commun. 89, 115978 (2020)

    Google Scholar 

  20. Li, C., Anwar, S., Porikli, F.: Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 98, 107038 (2020)

    Article  Google Scholar 

  21. Ueda, T., Yamada, K., Tanaka, Y.: Underwater image synthesis from RGB-D images and its application to deep underwater image restoration. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 2115–2119. IEEE (2019)

  22. Akkaynak, D., Treibitz, T., Shlesinger, T., Loya, Y., Tamir, R., Iluz, D.: What is the space of attenuation coefficients in underwater computer vision? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4931–4940 (2017)

  23. Liu, R., Fan, X., Zhu, M., Hou, M., Luo, Z.: Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Trans. Circuits Syst. Video Technol. 30(12), 4861–4875 (2020)

    Article  Google Scholar 

  24. Zhou, R., Susstrunk, S.: Kernel modeling super-resolution on real low-resolution images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2433–2443 (2019)

  25. Pan, J., Sun, D., Pfister, H., Yang, M.-H.: Blind image deblurring using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1628–1636 (2016)

  26. Li, C., Guo, C., Ren, W., Cong, R., Hou, J., Kwong, S., Tao, D.: An underwater image enhancement benchmark dataset and beyond. IEEE Trans. Image Process. 29, 4376–4389 (2019)

    Article  Google Scholar 

  27. Zhang, H., Patel, V.M.: Densely connected pyramid dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3194–3203 (2018)

  28. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  29. Zhu, H., Wang, C., Zhang, Y., Su, Z., Zhao, G.: Physical model guided deep image deraining. In: 2020 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2020)

  30. Ji, X., Cao, Y., Tai, Y., Wang, C., Li, J., Huang, F.: Real-world super-resolution via kernel estimation and noise injection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 466–467 (2020)

  31. Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Change Loy, C.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops (2018)

  32. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14, pp. 694–711. Springer (2016)

  33. Li, S., Liu, F., Wei, J.: Recovery of underwater images based on the attention mechanism and SOS mechanism. KSII Trans. Internet Inf. Syst. 16(8), 2552–2570 (2022)

    Google Scholar 

  34. Fu, X., Zhuang, P., Huang, Y., Liao, Y., Zhang, X.-P., Ding, X.: A retinex-based enhancing approach for single underwater image. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 4572–4576. IEEE (2014)

  35. Zhang, W., Jin, S., Zhuang, P., Liang, Z., Li, C.: Underwater image enhancement via piecewise color correction and dual prior optimized contrast enhancement. IEEE Signal Process. Lett. 30, 229–233 (2023)

    Article  Google Scholar 

  36. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  37. Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Ocean. Eng. 41(3), 541–551 (2016)

    Article  Google Scholar 

  38. Choi, L.K., You, J., Bovik, A.C.: Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Trans. Image Process. 24(11), 3888–3901 (2015)

    Article  MathSciNet  Google Scholar 

  39. Wang, S., Ma, K., Yeganeh, H., Wang, Z., Lin, W.: A patch-structure representation method for quality assessment of contrast changed images. IEEE Signal Process. Lett. 22(12), 2387–2390 (2015)

    Article  Google Scholar 

  40. Yang, M., Sowmya, A.: An underwater color image quality evaluation metric. IEEE Trans. Image Process. 24(12), 6062–6071 (2015)

    Article  MathSciNet  Google Scholar 

  41. NREL: Reference air mass 1.5 spectra. https://www.nrel.gov/grid/solar-resource/spectra-am1.5.html/ (2018)

  42. Berman, D., Levy, D., Avidan, S., Treibitz, T.: Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans. Pattern Anal. Mach. Intell. 43(8), 2822–2837 (2020)

    Google Scholar 

  43. Zhou, Y., Yan, K., Li, X.: Underwater image enhancement via physical-feedback adversarial transfer learning. IEEE J. Ocean. Eng. 47(1), 76–87 (2022)

  44. Gonzalez-Sabbagh, S., Robles-Kelly, A., Gao, S.: Dgd-cgan: a dual generator for image dewatering and restoration. arXiv preprint arXiv:2211.10026 (2022)

Download references

Funding

This study was funded by Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX20_0722).

Author information

Authors and Affiliations

  1. School of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, No.66 Xin Mofan Road, Nanjing, 210003, Jiangsu, China

    Shiwen Li, Feng Liu & Jian Wei

  2. School of Electronics and Information Engineering, Heyuan Polytechnic, Yuancheng District, Heyuan, 517000, Guangdong, China

    Shiwen Li

  3. Jiangsu KeyLab of Image Processing and Image Communications, No.66 Xin Mofan Road, Nanjing, 210003, Jiangsu, China

    Feng Liu & Jian Wei

Authors
  1. Shiwen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SL, FL, JW wrote the main manuscript text and prepared the figures. SL and FL conducted the analysis of underwater images. SL performed deep learning experiments. JW contributed to writing the manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Feng Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, S., Liu, F. & Wei, J. Two-stage underwater image restoration based on gan and optical model. SIViP 18, 379–388 (2024). https://doi.org/10.1007/s11760-023-02718-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02718-5

Keywords

Search

Navigation