Skip to main content
SpringerLink
Log in

Underwater video consistent enhancement: a real-world dataset and solution with progressive quality learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Underwater image and video enhancement task is of great significance for ocean exploration. Compared with a single image, underwater video enhancement is more susceptible to scene and light changes, which causes inconsistency between the frames of enhanced video. In this paper, we construct a high-quality dataset named UVE38K to establish a benchmark for underwater video enhancement, which consists of 50 real-world videos from various environments. To understand the difference in quality between underwater video frames. We propose a quality superiority decision network (QSDNet) to distinguish the high-quality and low-quality frames in enhanced videos. Our QSDNet can achieve an accuracy of 87.9%. We also propose two underwater video enhancement algorithms PUVE and BUVE for online and offline situations respectively. Experiments on the UVE38K dataset show that our methods outperform existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Algorithm 1
Algorithm 2
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability Statements

The datasets supporting the conclusions of this article are included within the article.

Notes

  1. http://www.diveplus.cn/app

  2. https://homepages.inf.ed.ac.uk/rbf/Fish4Knowledge/

  3. https://habcam.whoi.edu/

  4. https://www.aamvem.com/datachallenge

  5. https://www.mbari.org

  6. https://github.com/dlut-dimt/Realworld-Underwater-Image-Enhancement-RUIE-Benchmark

  7. https://github.com/IPNUISTlegal/underwater-test-dataset-U45-

  8. https://li-chongyi.github.io/proj_underwater_image_synthesis.html

  9. http://irvlab.cs.umn.edu/resources

  10. http://irvlab.cs.umn.edu/resources/euvp-dataset

  11. http://irvlab.cs.umn.edu/resources/ufo-120-dataset

  12. https://ieee-dataport.org/open-access/suid-synthetic-underwater-image-dataset

  13. http://csms.haifa.ac.il/profiles/tTreibitz/datasets/sea_thru/index.html

  14. http://csms.haifa.ac.il/profiles/tTreibitz/datasets/ambient_forwardlo-oking/index.html

  15. https://li-chongyi.github.io/proj_benchmark.html

  16. https://diveplus.cn/#community

  17. http://www.cnurpc.org/index.html

References

  1. Abdul Ghani A S, Mat Isa N A (2015) Underwater image quality enhancement through integrated color model with rayleigh distribution. Appl Soft Comput 27:219–230

    Article  Google Scholar 

  2. Akkaynak D, Treibitz T (2019) Sea-thru: a method for removing water from underwater images. In: IEEE Conference on computer vision and pattern recognition, pp 1682–1691

  3. Ancuti C, Ancuti C O, Haber T, Bekaert P (2012) Enhancing underwater images and videos by fusion. In: IEEE Conference on computer vision and pattern recognition, pp 81–88

  4. Anwar S, Li C (2020) Diving deeper into underwater image enhancement: a survey. Signal Process: Image Commun 89:115978

    Google Scholar 

  5. Anwar S, Li C, Porikli F (2018) Deep underwater image enhancement. arXiv:1807.03528

  6. Bai L, Zhang W, Pan X, Zhao C (2020) Underwater image enhancement based on global and local equalization of histogram and dual-image multi-scale fusion. IEEE Access 8:128973–128990

    Article  Google Scholar 

  7. Berman D, Levy D, Avidan S, Treibitz T (2020) Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans Pattern Anal Mach Intell

  8. Berman D, Treibitz T, Avidan S (2017) Diving into haze-lines: color restoration of underwater images. In: British machine vision conference

  9. Boom B J, He J, Palazzo S, Huang P X, Beyan C, Chou H-M, Lin F-P, Spampinato C, Fisher R B (2014) A research tool for long-term and continuous analysis of fish assemblage in coral-reefs using underwater camera footage. Eco Inform 23(9):83–97

    Article  Google Scholar 

  10. Buchsbaum G (1980) A spatial processor model for object colour perception. J Franklin Inst 310(1):1–26

    Article  Google Scholar 

  11. Cao K, Peng Y, Cosman P C (2018) Underwater image restoration using deep networks to estimate background light and scene depth. In: IEEE Southwest symposium on image analysis and interpretation, pp 1–4

  12. Carlevaris-Bianco N, Mohan A, Eustice R M (2010) Initial results in underwater single image dehazing. In: MTS/IEEE SEATTLE, pp 1–8

  13. Chambah M, Semani D, Renouf A, Courtellemont P, Rizzi A (2003) Underwater color constancy: enhancement of automatic live fish recognition. In: Color imaging IX: processing, hardcopy, and applications, vol 5293, pp 157–168

  14. Chiang J Y, Chen Y (2012) Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans Image Process 21(4):1756–1769

    Article  MathSciNet  Google Scholar 

  15. Cutter G, Stierhoff K, Zeng J (2015) Automated detection of rockfish in unconstrained underwater videos using haar cascades and a new image dataset: labeled fishes in the wild. In: IEEE Winter applications and computer vision workshops, pp 57–62

  16. Dai C, Lin M, Wu X, Wang Z, Guan Z (2020) Single underwater image restoration by decomposing curves of attenuating color. Opt Laser Technol 123:105947

    Article  Google Scholar 

  17. Ding D, Wang W, Tong J, Gao X, Liu Z, Fang Y (2022) Biprediction-based video quality enhancement via learning. IEEE Trans Cybern 52(2):1207–1220

    Article  Google Scholar 

  18. Drews P L J, Nascimento E R, Botelho S S C, Montenegro Campos M F (2016) Underwater depth estimation and image restoration based on single images. IEEE Comput Graphics Appl 36(2):24–35

    Article  Google Scholar 

  19. Fabbri C, Islam M J, Sattar J (2018) Enhancing underwater imagery using generative adversarial networks. In: IEEE International conference on robotics and automation, pp 7159–7165

  20. Fu X, Fan Z, Ling M, Huang Y, Ding X (2017) Two-step approach for single underwater image enhancement. In: International symposium on intelligent signal processing and communication systems , pp 789–794

  21. Fu X, Zhuang P, Huang Y, Liao Y, Zhang X, Ding X (2014) A retinex-based enhancing approach for single underwater image. In: IEEE International conference on image processing, pp 4572–4576

  22. Fu X, Cao X (2020) Underwater image enhancement with global–local networks and compressed-histogram equalization. Signal Process: Image Commun 86:115892

    Google Scholar 

  23. Ghani A S A, Isa N A M (2014) Underwater image quality enhancement through composition of dual-intensity images and rayleigh-stretching. In: IEEE International conference on consumer electronics. Berlin, pp 219–220

  24. Guan Z, Xing Q, Xu M, Yang R, Liu T, Wang Z (2021) Mfqe 2.0: a new approach for multi-frame quality enhancement on compressed video. IEEE Trans Pattern Anal Mach Intell 43(3):949–963

    Article  Google Scholar 

  25. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Article  Google Scholar 

  26. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE Conference on computer vision and pattern recognition, pp 770–778

  27. Hou G, Zhao X, Pan Z, Yang H, Tan L, Li J (2020) Benchmarking underwater image enhancement and restoration, and beyond. IEEE Access 8:122078–122091

    Article  Google Scholar 

  28. Hou G, Li J, Wang G, Yang H, Huang B, Pan Z (2020) A novel dark channel prior guided variational framework for underwater image restoration. J Vis Commun Image Represent 66:102732

    Article  Google Scholar 

  29. Hou G, Pan Z, Wang G, Yang H, Duan J (2019) An efficient nonlocal variational method with application to underwater image restoration. Neurocomputing 369:106–121

    Article  Google Scholar 

  30. Hou M, Liu R, Fan X, Luo Z (2018) Joint residual learning for underwater image enhancement. In: IEEE International conference on image processing, pp 4043–4047

  31. Hu J, Jiang Q, Cong R, Gao W, Shao F (2021) Two-branch deep neural network for underwater image enhancement in hsv color space. IEEE Signal Process Lett 28:2152–2156

    Article  Google Scholar 

  32. Huang D, Wang Y, Song W, Sequeira J, Mavromatis S (2018) Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition. In: MultiMedia Modeling, pp 453–465

  33. Iqbal K, Odetayo M, James A, Salam RA, Talib AZH (2010) Enhancing the low quality images using unsupervised colour correction method. In: IEEE International conference on systems, man and cybernetics, pp 1703–1709

  34. Islam M J, Xia Y, Sattar J (2020) Fast underwater image enhancement for improved visual perception. IEEE Robot Autom Lett 5(2):3227–3234

    Article  Google Scholar 

  35. Islam M J, Luo P, Sattar J (2020) Simultaneous enhancement and super-resolution of underwater imagery for improved visual perception. In: Robotics: science and systems

  36. Jaffe J S (1990) Computer modeling and the design of optimal underwater imaging systems. IEEE J Oceanic Eng 15(2):101–111

    Article  Google Scholar 

  37. Jian M, Qi Q, Dong J, Yin Y, Zhang W, Lam K (2017) The ouc-vision large-scale underwater image database. In: IEEE International conference on multimedia and expo, pp 1297–1302

  38. Kashif I, Salam R A, Azam O, Talib A Z (2007) Underwater image enhancement using an integrated colour model. Iaeng Int J Comput Sci 34(2):239–244

    Google Scholar 

  39. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  40. Kumar N, Sardana H K, Shome S N (2019) Saliency based shape extraction of objects in unconstrained underwater environment. Multimed Tools Applic 78(11):15121–15139

    Article  Google Scholar 

  41. Li C, Guo C, Ren W, Cong R, Hou J, Kwong S, Tao D (2020) An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Process 29:4376–4389

    Article  Google Scholar 

  42. Li C, Guo J, Cong R, Pang Y, Wang B (2016) Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans Image Process 25(12):5664–5677

    Article  MathSciNet  Google Scholar 

  43. Li C, Guo J, Guo C (2018) Emerging from water: underwater image color correction based on weakly supervised color transfer. IEEE Signal Process Lett 25(3):323–327

    Article  Google Scholar 

  44. Li C, Quo J, Pang Y, Chen S, Wang J (2016) Single underwater image restoration by blue-green channels dehazing and red channel correction. In: IEEE International conference on acoustics, speech and signal processing, pp 1731–1735

  45. Li C, Anwar S, Porikli F (2020) Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn 98:107038

    Article  Google Scholar 

  46. Li H, Li J, Wang W (2019) A fusion adversarial underwater image enhancement network with a public test dataset. arXiv:1906.06819

  47. Li Y, Zhang Y, Xu X, He L, Serikawa S, Kim H (2019) Dust removal from high turbid underwater images using convolutional neural networks. Opti Laser Technol 110:2–6

    Article  Google Scholar 

  48. Lin W H, Zhong J X, Liu S, Li T, Li G (2020) Roimix: proposal-fusion among multiple images for underwater object detection. In: IEEE International conference on acoustics, speech and signal processing, pp 2588–2592

  49. Liu R, Fan X, Zhu M, Hou M, Luo Z (2020) Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Trans Circuits Syst Video Technol 30(12):4861–4875

    Article  Google Scholar 

  50. Liu X, Gao Z, Chen B M (2020) Mlfcgan: multilevel feature fusion-based conditional gan for underwater image color correction. IEEE Geosci Remote Sens Lett 17(9):1488–1492

    Article  Google Scholar 

  51. Liu X, Gao Z, Chen B M (2021) Ipmgan: integrating physical model and generative adversarial network for underwater image enhancement. Neurocomputing 453:538–551

    Article  Google Scholar 

  52. Liu C, Meng W (2010) Removal of water scattering. In: International conference on computer engineering and technology, vol 2, pp V2–35–V2–39

  53. Lu J, Li N, Zhang S, Yu Z, Zheng H, Zheng B (2019) Multi-scale adversarial network for underwater image restoration. Opt Laser Technol 110:105–113

    Article  Google Scholar 

  54. M Uplavikar P, Wu Z, Wang Z (2019) All-in-one underwater image enhancement using domain-adversarial learning. In: IEEE Conference on computer vision and pattern recognition workshops

  55. McGlamery B L (1980) A computer model for underwater camera systems. In: Ocean Optics VI, vol 0208, pp 221–231

  56. Mohd Azmi K Z, Abdul Ghani A S, Md Yusof Z, Ibrahim Z (2019) Natural-based underwater image color enhancement through fusion of swarm-intelligence algorithm. Appl Soft Comput 85:105810

    Article  Google Scholar 

  57. Oleari F, Kallasi F, Rizzini D L, Aleotti J, Caselli S (2015) An underwater stereo vision system: from design to deployment and dataset acquisition. In: OCEANS, pp 1–6

  58. Panetta K, Gao C, Agaian S (2016) Human-visual-system-inspired underwater image quality measures. IEEE J Oceanic Eng 41(3):541–551

    Article  Google Scholar 

  59. Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Graphics Appl 21(5):34–41

    Article  Google Scholar 

  60. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from rgbd images. In: European conference on computer vision, pp 746–760

  61. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image ecognition. In: International conference on learning representations

  62. Srividhya K, Ramya M M (2017) Accurate object recognition in the underwater images using learning algorithms and texture features. Multimed Tools Applic 76(24):25679–25695

    Article  Google Scholar 

  63. Tang C, Von Lukas U F, Vahl M, Wang S, Wang Y, Tan M (2019) Efficient underwater image and video enhancement based on retinex. SIViP 13(5):1011–1018

    Article  Google Scholar 

  64. Torres-Méndez L A, Dudek G (2005) Color correction of underwater images for aquatic robot inspection. In: Energy minimization methods in computer vision and pattern recognition. Springer, Berlin, pp 60–73

  65. Trucco E, Olmos-Antillon A T (2006) Self-tuning underwater image restoration. IEEE J Oceanic Eng 31(2):511–519

    Article  Google Scholar 

  66. Wang Y, Wu B (2010) Fast clear single underwater image. In: International conference on computational intelligence and software engineering, pp 1–4

  67. Yang M, Hu J, Li C, Rohde G, Du Y, Hu K (2019) An in-depth survey of underwater image enhancement and restoration. IEEE Access 7:123638–123657

    Article  Google Scholar 

  68. Yang M, Sowmya A (2015) An underwater color image quality evaluation metric. IEEE Trans Image Process 24(12):6062–6071

    Article  MathSciNet  Google Scholar 

  69. Yu H, Li X, Lou Q, Lei C, Liu Z (2020) Underwater image enhancement based on DCP and depth transmission map. Multimed Tools Applic 79 (27–28):20373–20390

    Article  Google Scholar 

  70. Zhou K, Qiao Y, Xiang T (2018) Deep reinforcement learning for unsupervised video summarization with diversity-representativeness reward. Proceedings of the AAAI Conference on Artificial Intelligence 32:1

    Article  Google Scholar 

  71. Zhou W, Bovik A C, Sheikh H R, Simoncelli E P (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  72. Zhu J, Park T, Isola P, Efros A A (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International conference on computer vision, pp 2242–2251

  73. Zuiderveld K (1994) Contrast limited adaptive histogram equalization. Graphics Gems, 474–485

Download references

Acknowledgments

The research has been supported by the National Natural Science Foundation of China under Grant 61906177, in part by the Natural Science Foundation of Shandong Province under Grant ZR2019BF034, in part by Significant Applied Technology Innovation Projects for Agriculture of Shandong Province under Grant SD2019NJ020, and in part by Fundamental Research Funds for the Central Universities, under Grants 201813022 and 201964013.

Author information

Authors and Affiliations

  1. School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China

    Yongchang Zhang

  2. School of Information and Control Engineering, Qingdao University of Technology, Qingdao, China

    Qi Qi

  3. College of Engineering, Ocean University of China, Qingdao, China

    Kunqian Li & Dandan Liu

  4. College of Electrical Engineering, Yancheng Institute of Technology, Yancheng, China

    Dandan Liu

Authors
  1. Yongchang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kunqian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dandan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qi Qi.

Ethics declarations

Conflict of Interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Qi, Q., Li, K. et al. Underwater video consistent enhancement: a real-world dataset and solution with progressive quality learning. Multimed Tools Appl 83, 7335–7361 (2024). https://doi.org/10.1007/s11042-023-15542-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15542-3

Keywords

Search

Navigation