Skip to main content
SpringerLink
Log in

LLCaps: Learning to Illuminate Low-Light Capsule Endoscopy with Curved Wavelet Attention and Reverse Diffusion

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 (MICCAI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14229))

Abstract

Wireless capsule endoscopy (WCE) is a painless and non-invasive diagnostic tool for gastrointestinal (GI) diseases. However, due to GI anatomical constraints and hardware manufacturing limitations, WCE vision signals may suffer from insufficient illumination, leading to a complicated screening and examination procedure. Deep learning-based low-light image enhancement (LLIE) in the medical field gradually attracts researchers. Given the exuberant development of the denoising diffusion probabilistic model (DDPM) in computer vision, we introduce a WCE LLIE framework based on the multi-scale convolutional neural network (CNN) and reverse diffusion process. The multi-scale design allows models to preserve high-resolution representation and context information from low-resolution, while the curved wavelet attention (CWA) block is proposed for high-frequency and local feature learning. Moreover, we combine the reverse diffusion procedure to optimize the shallow output further and generate images highly approximate to real ones. The proposed method is compared with eleven state-of-the-art (SOTA) LLIE methods and significantly outperforms quantitatively and qualitatively. The superior performance on GI disease segmentation further demonstrates the clinical potential of our proposed model. Our code is publicly accessible at github.com/longbai1006/LLCaps.

L. Bai and T. Chen—are co-first authors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of 1st International Conference on Image Processing, vol. 2, pp. 168–172. IEEE (1994)

    Google Scholar 

  2. Chen, W., Liu, Y., Hu, J., Yuan, Y.: Dynamic depth-aware network for endoscopy super-resolution. IEEE J. Biomed. Health Inform. 26(10), 5189–5200 (2022)

    Article  Google Scholar 

  3. Coelho, P., Pereira, A., Leite, A., Salgado, M., Cunha, A.: A deep learning approach for red lesions detection in video capsule endoscopies. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds.) ICIAR 2018. LNCS, vol. 10882, pp. 553–561. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93000-8_63

    Chapter  Google Scholar 

  4. Fan, C.M., Liu, T.J., Liu, K.H.: Half wavelet attention on M-Net+ for low-light image enhancement. In: 2022 IEEE International Conference on Image Processing (ICIP), pp. 3878–3882. IEEE (2022)

    Google Scholar 

  5. Gómez, P., Semmler, M., Schützenberger, A., Bohr, C., Döllinger, M.: Low-light image enhancement of high-speed endoscopic videos using a convolutional neural network. Med. Biol. Eng. Comput. 57(7), 1451–1463 (2019). https://doi.org/10.1007/s11517-019-01965-4

    Article  Google Scholar 

  6. Guo, C., et al.: Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789 (2020)

    Google Scholar 

  7. Guo, X., Li, Y., Ling, H.: LIME: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural. Inf. Process. Syst. 33, 6840–6851 (2020)

    Google Scholar 

  9. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008)

    Article  Google Scholar 

  10. Jiang, Y., et al.: EnlightenGAN: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340–2349 (2021)

    Article  Google Scholar 

  11. Li, C., et al.: Low-light image and video enhancement using deep learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(12), 9396–9416 (2021)

    Article  Google Scholar 

  12. Li, J., Fang, F., Mei, K., Zhang, G.: Multi-scale residual network for image super-resolution. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 517–532 (2018)

    Google Scholar 

  13. Li, M., Liu, J., Yang, W., Sun, X., Guo, Z.: Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans. Image Process. 27(6), 2828–2841 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  14. Liu, Y.F., Guo, J.M., Yu, J.C.: Contrast enhancement using stratified parametric-oriented histogram equalization. IEEE Trans. Circuits Syst. Video Technol. 27(6), 1171–1181 (2016)

    Article  Google Scholar 

  15. Long, M., Li, Z., Xie, X., Li, G., Wang, Z.: Adaptive image enhancement based on guide image and fraction-power transformation for wireless capsule endoscopy. IEEE Trans. Biomed. Circuits Syst. 12(5), 993–1003 (2018)

    Article  Google Scholar 

  16. Lore, K.G., Akintayo, A., Sarkar, S.: LLNet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recogn. 61, 650–662 (2017)

    Article  Google Scholar 

  17. Ma, Y., et al.: Structure and illumination constrained GAN for medical image enhancement. IEEE Trans. Med. Imaging 40(12), 3955–3967 (2021)

    Article  Google Scholar 

  18. Ma, Y., et al.: Cycle structure and illumination constrained GAN for medical image enhancement. In: Martel, A.L., et al. (eds.) MICCAI 2020, Part II. LNCS, vol. 12262, pp. 667–677. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_64

    Chapter  Google Scholar 

  19. Pandey, K., Mukherjee, A., Rai, P., Kumar, A.: DiffuseVAE: efficient, controllable and high-fidelity generation from low-dimensional latents. arXiv preprint arXiv:2201.00308 (2022)

  20. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  21. Sliker, L.J., Ciuti, G.: Flexible and capsule endoscopy for screening, diagnosis and treatment. Expert Rev. Med. Devices 11(6), 649–666 (2014)

    Article  Google Scholar 

  22. Smedsrud, P.H., et al.: Kvasir-capsule, a video capsule endoscopy dataset. Sci. Data 8(1), 142 (2021)

    Article  Google Scholar 

  23. Venkatanath, N., Praneeth, D., Bh, M.C., Channappayya, S.S., Medasani, S.S.: Blind image quality evaluation using perception based features. In: 2015 Twenty First National Conference on Communications (NCC), pp. 1–6. IEEE (2015)

    Google Scholar 

  24. Wang, Y., Wan, R., Yang, W., Li, H., Chau, L.P., Kot, A.: Low-light image enhancement with normalizing flow. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 2604–2612 (2022)

    Google Scholar 

  25. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  26. Wu, J., Fu, R., Fang, H., Zhang, Y., Xu, Y.: MedSegDiff-V2: diffusion based medical image segmentation with transformer. arXiv preprint arXiv:2301.11798 (2023)

  27. Xiao, T., Singh, M., Mintun, E., Darrell, T., Dollár, P., Girshick, R.: Early convolutions help transformers see better. Adv. Neural. Inf. Process. Syst. 34, 30392–30400 (2021)

    Google Scholar 

  28. Xu, X., Wang, R., Fu, C.W., Jia, J.: SNR-aware low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17714–17724 (2022)

    Google Scholar 

  29. Zamir, S.W., et al.: Learning enriched features for real image restoration and enhancement. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part XXV. LNCS, vol. 12370, pp. 492–511. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58595-2_30

    Chapter  Google Scholar 

  30. Zamir, S.W., et al.: Learning enriched features for fast image restoration and enhancement. IEEE Trans. Pattern Anal. Mach. Intell. 45(2), 1934–1948 (2022)

    Article  Google Scholar 

  31. Zhang, Q., Nie, Y., Zheng, W.S.: Dual illumination estimation for robust exposure correction. In: Computer Graphics Forum, vol. 38, pp. 243–252 (2019)

    Google Scholar 

  32. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)

    Google Scholar 

  33. Zhou, S., Li, C., Change Loy, C.: LEDNet: joint low-light enhancement and deblurring in the dark. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13666, pp. 573–589. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20068-7_33

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was supported by Hong Kong RGC CRF C4063-18G, CRF C4026-21GF, RIF R4020-22, GRF 14216022, GRF 14211420, NSFC/RGC JRS N_CUHK420/22; Shenzhen-Hong Kong-Macau Technology Research Programme (Type C 202108233000303); GBABF #2021B1515120035.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, The Chinese University of Hong Kong (CUHK), Hong Kong SAR, China

    Long Bai, Yanan Wu, An Wang & Hongliang Ren

  2. The University of Sydney, Sydney, NSW, Australia

    Tong Chen

  3. Northeastern University, Shenyang, China

    Yanan Wu

  4. Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK

    Mobarakol Islam

  5. Shun Hing Institute of Advanced Engineering, CUHK, Hong Kong SAR, China

    Hongliang Ren

Authors
  1. Long Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. An Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mobarakol Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongliang Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongliang Ren .

Editor information

Editors and Affiliations

  1. Icahn School of Medicine, Mount Sinai, NYC, NY, USA, Tel Aviv University, Tel Aviv, Israel

    Hayit Greenspan

  2. Emory University, Atlanta, GA, USA

    Anant Madabhushi

  3. Queen's University, Kingston, ON, Canada

    Parvin Mousavi

  4. The University of British Columbia, Vancouver, BC, Canada

    Septimiu Salcudean

  5. Yale University, New Haven, CT, USA

    James Duncan

  6. IBM Research, San Jose, CA, USA

    Tanveer Syeda-Mahmood

  7. Johns Hopkins University, Baltimore, MD, USA

    Russell Taylor

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 581 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bai, L., Chen, T., Wu, Y., Wang, A., Islam, M., Ren, H. (2023). LLCaps: Learning to Illuminate Low-Light Capsule Endoscopy with Curved Wavelet Attention and Reverse Diffusion. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14229. Springer, Cham. https://doi.org/10.1007/978-3-031-43999-5_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43999-5_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43998-8

  • Online ISBN: 978-3-031-43999-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation