Skip to main content
SpringerLink
Log in

Lighting Enhancement Aids Reconstruction of Colonoscopic Surfaces

  • Conference paper
  • First Online:
Information Processing in Medical Imaging (IPMI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12729))

Included in the following conference series:

Abstract

High screening coverage during colonoscopy is crucial to effectively prevent colon cancer. Previous work has allowed alerting the doctor to unsurveyed regions by reconstructing the 3D colonoscopic surface from colonoscopy videos in real-time. However, the lighting inconsistency of colonoscopy videos can cause a key component of the colonoscopic reconstruction system, the SLAM optimization, to fail. In this work we focus on the lighting problem in colonoscopy videos. To successfully improve the lighting consistency of colonoscopy videos, we have found necessary a lighting correction that adapts to the intensity distribution of recent video frames. To achieve this in real-time, we have designed and trained an RNN network. This network adapts the gamma value in a gamma-correction process. Applied in the colonoscopic surface reconstruction system, our light-weight model significantly boosts the reconstruction success rate, making a larger proportion of colonoscopy video segments reconstructable and improving the reconstruction quality of the already reconstructed segments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    We implemented the training in [6] without color constancy loss using the colonoscopy images from our training set.

References

  1. Chen, Y.S., Wang, Y.C., Kao, M.H., Chuang, Y.Y.: Deep photo enhancer: Unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6306–6314 (2018)

    Google Scholar 

  2. Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2017)

    Article  Google Scholar 

  3. Engel, J., Sturm, J., Cremers, D.: Semi-dense visual odometry for a monocular camera. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449–1456 (2013)

    Google Scholar 

  4. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: The kitti dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

  5. Gharbi, M., Chen, J., Barron, J.T., Hasinoff, S.W., Durand, F.: Deep bilateral learning for real-time image enhancement. ACM Trans. Graph. (TOG) 36(4), 1–12 (2017)

    Article  Google Scholar 

  6. Guo, C., et al.: Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789 (2020)

    Google Scholar 

  7. Huang, S.C., Cheng, F.C., Chiu, Y.S.: Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE Trans. Image Process. 22(3), 1032–1041 (2012)

    Article  MathSciNet  Google Scholar 

  8. Jiang, Y., et al.: Enlightengan: deep light enhancement without paired supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  9. Ke, Z., Qiu, D., Li, K., Yan, Q., Lau, R.W.: Guided collaborative training for pixel-wise semi-supervised learning. In: European Conference on Computer Vision (ECCV) (August 2020)

    Google Scholar 

  10. Luft, T., Colditz, C., Deussen, O.: Image enhancement by un sharp masking the depth buffer. ACM Trans. Graph. (TOG) 25(3), 1206–1213 (2006)

    Article  Google Scholar 

  11. Ma, R., Wang, R., Pizer, S., Rosenman, J., McGill, S.K., Frahm, J.-M.: Real-Time 3D reconstruction of colonoscopic surfaces for determining missing regions. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 573–582. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_64

    Chapter  Google Scholar 

  12. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  13. Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: Dtam: dense tracking and mapping in real-time. In: 2011 International Conference on Computer Vision, pp. 2320–2327. IEEE (2011)

    Google Scholar 

  14. Pizer, S.M., et al.: Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987)

    Article  Google Scholar 

  15. Polesel, A., Ramponi, G., Mathews, V.J.: Image enhancement via adaptive unsharp masking. IEEE Trans. Image Process. 9(3), 505–510 (2000)

    Article  Google Scholar 

  16. Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  17. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-d slam systems. In: Proceedings of the International Conference on Intelligent Robot Systems (IROS) (October 2012)

    Google Scholar 

  18. Tateno, K., Tombari, F., Laina, I., Navab, N.: CNN-slam: Real-time dense monocular slam with learned depth prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6243–6252 (2017)

    Google Scholar 

  19. Wang, R., Zhang, Q., Fu, C.W., Shen, X., Zheng, W.S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6849–6857 (2019)

    Google Scholar 

  20. Wang, S., Clark, R., Wen, H., Trigoni, N.: Deepvo: towards end-to-end visual odometry with deep recurrent convolutional neural networks. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2043–2050. IEEE (2017)

    Google Scholar 

  21. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  22. Xiao, B., Xu, Y., Tang, H., Bi, X., Li, W.: Histogram learning in image contrast enhancement. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  23. Xingjian, S., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, pp. 802–810 (2015)

    Google Scholar 

  24. Yang, N., Wang, R., Stuckler, J., Cremers, D.: Deep virtual stereo odometry: leveraging deep depth prediction for monocular direct sparse odometry. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 817–833 (2018)

    Google Scholar 

Download references

Acknowledgement

We thank Prof. Jan-Michael Frahm for useful consultations. This work was carried out with financial support from the Olympus Corporation, the UNC Kenan Professorship Fund, and the UNC Lineberger Cancer Center.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, USA

    Yubo Zhang, Shuxian Wang, Ruibin Ma & Stephen M. Pizer

  2. Department of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, USA

    Sarah K. McGill

  3. Department of Radiation Oncology, University of North Carolina at Chapel Hill, Chapel Hill, USA

    Julian G. Rosenman

Authors
  1. Yubo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuxian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruibin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sarah K. McGill
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Julian G. Rosenman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stephen M. Pizer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yubo Zhang .

Editor information

Editors and Affiliations

  1. Technical University of Denmark, Kgs Lyngby, Denmark

    Aasa Feragen

  2. University of Copenhagen, Copenhagen, Denmark

    Stefan Sommer

  3. King's College London, London, UK

    Julia Schnabel

  4. University of Copenhagen, Copenhagen, Denmark

    Mads Nielsen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, Y., Wang, S., Ma, R., McGill, S.K., Rosenman, J.G., Pizer, S.M. (2021). Lighting Enhancement Aids Reconstruction of Colonoscopic Surfaces. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds) Information Processing in Medical Imaging. IPMI 2021. Lecture Notes in Computer Science(), vol 12729. Springer, Cham. https://doi.org/10.1007/978-3-030-78191-0_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78191-0_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78190-3

  • Online ISBN: 978-3-030-78191-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation