Skip to main content
SpringerLink
Log in

Colonoscopy Coverage Revisited: Identifying Scanning Gaps in Real-Time

  • Conference paper
  • First Online:
Cancer Prevention Through Early Detection (CaPTion 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14295))

Included in the following conference series:

Abstract

Colonoscopy is the most widely used medical technique for preventing Colorectal Cancer, by detecting and removing polyps before they become malignant. Recent studies show that around 25% of the existing polyps are routinely missed. While some of these do appear in the endoscopist’s field of view, others are missed due to a partial coverage of the colon. The task of detecting and marking unseen regions of the colon has been addressed in recent work, where the common approach is based on dense 3D reconstruction, which proves to be challenging due to lack of 3D ground truth and periods with poor visual content. In this paper we propose a novel and complementary method to detect deficient local coverage in real-time for video segments where a reliable 3D reconstruction is impossible. Our method aims to identify skips along the colon caused by a drifted position of the endoscope during poor visibility time intervals. The proposed solution consists of two phases. During the first, time segments with good visibility of the colon and gaps between them are identified. During the second phase, a trained model operates on each gap, answering the question: “Do you observe the same scene before and after the gap?” If the answer is negative, the endoscopist is alerted and can be directed to the appropriate area in real-time. The second phase model is trained using a contrastive loss based on the auto-generated examples. Our method evaluation on a dataset of 250 procedures annotated by trained physicians provides sensitivity of 75% with specificity of 90%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adjabi, I., Ouahabi, A., Benzaoui, A., Taleb-Ahmed, A.: Past, present, and future of face recognition: a review. Electronics 9(8), 1188 (2020)

    Article  Google Scholar 

  2. Ali, S., Rittscher, J.: Efficient video indexing for monitoring disease activity and progression in the upper gastrointestinal tract. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 91–95. IEEE (2019)

    Google Scholar 

  3. Bachman, P., Hjelm, R.D., Buchwalter, W.: Learning representations by maximizing mutual information across views. Adv. Neural Inf. Process. Syst. 32, 1–11 (2019)

    Google Scholar 

  4. Bae, G., et al.: Digiface-1m: 1 million digital face images for face recognition. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3526–3535 (2023)

    Google Scholar 

  5. Berton, G., Masone, C., Paolicelli, V., Caputo, B.: Viewpoint invariant dense matching for visual geolocalization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12169–12178 (2021)

    Google Scholar 

  6. Brownlee, J.: XGBoost With python: gradient boosted trees with XGBoost and scikit-learn. In: Machine Learning Mastery (2016)

    Google Scholar 

  7. Chen, H., Wang, Y., Lagadec, B., Dantcheva, A., Bremond, F.: Joint generative and contrastive learning for unsupervised person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2004–2013 (2021)

    Google Scholar 

  8. Chen, R.J., Bobrow, T.L., Athey, T., Mahmood, F., Durr, N.J.: Slam endoscopy enhanced by adversarial depth prediction. arXiv preprint arXiv:1907.00283 (2019)

  9. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)

    Google Scholar 

  10. Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4690–4699 (2019)

    Google Scholar 

  11. Freedman, D., et al.: Detecting deficient coverage in colonoscopies. IEEE Trans. Med. Imaging 39(11), 3451–3462 (2020)

    Article  Google Scholar 

  12. Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 1735–1742. IEEE (2006)

    Google Scholar 

  13. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)

    Google Scholar 

  14. Howard, A., et al.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019)

    Google Scholar 

  15. Kelner, O., Weinstein, O., Rivlin, E., Goldenberg, R.: Motion-based weak supervision for video parsing with application to colonoscopy. In: Proceedings of the “What is Motion for?” Workshop, ECCV (2022)

    Google Scholar 

  16. Kim, N.H., et al.: Miss rate of colorectal neoplastic polyps and risk factors for missed polyps in consecutive colonoscopies. Intest. Res. 15(3), 411 (2017)

    Article  Google Scholar 

  17. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  18. Kortli, Y., Jridi, M., Al Falou, A., Atri, M.: Face recognition systems: a survey. Sensors 20(2), 342 (2020)

    Article  Google Scholar 

  19. Lin, T.Y., Belongie, S., Hays, J.: Cross-view image geolocalization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 891–898 (2013)

    Google Scholar 

  20. Lin, T.Y., Cui, Y., Belongie, S., Hays, J.: Learning deep representations for ground-to-aerial geolocalization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5007–5015 (2015)

    Google Scholar 

  21. Lin, Y., Xie, L., Wu, Y., Yan, C., Tian, Q.: Unsupervised person re-identification via softened similarity learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3390–3399 (2020)

    Google Scholar 

  22. Ma, R., et al.: Colon10k: a benchmark for place recognition in colonoscopy. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1279–1283. IEEE (2021)

    Google Scholar 

  23. Oliveira, M., Araujo, H., Figueiredo, I.N., Pinto, L., Curto, E., Perdigoto, L.: Registration of consecutive frames from wireless capsule endoscopy for 3d motion estimation. IEEE Access 9, 119533–119545 (2021)

    Article  Google Scholar 

  24. Posner, E., Zholkover, A., Frank, N., Bouhnik, M.: C3 fusion: consistent contrastive colon fusion, towards deep slam in colonoscopy. arXiv preprint arXiv:2206.01961 (2022)

  25. Radenović, F., Tolias, G., Chum, O.: Fine-tuning CNN image retrieval with no human annotation. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1655–1668 (2018)

    Article  Google Scholar 

  26. Rau, A., et al.: Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. Int. J. Comput. Assist. Radiol. Surg. 14(7), 1167–1176 (2019)

    Article  Google Scholar 

  27. Shao, S., et al.: Self-supervised monocular depth and ego-motion estimation in endoscopy: appearance flow to the rescue. Med. Image Anal. 77, 102338 (2022)

    Article  Google Scholar 

  28. Sun, D., Yang, X., Liu, M.Y., Kautz, J.: Pwc-net: cnns for optical flow using pyramid, warping, and cost volume. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8934–8943 (2018)

    Google Scholar 

  29. Wang, D., Zhang, S.: Unsupervised person re-identification via multi-label classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10981–10990 (2020)

    Google Scholar 

  30. Wang, F., Liu, H.: Understanding the behaviour of contrastive loss. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2495–2504 (2021)

    Google Scholar 

  31. Wolf, L., Hassner, T., Taigman, Y.: Descriptor based methods in the wild. In: Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition (2008)

    Google Scholar 

  32. Yan, C., Gong, B., Wei, Y., Gao, Y.: Deep multi-view enhancement hashing for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 43(4), 1445–1451 (2020)

    Article  Google Scholar 

  33. Zhang, S., Zhao, L., Huang, S., Ye, M., Hao, Q.: A template-based 3d reconstruction of colon structures and textures from stereo colonoscopic images. IEEE Trans. Med. Rob. Bionics 3(1), 85–95 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Verily Research, Haifa, Israel

    George Leifman, Idan Kligvasser, Roman Goldenberg, Ehud Rivlin & Michael Elad

Authors
  1. George Leifman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Idan Kligvasser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roman Goldenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ehud Rivlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Elad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George Leifman .

Editor information

Editors and Affiliations

  1. University of Leeds, Leeds, UK

    Sharib Ali

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Fons van der Sommen

  3. Eindhoven University of Technology, Eindhoven, The Netherlands

    Maureen van Eijnatten

  4. University of Oxford, Oxford, UK

    Bartłomiej W. Papież

  5. National University of Singapore, Singapore, Singapore

    Yueming Jin

  6. Eindhoven University of Technology, Eindhoven, The Netherlands

    Iris Kolenbrander

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Leifman, G., Kligvasser, I., Goldenberg, R., Rivlin, E., Elad, M. (2023). Colonoscopy Coverage Revisited: Identifying Scanning Gaps in Real-Time. In: Ali, S., van der Sommen, F., van Eijnatten, M., Papież, B.W., Jin, Y., Kolenbrander, I. (eds) Cancer Prevention Through Early Detection. CaPTion 2023. Lecture Notes in Computer Science, vol 14295. Springer, Cham. https://doi.org/10.1007/978-3-031-45350-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45350-2_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45349-6

  • Online ISBN: 978-3-031-45350-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation