Skip to main content
SpringerLink
Log in

Artificial Intelligence for Colorectal Polyps in Colonoscopy

  • Living reference work entry
  • First Online:
Artificial Intelligence in Medicine

Abstract

Colorectal cancer (CRC) is one of the leading causes of death worldwide. Fortunately, its early detection and treatment highly improves the survival rates and reduces costs. In this regard, screening programs and colonoscopy play an essential role. Artificial intelligence (AI) and deep learning (DL) have arisen with great success and been widely applied to medical imaging for the last few years and efforts have also been placed to develop methods to improve the adenoma detection rate in colonoscopy. In this chapter, polyp detection, localization, segmentation, and classification using colonoscopy are addressed. Works on these tasks have shown an exponential growth in the last few years. In the first place, the elements required for applying supervised DL methods for colorectal polyps in colonoscopy are introduced. The focus is placed on the model architecture, the dataset, the loss function, data augmentation, and metrics. Next, the currently openly available datasets that might be useful for future research are presented, before discussing methods for polyp detection, localization, segmentation, and classification. Lastly, some challenges and future trends in these fields are commented. This chapter about AI and DL for colorectal polyps in colonoscopy put forward the wide range of applications and research lines that can be further investigated with promising results as shown by the already currently published works, and always with the ultimate goal of assisting the endoscopist to improve CRC detection and diagnosis, which would eventually lead to better patient outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. World Health Organization. World cancer report 2014. 2014.

    Google Scholar 

  2. International Agency for Research on Cancer. Cancer tomorrow. 2020. https://gco.iarc.fr/tomorrow/home. Accessed 30 Nov 2020.

  3. Wiegering A, Ackermann S, Riegel J, et al. Improved survival of patients with colon cancer detected by screening colonoscopy. Int J Colorectal Dis. 2016;31:1039–45. https://doi.org/10.1007/s00384-015-2501-6.

    Article  PubMed  Google Scholar 

  4. Mar J, Errasti J, Soto-Gordoa M, et al. The cost of colorectal cancer according to the TNM stage. Cirugía Española (English Ed). 2017;95:89–96. https://doi.org/10.1016/j.cireng.2017.01.001.

    Article  Google Scholar 

  5. Williams CB. Insertion technique. In: Waye JD, Rex DK, Williams CB, editors. Colonoscopy. Principles and practice. Blackwell Publishing; 2005.

    Google Scholar 

  6. Ferlitsch M, Moss A, Hassan C, et al. Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline. Endoscopy. 2017;49:270–97.

    Article  Google Scholar 

  7. Berros Fombella JP, Aguilar Huergo S, García Teijido P. Enfermedades premalignas. In: Sociedad Española de Oncología Médica (ed) Manual SEOM de prevención y diagnóstico precoz del cáncer. 2017.

    Google Scholar 

  8. Lund M, Trads M, Njor SH, et al. Quality indicators for screening colonoscopy and colonoscopist performance and the subsequent risk of interval colorectal cancer: a systematic review. JBI Database Syst Rev Implement Reports Online Fir. 2019.

    Google Scholar 

  9. Ertem FU, Ladabaum U, Mehrotra A, et al. Incidence of interval colorectal cancer attributable to an endoscopist in clinical practice. Gastrointest Endose. 2019;88:705–11. https://doi.org/10.1016/j.gie.2018.05.012.Incidence.

    Article  Google Scholar 

  10. Byrne MF, Shahidi N, Rex DK. Will computer-aided detection and diagnosis revolutionize colonoscopy? Gastroenterology. 2017;153:1460–1464.e1. https://doi.org/10.1053/j.gastro.2017.10.026.

    Article  PubMed  Google Scholar 

  11. Aziz M, Fatima R, Dong C, et al. The impact of deep convolutional neural network-based artificial intelligence on colonoscopy outcomes: a systematic review with meta-analysis. J Gastroenterol Hepatol. 2020;1–8. https://doi.org/10.1111/jgh.15070.

  12. Endoscopic Classification Review Group. Update on the Paris classification of superficial neoplastic lesions in the digestive tract. Endoscopy. 2005;37:570–8. https://doi.org/10.1055/s-2005-861352.

    Article  Google Scholar 

  13. Kudo SE, Tamura S, Nakajima T, et al. Diagnosis of colorectal tumorous lesions by magnifying endoscopy. Gastrointest Endosc. 1996;44:8–14. https://doi.org/10.1016/S0016-5107(96)70222-5.

    Article  CAS  PubMed  Google Scholar 

  14. Hayashi N, Tanaka S, Hewett DG, et al. Endoscopic prediction of deep submucosal invasive carcinoma: validation of the Narrow-Band Imaging International Colorectal Endoscopic (NICE) classification. Gastrointest Endosc. 2013;78:625–32. https://doi.org/10.1016/j.gie.2013.04.185.

    Article  PubMed  Google Scholar 

  15. Sánchez-Peralta LF, Bote-Curiel L, Picón A, et al. Deep learning to find colorectal polyps in colonoscopy: a systematic literature review. Artif Intell Med. 2020;108. https://doi.org/10.1016/j.artmed.2020.101923.

  16. Nogueira-Rodríguez A, Domínguez-Carbajales R, López-Fernández H, et al. Deep neural networks approaches for detecting and classifying colorectal polyps. Neurocomputing. 2021;423:723–34. https://doi.org/10.1016/j.neucom.2020.02.123.

    Article  Google Scholar 

  17. Sánchez-Montes C, Bernal J, García-Rodríguez A, et al. Review of computational methods for the detection and classification of polyps in colonoscopy imaging. Gastroenterol Hepatol (N Y). 2020;43:222–32.

    Article  Google Scholar 

  18. Pacal I, Karaboga D, Basturk A, et al. A comprehensive review of deep learning in colon cancer. Comput Biol Med. 2020;126:104003. https://doi.org/10.1016/j.compbiomed.2020.104003.

    Article  PubMed  Google Scholar 

  19. Wittenberg T, Raithel M. Artificial intelligence-based polyp detection in colonoscopy: where have we been, where do we stand, and where are we headed? Visc Med. 2020. https://doi.org/10.1159/000512438.

  20. Mori Y, Kudo S ei, East JE, et al. Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video). Gastrointest Endosc. 2020;92:905–11.e1. https://doi.org/10.1016/j.gie.2020.03.3759.

  21. Tajbakhsh N, Jeyaseelan L, Li Q, et al. Embracing imperfect datasets: a review of deep learning solutions for medical image segmentation. Med Image Anal. 2020;63:101693. https://doi.org/10.1016/j.media.2020.101693.

    Article  PubMed  Google Scholar 

  22. Sánchez-Peralta LF, Picón A, Sánchez-Margallo FM, Pagador JB. Unravelling the effect of data augmentation transformations in polyp segmentation. Int J Comput Assist Radiol Surg. 2020. https://doi.org/10.1007/s11548-020-02262-4.

  23. Sánchez-Peralta LF, Picón A, Antequera-Barroso JA, et al. Eigenloss: combined PCA-based loss function for polyp segmentation. Mathematics. 2020;8:1316. https://doi.org/10.3390/math8081316.

    Article  Google Scholar 

  24. Vázquez D, Bernal J, Sánchez FJ, et al. A benchmark for endoluminal scene segmentation of colonoscopy images. J Healthc Eng. 2017. https://doi.org/10.1155/2017/4037190

  25. Bernal J, Histace A, Masana M, et al. GTCreator: a flexible annotation tool for image-based datasets. Int J Comput Assist Radiol Surg. 2019;14:191–201. https://doi.org/10.1007/s11548-018-1864-x.

    Article  PubMed  Google Scholar 

  26. Angermann Q, Bernal J, Sánchez-Montes C, et al. Towards real-time polyp detection in colonoscopy videos: adapting still frame-based methodologies for video sequences analysis. In: Computer assisted and robotic endoscopy and clinical image-based procedures. 2017. p. 29–41.

    Google Scholar 

  27. Silva JS, Histace A, Romain O, et al. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg. 2014;9:283–93. https://doi.org/10.1007/s11548-013-0926-3.

    Article  PubMed  Google Scholar 

  28. Bernal J, Tajbakhsh N, Sánchez FJ, et al. Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge. IEEE Trans Med Imaging. 2017;36:1231–49. https://doi.org/10.1109/TMI.2017.2664042.

    Article  PubMed  Google Scholar 

  29. Tajbakhsh N, Gurudu SR, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging. 2016;35:630–44. https://doi.org/10.1109/TMI.2015.2487997.

    Article  PubMed  Google Scholar 

  30. Jha D, Smedsrud PH, Riegler MA, et al. Kvasir-SEG: a segmented polyp dataset. In: Proceedings of the international conference on multimedia modeling (MMM). 2020.

    Google Scholar 

  31. Borgli H, Thambawita V, Smedsrud P, et al. HyperKvasir: a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data. 2020;7:1. https://doi.org/10.1038/s41597-020-00622-y.

    Article  Google Scholar 

  32. Sánchez-Peralta LF, Pagador JB, Picón A, et al. PICCOLO white-light and narrow-band imaging colonoscopic dataset: a performance comparative of models and datasets. Appl Sci. 2020;10:8501. https://doi.org/10.3390/app10238501.

    Article  CAS  Google Scholar 

  33. Mesejo P, Pizarro D, Abergel A, et al. Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE Trans Med Imaging. 2016;35:2051–63. https://doi.org/10.1109/TMI.2016.2547947.

    Article  PubMed  Google Scholar 

  34. Guo Z, Nemoto D, Zhu X, et al. Polyp detection algorithm can detect small polyps: ex vivo reading test compared with endoscopists. Dig Endosc. 2020. https://doi.org/10.1111/den.13670.

  35. Barua I, Vinsard D, Jodal H, et al. Artificial intelligence for polyp detection during colonoscopy: a systematic review and meta-analysis. Endoscopy. 2020. https://doi.org/10.1055/a-1201-7165.

  36. González-Bueno Puyal J, Bhatia KK, Brandao P, et al. Endoscopic polyp segmentation using a hybrid 2D/3D CNN. In: Medical image computing and computer assisted intervention – MICCAI 2020. MICCAI 2020. Lecture notes in computer science, vol. 12266. Cham: Springer; 2020. p. 295–305.

    Google Scholar 

  37. Qadir HA, Balasingham I, Solhusvik J, et al. Improving automatic polyp detection using CNN by exploiting temporal dependency in colonoscopy video. IEEE J Biomed Heal Informatics. 2020;24:180–93. https://doi.org/10.1109/JBHI.2019.2907434.

    Article  Google Scholar 

  38. Holzwanger EA, Bilal M, Glissen Brown JR, Singh S, Becq A, Ernest-Suarez K, Berzin TM. Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy. Endoscopy 2021;53(9):937–940. https://doi.org/10.1055/a-1302-2942.

  39. Misawa M, Kudo S ei, Mori Y, et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology. 2018;154:2027–9. https://doi.org/10.1053/j.gastro.2018.04.003.

  40. Shin Y, Qadir HA, Aabakken L, et al. Automatic colon polyp detection using region based deep CNN and post learning approaches. IEEE Access. 2018;6:40950–62. https://doi.org/10.1109/ACCESS.2018.2856402.

    Article  Google Scholar 

  41. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. 2015. p. 3431–40.

    Google Scholar 

  42. Huang G, Liu Z, van der Maaten L, Weinberger KQ. Densely connected convolutional networks. 2018. arXiv. https://doi.org/10.1109/CVPR.2017.243.

  43. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Medical image computing and computer-assisted intervention – MICCAI 2015. Lecture notes in computer science, vol. 9351. Springer; 2015. p. 234–41.

    Google Scholar 

  44. Chaurasia A, Culurciello E. LinkNet: exploiting encoder representations for efficient semantic segmentation. 2017 IEEE Vis Commun Image Process VCIP 2017 2018-January:1–4. 2018. https://doi.org/10.1109/VCIP.2017.8305148.

  45. Mahmud T, Paul B, Anowarul S. PolypSegNet: a modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images. Comput Biol Med. 2021;128:104119.

    Article  Google Scholar 

  46. Wilson A. Optical diagnosis of small colorectal polyps during colonoscopy: when to resect and discard? Best Pract Res Clin Gastroenterol. 2015;29:639–49. https://doi.org/10.1016/j.bpg.2015.06.007.

    Article  PubMed  Google Scholar 

  47. Goyal H, Mann R, Gandhi Z, et al. Scope of artificial intelligence in screening and diagnosis of colorectal cancer. J Clin Med. 2020;9:3313. https://doi.org/10.3390/jcm9103313.

    Article  PubMed Central  Google Scholar 

  48. Patino-Barrientos S, Sierra-Sosa D, Garcia-Zapirain B, et al. Kudo’s classification for colon polyps assessment using a deep learning approach. Appl Sci. 2020;10:501. https://doi.org/10.3390/app10020501.

    Article  Google Scholar 

  49. Rodriguez-Diaz E, Baffy G, Lo W-K, et al. Real-time artificial intelligence-based histological classification of colorectal polyps with augmented visualization. Gastrointest Endosc. 2020;1–9. https://doi.org/10.1016/j.gie.2020.09.018.

  50. Jin EH, Lee D, Bae JH, et al. Improved accuracy in optical diagnosis of colorectal polyps using convolutional neural networks with visual explanations. Gastroenterology. 2020;158:2169–2179.e8. https://doi.org/10.1053/j.gastro.2020.02.036.

    Article  PubMed  Google Scholar 

  51. Byrne MF, Chapados N, Soudan F, et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut. 2019;68:94–100. https://doi.org/10.1136/gutjnl-2017-314547.

    Article  PubMed  Google Scholar 

  52. Berzin TM, Parasa S, Wallace MB, et al. Position statement on priorities for artificial intelligence in GI endoscopy: a report by the ASGE Task Force. Gastrointest Endosc. 2020;92:951–9. https://doi.org/10.1016/j.gie.2020.06.035.

    Article  PubMed  Google Scholar 

  53. Ahmad OF, Mori Y, Misawa M, et al. Establishing key research questions for the implementation of artificial intelligence in colonoscopy – a modified Delphi method. Endoscopy. 2020. https://doi.org/10.1055/a-1306-7590.

  54. Hoerter N, Gross SA, Liang PS. Artificial intelligence and polyp detection. Curr Treat Options Gastroenterol. 2020;18:120–36. https://doi.org/10.1007/s11938-020-00274-2.

    Article  Google Scholar 

  55. Cheplygina V, de Bruijne M, Pluim JPW. Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med Image Anal. 2019;54:280–96. https://doi.org/10.1016/j.media.2019.03.009.

    Article  PubMed  Google Scholar 

  56. Golhar M, Bobrow TL, Khoshknab MP, et al. Improving colonoscopy lesion classification using semi-supervised deep learning. IEEE Access. 2021;9:631–40.

    Article  Google Scholar 

  57. Pogorelov K, Ostroukhova O, Jeppsson M, et al. Deep learning and hand-crafted feature based approaches for polyp detection in medical videos. In: 2018 IEEE 31st international symposium on computer-based medical systems (CBMS). Karlstad. 2018. p. 381–6.

    Google Scholar 

  58. Shin Y, Qadir HA, Balasingham I. Abnormal colon polyp image synthesis using conditional adversarial networks for improved detection performance. IEEE Access. 2018;6:56007–17. https://doi.org/10.1109/ACCESS.2018.2872717.

    Article  Google Scholar 

  59. De Almeida TV, Sierra-Franco CA, Raposo AB, et al. Training data enhancements for robust polyp segmentation in colonoscopy images. In: EEE symposium on computer-based medical systems. 2019. p. 192–7.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bioengineering and Health Technologies, Jesús Usón Minimally Invasive Surgery Centre, Cáceres, Spain

    Luisa F. Sánchez-Peralta & J. Blas Pagador

  2. Scientific Direction, Jesús Usón Minimally Invasive Surgery Centre, Cáceres, Spain

    Francisco M. Sánchez-Margallo

Authors
  1. Luisa F. Sánchez-Peralta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Blas Pagador
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francisco M. Sánchez-Margallo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luisa F. Sánchez-Peralta .

Editor information

Editors and Affiliations

  1. Karolinska Institute, Stockholm, Sweden

    Niklas Lidströmer

  2. Faculty of Medicine, Imperial College London, London, UK

    Hutan Ashrafian

1 Electronic Supplementary Materials

(MP4 25172 kb)

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Sánchez-Peralta, L.F., Pagador, J.B., Sánchez-Margallo, F.M. (2021). Artificial Intelligence for Colorectal Polyps in Colonoscopy. In: Lidströmer, N., Ashrafian, H. (eds) Artificial Intelligence in Medicine. Springer, Cham. https://doi.org/10.1007/978-3-030-58080-3_308-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58080-3_308-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58080-3

  • Online ISBN: 978-3-030-58080-3

  • eBook Packages: Springer Reference MedicineReference Module Medicine

Publish with us

Policies and ethics

Search

Navigation