Skip to main content
SpringerLink
Log in

Artificial Intelligence and Polyp Detection

  • Colon (J Anderson, Section Editor)
  • Published:
Current Treatment Options in Gastroenterology Aims and scope Submit manuscript

Abstract

Purpose of review

This review highlights the history, recent advances, and ongoing challenges of artificial intelligence (AI) technology in colonic polyp detection.

Recent findings

Hand-crafted AI algorithms have recently given way to convolutional neural networks with the ability to detect polyps in real-time. The first randomized controlled trial comparing an AI system to standard colonoscopy found a 9% increase in adenoma detection rate, but the improvement was restricted to polyps smaller than 10 mm and the results need validation. As this field rapidly evolves, important issues to consider include standardization of outcomes, dataset availability, real-world applications, and regulatory approval.

Summary

AI has shown great potential for improving colonic polyp detection while requiring minimal training for endoscopists. The question of when AI will enter endoscopic practice depends on whether the technology can be integrated into existing hardware and an assessment of its added value for patient care.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References and Recommended Reading

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68:394–424. https://doi.org/10.3322/caac.21492 .

    Article  PubMed  Google Scholar 

  2. Cronin KA, Lake AJ, Scott S, Sherman RL, Noone AM, Howlader N, et al. Annual report to the nation on the status of cancer, part I: national cancer statistics: annual report national cancer statistics. Cancer. 2018;124:2785–800. https://doi.org/10.1002/cncr.31551 .

    Article  PubMed  Google Scholar 

  3. Siegel RL, Miller KD, Fedewa SA, Ahnen DJ, Meester RGS, Barzi A, et al. Colorectal cancer statistics, 2017: colorectal cancer statistics, 2017. CA Cancer J Clin. 2017;67:177–93. https://doi.org/10.3322/caac.21395 .

    Article  PubMed  Google Scholar 

  4. Rex D, Cutler C, Lemmel G, Rahmani E.Y., Clark D.W., Helper D.J., Lehman G.A., Mark D.G. (1997) Colonoscopic miss rates of adenomas determined by back-to-back colonoscopies. Gastroenterology 112:24–28. https://doi.org/10.1016/S0016-5085(97)70214-2.

    Article  CAS  PubMed  Google Scholar 

  5. Heresbach D, Barrioz T, Lapalus MG, Coumaros D, Bauret P, Potier P, et al. Miss rate for colorectal neoplastic polyps: a prospective multicenter study of back-to-back video colonoscopies. Endoscopy. 2008;40:284–90. https://doi.org/10.1055/s-2007-995618 .

    Article  CAS  PubMed  Google Scholar 

  6. Rex DK, Boland CR, Dominitz JA, Giardiello FM, Johnson DA, Kaltenbach T, et al. Colorectal cancer screening: recommendations for physicians and patients from the U.S. multi-society task force on colorectal cancer. Gastroenterology. 2017;153:307–23. https://doi.org/10.1053/j.gastro.2017.05.013 .

    Article  PubMed  Google Scholar 

  7. Corley DA, Jensen CD, Marks AR, Zhao WK, Lee JK, Doubeni CA, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370:1298–306. https://doi.org/10.1056/NEJMoa1309086 .

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Castaneda D, Popov VB, Verheyen E, et al (2018) New technologies improve adenoma detection rate, adenoma miss rate, and polyp detection rate: a systematic review and meta-analysis. Gastrointest Endosc 88:209-222.e11. https://doi.org/10.1016/j.gie.2018.03.022 .

    Article  PubMed  Google Scholar 

  9. Wada Y, Fukuda M, Ohtsuka K, Watanabe M, Fukuma Y, Wada Y, et al. Efficacy of Endocuff-assisted colonoscopy in the detection of colorectal polyps. Endosc Int Open. 2018;06:E425–31. https://doi.org/10.1055/s-0044-101142 .

    Article  Google Scholar 

  10. Atkinson NSS, Ket S, Bassett P, Aponte D, de Aguiar S, Gupta N, et al. Narrow-band imaging for detection of neoplasia at colonoscopy: a meta-analysis of data from individual patients in randomized controlled trials. Gastroenterology. 2019;157:462–71. https://doi.org/10.1053/j.gastro.2019.04.014 .

    Article  PubMed  Google Scholar 

  11. Hassan C, Senore C, Radaelli F, de Pretis G, Sassatelli R, Arrigoni A, et al. Full-spectrum (FUSE) versus standard forward-viewing colonoscopy in an organised colorectal cancer screening programme. Gut. 2017;66:1949–55. https://doi.org/10.1136/gutjnl-2016-311906 .

    Article  PubMed  Google Scholar 

  12. Buchner AM, Shahid MW, Heckman MG, McNeil R, Cleveland P, Gill KR, et al. High-definition colonoscopy detects colorectal polyps at a higher rate than standard white-light colonoscopy. Clin Gastroenterol Hepatol. 2010;8:364–70. https://doi.org/10.1016/j.cgh.2009.11.009 .

    Article  PubMed  Google Scholar 

  13. Subramanian V, Mannath J, Hawkey CJ, Ragunath K. High definition colonoscopy vs. standard video endoscopy for the detection of colonic polyps: a meta-analysis. Endoscopy. 2011;43:499–505. https://doi.org/10.1055/s-0030-1256207 .

    Article  CAS  PubMed  Google Scholar 

  14. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–8. https://doi.org/10.1038/nature21056 .

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Thrall JH, Li X, Li Q, Cruz C, Do S, Dreyer K, et al. Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success. J Am Coll Radiol. 2018;15:504–8. https://doi.org/10.1016/j.jacr.2017.12.026 .

    Article  PubMed  Google Scholar 

  16. Karkanis SA, Iakovidis DK, Maroulis DE, Karras DA, Tzivras M. Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans Inf Technol Biomed. 2003;7:141–52. https://doi.org/10.1109/TITB.2003.813794 .

    Article  PubMed  Google Scholar 

  17. Maroulis DE, Iakovidis DK, Karkanis SA, Karras DA (2003) CoLD: a versatile detection system for colorectal lesions in endoscopy video-frames. Comput Methods Prog Biomed 70:151–166. https://doi.org/10.1016/S0169-2607(02)00007-X.

    Article  CAS  PubMed  Google Scholar 

  18. Iakovidis DK, Maroulis DE, Karkanis SA. An intelligent system for automatic detection of gastrointestinal adenomas in video endoscopy. Comput Biol Med. 2006;36:1084–103. https://doi.org/10.1016/j.compbiomed.2005.09.008 .

    Article  PubMed  Google Scholar 

  19. Angermann Q, Bernal J, Sánchez-Montes C, et al (2017) Towards real-time polyp detection in colonoscopy videos: adapting still frame-based methodologies for video sequences analysis. In: Computer assisted and robotic endoscopy and clinical image-based procedures. Springer, pp 29–41.

  20. Wang Y, Tavanapong W, Wong J, Oh JH, de Groen PC. Polyp-alert: near real-time feedback during colonoscopy. Comput Methods Prog Biomed. 2015;120:164–79.

    Article  Google Scholar 

  21. Kaiser D, Haselhuhn T. Facing a regular world: how spatial object structure shapes visual processing. J Neurosci. 2017;37:1965–7. https://doi.org/10.1523/JNEUROSCI.3441-16.2017 .

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Billah M, Waheed S, Rahman MM. An automatic gastrointestinal polyp detection system in video endoscopy using fusion of color wavelet and convolutional neural network features. Int J Biomed Imaging. 2017;2017:9.

    Article  Google Scholar 

  23. Zhang R, Zheng Y, Mak TWC, et al. Automatic detection and classification of colorectal polyps by transferring low-level CNN features from nonmedical domain. IEEE J Biomed Health Inform. 2017;21:41–7. https://doi.org/10.1109/JBHI.2016.2635662 .

    Article  PubMed  Google Scholar 

  24. • Urban G, Tripathi P, Alkayali T, et al (2018) Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 155:1069-1078.e8. https://doi.org/10.1053/j.gastro.2018.06.037A real-time polyp detection algorithm shown to improve polyp detection compared with expert review of colonoscopy video.

    Article  PubMed  Google Scholar 

  25. •• Klare P, Sander C, Prinzen M, et al (2019) Automated polyp detection in the colorectum: a prospective study (with videos). Gastrointest Endosc 89:576-582.e1. https://doi.org/10.1016/j.gie.2018.09.042A real-time algorithm that was tested in vivo during live colonoscopies; its ADR was comparable with, but slightly inferior to, that of endoscopists.

    Article  PubMed  Google Scholar 

  26. •• Wang P, Berzin TM, Glissen Brown JR, et al (2019) Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut gutjnl-2018-317500. https://doi.org/10.1136/gutjnl-2018-317500This is the only randomized clinical trial using AI for polyp detection in live patients and found increased ADR compared with standard colonoscopy.

    Article  PubMed  Google Scholar 

  27. Atkin WS, Morson BC, Cuzick J. Long-term risk of colorectal cancer after excision of rectosigmoid adenomas. N Engl J Med. 1992;326:658–62. https://doi.org/10.1056/NEJM199203053261002 .

    Article  CAS  PubMed  Google Scholar 

  28. Noshirwani KC, van Stolk RU, Rybicki LA, Beck GJ (2000) Adenoma size and number are predictive of adenoma recurrence: implications for surveillance colonoscopy. Gastrointest Endosc 51:433–437. https://doi.org/10.1016/S0016-5107 (00)70444-5.

  29. Martínez ME, Baron JA, Lieberman DA, Schatzkin A, Lanza E, Winawer SJ, et al. A pooled analysis of advanced colorectal neoplasia diagnoses after colonoscopic polypectomy. Gastroenterology. 2009;136:832–41. https://doi.org/10.1053/j.gastro.2008.12.007 .

    Article  PubMed  Google Scholar 

  30. Saini SD, Kim HM, Schoenfeld P. Incidence of advanced adenomas at surveillance colonoscopy in patients with a personal history of colon adenomas: a meta-analysis and systematic review. Gastrointest Endosc. 2006;64:614–26. https://doi.org/10.1016/j.gie.2006.06.057 .

    Article  PubMed  Google Scholar 

  31. Rex DK, Schoenfeld PS, Cohen J, Pike IM, Adler DG, Fennerty MB, et al. Quality indicators for colonoscopy. Gastrointest Endosc. 2015;81:31–53. https://doi.org/10.1016/j.gie.2014.07.058 .

    Article  PubMed  Google Scholar 

  32. • Bernal J, Tajkbaksh N, Sánchez FJ, et al (2017) Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge. IEEE Trans Med Imaging 36:1231–1249 Describes results of the first and only attempt to compare the performance of multiple algorithms directly in a standardized manner.

    Article  PubMed  Google Scholar 

  33. • Rex DK, Kahi C, O’Brien M, et al (2011) The American Society for Gastrointestinal Endoscopy PIVI (preservation and incorporation of valuable endoscopic innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc 73:419–422. https://doi.org/10.1016/j.gie.2011.01.023ASGE statement establishing criteria for incorporation of polyp classification technology into practice.

    Article  PubMed  Google Scholar 

  34. Russakovsky O, Deng J, Su H, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis. 2015;115:211–52. https://doi.org/10.1007/s11263-015-0816-y .

    Article  Google Scholar 

  35. de Lange T, Halvorsen P, Riegler M. Methodology to develop machine learning algorithms to improve performance in gastrointestinal endoscopy. World J Gastroenterol. 2018;24:5057–62. https://doi.org/10.3748/wjg.v24.i45.5057 .

    Article  PubMed  PubMed Central  Google Scholar 

  36. Yu L, Chen H, Dou Q, et al. Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J Biomed Health Inform. 2017;21:65–75. https://doi.org/10.1109/JBHI.2016.2637004 .

    Article  Google Scholar 

  37. Artificial Intelligence for the American People. In: White House. https://www.whitehouse.gov/ai/. Accessed 12 Oct 2019.

  38. Edwards L, Veale M. Enslaving the algorithm: from a “Right to an Explanation” to a “Right to Better Decisions”? IEEE Secur Priv. 2018;16:46–54. https://doi.org/10.1109/MSP.2018.2701152 .

    Article  Google Scholar 

  39. Price WN, Gerke S, Cohen IG. Potential liability for physicians using artificial intelligence. JAMA. 2019. https://doi.org/10.1001/jama.2019.15064.

    Article  PubMed  Google Scholar 

  40. Chen P-J, Lin M-C, Lai M-J, Lin JC, Lu HH, Tseng VS. Accurate classification of diminutive colorectal polyps using computer-aided analysis. Gastroenterology. 2018;154:568–75.

    Article  PubMed  Google Scholar 

  41. Byrne MF, Chapados N, Soudan F, Oertel C, Linares Pérez M, Kelly R, et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut. 2019;68:94–100. https://doi.org/10.1136/gutjnl-2017-314547 .

    Article  PubMed  Google Scholar 

  42. Gross S, Trautwein C, Behrens A, Winograd R, Palm S, Lutz HH, et al. Computer-based classification of small colorectal polyps by using narrow-band imaging with optical magnification. Gastrointest Endosc. 2011;74:1354–9. https://doi.org/10.1016/j.gie.2011.08.001 .

    Article  PubMed  Google Scholar 

  43. Kominami Y, Yoshida S, Tanaka S, Sanomura Y, Hirakawa T, Raytchev B, et al. Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc. 2016;83:643–9. https://doi.org/10.1016/j.gie.2015.08.004 .

    Article  PubMed  Google Scholar 

  44. Mori Y, Kudo S, Misawa M, Saito Y, Ikematsu H, Hotta K, et al. Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Intern Med. 2018;169:357–66. https://doi.org/10.7326/M18-0249 .

    Article  PubMed  Google Scholar 

  45. Takemura Y, Yoshida S, Tanaka S, Onji K, Oka S, Tamaki T, et al. Quantitative analysis and development of a computer-aided system for identification of regular pit patterns of colorectal lesions. Gastrointest Endosc. 2010;72:1047–51. https://doi.org/10.1016/j.gie.2010.07.037 .

    Article  PubMed  Google Scholar 

  46. Karkanis SA, Iakovidis DK, Maroulis DE, Karras DA, Tzivras M. Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans Inf Technol Biomed. 2003;7:141–52.

    Article  PubMed  Google Scholar 

  47. Hwang S, Oh J, Tavanapong W, et al (2007) Polyp detection in colonoscopy video using elliptical shape feature. In: 2007 IEEE International Conference on Image Processing. IEEE, pp II-465-II–468.

  48. Park SY, Sargent D, Spofford I, Vosburgh KG, A-Rahim Y. A colon video analysis framework for polyp detection. IEEE Trans Biomed Eng. 2012;59:1408–18. https://doi.org/10.1109/TBME.2012.2188397 .

    Article  PubMed  Google Scholar 

  49. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F. WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imaging Graph. 2015;43:99–111.

    Article  PubMed  Google Scholar 

  50. Tajbakhsh N, Gurudu SR, Liang J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging. 2015;35:630–44.

    Article  PubMed  Google Scholar 

  51. Geetha K, Rajan C. Automatic colorectal polyp detection in colonoscopy video frames. Asian Pac J Cancer Prev APJCP. 2016;17:4869–73. https://doi.org/10.22034/APJCP.2016.17.11.4869.

    Article  Google Scholar 

  52. Misawa M, Kudo S, Mori Y, et al (2018) Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 154:2027-2029.e3. https://doi.org/10.1053/j.gastro.2018.04.003 .

    Article  PubMed  Google Scholar 

  53. Park SY, Sargent D (2016) Colonoscopic polyp detection using convolutional neural networks. In: Medical imaging 2016: computer-aided diagnosis. International Society for Optics and Photonics, p 978528.

  54. Pogorelov K, Ostroukhova O, Jeppsson M, et al (2018) Deep learning and hand-crafted feature based approaches for polyp detection in medical videos. In: 2018 IEEE 31st international symposium on computer-based medical systems (CBMS). IEEE, Karlstad, pp 381–386.

  55. Ahmad OF, Brandao P, Sami SS, et al. Tu1991 Artificial intelligence for real-time polyp localisation in colonoscopy withdrawal videos. Gastrointest Endosc. 2019;89:AB647. https://doi.org/10.1016/j.gie.2019.03.1135 .

    Article  Google Scholar 

  56. Eelbode T, Hassan C, Demedts I, et al. Tu1959 BLI and LCI improve polyp detection and delineation accuracy for deep learning networks. Gastrointest Endosc. 2019;89:AB632. https://doi.org/10.1016/j.gie.2019.03.1103 .

    Article  Google Scholar 

  57. Ka-Luen Lui T, Yee K, Wong K, Leung WK. 1062 Use of artificial intelligence image classifier for real-time detection of colonic polyps. Gastrointest Endosc. 2019;89:AB135. https://doi.org/10.1016/j.gie.2019.04.175 .

    Article  Google Scholar 

  58. Misawa M, Kudo S, Mori Y, et al. Tu1990 Artificial intelligence-assisted polyp detection system for colonoscopy, based on the largest available collection of clinical video data for machine learning. Gastrointest Endosc. 2019;89:AB646–7. https://doi.org/10.1016/j.gie.2019.03.1134 .

    Article  Google Scholar 

  59. Ozawa T, Ishihara S, Fujishiro M, et al (2018) Novel computer-assisted system for the detection and classification of colorectal polyps using artificial intelligence. UEG Week 2018 Oral Presentations.pdf. United European Gastroenterology Journal, Austria, p A98.

  60. Ozawa T, Ishihara S, Fujishiro M, et al. Sa1971 Automated endoscopic detection and classification of colorectal polyps using convolutional neural networks. Gastrointest Endosc. 2018;87:AB271. https://doi.org/10.1016/j.gie.2018.04.1585 .

    Article  Google Scholar 

  61. Repici A, Dinh NN, Cherubini A, et al. Su1716 Artificial intelligence for colorectal polyp detection: high accuracy and detection anticipation with CB-17-08 performance. Gastrointest Endosc. 2019;89:AB391–2. https://doi.org/10.1016/j.gie.2019.03.589 .

    Article  Google Scholar 

  62. Shichijo S, Aoyama K, Ozawa T, et al. Tu2003 Application of convolutional neural networks could detect all laterally spreading tumor in colonoscopic images. Gastrointest Endosc. 2019;89:AB653. https://doi.org/10.1016/j.gie.2019.03.1147 .

    Article  Google Scholar 

  63. Yamada M, Saito Y, Imaoka H, et al. Development of a real-time endoscopic image diagnosis support system using deep learning technology in colonoscopy. Austria: United European Gastroenterology Journal; 2018. p. A190.

    Google Scholar 

  64. Zheng Y, Mak T, Jiang Y, et al. A study comparing colorectal polyp detection rates between endoscopists and artificial intelligence-doscopist. France: Colorectal Disease; 2018. p. 22.

    Google Scholar 

  65. Zhu X, Nemoto D, Wang Y, et al. Sa1923 Detection and diagnosis of sessile serrated adenoma/polyps using convolutional neural network (artificial intelligence). Gastrointest Endosc. 2018;87:AB251 https://doi.org/10.1016/j.gie.2018.04.445.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NYU Langone Health, New York, NY, USA

    Nicholas Hoerter, Seth A. Gross & Peter S. Liang

  2. VA New York Harbor Health Care System, New York, NY, USA

    Peter S. Liang

Authors
  1. Nicholas Hoerter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seth A. Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter S. Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter S. Liang.

Ethics declarations

Conflict of Interest

Nicholas Hoerter declares no conflict of interest. Peter Liang reports grants from Epigenomics, outside the submitted work. Seth Gross reports personal fees from Olympus, outside the submitted work.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Colon

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoerter, N., Gross, S.A. & Liang, P.S. Artificial Intelligence and Polyp Detection. Curr Treat Options Gastro 18, 120–136 (2020). https://doi.org/10.1007/s11938-020-00274-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11938-020-00274-2

Keywords

Search

Navigation