Skip to main content
SpringerLink
Log in

State of the Art: The Impact of Artificial Intelligence in Endoscopy 2020

  • ENDOSCOPY AND SURGERY (S KOMANDURI, SECTION EDITOR)
  • Published:
Current Gastroenterology Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

Recently numerous researchers have shown remarkable progress using convolutional neural network-based artificial intelligence (AI) for endoscopy. In this manuscript we aim to summarize recent AI impact on endoscopy.

Recent Findings

AI for detecting colon polyps has been the most promising area for application of AI in endoscopy. Recent prospective randomized studies showed that AI assisted colonoscopy increased adenoma detection rate and the mean number of adenomas per patient compared to standard colonoscopy alone. AI for optical biopsy of colon polyp showed a negative predictive value of ≥90%. For capsule endoscopy, applying AI to pre-read the video images decreased physician reading time significantly. Recently, researchers are broadening the area of AI to quality assessment of endoscopy such as bowel preparation and automated report generation.

Summary

AI systems have shown great potential to increase physician performance by enhancing detection, reducing procedure time, and providing real-time feedback of endoscopy quality. To build a generally applicable AI, we need further investigations in real world settings and also integration of AI tools into pragmatic platforms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

Papers of particular interest, published recently, have been highlighted as: •• Of major importance

  1. Hirschowitz BI. A personal history of the fiberscope. Gastroenterology. 1979;76(4):864–9.

    Article  CAS  Google Scholar 

  2. Demling L, Hagel HJ. Video endoscopy. Fundamentals and problems Endoscopy. 1985;17(5):167–9. https://doi.org/10.1055/s-2007-1018491.

    Article  CAS  PubMed  Google Scholar 

  3. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44. https://doi.org/10.1038/nature14539.

    Article  CAS  PubMed  Google Scholar 

  4. Corley DA, Jensen CD, Marks AR, Zhao WK, Lee JK, Doubeni CA, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370(14):1298–306. https://doi.org/10.1056/NEJMoa1309086.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Misawa M, Kudo SE, Mori Y, Cho T, Kataoka S, Yamauchi A, et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology. 2018;154(8):2027–9.e3. https://doi.org/10.1053/j.gastro.2018.04.003.

    Article  PubMed  Google Scholar 

  6. Urban G, Tripathi P, Alkayali T, Mittal M, Jalali F, Karnes W, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology. 2018;155(4):1069–78.e8. https://doi.org/10.1053/j.gastro.2018.06.037.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Wang P, Xiao X, Glissen Brown JR, Berzin TM, Tu M, Xiong F, et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat Biomed Eng. 2018;2(10):741–8. https://doi.org/10.1038/s41551-018-0301-3.

    Article  PubMed  Google Scholar 

  8. Hassan C, Wallace MB, Sharma P, Maselli R, Craviotto V, Spadaccini M, et al. New artificial intelligence system: first validation study versus experienced endoscopists for colorectal polyp detection. Gut. 2020;69(5):799–800. https://doi.org/10.1136/gutjnl-2019-319914.

    Article  PubMed  Google Scholar 

  9. Klare P, Sander C, Prinzen M, Haller B, Nowack S, Abdelhafez M, et al. Automated polyp detection in the colorectum: a prospective study (with videos). Gastrointest Endosc. 2019;89(3):576–82.e1. https://doi.org/10.1016/j.gie.2018.09.042.

    Article  PubMed  Google Scholar 

  10. •• Wang P, Berzin TM, Glissen Brown JR, Bharadwaj S, Becq A, Xiao X, et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 2019;68(10):1813–9. https://doi.org/10.1136/gutjnl-2018-317500Large prospective randomised controlled study for AI.

  11. Enzinger PC, Mayer RJ. Esophageal cancer. N Engl J Med. 2003;349(23):2241–52. https://doi.org/10.1056/NEJMra035010.

    Article  CAS  PubMed  Google Scholar 

  12. Ishihara R, Takeuchi Y, Chatani R, Kidu T, Inoue T, Hanaoka N, et al. Prospective evaluation of narrow-band imaging endoscopy for screening of esophageal squamous mucosal high-grade neoplasia in experienced and less experienced endoscopists. Dis Esophagus. 2010;23(6):480–6. https://doi.org/10.1111/j.1442-2050.2009.01039.x.

    Article  PubMed  Google Scholar 

  13. Horie Y, Yoshio T, Aoyama K, Yoshimizu S, Horiuchi Y, Ishiyama A, et al. Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks. Gastrointest Endosc. 2019;89(1):25–32. https://doi.org/10.1016/j.gie.2018.07.037.

    Article  PubMed  Google Scholar 

  14. Cai SL, Li B, Tan WM, Niu XJ, Yu HH, Yao LQ, et al. Using a deep learning system in endoscopy for screening of early esophageal squamous cell carcinoma (with video). Gastrointest Endosc. 2019;90(5):745–53.e2. https://doi.org/10.1016/j.gie.2019.06.044.

    Article  PubMed  Google Scholar 

  15. Guo L, Xiao X, Wu C, Zeng X, Zhang Y, Du J, et al. Real-time automated diagnosis of precancerous lesions and early esophageal squamous cell carcinoma using a deep learning model (with videos). Gastrointest Endosc. 2020;91(1):41–51. https://doi.org/10.1016/j.gie.2019.08.018.

    Article  PubMed  Google Scholar 

  16. de Groof J, van der Sommen F, van der Putten J, Struyvenberg MR, Zinger S, Curvers WL, et al. The Argos project: the development of a computer-aided detection system to improve detection of Barrett's neoplasia on white light endoscopy. United European Gastroenterol J. 2019;7(4):538–47. https://doi.org/10.1177/2050640619837443.

    Article  PubMed  PubMed Central  Google Scholar 

  17. van der Sommen F, Zinger S, Curvers WL, Bisschops R, Pech O, Weusten BL, et al. Computer-aided detection of early neoplastic lesions in Barrett's esophagus. Endoscopy. 2016;48(7):617–24. https://doi.org/10.1055/s-0042-105284.

    Article  PubMed  Google Scholar 

  18. de Groof AJ, Struyvenberg MR, Fockens KN, van der Putten J, van der Sommen F, Boers TG, et al. Deep learning algorithm detection of Barrett's neoplasia with high accuracy during live endoscopic procedures: a pilot study (with video). Gastrointest Endosc. 2020;91(6):1242–50. https://doi.org/10.1016/j.gie.2019.12.048.

    Article  PubMed  Google Scholar 

  19. Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Shichijo S, Ozawa T, et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer. 2018;21(4):653–60. https://doi.org/10.1007/s10120-018-0793-2.

    Article  PubMed  Google Scholar 

  20. Cho BJ, Bang CS, Park SW, Yang YJ, Seo SI, Lim H, et al. Automated classification of gastric neoplasms in endoscopic images using a convolutional neural network. Endoscopy. 2019;51(12):1121–9. https://doi.org/10.1055/a-0981-6133.

    Article  PubMed  Google Scholar 

  21. Luo H, Xu G, Li C, He L, Luo L, Wang Z, et al. Real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy: a multicentre, case-control, diagnostic study. Lancet Oncol. 2019;20(12):1645–54. https://doi.org/10.1016/s1470-2045(19)30637-0.

    Article  CAS  PubMed  Google Scholar 

  22. Abu Dayyeh BK, Thosani N, Konda V, Wallace MB, Rex DK, Chauhan SS, et al. ASGE Technology Committee systematic review and meta-analysis assessing the ASGE PIVI thresholds for adopting real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc. 2015;81(3):502–16.e1. https://doi.org/10.1016/j.gie.2014.12.022.

    Article  PubMed  Google Scholar 

  23. Rex DK, Kahi C, O'Brien M, Levin TR, Pohl H, Rastogi A, et al. The American Society for Gastrointestinal Endoscopy PIVI (preservation and incorporation of valuable endoscopic innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc. 2011;73(3):419–22. https://doi.org/10.1016/j.gie.2011.01.023.

    Article  PubMed  Google Scholar 

  24. Ignjatovic A, Thomas-Gibson S, East JE, Haycock A, Bassett P, Bhandari P, et al. Development and validation of a training module on the use of narrow-band imaging in differentiation of small adenomas from hyperplastic colorectal polyps. Gastrointest Endosc. 2011;73(1):128–33. https://doi.org/10.1016/j.gie.2010.09.021.

    Article  PubMed  Google Scholar 

  25. Sikka S, Ringold DA, Jonnalagadda S, Banerjee B. Comparison of white light and narrow band high definition images in predicting colon polyp histology, using standard colonoscopes without optical magnification. Endoscopy. 2008;40(10):818–22. https://doi.org/10.1055/s-2008-1077437.

    Article  CAS  PubMed  Google Scholar 

  26. Kominami Y, Yoshida S, Tanaka S, Sanomura Y, Hirakawa T, Raytchev B, et al. Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc. 2016;83(3):643–9. https://doi.org/10.1016/j.gie.2015.08.004.

    Article  PubMed  Google Scholar 

  27. Chen PJ, Lin MC, Lai MJ, Lin JC, Lu HH, Tseng VS. Accurate classification of diminutive colorectal polyps using computer-aided analysis. Gastroenterology. 2018;154(3):568–75. https://doi.org/10.1053/j.gastro.2017.10.010.

    Article  PubMed  Google Scholar 

  28. Kudo SE, Misawa M, Mori Y, Hotta K, Ohtsuka K, Ikematsu H, et al. Artificial Intelligence-assisted System Improves Endoscopic Identification of Colorectal Neoplasms. Clin Gastroenterol Hepatol. 2020;18(8):1874–81.e2. https://doi.org/10.1016/j.cgh.2019.09.009.

    Article  PubMed  Google Scholar 

  29. Mori Y, Kudo SE, Wakamura K, Misawa M, Ogawa Y, Kutsukawa M, et al. Novel computer-aided diagnostic system for colorectal lesions by using endocytoscopy (with videos). Gastrointest Endosc. 2015;81(3):621–9. https://doi.org/10.1016/j.gie.2014.09.008.

    Article  PubMed  Google Scholar 

  30. Komeda Y, Handa H, Watanabe T, Nomura T, Kitahashi M, Sakurai T, et al. Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: preliminary experience. Oncology. 2017;93(Suppl 1):30–4. https://doi.org/10.1159/000481227.

    Article  PubMed  Google Scholar 

  31. Byrne MF, Chapados N, Soudan F, Oertel C, Linares Perez M, Kelly R, et al. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut. 2019;68(1):94–100. https://doi.org/10.1136/gutjnl-2017-314547.

    Article  PubMed  Google Scholar 

  32. Sanchez-Montes C, Sanchez FJ, Bernal J, Cordova H, Lopez-Ceron M, Cuatrecasas M, et al. Computer-aided prediction of polyp histology on white light colonoscopy using surface pattern analysis. Endoscopy. 2019;51(3):261–5. https://doi.org/10.1055/a-0732-5250.

    Article  PubMed  Google Scholar 

  33. Zachariah R, Samarasena J, Luba D, Duh E, Dao T, Requa J, et al. Prediction of Polyp Pathology Using Convolutional Neural Networks Achieves "Resect and Discard" Thresholds. Am J Gastroenterol. 2020;115(1):138–44. https://doi.org/10.14309/ajg.0000000000000429.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Aoki T, Yamada A, Aoyama K, Saito H, Tsuboi A, Nakada A, et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointestinal Endoscopy. 2019;89(2):357–63. e2.

    Article  Google Scholar 

  35. Ding Z, Shi H, Zhang H, Meng L, Fan M, Han C, et al. Gastroenterologist-Level Identification of Small-Bowel Diseases and Normal Variants by Capsule Endoscopy Using a Deep-Learning Model. Gastroenterology. 2019;157(4):1044–54.e5. https://doi.org/10.1053/j.gastro.2019.06.025.

    Article  PubMed  Google Scholar 

  36. He JY, Wu X, Jiang YG, Peng Q, Jain R. Hookworm detection in wireless capsule endoscopy images with deep learning. IEEE Trans Image Process. 2018;27(5):2379–92. https://doi.org/10.1109/TIP.2018.2801119.

    Article  PubMed  Google Scholar 

  37. Iakovidis DK, Georgakopoulos SV, Vasilakakis M, Koulaouzidis A, Plagianakos VP. Detecting and locating gastrointestinal anomalies using deep learning and iterative cluster unification. IEEE Trans Med Imaging. 2018;37(10):2196–210. https://doi.org/10.1109/tmi.2018.2837002.

    Article  PubMed  Google Scholar 

  38. Leenhardt R, Vasseur P, Li C, Saurin JC, Rahmi G, Cholet F, et al. A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc. 2019;89(1):189–94. https://doi.org/10.1016/j.gie.2018.06.036.

    Article  PubMed  Google Scholar 

  39. Zhou J, Wu L, Wan X, Shen L, Liu J, Zhang J, et al. A novel artificial intelligence system for the assessment of bowel preparation (with video). Gastrointest Endosc. 2020;91(2):428–35.e2. https://doi.org/10.1016/j.gie.2019.11.026.

    Article  PubMed  Google Scholar 

  40. Wu L, Zhang J, Zhou W, An P, Shen L, Liu J, et al. Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy. Gut. 2019;68(12):2161–9. https://doi.org/10.1136/gutjnl-2018-317366.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Gastroenterology and Hepatology, Endoscopy Unit, Mayo Clinic Jacksonville, 4500 San Pablo Road, Jacksonville, FL, 32224, USA

    Jiyoung Lee & Michael B. Wallace

  2. Health Screening and Promotion Center, Asan Medical Center, Seoul, South Korea

    Jiyoung Lee

  3. Center of Research in Computer Vision, University of Central Florida, Orlando, FL, USA

    Michael B. Wallace

Authors
  1. Jiyoung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael B. Wallace
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael B. Wallace.

Ethics declarations

Conflict of Interest

Dr. Wallace reports grants from Fujifilm, Boston Scientific, Olympus, Medtronic, Ninepoint Medical, Cosmo/Aries Pharmaceuticals, personal fees from Virgo Inc., Cosmo/Aries Pharmaceuticals, Anx Robotica (2019), Covidien, GI Supply, other from Virgo Inc., other from Synergy Pharmaceuticals, Boston Scientific, Cook Medical, outside the submitted work

Dr. Lee has nothing to disclose.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Endoscopy and Surgery

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, J., Wallace, M.B. State of the Art: The Impact of Artificial Intelligence in Endoscopy 2020. Curr Gastroenterol Rep 23, 7 (2021). https://doi.org/10.1007/s11894-021-00810-9

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11894-021-00810-9

Keywords

Search

Navigation