Skip to main content

Advertisement

SpringerLink
Log in

Development and validation of a deep learning-based algorithm for colonoscopy quality assessment

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Quality indicators should be assessed and monitored to improve colonoscopy quality in clinical practice. Endoscopists must enter relevant information in the endoscopy reporting system to facilitate data collection, which may be inaccurate. The current study aimed to develop a full deep learning-based algorithm to identify and analyze intra-procedural colonoscopy quality indicators based on endoscopy images obtained during the procedure.

Methods

A deep learning system for classifying colonoscopy images for quality assurance purposes was developed and its performance was assessed with an independent dataset. The system was utilized to analyze captured images and results were compared with those of real-world reports.

Results

In total, 10,417 images from the hospital endoscopy database and 3157 from Hyper-Kvasir open dataset were utilized to develop the quality assurance algorithm. The overall accuracy of the algorithm was 96.72% and that of the independent test dataset was 94.71%. Moreover, 761 real-world reports and colonoscopy images were analyzed. The accuracy of electronic reports about cecal intubation rate was 99.34% and that of the algorithm was 98.95%. The agreement rate for the assessment of polypectomy rates using the electronic reports and the algorithm was 0.87 (95% confidence interval 0.83–0.90). A good correlation was found between the withdrawal time calculated using the algorithm and that entered by the physician (correlation coefficient r = 0.959, p < 0.0001).

Conclusion

We proposed a novel deep learning-based algorithm that used colonoscopy images for quality assurance purposes. This model can be used to automatically assess intra-procedural colonoscopy quality indicators in clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cohen J, Pike IM (2015) Defining and measuring quality in endoscopy. Gastrointest Endosc 81:1–2

    Article  Google Scholar 

  2. Kaminski MF, Thomas-Gibson S, Bugajski M, Bretthauer M, Rees CJ, Dekker E et al (2017) Performance measures for lower gastrointestinal endoscopy: a European society of gastrointestinal endoscopy (ESGE) quality improvement initiative. Endoscopy 49:378–397

    Article  Google Scholar 

  3. Rees CJ, Thomas Gibson S, Rutter MD, Baragwanath P, Pullan R, Feeney M et al (2016) UK key performance indicators and quality assurance standards for colonoscopy. Gut 65:1923–1929

    Article  Google Scholar 

  4. Rizk MK, Sawhney MS, Cohen J, Pike IM, Adler DG, Dominitz JA et al (2015) Quality indicators common to all GI endoscopic procedures. Gastrointest Endosc 81:3–16

    Article  Google Scholar 

  5. Muthukuru S, Alomari M, Bisen R, Parikh MP, Al Momani L, Talal Sarmini M et al (2020) Quality of colonoscopy: a comparison between gastroenterologists and nongastroenterologists. Dis Colon Rectum 63:980–987

    Article  Google Scholar 

  6. Bretthauer M, Aabakken L, Dekker E, Kaminski MF, Rosch T, Hultcrantz R et al (2016) Reporting systems in gastrointestinal endoscopy: requirements and standards facilitating quality improvement: European society of gastrointestinal endoscopy position statement. United European Gastroenterol J 4:172–176

    Article  CAS  Google Scholar 

  7. Lieberman DA, Faigel DO, Logan JR, Mattek N, Holub J, Eisen G et al (2009) Assessment of the quality of colonoscopy reports: results from a multicenter consortium. Gastrointest Endosc 69:645–653

    Article  Google Scholar 

  8. Marshall JB, Brown DN (1996) Photodocumentation of total colonoscopy: how successful are endoscopists? Do reviewers agree? Gastrointest Endosc 44:243–248

    Article  CAS  Google Scholar 

  9. Thomas-Gibson S (2008) The caecum or not the caecum? Eur J Gastroenterol Hepatol 20:500–502

    Article  Google Scholar 

  10. Thuraisingam AI, Brown JL, Anderson JT (2008) What are the sensitivity and specificity of endoscopic photographs in determining completion of colonoscopy? Results from an online questionnaire. Eur J Gastroenterol Hepatol 20:567–571

    Article  Google Scholar 

  11. May FP, Shaukat A (2020) State of the science on quality indicators for colonoscopy and how to achieve them. Am J Gastroenterol 115:1183–1190

    Article  Google Scholar 

  12. Zhou J, Wu L, Wan X, Shen L, Liu J, Zhang J et al (2020) A novel artificial intelligence system for the assessment of bowel preparation (with video). Gastrointest Endosc 91:428–35 e2

    Article  Google Scholar 

  13. Attardo S, Chandrasekar VT, Spadaccini M, Maselli R, Patel HK, Desai M et al (2020) Artificial intelligence technologies for the detection of colorectal lesions: the future is now. World J Gastroenterol 26:5606–5616

    Article  Google Scholar 

  14. Milluzzo SM, Cesaro P, Grazioli LM, Olivari N, Spada C (2021) Artificial intelligence in lower gastrointestinal endoscopy: the current status and future perspective. Clin Endosc 54:329–339

    Article  Google Scholar 

  15. Yen H-H, Wu P-Y, Su P-Y, Yang C-W, Chen Y-Y, Chen M-F et al (2021) Performance comparison of the deep learning and the human endoscopist for bleeding peptic ulcer disease. J Med Biol Eng 41:504–513. https://doi.org/10.1007/s40846-021-00608-0

    Article  Google Scholar 

  16. Luo H, Xu G, Li C, He L, Luo L, Wang Z et al (2019) Real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy: a multicentre, case-control, diagnostic study. Lancet Oncol 20:1645–1654

    Article  CAS  Google Scholar 

  17. Chang YY, Li PC, Chang RF, Yao CD, Chen YY, Chang WY et al (2021) Deep learning-based endoscopic anatomy classification: an accelerated approach for data preparation and model validation. Surg Endosc. https://doi.org/10.1007/s00464-021-08698-2

    Article  PubMed  PubMed Central  Google Scholar 

  18. Chang YY, Yen HH, Li PC, Chang RF, Yang CW, Chen YY et al (2021) Upper endoscopy photodocumentation quality evaluation with novel deep learning system. Dig Endosc. https://doi.org/10.1111/den.14179

    Article  PubMed  Google Scholar 

  19. Lai EJ, Calderwood AH, Doros G, Fix OK, Jacobson BC (2009) The Boston bowel preparation scale: a valid and reliable instrument for colonoscopy-oriented research. Gastrointest Endosc 69:620–625

    Article  Google Scholar 

  20. Borgli H, Thambawita V, Smedsrud PH, Hicks S, Jha D, Eskeland SL et al (2020) HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci Data 7:283

    Article  Google Scholar 

  21. Zhang H, Wu C, Zhang Z, Zhu Y, Lin H, Zhang Z et al. (2020) ResNeSt: split-attention networks. arxiv:2004.08955

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conf Comput Vision Pattern Recognit (CVPR) 2016:770–778

    Google Scholar 

  23. Deng J, Dong W, Socher R, Li L, Kai L, Li F-F (2009) ImageNet: a large-scale hierarchical image database. IEEE Conf Comput Vision Pattern Recognit 2009:248–255

    Google Scholar 

  24. Pillai I, Fumera G, Roli F (2017) Designing multi-label classifiers that maximize F measures: state of the art. Pattern Recognit 61:394–404

    Article  Google Scholar 

  25. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. IEEE Int Conf Comput Vision (ICCV) 2017:618–626

    Google Scholar 

  26. Ribeiro MT, Singh S, Guestrin C (2018) Anchors: high-precision model-agnostic explanations. The Thirty-Second AAAI Conference on Artificial Intelligence, vol 32, No 1, Palo Alto, CA, pp 1527–1535

  27. Xie B, Mu Y, Tao D, Huang K (2011) m-SNE: multiview stochastic neighbor embedding. IEEE Trans Syst Man Cybern Part B (Cybernetics) 41:1088–1096

    Article  Google Scholar 

  28. Chiu HM, Jen GH, Wang YW, Fann JC, Hsu CY, Jeng YC et al (2021) Long-term effectiveness of faecal immunochemical test screening for proximal and distal colorectal cancers. Gut. https://doi.org/10.1136/gutjnl-2020-322545

    Article  PubMed  Google Scholar 

  29. Kaminski MF, Regula J, Kraszewska E, Polkowski M, Wojciechowska U, Didkowska J et al (2010) Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 362:1795–1803

    Article  CAS  Google Scholar 

  30. Spada C, Koulaouzidis A, Hassan C, Amaro P, Agrawal A, Brink L et al (2021) Colonoscopy quality across Europe: a report of the European colonoscopy quality investigation (ECQI) group. Endosc Int Open 9:E1456–E1462

    Article  Google Scholar 

  31. de Jonge V, Sint Nicolaas J, Cahen DL, Moolenaar W, Ouwendijk RJ, Tang TJ et al (2012) Quality evaluation of colonoscopy reporting and colonoscopy performance in daily clinical practice. Gastrointest Endosc 75:98–106

    Article  Google Scholar 

  32. Moran B, Sehgal R, O’Morain N, Slattery E, Collins C (2021) Impact of photodocumentation of caecal intubation on colonoscopy outcomes. Ir J Med Sci. https://doi.org/10.1007/s11845-020-02469-z

    Article  PubMed  Google Scholar 

  33. Aslinia FM, Bagi P, Sorkin JD, Williams RB, Knodell RG, Sorkin LF et al (2012) Anatomic classification of the endoscopic appearance of the normal appendiceal orifice: a novel tool for recognition and documentation of cecal intubation. Clin Anat 25:496–502

    Article  Google Scholar 

  34. Choi SJ, Khan MA, Choi HS, Choo J, Lee JM, Kwon S et al (2021) Development of artificial intelligence system for quality control of photo documentation in esophagogastroduodenoscopy. Surg Endosc. https://doi.org/10.1007/s00464-020-08236-6

    Article  PubMed  PubMed Central  Google Scholar 

  35. Takiyama H, Ozawa T, Ishihara S, Fujishiro M, Shichijo S, Nomura S et al (2018) Automatic anatomical classification of esophagogastroduodenoscopy images using deep convolutional neural networks. Sci Rep 8:7497

    Article  Google Scholar 

  36. Jheng YC, Wang YP, Lin HE, Sung KY, Chu YC, Wang HS et al (2021) A novel machine learning-based algorithm to identify and classify lesions and anatomical landmarks in colonoscopy images. Surg Endosc. https://doi.org/10.1007/s00464-021-08331-2

    Article  PubMed  Google Scholar 

  37. Yao L, Liu J, Wu L, Zhang L, Hu X, Liu J et al (2021) A Gastrointestinal endoscopy quality control system incorporated with deep learning improved endoscopist performance in a pretest and post-test trial. Clin Transl Gastroenterol 12:e00366

    Article  Google Scholar 

  38. Ouazzani S, Lemmers A, Martinez F, Kindt R, Le Moine O, Delhaye M et al (2021) Implementation of colonoscopy quality monitoring in a Belgian university hospital with integrated computer-based extraction of adenoma detection rate. Endosc Int Open 9:E197–E202

    Article  Google Scholar 

  39. Gohel TD, Burke CA, Lankaala P, Podugu A, Kiran RP, Thota PN et al (2014) Polypectomy rate: a surrogate for adenoma detection rate varies by colon segment, gender, and endoscopist. Clin Gastroenterol Hepatol 12:1137–1142

    Article  Google Scholar 

  40. Holzwanger EA, Bilal M, Glissen Brown JR, Singh S, Becq A, Ernest-Suarez K et al (2020) Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy. Endoscopy. https://doi.org/10.1055/a-1302-2942

    Article  PubMed  Google Scholar 

  41. Deliwala SS, Hamid K, Barbarawi M, Lakshman H, Zayed Y, Kandel P et al (2021) Artificial intelligence (AI) real-time detection vs. routine colonoscopy for colorectal neoplasia: a meta-analysis and trial sequential analysis. Int J Colorectal Dis. https://doi.org/10.1007/s00384-021-03929-3

    Article  PubMed  Google Scholar 

  42. Kaltenbach TR, Soetikno RM, DeVivo R, Laine LA, Barkun A, McQuaid KR et al (2017) Optimizing the quality of endoscopy in inflammatory bowel disease: focus on surveillance and management of colorectal dysplasia using interactive image- and video-based teaching. Gastrointest Endosc 86:1107–17 e1

    Article  Google Scholar 

  43. Iacucci M, Cannatelli R, Tontini GE, Panaccione R, Danese S, Fiorino G et al (2019) Improving the quality of surveillance colonoscopy in inflammatory bowel disease. Lancet Gastroenterol Hepatol 4:971–983

    Article  Google Scholar 

  44. Yen HH, Weng MT, Tung CC, Wang YT, Chang YT, Chang CH et al (2019) Epidemiological trend in inflammatory bowel disease in Taiwan from 2001 to 2015: a nationwide populationbased study. Intest Res 17:54–62

    Article  Google Scholar 

  45. Yen HH, Hsu TC, Chen MW, Su PY, Chen YY (2021) Clinical features and treatment of inflammatory bowel disease in a low-incidence area: a hospital-based retrospective cohort study in Taiwan. Medicine (Baltimore) 100:e25090

    Article  CAS  Google Scholar 

  46. Cho M, Kim JH, Hong KS, Kim JS, Kong HJ, Kim S (2019) Identification of cecum time-location in a colonoscopy video by deep learning analysis of colonoscope movement. Peer J 7:e7256

    Article  Google Scholar 

Download references

Funding

This study received funding from the Ministry of Science and Technology of Taiwan (MOST 110-2634-F-002-009) and Changhua Christian Hospital (110-CCH-IRP-020).

Author information

Authors and Affiliations

  1. Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan

    Yuan-Yen Chang, Pai-Chi Li & Ruey-Feng Chang

  2. Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan

    Ruey-Feng Chang

  3. Artificial Intelligence Development Center, Changhua Christian Hospital, Changhua, Taiwan

    Ruey-Feng Chang & Hsu-Heng Yen

  4. Department of Colorectal Surgery, Changhua Christian Hospital, Changhua, Taiwan

    Yu-Yao Chang & Hsu-Heng Yen

  5. Division of Gastroenterology, Changhua Christian Hospital, Changhua, Taiwan

    Siou-Ping Huang, Yang-Yuan Chen & Hsu-Heng Yen

  6. Department of Medical Education, National Taiwan University Hospital, Taipei, Taiwan

    Wen-Yen Chang

  7. Department of Electrical Engineering, Chung Yuan University, Taoyuan, Taiwan

    Hsu-Heng Yen

  8. Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung, Taiwan

    Hsu-Heng Yen

Authors
  1. Yuan-Yen Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pai-Chi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruey-Feng Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu-Yao Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Siou-Ping Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yang-Yuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wen-Yen Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hsu-Heng Yen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hsu-Heng Yen.

Ethics declarations

Disclosures

Mr. Yuan-Yen Chang, Prof. Pai-Chi Li, Prof. Ruey-Feng Chang, Dr. Yu-Yao Chang, Ms. Siou-Ping Huan, Drs. Yang-Yuan Chen, Wen-Yen Chang, and Hsu-Heng Yen have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, YY., Li, PC., Chang, RF. et al. Development and validation of a deep learning-based algorithm for colonoscopy quality assessment. Surg Endosc 36, 6446–6455 (2022). https://doi.org/10.1007/s00464-021-08993-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-021-08993-y

Keywords

Search

Navigation