Skip to main content

Advertisement

SpringerLink
Log in

Exploring the challenge of early gastric cancer diagnostic AI system face in multiple centers and its potential solutions

  • Original Article—Alimentary Tract
  • Published:
Journal of Gastroenterology Aims and scope Submit manuscript

Abstract

Background

Artificial intelligence (AI) performed variously among test sets with different diversity due to sample selection bias, which can be stumbling block for AI applications. We previously tested AI named ENDOANGEL, diagnosing early gastric cancer (EGC) on single-center videos in man–machine competition. We aimed to re-test ENDOANGEL on multi-center videos to explore challenges applying AI in multiple centers, then upgrade ENDOANGEL and explore solutions to the challenge.

Methods

ENDOANGEL was re-tested on multi-center videos retrospectively collected from 12 institutions and compared with performance in previously reported single-center videos. We then upgraded ENDOANGEL to ENDOANGEL-2022 with more training samples and novel algorithms and conducted competition between ENDOANGEL-2022 and endoscopists. ENDOANGEL-2022 was then tested on single-center videos and compared with performance in multi-center videos; the two AI systems were also compared with each other and endoscopists.

Results

Forty-six EGCs and 54 non-cancers were included in multi-center video cohort. On diagnosing EGCs, compared with single-center videos, ENDOANGEL showed stable sensitivity (97.83% vs. 100.00%) while sharply decreased specificity (61.11% vs. 82.54%); ENDOANGEL-2022 showed similar tendency while achieving significantly higher specificity (79.63%, p < 0.01) making fewer mistakes on typical lesions than ENDOANGEL. On detecting gastric neoplasms, both AI showed stable sensitivity while sharply decreased specificity. Nevertheless, both AI outperformed endoscopists in the two competitions.

Conclusions

Great increase of false positives is a prominent challenge for applying EGC diagnostic AI in multiple centers due to high heterogeneity of negative cases. Optimizing AI by adding samples and using novel algorithms is promising to overcome this challenge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

AI:

Artificial intelligence

GC:

Gastric cancer

EGC:

Early gastric cancer

LNM:

Lymph node metastasis

SM:

Submucosal

CNN:

Conventional neural network

WLE:

White light endoscopy

M-NBI:

Magnifying narrow-band imaging

PPV:

Positive predictive value

NPV:

Negative predictive value

References

  1. Smyth EC, Nilsson M, Grabsch HI, et al. Gastric cancer. Lancet. 2020;396:635–48.

    Article  CAS  PubMed  Google Scholar 

  2. Katai H, Ishikawa T, Akazawa K, et al. Five-year survival analysis of surgically resected gastric cancer cases in Japan: a retrospective analysis of more than 100,000 patients from the nationwide registry of the Japanese gastric cancer association (2001–2007). Gastric Cancer. 2018;21:144–54.

    Article  PubMed  Google Scholar 

  3. Banks M, Graham D, Jansen M, et al. British society of gastroenterology guidelines on the diagnosis and management of patients at risk of gastric adenocarcinoma. Gut. 2019;68:1545–75.

    Article  PubMed  Google Scholar 

  4. Bi WL, Hosny A, Schabath MB, et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin. 2019;69:127–57.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Niazi MKK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. Lancet Oncol. 2019;20:e253–61.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Yu TF, He W, Gan CG, et al. Deep learning applied to two-dimensional color Doppler flow imaging ultrasound images significantly improves diagnostic performance in the classification of breast masses: a multicenter study. Chin Med J (Engl). 2021;134:415–24.

    Article  PubMed  Google Scholar 

  7. Haggenmuller S, Maron RC, Hekler A, et al. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur J Cancer. 2021;156:202–16.

    Article  PubMed  Google Scholar 

  8. Hirasawa T, Aoyama K, Tanimoto T, et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer. 2018;21:653–60.

    Article  PubMed  Google Scholar 

  9. Niikura R, Aoki T, Shichijo S, et al. Artificial intelligence versus expert endoscopists for diagnosis of gastric cancer in patients who have undergone upper gastrointestinal endoscopy. Endoscopy. 2022;54:780–4.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Johnson KB, Wei WQ, Weeraratne D, et al. Precision medicine, AI, and the future of personalized health care. Cts-Clin Transl Sci. 2021;14:86–93.

    Article  Google Scholar 

  11. Chen M, Decary M. Artificial intelligence in healthcare: an essential guide for health leaders. Healthc Manage Forum. 2020;33:10–8.

    Article  PubMed  Google Scholar 

  12. Wu L, Wang J, He X, et al. Deep learning system compared with expert endoscopists in predicting early gastric cancer and its invasion depth and differentiation status (with videos). Gastrointest Endosc. 2022;95(92–104): e3.

    Google Scholar 

  13. Nagtegaal ID, Odze RD, Klimstra D, et al. The 2019 WHO classification of tumours of the digestive system. Histopathology. 2020;76:182–8.

    Article  PubMed  Google Scholar 

  14. Skrede OJ, De Raedt S, Kleppe A, et al. Deep learning for prediction of colorectal cancer outcome: a discovery and validation study. Lancet. 2020;395:350–60.

    Article  CAS  PubMed  Google Scholar 

  15. van Engelen JE, Hoos HH. A survey on semi-supervised learning. Mach Learn. 2020;109:373–440.

    Article  Google Scholar 

  16. Maron RC, Haggenmuller S, von Kalle C, et al. Robustness of convolutional neural networks in recognition of pigmented skin lesions. Eur J Cancer. 2021;145:81–91.

    Article  PubMed  Google Scholar 

  17. Sharmat P, Hassan C. Artificial intelligence and deep learning for upper gastrointestinal neoplasia. Gastroenterology. 2022;162:1056–66.

    Article  Google Scholar 

  18. Benkarim O, Paquola C, Park BY, et al. Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging. PLoS Biol. 2022;20: e3001627.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Norgeot B, Quer G, Beaulieu-Jones BK, et al. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nat Med. 2020;26:1320–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Ryan M, Stahl BC. Artificial intelligence ethics guidelines for developers and users: clarifying their content and normative implications. J Inf Commun Ethics Soc. 2021;19:61–86.

    Article  Google Scholar 

  21. de Hond AAH, Leeuwenberg AM, Hooft L, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. Npj Digital Med. 2022;5:2.

    Article  Google Scholar 

  22. Luo H, Xu G, Li C, et al. Real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy: a multicentre, case-control, diagnostic study. Lancet Oncol. 2019;20:1645–54.

    Article  CAS  PubMed  Google Scholar 

  23. Wu L, Zhou W, Wan X, et al. A deep neural network improves endoscopic detection of early gastric cancer without blind spots. Endoscopy. 2019;51:522–31.

    Article  PubMed  Google Scholar 

  24. Yu G, Sun K, Xu C, et al. Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images. Nat Commun. 2021;12:6311.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Wu L, Xu M, Jiang X, et al. Real-time artificial intelligence for detecting focal lesions and diagnosing neoplasms of the stomach by white-light endoscopy (with videos). Gastrointest Endosc. 2022;95:269-80.e6.

    Article  PubMed  Google Scholar 

  26. Chiu PWY, Uedo N, Singh R, et al. An Asian consensus on standards of diagnostic upper endoscopy for neoplasia. Gut. 2019;68:186–97.

    Article  PubMed  Google Scholar 

  27. Nakanishi H, Doyama H, Ishikawa H, et al. Evaluation of an e-learning system for diagnosis of gastric lesions using magnifying narrow-band imaging: a multicenter randomized controlled study. Endoscopy. 2017;49:957–67.

    Article  PubMed  Google Scholar 

  28. Tang D, Ni M, Zheng C, et al. A deep learning-based model improves diagnosis of early gastric cancer under narrow band imaging endoscopy. Surg Endosc. 2022;36:7800–10.

    Article  PubMed  Google Scholar 

  29. Yuan XL, Liu W, Liu Y, et al. Artificial intelligence for diagnosing microvessels of precancerous lesions and superficial esophageal squamous cell carcinomas: a multicenter study. Surg Endosc. 2022;36:8651–62.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

College-enterprise Deepening Reform Project of Wuhan University (to Honggang Yu). Artificial Intelligence Application Demonstration Scenario Project Wuhan (grant no.2022YYCJ01) (to Honggang Yu). National Natural Science Foundation of China-Youth Science Fund (grant no.82202257) (To Lianlian Wu). Special projects for knowledge innovation of Wuhan (grant no.2022020801020482) (To Lianlian Wu).

Funding

College-enterprise Deepening Reform Project of Wuhan Universit, Artificial Intelligence Application Demonstration Scenario Project Wuhan, 2022YYCJ01, Hongang Yu, National Natural Science Foundation of China-Youth Science Fund, 82202257, Lianlian Wu, Special projects for knowledge innovation of Wuhan, 2022020801020482, Lianlian Wu.

Author information

Author notes

  1. Zehua Dong and Xiao Tao authors have contributed equally to this work.

Authors and Affiliations

  1. Renmin Hospital of Wuhan University, Wuhan, China

    Zehua Dong, Xiao Tao, Hongliu Du, Junxiao Wang, Li Huang, Chaijie Luo, Yanxia Li, Xiaoquan Zeng, Jun Liu, Yijie Zhu, Lianlian Wu & Honggang Yu

  2. Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Zehua Dong, Xiao Tao, Hongliu Du, Junxiao Wang, Li Huang, Chaijie Luo, Yanxia Li, Xiaoquan Zeng, Jun Liu, Yijie Zhu, Lianlian Wu & Honggang Yu

  3. Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China

    Zehua Dong, Xiao Tao, Hongliu Du, Junxiao Wang, Li Huang, Chaijie Luo, Yanxia Li, Xiaoquan Zeng, Jun Liu, Yijie Zhu, Lianlian Wu & Honggang Yu

  4. Department of Gastroenterology, Yijishan Hospital of Wannan Medical College, Wuhu, 241001, Anhui, People’s Republic of China

    Chiyi He

  5. Department of Digestive Endoscopy, The Fourth Hospital of China Medical University, Shenyang, 110032, Liaoning Province, People’s Republic of China

    Zhifeng Zhao

  6. Department of Gastroenterology, Taizhou Hospital of Zhejiang Province Affiliated to Wenzhou Medical University, Linhai, Zhejiang, China

    Xinli Mao

  7. Department of Gastroenterology, The People’s Hospital of China Three Gorges University, The First People’s Hospital of Yichang, Yichang, China

    Yaowei Ai

  8. Department of Gastroenterology, The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China

    Beiping Zhang

  9. Department of Gastroenterology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China

    Mei Liu

  10. Department of Endoscopy, The First Hospital of Jilin University, Changchun, China

    Hong Xu

  11. Department of Gastroenterology, The Second Affiliated Hospital of Baotou Medical College, Baotou, Inner Mongolia, China

    Zhenyu Jiang

  12. Department of Gastroenterology, Ruijin Hospital, Shanghai Jiaotong University, Gubei Branch, Shanghai, People’s Republic of China

    Yunwei Sun

  13. Department of Gastroenterology, School of Clinical Medicine, Henan Provincial People’s Hospital, People’s Hospital of Zhengzhou University, Henan University, Zhengzhou, Henan, China

    Xiuling Li

  14. Department of Gastroenterology, Jilin City People’s Hospital, Jilin, China

    Zhihong Liu

  15. Endoscopy Center, School of Medicine, The First Affiliated Hospital of Xiamen University, Xiamen University, Xiamen, China

    Jinzhong Chen

  16. Department of Gastroenterology, Xi’an Gaoxin Hospital, Xi’an, 710032, Shaanxi Province, China

    Ying Song

  17. Yi Xin Clinic, Changzhou, Jiangsu, China

    Guowei Liu

  18. Department of Gastroenterology, Renmin Hospital of Wuhan University, 99 Zhangzhidong Road, Wuhan, 430060, Hubei Province, China

    Lianlian Wu & Honggang Yu

Authors
  1. Zehua Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongliu Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junxiao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Li Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chiyi He
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhifeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xinli Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yaowei Ai
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Beiping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Mei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Hong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Zhenyu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Yunwei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Xiuling Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Zhihong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Jinzhong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Ying Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Guowei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Chaijie Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Yanxia Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Xiaoquan Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Yijie Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Lianlian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Honggang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HGY and LLW made a big picture of the work and supervised the overall study. ZHD designed and did the experiments. ZHD and XT developed the system. CYH, ZFZ, XLM, YWA, BPZ, ML, HX, ZYJ, YWS, XLL, ZHL, JZC, YS, and GWL were involved in the data collection. ZHD and XT wrote the original draft. ZHD, HLD, CJL, LH, and XT analyzed the data. HGY and LLW revised the manuscript. JXW, XQZ, YXL, JL, and YJZ were involved in the format check and reviewing. HGY was responsible for the overall content as guarantor. All authors approved the final version of the report.

Corresponding authors

Correspondence to Lianlian Wu or Honggang Yu.

Ethics declarations

Conflict of interest

The authors have nothing to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Video 1. Use of ENDOANGEL-2022 on detecting gastric neoplasm and diagnosing early gastric cancer. A pathologically confirmed high-grade intraepithelial neoplasia is shown. ENDOANGEL-2022 successfully detected it as a gastric neoplasm under white-light endoscopy, diagnosed it as an EGC under magnifying narrow-band imaging. Supplementary file1 (MP4 12917 KB)

Video 2. Use of ENDOANGEL-2022 for detecting and diagnosing chronic inflammation. A pathologically confirmed chronic inflammation lesion is shown. ENDOANGEL-2022 successfully detected the lesion as non-neoplasm under white-light endoscopy and diagnosed it as noncancerous under magnifying narrow-band imaging. Supplementary file2 (MP4 59998 KB)

Supplementary file3 (DOCX 35 KB)

535_2023_2025_MOESM4_ESM.tif

Figure S1. Flow chart of the eligibility of the videos and lesions in the present competition. Supplementary file4 (TIF 8192 KB)

535_2023_2025_MOESM5_ESM.tif

Figure S2. Flow chart of the eligibility of the endoscopists in the present competition. Supplementary file5 (TIF 9383 KB)

535_2023_2025_MOESM6_ESM.tif

Figure S3. Heat maps and bar graph showing answers of two AI systems and the error rate of endoscopists on each case on detecting gastric neoplasms. A. Single-center videos. B. Multi-center videos. Each column represents a different case. The 1st and 2nd rows showed the results of the two AI systems. The 3rd row showed the gold standard, neoplasms were shown in red color, and non-neoplasms were shown in blue color. Supplementary file6 (TIF 13191 KB)

535_2023_2025_MOESM7_ESM.tif

Figure S4. Heat maps and bar graph showing answers of two AI systems and the error rate of endoscopists in each case on diagnosing early gastric cancer. A. Single-center videos. B. Multi-center videos. Each column represents a different case. The 1st and 2nd rows showed the results of the two AI systems. The 3rd row showed the gold standard, early gastric cancers was shown in red color, and non-cancers were shown in blue color. Supplementary file7 (TIF 11925 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, Z., Tao, X., Du, H. et al. Exploring the challenge of early gastric cancer diagnostic AI system face in multiple centers and its potential solutions. J Gastroenterol 58, 978–989 (2023). https://doi.org/10.1007/s00535-023-02025-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00535-023-02025-3

Keywords

Search

Navigation