Skip to main content

Advertisement

SpringerLink
Log in

Diagnosis using deep-learning artificial intelligence based on the endocytoscopic observation of the esophagus

  • Original Article
  • Published:
Esophagus Aims and scope Submit manuscript

Abstract

Background and aims

The endocytoscopic system (ECS) helps in virtual realization of histology and can aid in confirming histological diagnosis in vivo. We propose replacing biopsy-based histology for esophageal squamous cell carcinoma (ESCC) by using the ECS. We applied deep-learning artificial intelligence (AI) to analyse ECS images of the esophagus to determine whether AI can support endoscopists for the replacement of biopsy-based histology.

Methods

A convolutional neural network-based AI was constructed based on GoogLeNet and trained using 4715 ECS images of the esophagus (1141 malignant and 3574 non-malignant images). To evaluate the diagnostic accuracy of the AI, an independent test set of 1520 ECS images, collected from 55 consecutive patients (27 ESCCs and 28 benign esophageal lesions) were examined.

Results

On the basis of the receiver-operating characteristic curve analysis, the areas under the curve of the total images, higher magnification pictures, and lower magnification pictures were 0.85, 0.90, and 0.72, respectively. The AI correctly diagnosed 25 of the 27 ESCC cases, with an overall sensitivity of 92.6%. Twenty-five of the 28 non-cancerous lesions were diagnosed as non-malignant, with a specificity of 89.3% and an overall accuracy of 90.9%. Two cases of malignant lesions, misdiagnosed as non-malignant by the AI, were correctly diagnosed as malignant by the endoscopist. Among the 3 cases of non-cancerous lesions diagnosed as malignant by the AI, 2 were of radiation-related esophagitis and one was of gastroesophageal reflux disease.

Conclusion

AI is expected to support endoscopists in diagnosing ESCC based on ECS images without biopsy-based histological reference.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Kumagai Y, Monma K, Kawada K. Magnifying chromoendoscopy of the esophagus: in vivo pathological diagnosis using an endocytoscopy system. Endoscopy. 2004;36:590–4.

    Article  CAS  PubMed  Google Scholar 

  2. Kumagai Y, Kawada K, Yamazaki S, et al. Endocytoscopic observation for esophageal squamous cell carcinoma: can biopsy histology be omitted? Dis Esophagus. 2009;22:505–12.

    Article  CAS  PubMed  Google Scholar 

  3. Kumagai Y, Kawada K, Yamazaki S, et al. Prospective replacement of magnifying endoscopy by a newly developed endocytoscope, the ‘GIF-Y0002’. Dis Esophagus. 2010;23:627–32.

    Article  CAS  PubMed  Google Scholar 

  4. Kumagai Y, Kawada K, Yamazaki S, et al. Current status and limitations of the newly developed endocytoscope GIF-Y0002 with reference to its diagnostic performance for common esophageal lesions. J Dig Dis. 2012;13:393–400.

    Article  PubMed  Google Scholar 

  5. Kumagai Y, Kawada K, Yamazaki S, et al. Endocytoscopic observation of esophageal squamous cell carcinoma. Dig Endosc. 2010;22:10–6.

    Article  PubMed  Google Scholar 

  6. Kumagai Y, Kawada K, Higashi M, et al. Endocytoscopic observation of various esophageal lesions at ×600: can nuclear abnormality be recognized? Dis Esophagus. 2015;28:269–75.

    Article  CAS  PubMed  Google Scholar 

  7. Kumagai Y, Takubo K, Kawada K, et al. Endocytoscpic observation of various types of esophagitis. Esophagus. 2016;13:200–7. https://doi.org/10.1007/s10388-015-0517-1.

    Article  Google Scholar 

  8. Kumagai Y, Takubo K, Kawada K, et al. A newly developed continuous zoom-focus endocytoscope. Endoscopy. 2017;49(2):176–80.

    PubMed  Google Scholar 

  9. Inoue H, Kazawa T, Sato Y, et al. In vivo observation of living cancer cells in the esophagus, stomach, and colon using catheter-type contact endoscope, “Endo-Cytoscopy system”. Gastrointest Endosc Clin N Am. 2004;14:589–94.

    Article  PubMed  Google Scholar 

  10. Sasajima K, Kudo SE, Inoue H, et al. Real-time in vivo virtual histology of colorectal lesions when using the endocytoscopy system. Gastrointest Endosc. 2006;63(7):1010–7.

    Article  PubMed  Google Scholar 

  11. Bibault JE, Giraud P, Burgun A. Big Data and machine learning in radiation oncology: state of the art and future prospects. Cancer Lett. 2016;382:110–7.

    Article  CAS  PubMed  Google Scholar 

  12. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–10.

    Article  PubMed  Google Scholar 

  14. Yoshida H, Shimazu T, Kiyuna T, et al. Automated histological classification of whole-slide images of gastric biopsy specimens. Gastric Cancer. 2018;21(2):249–57. https://doi.org/10.1007/s10120-017-0731-8.

    Article  PubMed  Google Scholar 

  15. Misawa M, Kudo S, Mori Y, et al. Accuracy of computer-aided diagnosis based on narrow-band imaging endocytoscopy for diagnosing colorectal lesions: comparison with experts. Int J Comput Assist Radiol Surg. 2017;12:757–66.

    Article  PubMed  Google Scholar 

  16. Shichijo S, Nomura S, Aoyama K, et al. Application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images. EBioMedicine. 2017;25:106–11.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Hirasawa T, Aoyama K, Tanimoto T, et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer. 2018;21(4):653–60. https://doi.org/10.1007/s10120-018-0793-2.

    Article  PubMed  Google Scholar 

  18. Inoue H, Kazawa T, Sato Y, et al. In vivo observation of living cancer cells in the esophagus, stomach, and colon using catheter-type contact endoscope, “Endo-Cytoscopy system”. Gastrointest Endosc Clin N Am. 2004;14:589–94.

    Article  PubMed  Google Scholar 

  19. Misawa M, Kudo S, Mori Y, et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology. 2018;154(8):2027.e3–2029.e3. https://doi.org/10.1053/j.gastro.2018.04.003.

    Article  Google Scholar 

  20. Takubo K. Pathology of the esophagus. 3rd ed. Tokyo: Wiley Publishing Japan; 2017. p. 88–102 (131–134 ).

    Google Scholar 

Download references

Acknowledgements

We are grateful to Tomoyuki Kawada for his kind advice and confirmation of statistical analysis.

Author information

Authors and Affiliations

  1. Department of Digestive Tract and General Surgery, Saitama Medical Center, Saitama Medical University, 1981 Kamoda, Kawagoe, Saitama, 350-8550, Japan

    Youichi Kumagai, Erito Mochiki & Hideyuki Ishida

  2. Research Team for Geriatric Pathology, Tokyo Metropolitan Institute of Gerontology, Tokyo, Japan

    Kaiyo Takubo

  3. Department of Esophageal and General Surgery, Tokyo Medical and Dental University, Tokyo, Japan

    Kenro Kawada

  4. AI Medical Service Inc., Tokyo, Japan

    Kazuharu Aoyama, Yuma Endo & Tomohiro Tada

  5. Department of Surgery, Teikyo University Hospital, Tokyo, Japan

    Tsuyoshi Ozawa

  6. Tada Tomohiro Institute of Gastroenterology and Proctology, Saitama, Japan

    Tsuyoshi Ozawa, Toshiaki Hirasawa, Toshiyuki Yoshio, Soichiro Ishihara & Tomohiro Tada

  7. Department of Gastroenterology, Cancer Institute Hospital, Japanese Foundation for Cancer Research, Tokyo, Japan

    Toshiaki Hirasawa & Toshiyuki Yoshio

  8. Surgery Department, Sanno Hospital, International University of Health and Welfare, Tokyo, Japan

    Soichiro Ishihara

  9. Department of Gastroenterology, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan

    Mitsuhiro Fujishiro

  10. Department of Pathology, Saitama Medical Center, Saitama Medical University, Saitama, Japan

    Jun-ichi Tamaru

Authors
  1. Youichi Kumagai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaiyo Takubo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kenro Kawada
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kazuharu Aoyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuma Endo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tsuyoshi Ozawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Toshiaki Hirasawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Toshiyuki Yoshio
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Soichiro Ishihara
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Mitsuhiro Fujishiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jun-ichi Tamaru
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Erito Mochiki
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Hideyuki Ishida
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Tomohiro Tada
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Youichi Kumagai.

Ethics declarations

Conflict of interest

T. Tada, Y. Endo and K. Aoyama are employed by AI Medical Service Inc. The other authors declare no conflict of interest.

Human rights statement

All procedures followed were in accordance with the ethical standards of our hospital ethics committee on human experimentation and with the Helsinki Declaration of 1964 and later versions.

Informed consent

Written informed consent for endocytoscopic observation was obtained from all patients. Opt out consent was obtained for AI analysis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumagai, Y., Takubo, K., Kawada, K. et al. Diagnosis using deep-learning artificial intelligence based on the endocytoscopic observation of the esophagus. Esophagus 16, 180–187 (2019). https://doi.org/10.1007/s10388-018-0651-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10388-018-0651-7

Keywords

Search

Navigation