Advertisement

Towards Automated Colonoscopy Diagnosis: Binary Polyp Size Estimation via Unsupervised Depth Learning

  • Hayato ItohEmail author
  • Holger R. Roth
  • Le Lu
  • Masahiro Oda
  • Masashi Misawa
  • Yuichi Mori
  • Shin-ei Kudo
  • Kensaku Mori
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11071)

Abstract

In colon cancer screening, polyp size estimation using only colonoscopy images or videos is difficult even for expert physicians although the size information of polyps is important for diagnosis. Towards the fully automated computer-aided diagnosis (CAD) pipeline, a robust and precise polyp size estimation method is highly desired. However, the size estimation problem of a three-dimensional object from a two-dimensional image is ill-posed due to the lack of three-dimensional spatial information. To circumvent this challenge, we formulate a relaxed form of size estimation as a binary classification problem and solve it by a new deep neural network architecture: BseNet. This relaxed form of size estimation is defined as a two-category classification: under and over a certain polyp dimension criterion that would provoke different clinical treatments (resecting the polyp or not). BseNet estimates the depth map image from an input colonoscopic RGB image using unsupervised deep learning, and integrates RGB with the computed depth information to produce a four-channel RGB-D imagery data, that is subsequently encoded by BseNet to extract deep RGB-D image features and facilitate the size classification into two categories: under and over 10 mm polyps. For the evaluation of BseNet, a large dataset of colonoscopic videos of totally over 16 h is constructed. We evaluate the accuracies of both binary polyp size estimation and polyp detection performance since detection is a prerequisite step of a fully automated CAD system. The experimental results show that our proposed BseNet achieves 79.2 % accuracy for binary polyp-size classification. We also combine the image feature extraction by BseNet and classification of short video clips using a long short-term memory (LSTM) network. Polyp detection (if the video clip contains a polyp or not) shows 88.8 % sensitivity when employing the spatio-temporal image feature extraction and classification.

Keywords

Size estimation Depth estimation Deep neural networks Long short-term memory (LSTM) Polyp detection 

References

  1. 1.
    Winawer, S.: Guidelines for colonoscopy surveillance after polypectomy: a consensus update by the us multi-society task force on colorectal cancer and the american cancer society. Gastroenterology 130, 1872–1885 (2006)CrossRefGoogle Scholar
  2. 2.
    Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: Proceedings of the CVPR (2016)Google Scholar
  3. 3.
    Zhou, T., Brown, M., Snavely, N., Lowe, D.: Unsupervised learning of depth and ego-motion from video. In: Proceedings of the CVPR (2017)Google Scholar
  4. 4.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2011)zbMATHGoogle Scholar
  5. 5.
    Visentini-Scarzanella, M., Stoyanov, D., Yang, G.Z.: Metric depth recovery from monocular images using shape-from-shading and specularities. In: Proceedings of the IEEE ICIP (2012)Google Scholar
  6. 6.
    Bernal, J., Sánchez, J., Vilariño, F.: Towards automatic polyp detection with a polyp appearance model. Pattern Recognit. 45(9), 3166–3182 (2012)CrossRefGoogle Scholar
  7. 7.
    Bernal, J.: Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge. IEEE Trans. Med. Imaging 36(6), 1231–1249 (2017)CrossRefGoogle Scholar
  8. 8.
    Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3d convolutional networks. In: Proceedings of the ICCV (2015)Google Scholar
  9. 9.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Hayato Itoh
    • 1
    Email author
  • Holger R. Roth
    • 1
    • 2
  • Le Lu
    • 2
  • Masahiro Oda
    • 1
  • Masashi Misawa
    • 3
  • Yuichi Mori
    • 3
  • Shin-ei Kudo
    • 3
  • Kensaku Mori
    • 1
    • 4
    • 5
  1. 1.Graduate School of InformaticsNagoya UniversityNagoyaJapan
  2. 2.AI-InfraNVIDIA CorporationSanta ClaraUSA
  3. 3.Digestive Disease CenterShowa University Northern Yokohama HospitalYokohamaJapan
  4. 4.Information Technology CenterNagoya UniversityNagoyaJapan
  5. 5.Research Center for Medical BigdataNational Institute of InformaticsTokyoJapan

Personalised recommendations