Skip to main content
SpringerLink
Log in

Deep Learning-Driven Models for Endoscopic Image Analysis

  • Chapter
  • First Online:
Advances in Artificial Intelligence, Computation, and Data Science

Part of the book series: Computational Biology ((COBO,volume 31))

Abstract

The advent of video endoscopy has led to an increased interest in the development of computer-aided diagnosis (CAD) approaches. Many of these focus on the use of deep learning methods as a means of automatically identifying abnormalities during endoscopy to lessen the workload on doctors. In this chapter, we take two tasks in endoscopic image analysis as examples, to survey the state of the art, recent advances, and future directions of CAD applications, especially with regard to deep learning models. We introduce the fundamentals of deep learning-driven methods and elaborate on their success in areas such as endoscopic image classification, detection of abnormal regions, and lesion boundary segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://endovissub2017-giana.grand-challenge.org/polypsegmentation/.

References

  1. Karkanis SA, Iakovidis DK, Maroulis DE, Magoulas GD, Theofanous NG (2000) Tumor recognition in endoscopic video images using artificial neural network architectures. In: Proceedings of the 26th Euromicro Conference. EUROMICRO 2000. Informatics: inventing the future, vol. 2. IEEE, New York, pp 423–429

    Google Scholar 

  2. Karkanis SA, Iakovidis DK, Maroulis DE, Karras DA, Tzivras M (2003) Computer-aided tumor detection in endoscopic video using color wavelet features. In: IEEE Transactions on information technology in biomedicine, vol 7. IEEE, New York, pp 141–152

    Google Scholar 

  3. Iakovidis DK, Koulaouzidis A (2015) Software for enhanced video capsule endoscopy: challenges for essential progress. In: Nature reviews gastroenterology & hepatology, vol 12. Nature Publishing Group, Berlin, pp 172–186

    Google Scholar 

  4. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. Curran Associates, Inc., Red Hook, NY, pp 1097–1105

    Google Scholar 

  5. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 3431–3440

    Google Scholar 

  6. Lei H, Han T, Zhou F, Yu Z, Qin J, Elazab A, Lei B (2018) A deeply supervised residual network for hep-2 cell classification via cross-modal transfer learning. In: Pattern recognition, vol 79. Elsevier, Amsterdam, pp 290–302

    Google Scholar 

  7. Sarikaya D, Corso JJ, Guru KA (2017) Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection. In: IEEE transactions on medical imaging, vol 36. IEEE, New York, pp 1542–1549

    Google Scholar 

  8. Xu Y, Li Y, Liu M, Wang Y, Lai M, Eric I, Chang C (2016) Gland instance segmentation by deep multichannel side supervision. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 496–504

    Google Scholar 

  9. Jia X, Meng MQ-H (2016) A deep convolutional neural network for bleeding detection in wireless capsule endoscopy images. In: 2016 38th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, Orlando, FL, pp 639–642

    Google Scholar 

  10. Jia X, Meng MQ-H (2017) Gastrointestinal bleeding detection in wireless capsule endoscopy images using handcrafted and cnn features. In: 2017 39th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, South Korea, pp 3154–3157

    Google Scholar 

  11. Jia X, Meng MQ-H (2017) A study on automated segmentation of blood regions in wireless capsule endoscopy images using fully convolutional networks. In: 2017 IEEE 14th international symposium on biomedical imaging (ISBI). IEEE, Melbourne, pp 179–182

    Google Scholar 

  12. Jia X, Cai L, Liu J, Dai W, Meng MQ-H (2016) GI bleeding detection in wireless capsule endoscopy images based on pattern recognition and a MapReduce framework. In: 2016 IEEE international conference on real-time computing and robotics (RCAR). IEEE, Cambodia, pp 266–271

    Google Scholar 

  13. Jia X, Mai X, Cui Y, Yuan Y, Xing X, Seo H, Xing L, Meng MQ-H (2020) Automatic polyp recognition in colonoscopy images using deep learning and two-stage pyramidal feature prediction. In: IEEE transactions on automation science and engineering. IEEE

    Google Scholar 

  14. Jia X, Xing X, Yuan Y, Xing L, Meng MQ-H (2019) Wireless capsule endoscopy: a new tool for cancer screening in the colon with deep-learning-based polyp recognition. In: Proceedings of the IEEE, vol 108. IEEE, pp 178–197

    Google Scholar 

  15. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. Curran Associates, Inc., Red Hook, NY, pp 91–99

    Google Scholar 

  16. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision. IEEE, Cambridge, MA, pp 1440–1448

    Google Scholar 

  17. Bovik AC (2010) Handbook of image and video processing. Academic press, Cambridge

    Google Scholar 

  18. Iddan G, Meron G, Glukhovsky A, Swain P (2000) Wireless capsule endoscopy. Nature 405:417. Nature Research

    Google Scholar 

  19. Van Gossum A, Munoz-Navas M, Fernandez-Urien I, Carretero C, Gay G, Delvaux M, Lapalus MG, Ponchon T, Neuhaus H, Philipper M, et al (2009) Capsule endoscopy versus colonoscopy for the detection of polyps and cancer. N Engl J Med 361:264–270. Mass Medical Soc

    Google Scholar 

  20. Hwang S (2011) Bag-of-visual-words approach to abnormal image detection in wireless capsule endoscopy videos. In: International symposium on visual computing. Springer, Berlin, pp 320–327

    Google Scholar 

  21. Yu M (2002) \(\text{M2A}^{\text{ TM }}\) capsule endoscopy: a breakthrough diagnostic tool for small intestine imaging. Gastroenterol Nurs 25:24–27. LWW

    Google Scholar 

  22. Fisher L, Krinsky ML, Anderson MA, Appalaneni V, Banerjee S, Ben-Menachem T, Cash BD, Decker GA, Fanelli RD, Friis C, et al (2010) The role of endoscopy in the management of obscure GI bleeding. Gastrointest Endosc 72:471–479. Elsevier

    Google Scholar 

  23. Fu Y, Zhang W, Mandal M, Meng MQ-H (2014) Computer-aided bleeding detection in WCE video. IEEE J Biomed Health Inf 18:636–642. IEEE

    Google Scholar 

  24. Mathew M, Gopi VP (2015) Transform based bleeding detection technique for endoscopic images. In: 2015 2nd international conference on electronics and communication systems (ICECS). IEEE, Piscataway, pp 1730–1734

    Google Scholar 

  25. Ghosh T, Bashar SK, Alam MS, Wahid K, Fattah SA (2014) A statistical feature based novel method to detect bleeding in wireless capsule endoscopy images. In: 2014 international conference on informatics, electronics & vision (ICIEV). IEEE, Dhaka, pp 1–4

    Google Scholar 

  26. Yuan Y, Meng MQ-H (2015) Automatic bleeding frame detection in the wireless capsule endoscopy images. In: 2015 IEEE international conference on robotics and automation (ICRA). IEEE, Seattle, pp 1310–1315

    Google Scholar 

  27. Yuan Y, Li B, Meng Q (2015) Bleeding frame and region detection in the wireless capsule endoscopy video. IEEE J Biomed Health Inf 20:624–630. IEEE

    Google Scholar 

  28. Cancer Facts & Figures (2019) American cancer society. Atlanta, GA, USA

    Google Scholar 

  29. Colorectal Cancer Facts & Figures 2017–2019 (2017) American cancer society, Atlanta, GA, USA

    Google Scholar 

  30. Silva J, Histace A, Romain O, Dray X, Granado B (2014) Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg 9:283–293. Springer

    Google Scholar 

  31. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F (2015) Wm-dova maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imaging Graph 43:99–111. Elsevier

    Google Scholar 

  32. Park SY, Sargent D (2016) Colonoscopic polyp detection using convolutional neural networks. In: Medical imaging 2016: computer-aided diagnosis. International society for optics and photonics, vol 9785, p 978528

    Google Scholar 

  33. Shin Y, Qadir HA, Aabakken L, Bergsland J, Balasingham I (2018) Automatic colon polyp detection using region based deep CNN and post learning approaches. IEEE Access 6:40950–40962. IEEE

    Google Scholar 

  34. Yu L, Chen H, Dou Q, Qin J, Heng PA (2017) Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J Biomed Health Inf 21:65–75. IEEE

    Google Scholar 

  35. Vázquez D, Bernal J, Sánchez FJ, Fernández-Esparrach G, López AM, Romero A, Drozdzal M, Courville A (2017) A benchmark for endoluminal scene segmentation of colonoscopy images. J Healthcare Eng 2017. Hindawi

    Google Scholar 

  36. Zhang L, Dolwani S, Ye X (2017) Automated polyp segmentation in colonoscopy frames using fully convolutional neural network and textons. In: Annual conference on medical image understanding and analysis. Springer, Berlin, pp 707–717

    Google Scholar 

  37. Brandao P, Zisimopoulos O, Mazomenos E, Ciuti G, Bernal J, Visentini-Scarzanella M, Menciassi A, Dario P, Koulaouzidis A, Arezzo A, et al (2018) Towards a computed-aided diagnosis system in colonoscopy: automatic polyp segmentation using convolution neural networks. J Med Robot Res 3:1840002. World Scientific

    Google Scholar 

  38. Chen L-C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587

  39. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 39. IEEE, pp 2481–2495

    Google Scholar 

  40. Xiao W-T, Chang L-J, Liu W-M (2018) Semantic segmentation of colorectal polyps with DeepLab and LSTM networks. In: 2018 IEEE international conference on consumer electronics-Taiwan (ICCE-TW). IEEE, pp 1–2

    Google Scholar 

  41. Wang P, Xiao X, Brown JRG, Berzin TM, Tu M, Xiong F, Hu X, Liu P, Song Y, Zhang D, et al (2018) Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nature Biomed Eng 2:741. Nature Publishing Group

    Google Scholar 

  42. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 770–778

    Google Scholar 

  43. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 1. IEEE, p 4

    Google Scholar 

  44. Bernal J, Tajkbaksh N, Sánchez FJ, Matuszewski BJ, Chen H, Yu L, Angermann Q, Romain O, Rustad B, Balasingham I, et al (2017) Comparative validation of polyp detection methods in video colonoscopy: results from the miccai 2015 endoscopic vision challenge. In: IEEE transactions on medical imaging, vol 36. IEEE, pp 1231–1249

    Google Scholar 

  45. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 2921–2929

    Google Scholar 

  46. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 2881–2890

    Google Scholar 

  47. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV). Springer, Berlin, pp 801–818

    Google Scholar 

  48. Zhou B, Li Y, Wang J (2018) A weakly supervised adaptive densenet for classifying thoracic diseases and identifying abnormalities. arXiv:1807.01257

  49. Donahue J, Anne Hendricks L, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K, Darrell T (2015) Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Boston, MA, pp 2625–2634

    Google Scholar 

  50. Yueming J, Qi D, Hao C, Yu L, Jing Q, Fu C-W, Pheng-Ann H (2017) Sv-rcnet: workflow recognition from surgical videos using recurrent convolutional network. IEEE Trans Med Imag. IEEE

    Google Scholar 

  51. Xing X, Yuan Y, Meng MQ-H (2020) Zoom in lesions for better diagnosis: attention guided deformation network for WCE image classification. IEEE Trans Med Imag

    Google Scholar 

  52. Guo X, Yuan Y (2020) Semi-supervised WCE image classification with adaptive aggregated attention. Med Image Anal 64:101733

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Key R&D program of China under Grant 2019YFB1312400, the Hong Kong Research Grants Council (RGC) Collaborative Research Fund (CRF) Project under Grant C4063-18G, and the Shenzhen Science and Technology Innovation Project under Grant JCYJ20170413161503220.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China

    Xiao Jia & Xiaohan Xing

  2. Department of Electrical Engineering, City University of Hong Kong, Hong Kong, China

    Yixuan Yuan

  3. Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China

    Max Q.-H Meng

Authors
  1. Xiao Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaohan Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yixuan Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Max Q.-H Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Jia .

Editor information

Editors and Affiliations

  1. Center for Artificial Intelligence, Prince Mohammad Bin Fahd University, Khobar, Saudi Arabia

    Tuan D. Pham

  2. Department of Electrical Engineering, City University of Hong Kong, Kowloon, Hong Kong

    Hong Yan

  3. College of Sciences and Human Studies, Prince Mohammad Bin Fahd University, Khobar, Saudi Arabia

    Muhammad W. Ashraf

  4. Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden

    Folke Sjöberg

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jia, X., Xing, X., Yuan, Y., Meng, M.QH. (2021). Deep Learning-Driven Models for Endoscopic Image Analysis. In: Pham, T.D., Yan, H., Ashraf, M.W., Sjöberg, F. (eds) Advances in Artificial Intelligence, Computation, and Data Science. Computational Biology, vol 31. Springer, Cham. https://doi.org/10.1007/978-3-030-69951-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69951-2_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69950-5

  • Online ISBN: 978-3-030-69951-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation