Skip to main content
SpringerLink
Log in

Deep Transfer Learning for Automated Intestinal Bleeding Detection in Capsule Endoscopy Imaging

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Purpose: The objective of this paper was to develop a computer-aided diagnostic (CAD) tools for automated analysis of capsule endoscopic (CE) images, more precisely, detect small intestinal abnormalities like bleeding. Methods: In particular, we explore a convolutional neural network (CNN)-based deep learning framework to identify bleeding and non-bleeding CE images, where a pre-trained AlexNet neural network is used to train a transfer learning CNN that carries out the identification. Moreover, bleeding zones in a bleeding-identified image are also delineated using deep learning-based semantic segmentation that leverages a SegNet deep neural network. Results: To evaluate the performance of the proposed framework, we carry out experiments on two publicly available clinical datasets and achieve a 98.49% and 88.39% F1 score, respectively, on the capsule endoscopy.org and KID datasets. For bleeding zone identification, 94.42% global accuracy and 90.69% weighted intersection over union (IoU) are achieved. Conclusion: Finally, our performance results are compared to other recently developed state-of-the-art methods, and consistent performance advances are demonstrated in terms of performance measures for bleeding image and bleeding zone detection. Relative to the present and established practice of manual inspection and annotation of CE images by a physician, our framework enables considerable annotation time and human labor savings in bleeding detection in CE images, while providing the additional benefits of bleeding zone delineation and increased detection accuracy. Moreover, the overall cost of CE enabled by our framework will also be much lower due to the reduction of manual labor, which can make CE affordable for a larger population.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. I.N. Figueiredo, S. Kumar, C. Leal, P.N. Figueiredo, Computer-assisted bleeding detection in wireless capsule endoscopy images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 1(4), 198 (2013) 

    Google Scholar 

  2. Dr efe’s medical art bank (2018). URL http://medicalartbank.com/medical-illustration-shop/gastrointestinal-git/capsule-endoscopy/

  3. D.G. Adler, C.J. Gostout, Wireless capsule endoscopy. Hospital Physician 39(5), 14 (2003)

    Google Scholar 

  4. S.C. Park, H.J. Chun, E.S. Kim, B. Keum, Y.S. Seo, Y.S. Kim, Y.T. Jeen, H.S. Lee, S.H. Um, C.D. Kim, et al. (2012) Sensitivity of the suspected blood indicator: an experimental study. World Journal of Gastroenterology: WJG 18(31), 4169 

    Article  Google Scholar 

  5. T. Ghosh, S.A. Fattah, S. Bashar, C. Shahnaz, K. Wahid, W.P. Zhu, M.O. Ahmad, An automatic bleeding detection technique in wireless capsule endoscopy from region of interest. in 2015 IEEE International Conference on Digital Signal Processing (DSP), pp. 1293–1297 (2015)

  6. S. Suman, F.A.B. Hussin, N. Walter, A.S. Malik, S.H. Ho, K.L. Goh, Detection and classification of bleeding using statistical color features for wireless capsule endoscopy images. In: International Conference on Signal and Information Processing (IConSIP) , pp. 1–5 (2016)

  7. A. Novozámskỳ, J. Flusser, I. Tachecí, L. Sulík, J. Bureš, O. Krejcar, Automatic blood detection in capsule endoscopy video. Journal of Biomedical Optics 21(12), 126007 (2016)

    Article  Google Scholar 

  8. T. Ghosh, S. Fattah, C. Shahnaz, A. Kundu, M. Rizve, Block based histogram feature extraction method for bleeding detection in wireless capsule endoscopy. in 2015 IEEE Region 10 Conference (TENCON) (2015), pp. 1–4

  9. E. Hu, H. Sakanashi, H. Nosato, E. Takahashi, Y. Suzuki, K. Takeuchi, H. Aoki, M. Murakawa, Bleeding and tumor detection for capsule endoscopy images using improved geometric feature. Journal of Medical and Biological Engineering 36(3), 344 (2016)

    Article  Google Scholar 

  10. T. Ghosh, S.A. Fattah, K.A. Wahid, Automatic computer aided bleeding detection scheme for wireless capsule endoscopy (wce) video based on higher and lower order statistical features in a composite color. Journal of Medical and Biological Engineering 38(3), 482 (2018)

    Article  Google Scholar 

  11. X. Jia, M.Q.H. Meng, A deep convolutional neural network for bleeding detection in wireless capsule endoscopy images. in 2016 IEEE 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC) (2016), pp. 639–642

  12. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell, Caffe: Convolutional architecture for fast feature embedding. in Proceedings of the 22nd ACM International Conference on Multimedia (2014), pp. 675–678

  13. X. Jia, M.Q.H. Meng, Gastrointestinal bleeding detection in wireless capsule endoscopy images using handcrafted and cnn features. in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2017), pp. 3154–3157

  14. P. Li, Z. Li, F. Gao, L. Wan, J. Yu, Convolutional neural networks for intestinal hemorrhage detection in wireless capsule endoscopy images. in 2017 IEEE International Conference on Multimedia and Expo (ICME) (2017), pp. 1518–1523

  15. G. Pan, G. Yan, X. Qiu, J. Cui, Bleeding detection in wireless capsule endoscopy based on probabilistic neural network. Journal of Medical Systems 35(6), 1477 (2011)

    Article  Google Scholar 

  16. S. Sainju, F.M. Bui, K.A. Wahid, Automated bleeding detection in capsule endoscopy videos using statistical features and region growing. Journal of Medical Systems 38(4), 25 (2014)

    Article  Google Scholar 

  17. Y. Yuan, B. Li, M.Q.H. Meng, Bleeding frame and region detection in the wireless capsule endoscopy video. IEEE Journal of Biomedical and Health Informatics 20(2), 624 (2016)

    Article  Google Scholar 

  18. Y. Fu, W. Zhang, M. Mandal, M.Q.H. Meng, Computer-aided bleeding detection in WCE video. IEEE Journal of Biomedical and Health Informatics 18(2), 636 (2014)

    Article  Google Scholar 

  19. T. Ghosh, S.A. Fattah, K.A. Wahid, W.P. Zhu, M.O. Ahmad, Cluster based statistical feature extraction method for automatic bleeding detection in wireless capsule endoscopy video. Computers in Biology and Medicine 94, 41 (2018)

    Article  Google Scholar 

  20. A.K. Kundu, S.A. Fattah, M.N. Rizve, An automatic bleeding frame and region detection scheme for wireless capsule endoscopy videos based on interplane intensity variation profile in normalized rgb color space. Journal of Healthcare Engineering 2018 (2018)

  21. T. Ghosh, S.A. Fattah, K.A. Wahid, CHOBS: Color histogram of block statistics for automatic bleeding detection in wireless capsule endoscopy video. IEEE Journal of Translational Engineering in Health and Medicine 6, 1 (2018)

    Article  CAS  Google Scholar 

  22. B. Li, M.Q.H. Meng, Computer-aided detection of bleeding regions for capsule endoscopy images. IEEE Transactions on Biomedical Engineering 56(4), 1032 (2009)

    Article  Google Scholar 

  23. X. Jia, M.Q.H. Meng, A study on automated segmentation of blood regions in wireless capsule endoscopy images using fully convolutional networks. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI) (2017), pp. 179–182

  24. T. Ghosh, L. Li, J. Chakareski, Effective deep learning for semantic segmentation based bleeding zone detection in capsule endoscopy images. in 2018 25th IEEE International Conference on Image Processing (ICIP) (IEEE, 2018), pp. 3034–3038

  25. Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Nature 521(7553), 436 (2015)

    Article  CAS  Google Scholar 

  26. H.C. Shin, H.R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, R.M. Summers, Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging 35(5), 1285 (2016)

    Article  Google Scholar 

  27. A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks. in Advances in Neural Information Processing Systems (2012), pp. 1097–1105

  28. J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database. in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 248–255

  29. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions. in Proceedings of the IEEE conference on computer vision and pattern recognition (2015), pp. 1–9

  30. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 770–778

  31. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  32. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al., Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115(3), 211 (2015)

    Article  Google Scholar 

  33. L. Perez, J. Wang, The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 (2017)

  34. V. Badrinarayanan, A. Kendall, R. Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(12), 2481 (2017)

    Article  Google Scholar 

  35. R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation. in Proceedings of the IEEE conference on computer vision and pattern recognition (2014), pp. 580–587

  36. S. Ren, K. He, R. Girshick, J. Sun, Faster r-cnn: Towards real-time object detection with region proposal networks. in Advances in neural information processing systems (2015), pp. 91–99

  37. The capsule endoscopy website (2014). URL https://www.capsuleendoscopy.org

  38. A. Koulaouzidis. Kid: Koulaouzidis-iakovidis database for capsule endoscopy (2015)

  39. D.K. Iakovidis, A. Koulaouzidis, Software for enhanced video capsule endoscopy: challenges for essential progress. Nature Reviews Gastroenterology & Hepatology 12(3), 172 (2015)

  40. G. Csurka, D. Larlus, F. Perronnin, F. Meylan, What is a good evaluation measure for semantic segmentation?. In Proceedings of British Machine Vision Conference (BMVC), vol. 27 (Citeseer, 2013), vol. 27, pp. 1–11

Download references

Funding

This work has been supported in part by NSF Awards CCF-1528030, ECCS-1711592, CNS-1836909, and CNS-1821875, and in part by NIH Award R01EY030470.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Alabama, Alabama, 35401, Tuscaloosa, USA

    Tonmoy Ghosh

  2. Department of Informatics, College of Computing , New Jersey Institute of Technology, Newark, 07103, New Jersey, USA

    Jacob Chakareski

Authors
  1. Tonmoy Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacob Chakareski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tonmoy Ghosh.

Ethics declarations

Conflicts of Interest

Authors declare that they have no conflict of interest.

Ethical Standard

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghosh, T., Chakareski, J. Deep Transfer Learning for Automated Intestinal Bleeding Detection in Capsule Endoscopy Imaging. J Digit Imaging 34, 404–417 (2021). https://doi.org/10.1007/s10278-021-00428-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00428-3

Keywords

Search

Navigation