Skip to main content
SpringerLink
Log in

Classification of Helicobacter Pylori infection based on deep convolutional neural network with visual attention and self-supervised learning for endoscopic images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Helicobacter pylori (HP) is a major risk factor for chronic gastritis as well as gastroduenal disease and gastric endoscopic examination is the standard procedure to diagnose HP infection. A computer-aided diagnosis (CAD) scheme based on convolutional neural network (CNN) and self-supervised learning was proposed for classification of HP infection in this paper. The proposed scheme is composed of an encoder and a prediction head. The encoder composed of backbone network, visual attention module (VAM), and feature fusion module (FFM) is trained by using self-supervised contrastive learning for feature extraction. After obtaining the trained encoder, the whole network is trained by using a small image dataset with annotations. To evaluate the performance of the proposed scheme, some BLI (Blue Laser Imaging) images are collected for testing. According to experimental results of 5-fold cross validation, the average F1-score rates of the proposed scheme are from 0.885 to 0.915 for diagnosing HP infection. Therefore, the experimental results demonstrate that the proposed scheme can effectively extract visual representative features from BLI images for HP infection classification. Furthermore, the proposed scheme is superior to two existing methods in terms of accuracy and F1-score.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Azizi S, Mustafa B, Ryan F, Beaver Z, Freyberg J, Deaton J, Loh A, Karthikesalingam A, Kornblith S, Chen T, Natarajan V, Norouzi M (2021) Big Self-Supervised Models Advance Medical Image Classification. arXiv:2101.05224v2 [eess.IV]

  2. Bi Y, Min M, Zhang F, Li X (2019) The characteristics of blue laser imaging and the application in diagnosis of early digestive tract Cancer. Technol Cancer Res Treat 18:1–5

    Article  Google Scholar 

  3. Chen T, et al. (2020) A simple framework for contrastive learning of visual representations. International conference on machine learning. PMLR

  4. Gidaris S, Singh P, Komodakis N (2018) Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728

  5. Hassanin M, Anwar S, Radwan I, Khan FS, Mian A (2022) Visual Attention Methods in Deep Learning: An In-depth Survey. arXiv:2204.07756v2 [cs.CV]

  6. He K, et al. (2016) Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition

  7. He K, Fan H, Wu Y, Xie S, Girshick R (2020) Momentum Contrast for Unsupervised Visual Representation Learning. arXiv:1911.05722v3 [cs.CV]

  8. Hooi JKY, Lai WY, Ng WK, Suen MMY, Underwood FE, Tanyingoh D, Malfertheiner P, Graham DY, Wong VWS, Wu JCY, Chan FKL, Sung JJY, Kaplan GG, Ng SC (2017) Global prevalence of helicobacter pylori infection: systematic review and meta-analysis. Gastroenterology 153(2):420–429

    Article  Google Scholar 

  9. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. Proceedings of the IEEE conference on computer vision and pattern recognition

  10. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In CVPR

  11. Huang C-R, Sheu BS, Chung PC, Yang HB (2004) Computerized diagnosis of helicobacter pylori infection and associated gastric inflammation from endoscopic images by refined feature selection using a neural network. Endoscopy 36(7):601–608

    Article  Google Scholar 

  12. Huang C-R, Chung P-C, Sheu B-S, Kuo H-J, Popper M (2008) Helicobacter pylori-related gastric histology classification using support-vector-machine-based feature selection. IEEE Trans Inf Technol Biomed 12(4):523–531

    Article  Google Scholar 

  13. Itoh T et al. (2018) Deep learning analyzes Helicobacter pylori infection by upper gastrointestinal endoscopy images. Endoscopy international open. Vol. 6, No. 2

  14. Jetley S, Lord NA, Lee N, Torr PHS (2018) Learn to pay attention. Proceedings of International Conference on Learning Representations

  15. Jia S, Zhang Y (2018) Saliency-based deep convolutional neural network for no-reference image quality assessment. Multimed Tools Appl 77:14859–14872

    Article  Google Scholar 

  16. Jin Z, Gan T, Wang P, Fu Z, Zhang C, Yan Q, Zheng X, Liang X, Ye X (2022) Deep learning for gastroscopic images: computer-aided techniques for clinicians. Biomed Eng Online 21(1). https://doi.org/10.1186/s12938-022-00979-8

  17. Kaneko K, Oono Y, Yano T, Ikematsu H, Odagaki T, Yoda Y, Yagishita A, Sato A, Nomura S (2014) Effect of novel bright image enhanced endoscopy using blue laser imaging (BLI). Endoscop Int Open 2:E212–E219

    Article  Google Scholar 

  18. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst NIPS 2012(25):1097–1105

    Google Scholar 

  19. Li Z, Liu F, Yang W, Peng S, Zhou J (2021) A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems. pp. 1–21

  20. Lie W-N, C-K (2005) News video classification based on multi-modal information fusion, Proceedings of IEEE Conference on Image Processing

  21. Lin G-S, Chang J-F (2012) Detection of Frame Duplication Forgery in Videos Based on Spatial and Temporal Analysis. Int J Patt Recog Artifi Int 26(7):1250017

    Article  MathSciNet  Google Scholar 

  22. Lin G-S, Chang M-K, Chiu S-T (2009) A feature-based scheme for detecting and classifying video-shot transitions based on spatio-temporal analysis and fuzzy classification. Int J Pattern Recognit Artif Intell 23(6):1179–1200

    Article  Google Scholar 

  23. Lin G-S, Chang M-K, Chang Y-J, Yeh C-H (2016) A gender classification scheme based on multi-region feature extraction and information fusion for unconstrained images. Multimed Tools Appl 75(16):9775–9795

    Article  Google Scholar 

  24. Lin T, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature Pyramid Networks for Object Detection. arXiv:1612.03144

  25. Lin Y-W, Lin G-S, Daniel Chai S-K (2019) Helicobacter Pylori Classification based on Deep Neural Network. 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)

  26. Lin G-S , Chai S-K, Li H-M, Lin J-Y (2020) (2020) Vision-Based Patient Identification Recognition Based on Image Content Analysis and Support Vector Machine for Medical Information System. EURASIP Journal on Advances in Signal Processing. Vol. 2020

  27. Nakashima H, et al. (2018) Artificial intelligence diagnosis of Helicobacter pylori infection using blue laser imaging-bright and linked color imaging: a single-center prospective study Annals of gastroenterology. 31.4

  28. Noroozi M, Favaro P (2016) Unsupervised learning of visual representations by solving jigsaw puzzles. European conference on computer vision. Springer, Cham

  29. Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions. arXiv preprint arXiv:1710.05941

  30. Rodriguez JD, Perez A, Lozano JA (March 2010) Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans Pattern Anal Mach Intell 32(3):569–575

    Article  Google Scholar 

  31. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2019) Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. International Journal of Computer Vision (IJCV)

  32. Shichijo S et al. (2017) Application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images. EBioMedicine, Vol. 25. pp.106–111

  33. Shichijo S, Endo Y, Aoyama K, Takeuchi Y, Ozawa T, Takiyama H, Matsuo K, Fujishiro M, Ishihara S, Ishihara R, Tada T (2019) Application of convolutional neural networks for evaluating helicobacter pylori infection status on the basis of endoscopic images. Scand J Gastroenterol 54(2):158–163

    Article  Google Scholar 

  34. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  35. Tahara T, Takahama K, Horiguchi N, Yoshida D, Kawamura T, Okubo M, Ishizuka T, Nagasaka M, Nakagawa Y, Shibata T, Ohmiya N (2017) A comparative study of magnifying blue laser imaging and magnifying narrow-band imaging system for endoscopic diagnosis of helicobacter pylori infection. Biomed Rep 7(3):236–240

    Article  Google Scholar 

  36. Vondrick C, et al. (2018) Tracking emerges by colorizing videos. Proc Eur conf comput vis (ECCV).

  37. Woo S, Park J, Lee J-Y, Kweon IS (2018) CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV). pp. 3–190

  38. Yasuda T, Hiroyasu T, Hiwa S, Okada Y, Hayashi S, Nakahata Y, Yasuda Y, Omatsu T, Obora A, Kojima T, Ichikawa H, Yagi N (2020) Potential of automatic diagnosis system with linked color imaging for diagnosis of helicobacter pylori infection. Dig Endosc 32(3):373–381

    Article  Google Scholar 

  39. Zheng W et al. (2019) High accuracy of convolutional neural network for evaluation of Helicobacter pylori infection based on endoscopic images: preliminary experience. Clinical and translational gastroenterology

  40. Zhu Y et al. (2018) Attention couplenet: Fully convolutional attention coupling network for object detection. IEEE Transactions on Image Processing. pp. 113–126

Download references

Acknowledgements

Authors thank the editor and reviewers for their valuable suggestions. This research was partially supported by the National Science and Technology Council, Taiwan, under the grant of MOST 111-2221-E-167-006 -MY2 and by Chang Bing Show Chwan Memorial Hospital, Taiwan, under the grant of BRD-109009.

Data availability statement

Please contact authors for data requests.

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Chin-Yi University of Technology, Taichung City, Taiwan

    Guo-Zhang Jian, Guo-Shiang Lin & Chuin-Mu Wang

  2. Department of Internal Medicine Chang Bing Show Chwan Memorial Hospital, Changhua City, Taiwan

    Sheng-Lei Yan

Authors
  1. Guo-Zhang Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guo-Shiang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chuin-Mu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sheng-Lei Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guo-Shiang Lin.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jian, GZ., Lin, GS., Wang, CM. et al. Classification of Helicobacter Pylori infection based on deep convolutional neural network with visual attention and self-supervised learning for endoscopic images. Multimed Tools Appl 82, 37731–37754 (2023). https://doi.org/10.1007/s11042-023-14764-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14764-9

Keywords

Search

Navigation