Skip to main content
SpringerLink
Log in

Effective high-to-low-level feature aggregation network for endoscopic image classification

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

The accuracy improvement in endoscopic image classification matters to the endoscopists in diagnosing and choosing suitable treatment for patients. Existing CNN-based methods for endoscopic image classification tend to use the deepest abstract features without considering the contribution of low-level features, while the latter is of great significance in the actual diagnosis of intestinal diseases.

Methods

To make full use of both high-level and low-level features, we propose a novel two-stream network for endoscopic image classification. Specifically, the backbone stream is utilized to extract high-level features. In the fusion stream, low-level features are generated by a bottom-up multi-scale gradual integration (BMGI) method, and the input of BMGI is refined by top-down attention learning modules. Besides, a novel correction loss is proposed to clarify the relationship between high-level and low-level features.

Results

Experiments on the KVASIR dataset demonstrate that the proposed framework can obtain an overall classification accuracy of 97.33% with Kappa coefficient of 95.25%. Compared to the existing models, the two evaluation indicators have increased by 2% and 2.25%, respectively, at least.

Conclusion

In this study, we proposed a two-stream network that fuses the high-level and low-level features for endoscopic image classification. The experiment results show that the high-to-low-level feature can better represent the endoscopic image and enable our model to outperform several state-of-the-art classification approaches. In addition, the proposed correction loss could regularize the consistency between backbone stream and fusion stream. Thus, the fused feature can reduce the intra-class distances and make accurate label prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Favoriti P, Carbone G, Greco M, Pirozzi F, Pirozzi REM, Corcione F (2016) Worldwide burden of colorectal cancer: a review. Updat Surg 68(1):7–11

    Article  Google Scholar 

  2. Korbar B, Olofson AM, Miraflor AP, Nicka KM, Suriawinata MA Torresani L, Suriawinata AA, Hassanpour S (2017) Deep-learning for classifification of colorectal polyps on whole-slide images. J Pathol Inf 8(1):1–24

    Article  Google Scholar 

  3. Jia X, Xing X, Yuan Y, Xing L, Meng QH (2019) Wireless capsule endoscopy: a new tool for cancer screening in the colon with deep-learning-based polyp recognition. Proc IEEE PP(99):1–20

    Google Scholar 

  4. Karkanis SA, Iakovidis DK, Maroulis DE, Karras DA, Tzivras M (2003) Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans Inf Technol Biomed A Publ IEEE Eng Med Biol Soc 7(3):141

    Article  Google Scholar 

  5. Alexandre LA, Nobre N, Casteleiro J (2008) Color and position versus texture features for endoscopic polyp detection. In: International conference on biomedical engineering & informatics

  6. Yang J, Chang L, Li S et al (2020) WCE polyp detection based on novel feature descriptor with normalized variance locality-constrained linear coding. Int J Comput Assist Radiol Surg 15(8):1291–1302

    Article  Google Scholar 

  7. Bi L, Feng DDF, Fulham MJ, Kim J (2020) Multi-label classification of multi-modality skin lesion via hyper-connected convolutional neural network. Pattern Recogn 107:107502

    Article  Google Scholar 

  8. Ding X, Zhang X, Ma N, Han J, Ding G, Sun J (2021) Repvgg: making VGG-style convnets great again. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13 733–13 742

  9. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  10. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In IEEE conference on computer vision and pattern recognition (CVPR) 2016, pp 770–778

  11. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500

  12. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  13. Shin Y, Balasingham I (2017) Comparison of hand-craft feature based SVM and CNN based deep learning framework for automatic polyp classification. In: Engineering in medicine & biology society

  14. Guo X, Yuan Y (2019) Triple ANET: adaptive abnormal-aware attention network for WCE image classification. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 293–301

  15. Xing X, Yuan Y, Meng QH (2020) Zoom in lesions for better diagnosis: attention guided deformation network for WCE image classification. IEEE Trans Med Imaging PP(99):1

    Google Scholar 

  16. Guo X, Yuan Y (2020) Semi-supervised WCE image classification with adaptive aggregated attention. Med Image Anal 64:101733

    Article  Google Scholar 

  17. Poudel S, Kim YJ, Vo DM, Lee SW (2020) Colorectal disease classification using efficiently scaled dilation in convolutional neural network. IEEE Access PP(99):1

    Google Scholar 

  18. Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B, Glocker B, Rueckert D (2019) Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal 53:197–207

    Article  Google Scholar 

  19. Zhang J, Xie Y, Xia Y et al (2019) Attention residual learning for skin lesion classification. IEEE Trans Med Imaging 38(9):2092–2103

    Article  Google Scholar 

  20. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  21. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  22. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  23. Jia X, Mai X, Cui Y, Yuan Y, Xing X, Seo H, Xing L, Meng MQ (2020) Automatic polyp recognition in colonoscopy images using deep learning and two-stage pyramidal feature prediction. IEEE Trans Autom Sci Eng 17(3):1570–1584

    Google Scholar 

  24. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR)

  25. Kong T, Yao A, Chen Y, Sun F (2016) Hypernet: towards accurate region proposal generation and joint object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 845–853

  26. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 510–519

  27. Pogorelov K, Randel KR, Griwodz C, Eskeland SL, de Lange T, Johansen D, Spampinato C, Dang-Nguyen D-T, Lux M, Schmidt PT et al. (2017) Kvasir: a multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of the 8th ACM on multimedia systems conference, pp 164–169

  28. Alaskar H, Hussain A, Al-Aseem N et al (2019) Application of convolutional neural networks for automated ulcer detection in wireless capsule endoscopy images. Sensors 19(6):1265

    Article  Google Scholar 

  29. Sun G, Cholakkal H, Khan S, Khan F, Shao L (2020) Fine-grained recognition: accounting for subtle differences between similar classes. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, no 7, pp 12047–12054

  30. Klare P, Sander C, Prinzen M, Haller B, Nowack S, Abdelhafez M, Poszler A, Brown H, Wilhelm D, Schmid RM et al (2019) Automated polyp detection in the colorectum: a prospective study (with videos). Gastrointest Endosc 89(3):576–582

  31. Wang P, Berzin TM, Brown JRG, Bharadwaj S, Becq A, Xiao X, Liu P, Li L, Song Y, Zhang D et al (2019) Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 68(10):1813–1819

    Article  Google Scholar 

  32. Jorge Bernal, Fernando Vilarino, Gloria Fernandez-Esparrach, Debora Gil, Sanchez JF (2015) WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imaging Graph 43:99–111

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Science Foundation of P.R. China (Grants: 61873239), Key R& D Program Projects in Zhejiang Province (Grant: 2020C03074).

Author information

Authors and Affiliations

  1. College of Information Engineering, Zhejiang University of Technology, Hangzhou, 310023, Zhejiang, People’s Republic of China

    Sheng Li, Jiafeng Yao, Jing Cao & Xueting Kong

  2. The Second Affiliated Hospital of Hospital of Zhejiang University School of Medicine, Hangzhou, 310023, Zhejiang, People’s Republic of China

    Jinhui Zhu

Authors
  1. Sheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiafeng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xueting Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinhui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinhui Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, S., Yao, J., Cao, J. et al. Effective high-to-low-level feature aggregation network for endoscopic image classification. Int J CARS 17, 1225–1233 (2022). https://doi.org/10.1007/s11548-022-02591-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02591-6

Keywords

Search

Navigation