Skip to main content
SpringerLink
Log in

A deep representation to fully characterize hyperplastic, adenoma, and serrated polyps on narrow band imaging sequences

  • Original Paper
  • Published:
Health and Technology Aims and scope Submit manuscript

Abstract

An effective and early polypectomy of adenomatous and serrated polyps may dramatically reduce the incidence of colorectal cancer. Recently, Narrow-band imaging (NBI) sequences have emerged to enhance the description of these polyp types from the description of their microvascular and surface textural patterns. Despite the major observation capabilities, the in-situ analysis from the colonoscopy procedure remains challenging due to dependence on the expertise of gastroenterologists to localize and characterize polyps. This work introduces a robust frame-level strategy that achieves a full characterization of polyp patterns to differentiate among serrated, adenoma, and hyperplastic samples. The proposed strategy learns a deep convolutional representation that supports classification but also retrieves attention maps to localize main regions associated with the lesion. From this deep representation, it was also possible to build a low-dimensional representation space that allows visualizing a particular frame-video sample w.r.t to other diagnosed samples. From a total of 76 public available colonoscopies, the proposed strategy achieves an average classification accuracy of 90.79%. Besides, the proposed approach achieves a remarkable classification of polyps to be resected and also the masses diagnosed as serrated, a task with major diagnostic variability among experts.

Graphic Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://www.image-net.org/

References

  1. Hyuna S, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71(3):209–49.

    Article  Google Scholar 

  2. Ortega-Morán JF, et al. Medical needs related to the endoscopic technology and colonoscopy for colorectal cancer diagnosis. BMC Cancer. 2021;21(1):1–12.

    Article  Google Scholar 

  3. Kudo S, et al. Colorectal tumours and pit pattern. J Clin Pathol. 1994;47(10):880–5.

    Article  Google Scholar 

  4. El Hajjar A, Jean-François R. Artificial intelligence in gastrointestinal endoscopy: general overview. Chinese Med J. 2020;133(3):326.

  5. Van D, Sascha C, et al. Polyp morphology: an interobserver evaluation for the Paris classification among international experts. Official J Am College of Gastroenterology ACG. 2015;110(1):180–7.

    Google Scholar 

  6. Stehle T, et al. Classification of colon polyps in NBI endoscopy using vascularization features. Medical Imaging 2009: Computer-Aided Diagnosis. Vol. 7260. International Society for Optics and Photonics, 2009.

  7. Mesejo P, et al. Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE Trans Med Imaging. 2016;35(9):2051–63.

    Article  Google Scholar 

  8. Rogart JN, et al. Narrow-band imaging without high magnification to differentiate polyps during real-time colonoscopy: improvement with experience. Gastrointest Endosc. 2008;68(6):1136–45.

    Article  Google Scholar 

  9. Viovan II, et al. The role of narrow band imaging in colorectal polyp detection. Bosn J Basic Med Sci. 2017;17(2):152.

    Google Scholar 

  10. Kazuhiro G, et al. Appearance of enhanced tissue features in narrow-band endoscopic imaging. J Biomed Opt. 2004;9(3):568–77.

    Article  Google Scholar 

  11. Yoko K, et al. Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointest Endosc. 2016;83(3):643–9.

    Article  Google Scholar 

  12. Koichi O, et al. Clinicopathological characteristics of serrated polyps as precursors to colorectal cancer: current status and management. J Gastroenterol Hepatol. 2017;32(2):358–67.

    Article  Google Scholar 

  13. Yuichi M, et al. Computer-aided diagnosis for colonoscopy. Endoscopy. 2017;49(08):813–9.

    Article  Google Scholar 

  14. Yusuke H, et al. Convolutional neural network for differentiating gastric cancer from gastritis using magnified endoscopy with narrow band imaging. Dig Dis Sci. 2020;65(5):1355–63.

    Article  Google Scholar 

  15. Tischendorf JJW, et al. Computer-aided classification of colorectal polyps based on vascular patterns: a pilot study. Endoscopy. 2010;42(03):203–7.

    Article  Google Scholar 

  16. Sebastian G, et al. Computer-based classification of small colorectal polyps by using narrow-band imaging with optical magnification. Gastrointest Endosc. 2011;74(6):1354–9.

    Article  Google Scholar 

  17. Lan L, et al. Convolutional neural network for the diagnosis of early gastric cancer based on magnifying narrow band imaging. Gastric Cancer. 2020;23(1):126–32.

    Article  Google Scholar 

  18. Hiroya U, et al. Application of artificial intelligence using a convolutional neural network for diagnosis of early gastric cancer based on magnifying endoscopy with narrow-band imaging. J Gastroenterol Hepatol. 2021;36(2):482–9.

    Article  Google Scholar 

  19. Peng-Jen C, et al. Accurate classification of diminutive colorectal polyps using computer-aided analysis. Gastroenterology. 2018;154(3):568–75.

    Article  Google Scholar 

  20. Henrik T, et al. Serrated polyps-a concealed but prevalent precursor of colorectal cancer. Scand J Gastroenterol. 2017;52(6–7):654–61.

    Article  Google Scholar 

  21. Ruikai Z, et al. Automatic detection and classification of colorectal polyps by transferring low-level CNN features from nonmedical domain. IEEE J Biomed Health Inform. 2016;21(1):41–7.

    Google Scholar 

  22. Ribeiro E, Andreas U, Michael H. Colonic polyp classification with convolutional neural networks. 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS). IEEE, 2016.

  23. Haumaier F, Sterlacci W, Vieth M. Histological and molecular classification of gastrointestinal polyps. Best Pract Res Clin Gastroenterol. 2017;31(4):369–79.

    Article  Google Scholar 

  24. Gregor U, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology. 2018;155(4):1069–78.

    Article  Google Scholar 

  25. Kim DH, et al. Serrated polyps at CT colonography: prevalence and characteristics of the serrated polyp spectrum. Radiology. 2016;280(2):455–63.

    Article  Google Scholar 

  26. Longacre TA, Cecilia MFP. Mixed hyperplastic adenomatous polyps/serrated adenomas A distinct form of colorectal neoplasia. Am J Surg Pathol. 1990;14(6):524–37.

    Article  Google Scholar 

  27. Van der ML, Geoffrey H. Visualizing data using t-SNE. J Mach Learn Res. 2008;9(11).

  28. Selvaraju RR, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE international conference on computer vision. 2017.

  29. Simonyan K, Andrew Z. Very deep convolutional networks for large-scale image recognition. 2014. https://arxiv.org/abs/1409.1556.

  30. Howard AG, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. 2017.https://arxiv.org/abs/1704.04861.

  31. Chollet F. keras. 2015.

  32. Abadi M, et al. Tensorflow: A system for large-scale machine learning. 12th USENIX symposium on operating systems design and implementation (OSDI 16). 2016.

  33. Hiroyasu U, et al. Colorectal Polyp Classification Based On Latent Sharing Features Domain from Multiple Endoscopy Images. Procedia Computer Science. 2020;176:2507–14.

    Article  Google Scholar 

  34. Robin Z, et al. Prediction of Polyp Pathology Using Convolutional Neural Networks Achieves Resect and Discard Thresholds. Am J Gastroenterol. 2020;115(1):138.

    Article  MathSciNet  Google Scholar 

  35. Jin EH, et al. Improved accuracy in optical diagnosis of colorectal polyps using convolutional neural networks with visual explanations. Gastroenterology. 2020;158(8):2169–79.

    Article  Google Scholar 

  36. Pu LZCT, et al. Computer-aided diagnosis for characterization of colorectal lesions: comprehensive software that includes differentiation of serrated lesions. Gastrointest Endosc. 2020;92(4):891–9.

    Article  Google Scholar 

  37. Eladio RD, et al. Real-time artificial intelligence-based histologic classification of colorectal polyps with augmented visualization. Gastrointest Endosc. 2021;93(3):662–70.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Biomedical Imaging, Vision and Learning Laboratory (BivL2ab), Universidad Industrial de Santander (UIS), Cra. 27-Cl. 9, Bucaramanga, Colombia

    Franklin Sierra-Jerez & Fabio Martínez

Authors
  1. Franklin Sierra-Jerez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabio Martínez.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sierra-Jerez, F., Martínez, F. A deep representation to fully characterize hyperplastic, adenoma, and serrated polyps on narrow band imaging sequences. Health Technol. 12, 401–413 (2022). https://doi.org/10.1007/s12553-021-00633-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12553-021-00633-8

Keywords

Search

Navigation