Skip to main content
SpringerLink
Log in

Automated Colorectal Polyp Classification Using Deep Neural Networks with Colonoscopy Images

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

Colonoscopy screening test plays a crucial role in identifying and classifying possible cancerous polyps. The automation of polyp classification is challenging due to the limitations associated with the traditional handcrafted image features. This paper proposes a deep learning model for colorectal polyp classification, referred to as DeepCPC. This model consists of three main stages: image pre-processing using patch extraction and data augmentation; model initialization using six pre-trained CNNs to generate the fine-tuned baseline model; and formulating and learning generic yet discriminating image descriptors extracted and fused from the convolutional layers of two efficient CNNs architectures to classify colorectal polyps. The DeepCPC architecture has been fine-tuned on the CVC-Clinic dataset through a complete end-to-end training in which the patch extraction and image augmentation have been applied to generate more colonoscopy images for the patients. The experimental results show that the baseline fine-tuned model achieves an accuracy of 97.6%, and the final DeepCPC model achieves an accuracy of 98.4%. The reported results also demonstrate the capability of the proposed approach in identifying polyps in terms of precision, recall, and f-score. The DeepCPC helps the endoscopic physicians in classifying polyps and decreasing the colorectal polyp miss rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Siegel, R.L., Miller, K.D., Goding Sauer, A., Fedewa, S.A., Butterly, L.F., Anderson, J.C., et al.: Colorectal cancer statistics. CA 70(3), 145–164 (2020)

    Google Scholar 

  2. Nair, M., Peate, I.: Pathophysiology for Nurses at a Glance. Wiley, New York (2015)

    Google Scholar 

  3. Groff, R.J., Nash, R., Ahnen, D.J.: Significance of serrated polyps of the colon. Curr. Gastroenterol. Rep. 10(5), 490–498 (2008)

    Article  Google Scholar 

  4. Dekker, E., Rex, D.K.: Advances in CRC prevention: screening and surveillance. Gastroenterology 154(7), 1970–1984 (2018)

    Article  Google Scholar 

  5. Arnold, M., Sierra, M.S., Laversanne, M., Soerjomataram, I., Jemal, A., Bray, F.: Global patterns and trends in colorectal cancer incidence and mortality. Gut 66(4), 683–691 (2017)

    Article  Google Scholar 

  6. Imperiale, T.F., Ransohoff, D.F., Itzkowitz, S.H., Levin, T.R., Lavin, P., Lidgard, G.P., et al.: Multitarget stool DNA testing for colorectal-cancer screening. N. Engl. J. Med. 370(14), 1287–1297 (2014)

    Article  Google Scholar 

  7. Barancin, C., Pickhardt, P.J., Kim, D.H., Spier, B., Lindstrom, M., Reichelderfer, M., et al.: Prospective blinded comparison of polyp size on computed tomography colonography and endoscopic colonoscopy. Clin. Gastroenterol. Hepatol. 9(5), 443–445 (2011)

    Article  Google Scholar 

  8. Mir, A., Nguyen, V.Q., Soliman, Y., Sorrentino, D.: Wireless capsule endoscopy for diagnosis and management of post-operative recurrence of crohn’s disease. Life 11(7), 602 (2021)

    Article  Google Scholar 

  9. Zauber, A.G., Winawer, S.J., O’Brien, M.J., Lansdorp-Vogelaar, I., van Ballegooijen, M., Hankey, B.F., et al.: Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N. Engl. J. Med. 366, 687–696 (2012)

    Article  Google Scholar 

  10. Lieberman, D.A., Rex, D.K., Winawer, S.J., Giardiello, F.M., Johnson, D.A., Levin, T.R.: Guidelines for colonoscopy surveillance after screening and polypectomy: a consensus update by the US Multi-Society Task Force on Colorectal Cancer. Gastroenterology 143(3), 844–857 (2012)

    Article  Google Scholar 

  11. Rex, D.K., Cutler, C.S., Lemmel, G.T., Rahmani, E.Y., Clark, D.W., Helper, D.J., et al.: Colonoscopic miss rates of adenomas determined by back-to-back colonoscopies. Gastroenterology 112(1), 24–28 (1997)

    Article  Google Scholar 

  12. Kahi, C.J.: How does the serrated polyp pathway alter CRC screening and surveillance? Dig. Dis. Sci. 60(3), 773–780 (2015)

    Article  Google Scholar 

  13. Tholoor, S., Tsagkournis, O., Basford, P., Bhandari, P.: Managing difficult polyps: techniques and pitfalls. Ann. Gastroenterol. 26(2), 114 (2013)

    Google Scholar 

  14. Irshad, H., Veillard, A., Roux, L., Racoceanu, D.: Methods for nuclei detection, segmentation, and classification in digital histopathology: a review—current status and future potential. IEEE Rev. Biomed. Eng. 7, 97–114 (2013)

    Article  Google Scholar 

  15. Horváth, A., Spindler, S., Szalay, M., Rácz, I.: Preprocessing endoscopic images of colorectal polyps. Acta Technica Jaurinensis 9(1), 65–82 (2016)

    Article  Google Scholar 

  16. Srivastava, A.N., Han, J. (eds.): Machine Learning and Knowledge Discovery for Engineering Systems Health Management. CRC Press, Boca Raton (2016)

    Google Scholar 

  17. Alzu’bi, A., Abuarqoub, A.: Deep learning model with low-dimensional random projection for large-scale image search. Eng. Sci. Technol. Int. J. 23(4), 911–920 (2020)

    Google Scholar 

  18. Jerebko, A.K., Malley, J.D., Franaszek, M., Summers, R.M.: Support vector machines committee classification method for computer-aided polyp detection in CT colonography1. Acad. Radiol. 12(4), 479–486 (2005)

    Article  Google Scholar 

  19. Summers, R.M., Yao, J., Pickhardt, P.J., Franaszek, M., Bitter, I., Brickman, D., et al.: Computed tomographic virtual colonoscopy computer-aided polyp detection in a screening population. Gastroenterology 129(6), 1832–1844 (2005)

    Article  Google Scholar 

  20. Mori, Y., Kudo, S.E., Berzin, T.M., Misawa, M., Takeda, K.: Computer-aided diagnosis for colonoscopy. Endoscopy 49(08), 813–819 (2017)

    Article  Google Scholar 

  21. Ameling, S., Wirth, S., Paulus, D., Lacey, G., Vilarino, F.: Texture-based polyp detection in colonoscopy. In: Bildverarbeitung für die Medizin, pp. 346–350. Springer, Berlin (2009)

  22. Karkanis, S.A., Iakovidis, D.K., Maroulis, D.E., Karras, D.A., Tzivras, M.: Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans. Inf. Technol. Biomed. 7(3), 141–152 (2003)

    Article  Google Scholar 

  23. Pogorelov, K., Suman, S., Azmadi Hussin, F., Saeed Malik, A., Ostroukhova, O., Riegler, M., et al.: Bleeding detection in wireless capsule endoscopy videos—color versus texture features. J. Appl. Clin. Med. Phys. 20(8), 141–154 (2019)

    Article  Google Scholar 

  24. Nogueira-Rodríguez, A., Domínguez-Carbajales, R., López-Fernández, H., Iglesias, Á., Cubiella, J., Fdez-Riverola, F., et al.: Deep neural networks approaches for detecting and classifying colorectal polyps. Neurocomputing 423, 721–734 (2021)

    Article  Google Scholar 

  25. Obuch, J.C., Pigott, C.M., Ahnen, D.J.: Sessile serrated polyps: detection, eradication, and prevention of the evil twin. Curr. Treat. Options Gastroenterol. 13(1), 156–170 (2015)

    Article  Google Scholar 

  26. Mesejo, P., Pizarro, D., Abergel, A., Rouquette, O., Beorchia, S., Poincloux, L., Bartoli, A.: Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE Trans. Med. Imaging 35(9), 2051–2063 (2016)

    Article  Google Scholar 

  27. Tamaki, T., Yoshimuta, J., Kawakami, M., Raytchev, B., Kaneda, K., Yoshida, S., et al.: Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med. Image Anal. 17(1), 78–100 (2013)

    Article  Google Scholar 

  28. Stehle, T., Auer, R., Gross, S., Behrens, A., Wulff, J., Aach, T., et al.: Classification of colon polyps in NBI endoscopy using vascularization features. In: Medical Imaging 2009: Computer-Aided Diagnosis, International Society for Optics and Photonics, 7260, 72602S (2009)

  29. Li, P., Chan, K.L., Krishnan, S.M., Gao, Y.: Detecting abnormal regions in colonoscopic images by patch-based classifier ensemble. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 3, pp. 774–777 (2004)

  30. Manivannan, S., Wang, R., Trucco, E., Hood, A.: Automatic normal-abnormal video frame classification for colonoscopy. In: 2013 IEEE 10th International Symposium on Biomedical Imaging, pp. 644–647 (2013)

  31. Hilsden, R.J., Heitman, S.J., Mizrahi, B., Narod, S.A., Goshen, R.: Prediction of findings at screening colonoscopy using a machine learning algorithm based on complete blood counts (ColonFlag). PLoS ONE 13(11), e0207848 (2018)

    Article  Google Scholar 

  32. Ribeiro, E., Uhl, A., Häfner, M.: Colonic polyp classification with convolutional neural networks. In: 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS), pp. 253–258 (2016)

  33. Byrne, M.F., Chapados, N., Soudan, F., Oertel, C., Pérez, M.L., Kelly, R., et al.: Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 68(1), 94–100 (2019)

    Article  Google Scholar 

  34. Tian, Y., Pu, L. Z., Singh, R., Burt, A. D., Carneiro, G.: One-stage five-class polyp detection and classification. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 70–73 (2019)

  35. Komeda, Y., Handa, H., Matsui, R., Kashida, H., Watanabe, T., Sakurai, T., Kudo, M.: Computer-aided diagnosis (CAD) based on convolutional neural network (CNN) system using artificial intelligence (AI) for colorectal polyp classification. Endoscopy 51(04), OP2 (2019)

    Article  Google Scholar 

  36. Wang, L., Chen, R., Wang, S., Zeng, N., Huang, X., Liu, C.: Nested dilation network (NDN) for multi-task medical image segmentation. IEEE Access 7, 44676–44685 (2019)

    Article  Google Scholar 

  37. Wimmer, G., Vécsei, A., Uhl, A.: CNN transfer learning for the automated diagnosis of celiac disease. In: 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1–6 (2016)

  38. Urban, G., Tripathi, P., Alkayali, T., Mittal, M., Jalali, F., Karnes, W., Baldi, P.: Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 155(4), 1069–1078 (2018)

    Article  Google Scholar 

  39. Patino-Barrientos, S., Sierra-Sosa, D., Garcia-Zapirain, B., Castillo-Olea, C., Elmaghraby, A.: Kudo’s classification for colon polyps assessment using a deep learning approach. Appl. Sci. 10(2), 501 (2020)

    Article  Google Scholar 

  40. Patel, K., Li, K., Tao, K., Wang, Q., Bansal, A., Rastogi, A., Wang, G.: A comparative study on polyp classification using convolutional neural networks. PLoS ONE 15(7), e0236452 (2020)

    Article  Google Scholar 

  41. Alzu’bi, A., Amira, A., Ramzan, N.: Learning transfer using deep convolutional features for remote sensing image retrieval. Int. J. Comput. Sci. 46(4), 637–644 (2019)

    Google Scholar 

  42. Weiss, K., Khoshgoftaar, T.M., Wang, D.: A survey of transfer learning. J. Big data 3(1), 1–40 (2016)

    Article  Google Scholar 

  43. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

  44. Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., Gil, D., Rodríguez, C., Vilariño, F.: WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput. Med. Imaging Graphics 43, 99–111 (2015)

    Article  Google Scholar 

  45. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  46. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:abs/1409.1556 (2014)

  47. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  48. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

  49. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

  50. Bisong, E.: Building machine learning and deep learning models on Google cloud platform: a comprehensive guide for beginners. Apress. 7–10 (2019)

  51. Huang, Y., Gong, W., Su, B., Zhi, F., Liu, S., Jiang, B.: Risk and cause of interval colorectal cancer after colonoscopic polypectomy. Digestion 86(2), 148–154

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Middle East University, Amman, 11831, Jordan

    Dima Taha

  2. Department of Computer Science, Jordan University of Science and Technology, Irbid, 22110, Jordan

    Ahmad Alzu’bi

  3. Cardiff School of Technologies, Cardiff Metropolitan University, Cardiff, Wales, UK

    Abdelrahman Abuarqoub

  4. Department of Computing and Mathematics, Manchester Metropolitan University, Manchester, UK

    Mohammad Hammoudeh

  5. College of Computer Information Technology, American University in the Emirates, Dubai, United Arab Emirates

    Mohamed Elhoseny

  6. Faculty of Computers and Information, Mansoura University, Mansoura, Egypt

    Mohamed Elhoseny

Authors
  1. Dima Taha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmad Alzu’bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdelrahman Abuarqoub
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammad Hammoudeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohamed Elhoseny
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmad Alzu’bi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Taha, D., Alzu’bi, A., Abuarqoub, A. et al. Automated Colorectal Polyp Classification Using Deep Neural Networks with Colonoscopy Images. Int. J. Fuzzy Syst. 24, 2525–2537 (2022). https://doi.org/10.1007/s40815-021-01182-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-021-01182-y

Keywords

Search

Navigation