Skip to main content
SpringerLink
Log in

Automatic Detection of Colorectal Polyps with Mixed Convolutions and its Occlusion Testing

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Manual detection of colorectal polyps in the colonoscopy videos during diagnosis of colorectal cancer can be challenging due to the large no. of sequential frames and high chances of false positive rate. So, researchers are working to develop an automatic computer-aided decision system to successfully detect colorectal polyps for early diagnosis of colorectal cancer. This work proposes an end-to-end, automatic colorectal polyp detection architecture called ‘Window-based Detection after Mixed Convolutions Polyp Identification (WD-MCPI)’. It has been trained and validated on the Etis-Larib, CVC-Colon, and Kvasir v1 data. Varying sequential and non-sequential colonoscopy frames and developed occlusive frames have been considered in the test set. Systematic hyperparameter tuning of various convolutional layers, optimizers, kernel size, color space, image dimension, and filter size has been performed to build the proposed architecture. The robustness and explainability of the proposed architecture have been estimated through various evaluation metrics, feature mapping, ablation studies, occlusion testing, and class activation mapping. The proposed work has achieved an overall accuracy, precision, recall, specificity, F1 score, and area-under-curve score up to 94.23, 91.16, 94.00, 92.67, 91.75, and 92.53%, respectively. An average test set accuracy up to 93% has been achieved on a newly developed test set named ‘Gastrointestinal atlas-Colon Polyp’ (available here). Upon comparison with the existing state-of-the-art works and vanilla inception v3 architecture, the proposed architecture has been tested on varying colonoscopy frames, achieving optimum results with reduced trainable parameters, and inference time per epoch.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The datasets used in this work are freely available online. Refer link.

Code availability

Available on request from the corresponding author.

References

  1. Shah D, Makharia GK, Ghoshal UC, Varma S, Ahuja V, Hutfless S (2018) Burden of gastrointestinal and liver diseases in india, 1990–2016. Indian J Gastroenterol 37(5):439–445

    Article  Google Scholar 

  2. Mohandas KM (2011) Colorectal cancer in India: controversies, enigmas and primary prevention. Indian J Gastroenterol 30(1):3–6

    Article  MathSciNet  Google Scholar 

  3. Stryker SJ, Wolff BG, Culp CE, Libbe SD, Ilstrup DM, MacCarty RL (1987) Natural history of untreated colonic polyps. Gastroenterology 93(5):1009–1013

    Article  Google Scholar 

  4. Oh J, Hwang S, Cao Y, Tavanapong W, Liu D, Wong J, De Groen PC (2008) Measuring objective quality of colonoscopy. IEEE Trans Biomed Eng 56(9):2190–2196

    Article  Google Scholar 

  5. Chan MY, Cohen H, Spiegel BM (2009) Fewer polyps detected by colonoscopy as the day progresses at a veteran’s administration teaching hospital. Clin Gastroenterol Hepatol 7(11):1217–1223

    Article  Google Scholar 

  6. Goel N, Kaur S, Gunjan D, Mahapatra S (2022) Investigating the significance of color space for abnormality detection in wireless capsule endoscopy images. Biomed Signal Process Control 75:103624

    Article  Google Scholar 

  7. Nogueira-Rodriguez A, Dominguez-Carbajales R, Lopez-Fernandez H, Igle-sias A, Cubiella J, Fdez-Riverola F, Reboiro-Jato M, Glez-Pena D (2021) Deep neural networks approaches for detecting and classifying colorectal polyps. Neurocomputing 423:721–734

    Article  Google Scholar 

  8. Goel N, Kaur S, Gunjan D, Mahapatra S (2022) Dilated cnn for abnormality detection in wireless capsule endoscopy images. Soft Comput 26(3):1231–1247

    Article  Google Scholar 

  9. Ikematsu H, Murano T, Shinmura K (2022) Detection of colorectal lesions during colonoscopy. DEN open 2(1):68

    Article  Google Scholar 

  10. Houwen BB, Nass KJ, Vleugels JL, Fockens P, Hazewinkel Y, Dekker E (2022) Comprehensive review of publicly available colonoscopic imaging databases for artificial intelligence research: availability, accessibility and usability. Gastrointest Endosc 97(2):184–199

    Article  Google Scholar 

  11. Wickstrøm K, Kampffmeyer M, Jenssen R (2020) Uncertainty and interpretability in convolutional neural networks for semantic segmentation of colorectal polyps. Med Image Anal 60:101619

    Article  Google Scholar 

  12. Hicks S, Storas A, Riegler M, Midoglu C, Hammou M, Lange T, Parasa S, Halvorsen P, Strumke I (2022) Visual explanations for polyp detection: How medical doctors assess intrinsic versus extrinsic explanations. arXiv preprint arXiv:2204.00617

  13. Chen J, Li K, Rong H, Bilal K, Yang N, Li K (2018) A disease diagnosis and treatment recommendation system based on big data mining and cloud computing. Inf Sci 435:124–149

    Article  Google Scholar 

  14. Chen J, Yang N, Zhou M, Zhang Z, Yang X (2022) A configurable deep learning framework for medical image analysis. Neural Comput Appl 34(10):7375–7392

    Article  Google Scholar 

  15. Zhao L, Li N, Tan G, Chen J, Li S, Duan M (2022) The end-to-end fetal head circumference detection and estimation in ultrasound images. IEEE/ACM Trans Comput Biol Bioinf. https://doi.org/10.1109/TCBB.2022.3227037

    Article  Google Scholar 

  16. Jie L, Liang P, Zhao Z, Chen J, Chang Q, Zeng Z (2022) Adan: an adversarial domain adaptation neural network for early gastric cancer prediction. In: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 2169–2172 (2022)

  17. Handa P, Goel N, Indu S (2022) Automatic intestinal content classification using transfer learning architectures. In: 2022 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT), pp. 1–5

  18. ELKarazle K, Raman V, Then P, Chua C (2023) Detection of colorectal polyps from colonoscopy using machine learning: A survey on modern techniques. Sensors 23(3):1225

    Article  Google Scholar 

  19. Shah S, Park N, Chehade NEH, Chahine A, Monachese M, Tiritilli A, Moosvi Z, Ortizo R, Samarasena J (2023) Effect of computer-aided colonoscopy on adenoma miss rates and polyp detection: a systematic review and meta-analysis. J Gastroenterol Hepatol 38(2):162–176

    Article  Google Scholar 

  20. Hsu C-M, Hsu C-C, Hsu Z-M, Shih F-Y, Chang M-L, Chen T-H (2021) Colorectal polyp image detection and classification through grayscale images and deep learning. Sensors 21(18):5995

    Article  Google Scholar 

  21. Akbari M, Mohrekesh M, Rafiei S, Soroushmehr SR, Karimi N, Samavi S, Najarian K (2018) Classification of informative frames in colonoscopy videos using convolutional neural networks with binarized weights. In: 2018 40th Annual Inter- national Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 65–68

  22. Rahim T, Hassan SA, Shin SY (2021) A deep convolutional neural network for the detection of polyps in colonoscopy images. Biomed Signal Process Control 68:102654

    Article  Google Scholar 

  23. Gong R, He S, Tian T, Chen J, Hao Y, Qiao C (2023) FRCNN-AA-CIF: An automatic detection model of colon polyps based on attention awareness and context information fusion. Comput Biol Med 158:106787

    Article  Google Scholar 

  24. Kara OC, Venkatayogi N, Ikoma N, Alambeigi F (2023) A reliable and sensitive framework for simultaneous type and stage detection of colorectal cancer polyps. Annals Biomed Eng 51(7):1–14

    Article  Google Scholar 

  25. Chino A, Ide D, Abe S, Yoshinaga S, Ichimasa K, Kudo T, Ninomiya Y, Oka S, Tanaka S, Igarashi M (2023) Performance evaluation of a computer- aided polyp detection system with artificial intelligence for colonoscopy. Dig Endosc. https://doi.org/10.1111/den.14578

    Article  Google Scholar 

  26. Younas F, Usman M, Yan WQ (2023) A deep ensemble learning method for colorectal polyp classification with optimized network parameters. Appl Intell 53(2):2410–2433

    Article  Google Scholar 

  27. Mangotra H, Goel N et al (2023) Effect of selection bias on automatic colonoscopy polyp detection. Biomed Signal Process Control 85:104915

    Article  Google Scholar 

  28. Wang S, Yin Y, Wang D, Lv Z, Wang Y, Jin Y (2021) An interpretable deep neu- ral network for colorectal polyp diagnosis under colonoscopy. Knowl Based Syst 234:107568

    Article  Google Scholar 

  29. Kliegis L, Obst W, Bruns J, Weigt J (2022) Can a polyp detection and characterization system predict complete resection? Dig Dis 40(1):115–118

    Article  Google Scholar 

  30. Kaiser L, Gomez AN, Chollet F (2017) Depthwise separable convolutions for neural machine translation. arXiv preprint arXiv:1706.03059

  31. Dumoulin V, Visin F (2016) A guide to convolution arithmetic for deep learning. arXiv preprint arXiv:1603.07285

  32. Kavukcuoglu K, Sermanet P, Boureau Y-L, Gregor K, Mathieu M, Cun Y et al. (2010) Learning convolutional feature hierarchies for visual recognition. Advances in neural information processing

  33. Agarap AF (2018) Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375

  34. Basta N (2022) The Differences between Sigmoid and Softmax Activation Func- tions, Medium. https://medium.com/arteos-ai/the-differences-between-sigmoid-and-softmax-acti\protect\discretionary{\char\hyphenchar\font}{}{}vation-function-12adee8cf322. Accessed 21 Sept 2022

  35. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion. In: Proceedings of the IEEE International Conference on Computer Vision, pp 618–626

  36. Lafeuille P, Lambin T, Yzet C, Latif EH, Ordoqui N, Rivory J, Pioche M (2022) Flat colorectal sessile serrated polyp: an example of what artificial intelligence does not easily detect. Endoscopy 54(05):520–521

    Article  Google Scholar 

  37. Yu T, Lin N, Zhong X, Zhang X, Zhang X, Chen Y, Liu J, Hu W, Duan H, Si J (2022) Multi-label recognition of cancer-related lesions with clinical priors on white-light endoscopy. Comput Biol Med 143:105255

    Article  Google Scholar 

  38. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826

  39. Patel K, Li K, Tao K, Wang Q, Bansal A, Rastogi A, Wang G (2020) A com-parative study on polyp classification using convolutional neural networks. PLoS ONE 15(7):0236452

    Article  Google Scholar 

  40. Jheng Y-C, Wang Y-P, Lin H-E, Sung K-Y, Chu Y-C, Wang H-S, Jiang J-K, Hou M-C, Lee F-Y, Lu C-L (2022) A novel machine learning- based algorithm to identify and classify lesions and anatomical landmarks in colonoscopy images. Surg Endosc 36(1):640–650

    Article  Google Scholar 

  41. Jia X, Mai X, Cui Y, Yuan Y, Xing X, Seo H, Xing L, Meng MQ-H (2020) Automatic polyp recognition in colonoscopy images using deep learning and two- stage pyramidal feature prediction. IEEE Trans Auto Sci Eng 17(3):1570–1584

    Google Scholar 

  42. Ellahyani A, Jaafari IE, Charfi S, Ansari ME (2022) Fine-tuned deep neural networks for polyp detection in colonoscopy images. Pers Ubiquit Comput 27(2):1–13

    Google Scholar 

  43. Karaman A, Pacal I, Basturk A, Akay B, Nalbantoglu U, Coskun S, Sahin O, Karaboga D (2023) Robust real-time polyp detection system design based on yolo algorithms by optimizing activation functions and hyper-parameters with artificial bee colony (abc). Expert Syst Appl 221:119741

    Article  Google Scholar 

Download references

Funding

No funding was received for this work.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Delhi Technological University, New Delhi, India

    Palak Handa & Sreedevi Indu

  2. Department of Electronics and Communication Engineering, Indira Gandhi Delhi Technical University for Women, New Delhi, India

    Nidhi Goel

  3. Department of Gastroenterology and HNU, All India Institute of Medical Sciences, New Delhi, India

    Deepak Gunjan

Authors
  1. Palak Handa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nidhi Goel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sreedevi Indu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Deepak Gunjan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nidhi Goel.

Ethics declarations

Conflict of interest

The authors declare no personal, academic, or financial conflicts of interest associated with this paper.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 13 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Handa, P., Goel, N., Indu, S. et al. Automatic Detection of Colorectal Polyps with Mixed Convolutions and its Occlusion Testing. Neural Comput & Applic 35, 19409–19426 (2023). https://doi.org/10.1007/s00521-023-08762-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08762-z

Keywords

Search

Navigation