Skip to main content
SpringerLink
Log in

DeepCPD: deep learning with vision transformer for colorectal polyp detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

One of the most severe cancers worldwide is Colorectal Cancer (CRC), which has the third-highest incidence of cancer cases and the second-highest rate of cancer mortality. Early diagnosis and treatment are receiving much attention globally due to the increasing incidence and death rates. Colonoscopy is acknowledged as the gold standard for screening CRC. Despite early screening, doctors miss approximately 25% of polyps during a colonoscopy examination because the diagnosis varies from expert to expert. After a few years, this missing polyp may develop into cancer. This study is focused on addressing such diagnostic challenges, aiming to minimize the risk of misdiagnosis and enhance the overall accuracy of diagnostic procedures. Thus, we propose an efficient deep learning method, DeepCPD, combining transformer architecture and Linear Multihead Self-Attention (LMSA) mechanism with data augmentation to classify colonoscopy images into two categories: polyp versus non-polyp and hyperplastic versus adenoma based on the dataset. The experiments are conducted on four benchmark datasets: PolypsSet, CP-CHILD-A, CP-CHILD-B, and Kvasir V2. The proposed model demonstrated superior performance compared to the existing state-of-the-art methods with an accuracy above 98.05%, precision above 97.71%, and recall above 98.10%. Notably, the model exhibited a training time improvement of over 1.2x across all datasets. The strong performance of the recall metric shows the ability of DeepCPD to detect polyps by minimizing the false negative rate. These results indicate that this model can be used effectively to create a diagnostic tool with computer assistance that can be highly helpful to clinicians during the diagnosing process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Availability of data and materials

Public datasets are used for this study.

References

  1. Cancer IA Cancer Today. https://gco.iarc.fr/. Accessed 15 Nov 2022

  2. international W Colorectal cancer statistics. https://www.wcrf.org/cancer-trends/colorectal-cancer-statistics/. Accessed 15 Nov 2022

  3. Chen H, Li C, Li X, Rahaman MM, Hu W, Li Y, Liu W, Sun C, Sun H (2022) Huang X et al Il-mcam: an interactive learning and multi-channel attention mechanism-based weakly supervised colorectal histopathology image classification approach. Computers in Biology and Medicine 143:105265

  4. Society AC American Cancer Society Guideline for Colorectal Cancer Screening. https://www.cancer.org/cancer/colon-rectal-cancer/detection-diagnosis-staging/acs-recommendations.html. Accessed 15 Nov 2022

  5. Tan J, Gao Y, Liang Z, Cao W, Pomeroy MJ, Huo Y, Li L, Barish MA, Abbasi AF, Pickhardt PJ (2019) 3D-GLCM CNN: a 3-dimensional gray-level co-occurrence matrix-based cnn model for polyp classification via ct colonography. IEEE Transactions on Medical Imaging 39(6):2013–2024

  6. Nguyen H-G, Blank A, Lugli A, Zlobec I (2020) An effective deep learning architecture combination for tissue microarray spots classification of h &e stained colorectal images. In: 2020 IEEE 17th International symposium on biomedical imaging (ISBI), IEEE, pp 1271–1274

  7. Pacal I, Karaboga D, Basturk A, Akay B, Nalbantoglu U (2020) A comprehensive review of deep learning in colon cancer. Computers in Biology and Medicine 126:104003

    Article  PubMed  Google Scholar 

  8. Solak A, Ceylan R (2023) A sensitivity analysis for polyp segmentation with u-net. Multimed Tools Appl 1–29

  9. Jha D, Smedsrud PH, Johansen D, Lange T, Johansen HD, Halvorsen P, Riegler MA (2021) A comprehensive study on colorectal polyp segmentation with resunet++, conditional random field and test-time augmentation. IEEE journal of biomedical and health informatics 25(6):2029–2040

    Article  PubMed  Google Scholar 

  10. Tasnim Z, Chakraborty S, Shamrat F, Chowdhury AN, Nuha HA, Karim A, Zahir SB, Billah MM et al (2021) Deep learning predictive model for colon cancer patient using cnn-based classification. Int J Adv Comput Sci Appl 12

  11. Lorenzovici N, Dulf E-H, Mocan T, Mocan L (2021) Artificial intelligence in colorectal cancer diagnosis using clinical data: non-invasive approach. Diagnostics 11(3):514

  12. Younas F, Usman M, Yan WQ (2022) A deep ensemble learning method for colorectal polyp classification with optimized network parameters. Appl Intell 1–24

  13. Xie X, Xing J, Kong N, Li C, Li J, Zhang S (2017) Improving colorectal polyp classification based on physical examination data—an ensemble learning approach. IEEE Robot Automat Lett 3(1):434–441

  14. Chou Y-C, Chen C-C (2023) Improving deep learning-based polyp detection using feature extraction and data augmentation. Multimedia Tools and Applications 82(11):16817–16837

    Article  Google Scholar 

  15. Younas F, Usman M, Yan WQ (2023) An ensemble framework of deep neural networks for colorectal polyp classification. Multimedia Tools and Applications 82(12):18925–18946

    Article  Google Scholar 

  16. Fang Y, Zhu D, Yao J, Yuan Y, Tong K-Y (2020) Abc-net: area-boundary constraint network with dynamical feature selection for colorectal polyp segmentation. IEEE Sensors Journal 21(10):11799–11809

  17. Chandan S, Mohan BP, Khan SR, Bhogal N, Ramai D, Bilal M, Aziz M, Shah AR, Mashiana HS, Jha LK et al (2021) Adenoma and polyp detection rates during insertion versus withdrawal phase of colonoscopy: a systematic review and meta-analysis of randomized controlled trials. Gastrointestinal endoscopy 93(1):68–76

    Article  PubMed  Google Scholar 

  18. Mesejo P, Pizarro D, Abergel A, Rouquette O, Beorchia S, Poincloux L, Bartoli A (2016) Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE transactions on medical imaging 35(9):2051–2063

    Article  PubMed  Google Scholar 

  19. Aggarwal AK (2023) Thermal imaging for cancer detection. Imaging and Radiation Research 6(1):2638

    Article  MathSciNet  Google Scholar 

  20. Wang S, Li BZ, Khabsa M, Fang H, Ma H (2020) Linformer: self-attention with linear complexity. arXiv:2006.04768

  21. Alqudah AM, Alqudah A (2022) Improving machine learning recognition of colorectal cancer using 3d glcm applied to different color spaces. Multimedia Tools and Applications 81(8):10839–10860

    Article  MathSciNet  Google Scholar 

  22. Jheng Y-C, Wang Y-P, Lin H-E, Sung K-Y, Chu Y-C, Wang H-S, Jiang J-K, Hou M-C, Lee F-Y, Lu C-L (2022) A novel machine learning-based algorithm to identify and classify lesions and anatomical landmarks in colonoscopy images. Surgical Endoscopy 36(1):640–650

    Article  PubMed  Google Scholar 

  23. Koppad S, Basava A, Nash K, Gkoutos GV, Acharjee A (2022) Machine learning-based identification of colon cancer candidate diagnostics genes. Biology 11(3):365

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Su Y, Tian X, Gao R, Guo W, Chen C, Chen C, Jia D, Li H, Lv X (2022) Colon cancer diagnosis and staging classification based on machine learning and bioinformatics analysis. Computers in biology and medicine 145:105409

    Article  PubMed  Google Scholar 

  25. Mulenga M, Kareem SA, Sabri AQM, Seera M (2021) Stacking and chaining of normalization methods in deep learning-based classification of colorectal cancer using gut microbiome data. IEEE Access 9:97296–97319

    Article  Google Scholar 

  26. Sarwinda D, Paradisa RH, Bustamam A, Anggia P (2021) Deep learning in image classification using residual network (resnet) variants for detection of colorectal cancer. Procedia Computer Science 179:423–431

    Article  Google Scholar 

  27. Mulenga M, Kareem SA, Sabri AQM, Seera M, Govind S, Samudi C, Mohamad SB (2021) Feature extension of gut microbiome data for deep neural network-based colorectal cancer classification. IEEE Access 9:23565–23578

    Article  Google Scholar 

  28. Tang C-P, Chen K-H, Lin T-L (2021) Computer-aided colon polyp detection on high resolution colonoscopy using transfer learning techniques. Sensors 21(16):5315

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  29. Hsu C-M, Hsu C-C, Hsu Z-M, Shih F-Y, Chang M-L, Chen T-H (2021) Colorectal polyp image detection and classification through grayscale images and deep learning. Sensors 21(18):5995

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  30. Zhou C, Jin Y, Chen Y, Huang S, Huang R, Wang Y, Zhao Y, Chen Y, Guo L, Liao J (2021) Histopathology classification and localization of colorectal cancer using global labels by weakly supervised deep learning. Computerized Medical Imaging and Graphics 88:101861

    Article  PubMed  Google Scholar 

  31. Paladini E, Vantaggiato E, Bougourzi F, Distante C, Hadid A, Taleb-Ahmed A (2021) Two ensemble-CNN approaches for colorectal cancer tissue type classification. Journal of Imaging 7(3):51

    Article  PubMed  PubMed Central  Google Scholar 

  32. Liew WS, Tang TB, Lin C-H, Lu C-K (2021) Automatic colonic polyp detection using integration of modified deep residual convolutional neural network and ensemble learning approaches. Computer Methods and Programs in Biomedicine 206:106114

    Article  PubMed  Google Scholar 

  33. Zhang R, Zheng Y, Mak TWC, Yu R, Wong SH, Lau JY, Poon CC (2016) Automatic detection and classification of colorectal polyps by transferring low-level cnn features from nonmedical domain. IEEE journal of biomedical and health informatics 21(1):41–47

    Article  PubMed  Google Scholar 

  34. Liu X, Li Y, Yao J, Chen B, Song J, Yang X (2019) Classification of polyps and adenomas using deep learning model in screening colonoscopy. In: 2019 8th International symposium on next generation electronics (ISNE), IEEE, pp 1–3

  35. Nisha J, Gopi VP, Palanisamy P (2022) Automated colorectal polyp detection based on image enhancement and dual-path cnn architecture. Biomedical Signal Processing and Control 73:103465

    Article  Google Scholar 

  36. Patel K, Li K, Tao K, Wang Q, Bansal A, Rastogi A, Wang G (2020) A comparative study on polyp classification using convolutional neural networks. PloS one 15(7):0236452

    Article  Google Scholar 

  37. Lo C-M, Yang Y-W, Lin J-K, Lin T-C, Chen W-S, Yang S-H, Chang S-C, Wang H-S, Lan Y-T, Lin H-H et al (2023) Modeling the survival of colorectal cancer patients based on colonoscopic features in a feature ensemble vision transformer. Computer Med Imaging Graphics 107:102242

  38. Mali MT, Hancer E, Samet R, Yıldırım Z, Nemati N (2022) Detection of colorectal cancer with vision transformers. In: 2022 Innovations in intelligent systems and applications conference (ASYU), IEEE, pp 1–6

  39. Wang X, Yang S, Zhang J, Wang M, Zhang J, Yang W, Huang J, Han X (2022) Transformer-based unsupervised contrastive learning for histopathological image classification. Medical Image Analysis 81:102559

  40. Hossain MS, Rahman MM, Syeed MM, Uddin MF, Hasan M, Hossain MA, Ksibi A, Jamjoom MM, Ullah Z, Samad MA (2023) Deeppoly: deep learning based polyps segmentation and classification for autonomous colonoscopy examination. IEEE Access

  41. Zhang J (2023) Towards a high-performance object detector: insights from drone detection using vit and cnn-based deep learning models. In: 2023 IEEE International conference on sensors, electronics and computer engineering (ICSECE), IEEE, pp 141–147

  42. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: transformers for image recognition at scale. In: International conference on learning representations

  43. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inform Process Syst 30

  44. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  45. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: European conference on computer vision, Springer, pp 213–229

  46. Kaur A, Chauhan APS, Aggarwal AK (2021) An automated slice sorting technique for multi-slice computed tomography liver cancer images using convolutional network. Expert Systems with Applications 186:115686

    Article  Google Scholar 

  47. Li K, Fathan MI, Patel K, Zhang T, Zhong C, Bansal A, Rastogi A, Wang JS, Wang G (2021) Colonoscopy polyp detection and classification: dataset creation and comparative evaluations. Plos One 16(8):0255809

  48. Bernal J, Tajkbaksh N, Sanchez FJ, Matuszewski BJ, Chen H, Yu L, Angermann Q, Romain O, Rustad B (2017) Balasingham I et al Comparative validation of polyp detection methods in video colonoscopy: results from the miccai 2015 endoscopic vision challenge. IEEE transactions on medical imaging 36(6):1231–1249

    Article  PubMed  Google Scholar 

  49. Bernal J, Sánchez J, Vilarino F (2012) Towards automatic polyp detection with a polyp appearance model. Pattern Recognition 45(9):3166–3182

    Article  ADS  Google Scholar 

  50. Wang W, Tian J, Zhang C, Luo Y, Wang X, Li J (2020) An improved deep learning approach and its applications on colonic polyp images detection. BMC Medical Imaging 20:1–14

    Article  Google Scholar 

  51. Pogorelov K, Randel KR, Griwodz C, Eskeland SL, Lange T, Johansen D, Spampinato C, Dang-Nguyen D-T, Lux M, Schmidt PT et al (2017) Kvasir: a multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of the 8th ACM on multimedia systems conference, pp 164–169

  52. Zha Z, Tang H, Sun Y, Tang J (2023) Boosting few-shot fine-grained recognition with background suppression and foreground alignment. IEEE Trans Circuits Syst Video Technol

  53. Tang H, Liu J, Yan S, Yan R, Li Z, Tang J (2023) M3net: multi-view encoding, matching, and fusion for few-shot fine-grained action recognition. In: Proceedings of the 31st ACM international conference on multimedia, pp 1719–1728

Download references

Acknowledgements

This research was supported by the University Grants Commission of India under UGC-JRF grant 3640/(NET- JULY 2018), and the computing resources are provided by C- DAC and National Institute of Technology Tiruchirappalli.

Funding

This study was funded by the University Grants Commission (UGC) of India under UGC-JRF grant 3640/(NET-JULY 2018).

Author information

Authors and Affiliations

  1. Department of Computer Applications, National Institute of Technology Tiruchirappalli, Tiruchirappalli, Tamilnadu, 620015, India

    Raseena T.P, Jitendra Kumar & S. R. Balasundaram

Authors
  1. Raseena T.P
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jitendra Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. R. Balasundaram
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors equally contributed to this study.

Corresponding author

Correspondence to Raseena T.P.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

The authors of this study have not engaged in any research involving human subjects or animals.

Consent to participate

All individual participants included in the study provided informed consent.

Consent for publication

The participant has given consent for the submission of the case report to the journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Jitendra Kumar and S.R. Balasundaram are contributed equally to this work.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

T.P, R., Kumar, J. & Balasundaram, S.R. DeepCPD: deep learning with vision transformer for colorectal polyp detection. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18607-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11042-024-18607-z

Keywords

Search

Navigation