Skip to main content
SpringerLink
Log in

Improving deep learning-based polyp detection using feature extraction and data augmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, Colorectal Cancer (CRC) has been common reasons of lethal disease and cancer. However, colonoscopy can examine this disease, and the location of polyps and tumors can be detected. However, the early symptoms of CRC are not evident and specific, which is easy to be ignored by patients and doctors. As a result, the opportunity for early diagnosis and treatment was missed. This study aims to provide auxiliary detection to obtain accurate polyp diagnosis and assist clinicians in more precise detection. This paper proposes a novel polyp detection method through deep learning, which uses a fusion module combining feature extraction and data augmentation to enhance images. The Discrete Wavelet Transform (DWT) is applied to extract the texture features of polyps and strengthen the texture features that are not obvious in the polyp image. Then style-based GAN2 is used to enhance the image data, increase the image training data of YOLOv4, and let YOLOv4 learn more features of polyps. According to the experimental results, our method is better than state-of-the-art methods in polyp detection efficiency. In addition, because we have enhanced the image, the detection rate of small polyps is significantly improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The datasets analysed during the current study are available in the publicly archived datasets: CVC-ClinicDB: https://polyp.grand-challenge.org/CVCClinicDB/, CVC-ColonDB: http://mv.cvc.uab.es/projects/colon-qa/cvccolondb, ETIS-Larib: https://polyp.grand-challenge.org/EtisLarib/, and Kvasir-SEG: https://datasets.simula.no/kvasir-seg/.

References

  1. Ameling S, Wirth S, Paulus D, Lacey G, Vilarino F (2009) Texture-based polyp detection in colonoscopy. In: Bildverarbeitung für die medizin 2009. Springer, pp 346–350

  2. Bansal M, Kumar M (2021) Kumar, m.: 2d object recognition techniques: state-of-the-art work. Archives of Computational Methods in Engineering 28(3):1147–1161

    Article  MathSciNet  Google Scholar 

  3. Bansal M, Kumar M, Kumar M (2021) 2d object recognition: a comparative analysis of sift, surf and orb feature descriptors. Multimed Tools Appl 80 (12):18839–18857

    Article  Google Scholar 

  4. Bansal M, Kumar M, Kumar M, Kumar K (2021) An efficient technique for object recognition using shi-tomasi corner detection algorithm. Soft Comput 25(6):4423–4432

    Article  Google Scholar 

  5. Bernal J, Sanchez FJ, Fernandez-Esparrach G, Gil D, Rodriguez C, Vilarino F (2015) Wm-dova maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imaging Graph 43:99–111

    Article  Google Scholar 

  6. Bernal J, Sanchez J, Vilarino F (2012) Towards automatic polyp detection with a polyp appearance model. Pattern Recogn 45(9):3166–3182

    Article  Google Scholar 

  7. Billah M, Waheed S (2020) Minimum redundancy maximum relevance (mrmr) based feature selection from endoscopic images for automatic gastrointestinal polyp detection. Multimed Tools Appl 79(33):23633–23643

    Article  Google Scholar 

  8. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934

  9. Cancer IAFRO (2020) International Agency for Research on Cancer. https://gco.iarc.fr/today/home. Accessed Sept 2021

  10. Chhabra P, Garg NK, Kumar M (2020) Content-based image retrieval system using orb and sift features. Neural Comput Applic 32(7):2725–2733

    Article  Google Scholar 

  11. Costa P, Galdran A, Meyer MI, Niemeijer M, Abramoff M, Mendonca AM, Campilho A (2018) End-to-end adversarial retinal image synthesis. IEEE Trans Med Imaging 37(3):781–791

    Article  Google Scholar 

  12. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA (2018) Generative adversarial networks an overview. IEEE Signal Proc Mag 35(1):53–65

    Article  Google Scholar 

  13. Demirel H, Anbarjafari G (2011) Discrete wavelet transform-based satellite image resolution enhancement. IEEE Trans Geosci Remote Sens 49(6):1997–2004

    Article  MATH  Google Scholar 

  14. Durak S, Bayram B, Bakirman T, Erkut M, Dogan M, Gurturk M, Akpinar B (2021) Deep neural network approaches for detecting gastric polyps in endoscopic images. Med Biol Eng Comput 59(7-8):1563–1574

    Article  Google Scholar 

  15. Engelhardt S, Ameling S, Wirth S, Paulus D (2010) Features for classification of polyps in colonoscopy. Bildverarbeitung für die Medizin 574:350–354

    Google Scholar 

  16. Fetty L, Bylund M, Kuess P, Heilemann G, Nyholm T, Georg D, Lofstedt T (2020) Latent space manipulation for high-resolution medical image synthesis via the stylegan. Zeitschrift Fur Medizinische Physik 30(4):305–314

    Article  Google Scholar 

  17. Fonolla R, van der Zander QEW, Schreuder RM, Subramaniam S, Bhandari P, Masclee AAM, Schoon EJ, van Der Sommen F, de With PHN (2021) Automatic image and text-based description for colorectal polyps using basic classification. Artif Intell Med 121. https://doi.org/ARTN10217810.1016/j.artmed.2021.102178

  18. Gonzalez RC, Woods RE (2008) Digital image processing. Prentice-Hall

  19. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Advances in neural information processing systems 27

  20. Gupta S, Kumar M, Garg A (2019) Improved object recognition results using sift and orb feature detector. Multimed Tools Appl 78(23):34157–34171

    Article  Google Scholar 

  21. Gupta S, Mohan N, Kumar M (2021) A study on source device attribution using still images. Archives Comput Methods Eng 28(4):2209–2223

    Article  Google Scholar 

  22. Hasan M, Hossain MM, Mia S, Ahammad M, Rahman MM, et al. (2022) A combined approach of non-subsampled contourlet transform and convolutional neural network to detect gastrointestinal polyp. Multimed Tools Appl 1–20

  23. He W, Zi Y, Chen B, Wu F, He Z (2015) Automatic fault feature extraction of mechanical anomaly on induction motor bearing using ensemble super-wavelet transform. Mech Syst Signal Process 54:457–480

    Article  Google Scholar 

  24. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  25. Jha D, Ali S, Tomar NK, Johansen HD, Johansen D, Rittscher J, Riegler MA, Halvorsen P (2021) Real-time polyp detection, localization and segmentation in colonoscopy using deep learning. IEEE Access 9:40496–40510

    Article  Google Scholar 

  26. Jha D, Smedsrud PH, Riegler MA, Halvorsen P, Lange TD, Johansen D, Johansen HD (2020) Kvasir-seg: A segmented polyp dataset. In: International conference on multimedia modeling, pp 451–462

  27. Jiang G, Lu Y, Wei J, Xu Y (2019) Synthesize mammogram from digital breast tomosynthesis with gradient guided cgans. In: International conference on medical image computing and computer-assisted intervention, pp 801–809

  28. Kang J, Gwak J (2019) Ensemble of instance segmentation models for polyp segmentation in colonoscopy images. IEEE Access 7:26440–26447

    Article  Google Scholar 

  29. Karkanis SA, Iakovidis DK, Maroulis DE, Karras DA, Tzivras M (2003) Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans Inf Technol Biomed 7(3):141– 152

    Article  Google Scholar 

  30. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv:1710.10196

  31. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4401–4410

  32. Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8110–8119

  33. Kim S-H, Koh HM, Lee B-D (2021) Classification of colorectal cancer in histological images using deep neural networks: an investigation. Multimed Tools Appl 80(28):35941–35953

    Article  Google Scholar 

  34. Komeda Y, Handa H, Watanabe T, Nomura T, Kitahashi M, Sakurai T, Okamoto A, Minami T, Kono M, Arizumi T, Takenaka M, Hagiwara S, Matsui S, Nishida N, Kashida H, Kudo M (2017) Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: preliminary experience. Oncology 93:30–34

    Article  Google Scholar 

  35. Kumar M, Chhabra P, Garg NK (2018) An efficient content based image retrieval system using bayesnet and k-nn. Multimed Tools Appl 77(16):21557–21570

    Article  Google Scholar 

  36. Lee WL, Chen YC, Hsieh KS (2003) Ultrasonic liver tissues classification by fractal feature vector based on m-band wavelet transform. IEEE Trans Med Imaging 22(3):382–392

    Article  Google Scholar 

  37. Leufkens AM, van Oijen MGH, Vleggaar FP, Siersema PD (2012) Factors influencing the miss rate of polyps in a back-to-back colonoscopy study. Endoscopy 44(5):470–475

    Article  Google Scholar 

  38. Li BP, Meng MQH (2012) Automatic polyp detection for wireless capsule endoscopy images. Expert Syst Appl 39(12):10952–10958

    Article  Google Scholar 

  39. Lin TY, Goyal P, Girshick R, He KM, Dollar P (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 42(2):318–327

    Article  Google Scholar 

  40. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision, pp 21–37

  41. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8759–8768

  42. Liu DY, Rao NN, Mei XM, Jiang HX, Li QC, Luo CS, Li Q, Zeng CS, Zeng B, Gan T (2018) Annotating early esophageal cancers based on two saliency levels of gastroscopic images. J Med Syst 42(12)

  43. Mallat SG (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell 11(7):674–693. https://doi.org/10.1109/34.192463

    Article  MATH  Google Scholar 

  44. Maroulis DE, Iakovidis DK, Karkanis SA, Karras DA (2003) Cold: a versatile detection system for colorectal lesions in endoscopy video-frames. Comput Methods Prog Biomed 70(2):151–166

    Article  Google Scholar 

  45. Öztürk Ş, Özkaya U (2020) Gastrointestinal tract classification using improved lstm based cnn. Multimed Tools Appl 79(39):28825–28840

    Article  Google Scholar 

  46. Pacal I, Karaboga D (2021) A robust real-time deep learning based automatic polyp detection system. Comput Biol Med 134

  47. Pacal I, Karaman A, Karaboga D, Akay B, Basturk A, Nalbantoglu U, Coskun S (2022) An efficient real-time colonic polyp detection with yolo algorithms trained by using negative samples and large datasets. Comput Biology Med 141:105031

    Article  Google Scholar 

  48. Pannu HS, Ahuja S, Dang N, Soni S, Malhi AK (2020) Deep learning based image classification for intestinal hemorrhage. Multimed Tools Appl 79(29):21941–21966

    Article  Google Scholar 

  49. Pogorelov K, Riegler M, Eskeland SL, de Lange T, Johansen D, Griwodz C, Schmidt PT, Halvorsen P (2017) Efficient disease detection in gastrointestinal videos–global features versus neural networks. Multimed Tools Appl 76 (21):22493–22525

    Article  Google Scholar 

  50. Poorneshwaran J, Kumar SS, Ram K, Joseph J, Sivaprakasam M (2019) Polyp segmentation using generative adversarial network. In: 2019 41St annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 7201–7204

  51. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434

  52. Rasti P, Daneshmand M, Alisinanoglu F, Ozcinar C, Anbarjafari G (2016) Medical image illumination enhancement and sharpening by using stationary wavelet transform. In: 2016 24Th signal processing and communication application conference (SIU), pp 153–156

  53. Rasti P, Daneshmand M, Alisinanoglu F, Ozcinar C, Anbarjafari G (2016) Medical image illumination enhancement and sharpening by using stationary wavelet transform. In: 2016 24Th signal processing and communication application conference (SIU), pp 153–156

  54. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  55. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv:1804.02767

  56. Ren SQ, He KM, Girshick R, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  57. Rufai AM, Anbarjafari G, Demirel H (2014) Lossy image compression using singular value decomposition and wavelet difference reduction. Digital signal processing 24:117–123

    Article  Google Scholar 

  58. Schoofs N, Deviere J, Van Gossum A (2006) Pillcam colon capsule endoscopy compared with colonoscopy for colorectal tumor diagnosis: a prospective pilot study. Endoscopy 38(10):971–977

    Article  Google Scholar 

  59. Shin Y, Qadir HA, Aabakken L, Bergsland J, Balasingham I (2018) Automatic colon polyp detection using region based deep cnn and post learning approaches. IEEE Access 6:40950–40962

    Article  Google Scholar 

  60. Silva J, Histace A, Romain O, Dray X, Granado B (2014) Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J CARS 9(2):283–293

    Article  Google Scholar 

  61. Tan M, Pang R, Le QV (2020) Efficientdet: scalable and efficient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10781–10790

  62. Thomaz VD, Sierra-Franco CA, Raposo AB (2021) Training data enhancements for improving colonic polyp detection using deep convolutional neural networks. Artif Intell Med 111. https://doi.org/ARTN10198810.1016/j.artmed.2020.101988

  63. Tulum G, Bolat B, Osman O (2017) A cad of fully automated colonic polyp detection for contrasted and non-contrasted ct scans. Int J CARS 12(4):627–644

    Article  Google Scholar 

  64. Van Rijn JC, Reitsma JB, Stoker J, Bossuyt PM, van Deventer SJ, Dekker E (2006) Polyp miss rate determined by tandem colonoscopy: a systematic review. Am J Gastroenterol 101(2):343–350

    Article  Google Scholar 

  65. van Wijk C, van Ravesteijn VF, Vos FM, van Vliet LJ (2010) Detection and segmentation of colonic polyps on implicit isosurfaces by second principal curvature flow. IEEE Trans Med Imaging 29(3):688–698

    Article  Google Scholar 

  66. Vazquez D, Bernal J, Sanchez FJ, Fernandez-Esparrach G, Lopez AM, Romero A, Drozdzal M, Courville A (2017) A benchmark for endoluminal scene segmentation of colonoscopy images. Journal of Healthcare Engineering, 2017

  67. Velmurugan A, Kannan RJ (2013) Wavelet analysis for medical image denoising based on thresholding techniques. In: 2013 international conference on current trends in engineering and technology (ICCTET), pp 213–215

  68. Vieira PM, Freitas NR, Lima VB, Costa D, Rolanda C, Lima CS (2021) Multi-pathology detection and lesion localization in wce videos by using the instance segmentation approach. Artif Intell Med 119. https://doi.org/ARTN10214110.1016/j.artmed.2021.102141

  69. Wang TC, Karayiannis NB (1998) Detection of microcalcifications in digital mammograms using wavelets. IEEE Trans Med Imaging 17(4):498–509

    Article  Google Scholar 

  70. Wimmer G, Tamaki T, Tischendorf JJW, Hafner M, Yoshida S, Tanaka S, Uhl A (2016) Directional wavelet based features for colonic polyp classification. Med Image Anal 31:16–36. https://doi.org/10.1016/j.media.2016.02.001https://doi.org/10.1016/j.media.2016.02.001

    Article  Google Scholar 

  71. Yu LQ, Chen H, Dou Q, Qin J, Heng PA (2017) Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J Biomed Health Inform 21(1):65–75. https://doi.org/10.1109/Jbhi.2016.2637004

    Article  Google Scholar 

  72. Zhang T, Fu H, Zhao Y, Cheng J, Guo M, Gu Z, Yang B, Xiao Y, Gao S, Liu J (2019) Skrgan: sketching-rendering unconditional generative adversarial networks for medical image synthesis. In: International conference on medical image computing and computer-assisted intervention, pp 777–785

  73. Zhang RK, Zheng YL, Poon CCY, Shen DG, Lau JYW (2018) Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker. Pattern Recogn 83:209–219

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Science and Technology Council (NSTC) of Taiwan, under grants NSTC 111-2221-E-006-202 and 110-2221-E-006-124. This work was also supported by the “Intelligent Manufacturing Research Center” (iMRC) from The Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education in Taiwan.

Author information

Authors and Affiliations

  1. Institute of Manufacturing Information and Systems, National Cheng Kung University, Tainan, 701, Taiwan

    Yung-Chien Chou & Chao-Chun Chen

Authors
  1. Yung-Chien Chou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao-Chun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Chun Chen.

Ethics declarations

Conflict of Interests

All authors hereby declare that there are no conflicts of interest/competing interests in this research.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chou, YC., Chen, CC. Improving deep learning-based polyp detection using feature extraction and data augmentation. Multimed Tools Appl 82, 16817–16837 (2023). https://doi.org/10.1007/s11042-022-13995-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13995-6

Keywords

Search

Navigation