Skip to main content
SpringerLink
Log in

A DWT-based encoder-decoder network for Specularity segmentation in colonoscopy images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Specularity segmentation in colonoscopy images is a crucial pre-processing step for efficient computational diagnosis. The presence of these specular highlights could mislead the detectors that are intended towards precise identification of biomarkers. Conventional methods adopted so far do not provide satisfactory results, especially in the overexposed regions. The potential of deep learning methods is still unexplored in the related problem domain. Our work aims at providing a solution for more accurate highlights segmentation to assist surgeons. In this paper, we propose a novel deep learning based approach that performs segmentation following a multi-resolution analysis. This is achieved by introducing discrete wavelet transform (DWT) into the proposed model. We replace the standard pooling layers with DWTs, which helps preserve information and circumvent the effect of overexposed regions. All analytical experiments are performed using a publicly available benchmark dataset, and an F1-score (%) of 83.10 ± 0.14 is obtained on the test set. The experimental results show that this technique outperforms state-of-the-art methods and performs significantly better in overexposed regions. The proposed model also performed superior to some deep learning models (but applied in different domains) when tested with our problem specifications. Our method provides segmentation outcomes that are closer to the actual segmentation done by experts. This ensures improved pre-processed colonoscopy images that aid in better diagnosis of colorectal cancer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Akbari M, Mohrekesh M, Najariani K, Karimi N, Samavi S, Soroushmehr SR (2018) Adaptive specular reflection detection and inpainting in colonoscopy video frames. In: 2018 25th IEEE International Conference on Image Processing (ICIP), IEEE, pp 3134-3138

  2. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodriíguez C, Vilariño F (2015) Wm-dova maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imaging Graph 43:99–111. https://doi.org/10.1016/j.compmedimag.2015.02.007

    Article  Google Scholar 

  3. Bernal J, Sánchez J, Vilarino F (2013) Impact of image preprocessing methods on polyp localization in colonoscopy frames. In: 2013 35th Annual international conference of the ieee engineering in medicine and biology society (EMBC). IEEE, pp 7350–7354

  4. Bernal J, Sánchez FJ, de Miguel CR, Fernández-Esparrach G (2015) Building up the future of colonoscopy–a synergy between clinicians and computer scientists. Screening for Colorectal Cancer with Colonoscopy, pp 109

  5. Chahal ES, Patel A, Gupta A, Purwar A et al (2021) Unet based xception model for prostate cancer segmentation from mri images. Multimed Tools Appl, pp 1–17. https://doi.org/10.1007/s11042-021-11334-9

  6. Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818

  7. Choudhury AR, Vanguri R, Jambawalikar SR, Kumar P (2018) Segmentation of brain tumors using deeplabv3+. In: International MICCAI Brainlesion workshop. Springer, pp 154–167

  8. Figueiredo IN, Pinto L, Figueiredo PN, Tsai R (2019) Unsupervised segmentation of colonic polyps in narrow-band imaging data based on manifold representation of images and wasserstein distance. Biomed Signal Process Control 53:101577. https://doi.org/10.1016/j.bspc.2019.101577

    Article  Google Scholar 

  9. Ganz M, Yang X, Slabaugh G (2012) Automatic segmentation of polyps in colonoscopic narrow-band imaging data. IEEE Trans Biomed Eng 59 (8):2144–2151. https://doi.org/10.1109/TBME.2012.2195314

    Article  Google Scholar 

  10. Gross S, Palm S, Tischendorf JJ, Behrens A, Trautwein C, Aach T (2012) Automated classification of colon polyps in endoscopic image data. In: Medical imaging 2012: computer-aided diagnosis, international society for optics and Photonics, vol 8315, pp 83150W

  11. Hamaguchi R, Fujita A, Nemoto K, Imaizumi T, Hikosaka S (2018) Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery. In: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE, pp 1442-1450

  12. Hu P, Ramanan D (2017) Finding tiny faces. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 951–959

  13. Kang J, Gwak J (2019) Ensemble of instance segmentation models for polyp segmentation in colonoscopy images. IEEE Access 7:26440–26447. https://doi.org/10.1109/ACCESS.2019.2900672

    Article  Google Scholar 

  14. Li X, Chen H, Qi X, Dou Q, Fu CW, Heng PA (2018) H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes. IEEE Trans Med Imaging 37(12):2663–2674. https://doi.org/10.1109/TMI.2018.2845918

    Article  Google Scholar 

  15. Li R, Pan J, Si Y, Yan B, Hu Y, Qin H (2019) Specular reflections removal for endoscopic image sequences with adaptive-rpca decomposition. IEEE Trans Med Imaging 39(2):328–340. https://doi.org/10.1109/TMI.2019.2926501

    Article  Google Scholar 

  16. Li J, Yu ZL, Gu Z, Liu H, Li Y (2019) Dilated-inception net: multi-scale feature aggregation for cardiac right ventricle segmentation. IEEE Trans Biomed Eng 66(12):3499–3508. https://doi.org/10.1109/TBME.2019.2906667

    Article  Google Scholar 

  17. Liu Q, Dou Q, Yu L, Heng PA (2020) Ms-net: multi-site network for improving prostate segmentation with heterogeneous mri data. IEEE Trans Med Imaging. https://doi.org/10.1109/TMI.2020.2974574

  18. Liu P, Zhang H, Zhang K, Lin L, Zuo W (2018) Multi-level wavelet-cnn for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 773–782

  19. Park SY, Sargent D, Spofford I, Vosburgh K G, Yousif A et al (2012) A colon video analysis framework for polyp detection. IEEE Trans Biomed Eng 59(5):1408–1418. https://doi.org/10.1109/TBME.2012.2188397

    Article  Google Scholar 

  20. Pavan Kumar M, Jayagopal P (2021) Generative adversarial networks: a survey on applications and challenges. International Journal of Multimedia Information Retrieval 10(1):1–24. https://doi.org/10.1007/s13735-020-00196-w

    Article  Google Scholar 

  21. Prasath V (2017) Polyp detection and segmentation from video capsule endoscopy: a review. Journal of Imaging 3(1):1. https://doi.org/10.3390/jimaging3010001

    Article  Google Scholar 

  22. Priyadharsini R, Sharmila TS, Rajendran V (2015) Underwater acoustic image enhancement using wavelet and kl transform. In: 2015 International conference on applied and theoretical computing and communication technology (iCATccT). IEEE, pp 563–567

  23. Pyka K (2017) Wavelet-based local contrast enhancement for satellite, aerial and close range images. Remote Sens 9(1):25. https://doi.org/10.3390/rs9010025

    Article  Google Scholar 

  24. Ribeiro E, Uhl A, Wimmer G, Häfner M (2016) Exploring deep learning and transfer learning for colonic polyp classification. Computational and Mathematical Methods in Medicine, pp 2016. https://doi.org/10.1155/2016/6584725

  25. Ronneberger O, Fischer P, Brox T (2015) Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  26. Sánchez FJ, Bernal J, Sánchez-Montes C, de Miguel CR, Fernández-Esparrach G (2017) Bright spot regions segmentation and classification for specular highlights detection in colonoscopy videos. Mach Vis Appl 28 (8):917–936. https://doi.org/10.1007/s00138-017-0864-0

    Article  Google Scholar 

  27. Sasmal P, Bhuyan MK, Bora K, Iwahori Y, Kasugai K (2019) Colonoscopic image polyp classification using texture features. In: International conference on pattern recognition and machine intelligence. Springer, pp 96–101

  28. Shin Y, Qadir HA, Aabakken L, Bergsland J, Balasingham I (2018) Automatic colon polyp detection using region based deep cnn and post learning approaches. IEEE Access 6:40950–40962. https://doi.org/10.1109/ACCESS.2018.2856402

    Article  Google Scholar 

  29. Stehle T, Auer R, Gross S, Behrens A, Wulff J, Aach T, Winograd R, Trautwein C, Tischendorf J (2009) Classification of colon polyps in nbi endoscopy using vascularization features. In: Medical imaging 2009: computer-aided diagnosis, international society for optics and Photonics, vol 7260, pp 72602S

  30. Sun M, Zhang G, Dang H, Qi X, Zhou X, Chang Q (2019) Accurate gastric cancer segmentation in digital pathology images using deformable convolution and multi-scale embedding networks. IEEE access 7:75530–75541. https://doi.org/10.1109/ACCESS.2019.2918800

    Article  Google Scholar 

  31. Wang P, Berzin TM, Brown JRG, Bharadwaj S, Becq A, Xiao X, Liu P, Li L, Song Y, Zhang D et al (2019) Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut 68(10):1813–1819. https://doi.org/10.1136/gutjnl-2018-317500

    Article  Google Scholar 

  32. Weng Y, Zhou T, Li Y, Qiu X (2019) Nas-unet: neural architecture search for medical image segmentation. IEEE Access 7:44247–44257. https://doi.org/10.1109/ACCESS.2019.2908991

    Article  Google Scholar 

  33. Yu B, Chen W, Zhong Q, Zhang H (2021) Specular highlight detection based on color distribution for endoscopic images. Frontiers in Physics 8:575

    Article  Google Scholar 

  34. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv:https://doi.org/10.48550/arXiv.1511.07122

  35. Zeng Z, Xie W, Zhang Y, Lu Y (2019) Ric-unet: an improved neural network based on unet for nuclei segmentation in histology images. Ieee Access 7:21420–21428. https://doi.org/10.1109/ACCESS.2019.2896920

    Article  Google Scholar 

  36. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2014) Object detectors emerge in deep scene cnns. arXiv:https://doi.org/10.48550/arXiv.1412.6856

Download references

Acknowledgements

Vanshali Sharma would like to thank INSPIRE fellowship scheme of Department of Science and Technology, Government of India for providing research fellowship.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology Guwahati, Guwahati, Assam, India

    Vanshali Sharma & Pradip K. Das

  2. Department of Electronics and Electrical Engineering, Indian Institute of Technology Guwahati, Guwahati, Assam, India

    M. K. Bhuyan

  3. Department of Computer Science and Information Technology, Cotton University, Guwahati, Assam, India

    Kangkana Bora

Authors
  1. Vanshali Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. K. Bhuyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pradip K. Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kangkana Bora
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vanshali Sharma.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharma, V., Bhuyan, M.K., Das, P.K. et al. A DWT-based encoder-decoder network for Specularity segmentation in colonoscopy images. Multimed Tools Appl 82, 40065–40084 (2023). https://doi.org/10.1007/s11042-023-14564-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14564-1

Keywords

Search

Navigation