Skip to main content
SpringerLink
Log in

DCANet: deep context attention network for automatic polyp segmentation

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Automatic and accurate polyp segmentation is significant for diagnosis and treatment of colorectal cancer. This is a challenging task due to the polyp’s shape and size diversity. Recently, various deep convolutional neural networks have been developed for polyp segmentation. However, most state-of-the-art methods have suffered from a poor performance in the segmentation of smaller, flat, or noisy polyp objects. In the paper, we propose a novel Deep Context Attention Network (DCANet) for accurate polyp segmentation based on an encoder–decoder framework. ResNet34 is adopted as the encoder, and five functional modules are introduced to improve the performance. Specifically, the improved local context attention module (LCAM) and global context module (GCM) are exploited to efficiently extract the local multi-scale and global multi-receptive-field context information, respectively. Then, an enhanced feature fusion module (FFM) is devised to effectively select and aggregate context features through spatial-channel attention. Finally, equipped with elaborately designed multi-attention modules (MAM), new decoder and supervision blocks are developed to accurately predict polyp regions via powerful channel-spatial-channel attention. Extensive experiments are conducted on the Kvasir-SEG and EndoScene benchmark datasets. The results demonstrate that the proposed network achieves superior performance compared to other state-of-the-art models. The source code will be available at https://github.com/ZAKAUDD/DCANet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Asif, M., Chen, L., Song, H., Yang, J., Frangi, Alejandro F.: An automatic framework for endoscopic image restoration and enhancement. Appl. Intell. 51(4), 1959–1971 (2021)

    Article  Google Scholar 

  2. Gloria R.: What is colorectal cancer?, 06 (2020). URL https://www.cancer.org/cancer/colon-rectal-cancer/about/what-is-colorectal-cancer.html#references

  3. Angelica, B., Yvette, B.: Colorectal cancer: symptoms, treatment, risk factors and more, May (2021). URL https://www.medicalnewstoday.com/articles/155598#treatment

  4. Tomar, N.K., Jha, D., Ali, S., Johansen, H.D., Johansen, D., Riegler, M.A., Halvorsen, P.: Ddanet: dual decoder attention network for automatic polyp segmentation. In International conference on pattern recognition 307–314 Springer, (2021)

  5. Zhang, W., Dai, Y., Liu, M., Chen, Y., Zhong, J., Yi, Z.: Deepuwf-plus: automatic fundus identification and diagnosis system based on ultrawide-field fundus imaging. Appl Intell 51(10), 7533–7551 (2021)

    Article  Google Scholar 

  6. Ahuja, Sakshi, Panigrahi, Bijaya Ketan, Dey, Nilanjan, Rajinikanth, Venkatesan, Gandhi, Tapan Kumar: Deep transfer learning-based automated detection of covid-19 from lung ct scan slices. Appl Intell 51(1), 571–585 (2021)

    Article  Google Scholar 

  7. Jorge, B., Aymeric, H., Marc, M., Quentin, A., Cristina, S.M., Cristina, R., Maroua, H., Ana, G.R., Henry, C., Olivier, R., et al.: Polyp detection benchmark in colonoscopy videos using gtcreator: a novel fully configurable tool for easy and fast annotation of image databases. In Proceedings of 32nd CARS conference, (2018)

  8. Bychkov, Dmitrii, Linder, Nina, Turkki, Riku, Nordling, Stig, Kovanen, Panu E., Verrill, Clare, Walliander, Margarita, Lundin, Mikael, Haglund, Caj, Lundin, Johan: Deep learning based tissue analysis predicts outcome in colorectal cancer. Scient. Rep. 8(1), 1–11 (2018)

    Google Scholar 

  9. Min, Min, Song, Su., He, Wenrui, Bi, Yiliang, Ma, Zhanyu, Liu, Yan: Computer-aided diagnosis of colorectal polyps using linked color imaging colonoscopy to predict histology. Scient. Rep. 9(1), 1–8 (2019)

    Google Scholar 

  10. Mori, Y., Kudo, S., Misawa, M., Saito, Y., Ikematsu, H., Hotta, K., Ohtsuka, K., Urushibara, F., Kataoka, S., Ogawa, Y., et al.: Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Annals Intern Med 169(6), 357–366 (2018)

    Article  Google Scholar 

  11. Pogorelov, K., Riegler, M., Eskeland, S.L., de Lange, T., Johansen, D., Griwodz, C., Schmidt, P.T., Halvorsen, P.l.: Efficient disease detection in gastrointestinal videos-global features versus neural networks. Mult. Tool. Appl. 76(21), 22493–22525 (2017)

  12. Pogorelov, K., Eskeland, S.L., de Lange T., Griwodz, C., Randel, K.R., Stensland, H.K., Dang-Nguyen, D.T., Spampinato, C., Johansen, D., Riegler, M., Halvorsen. P., et al.: A holistic multimedia system for gastrointestinal tract disease detection. In Proceedings of the 8th ACM on Multimedia Systems Conference, pages 112–123, (2017b)

  13. Silva, Juan, Histace, Aymeric, Romain, Olivier, Dray, Xavier, Granado, Bertrand: Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J Comp Assist Radiol Surg 9(2), 283–293 (2014)

    Article  Google Scholar 

  14. Wang, Yi., Tavanapong, Wallapak, Wong, Johnny, Jung Hwan, Oh., De Groen, Piet C.: Polyp-alert: near real-time feedback during colonoscopy. Comp Meth Progr Biomed 1200(3), 164–179 (2015)

    Article  Google Scholar 

  15. Jha, D., Smedsrud, P.H., Riegler, M.A., Johansen, D., de Lange, T., Halvorsen, P., Johansen, H.D.: Resunet++: an advanced architecture for medical image segmentation. In 2019 IEEE International symposium on multimedia (ISM), pages 225–2255. IEEE, (2019)

  16. Jha, D., Riegler, M.A., Johansen, D., Halvorsen, P., Johansen, H.D.: Doubleu-net: a deep convolutional neural network for medical image segmentation. In 2020 IEEE 33rd International symposium on computer-based medical systems (CBMS), pages 558–564. IEEE, (2020a)

  17. Guo, Yunbo, Bernal, Jorge, Matuszewski, Bogdan J.: Polyp segmentation with fully convolutional deep neural networks extended evaluation study. J Imag 6(7), 69 (2020)

    Article  Google Scholar 

  18. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention 234–241 Springer, (2015)

  19. Badrinarayanan, Vijay, Kendall, Alex, Cipolla, Roberto: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Patt Anal Mach Intell 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  20. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: a nested u-net architecture for medical image segmentation. In Deep learning in medical image analysis and multimodal learning for clinical decision support, pages 3–11. Springer, (2018)

  21. Zongwei Zhou, Md., Siddiquee, Mahfuzur Rahman, Tajbakhsh, Nima, Liang, Jianming: Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imag 390(6), 1856–1867 (2020). https://doi.org/10.1109/TMI.2019.2959609

    Article  Google Scholar 

  22. Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.W., Wu, J.: Unet 3+: a full-scale connected unet for medical image segmentation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1055–1059. IEEE, (2020)

  23. Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d u-net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assisted intervention, pages 424–432. Springer, (2016)

  24. Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565–571. IEEE, (2016)

  25. Xiao, X., Lian, S., Luo, Z., Li, S.: Weighted res-unet for high-quality retina vessel segmentation. In 2018 9th international conference on information technology in medicine and education (ITME), pages 327–331. IEEE, (2018)

  26. Fan, D.P., Ji, G.P., Zhou, T., Chen, G., Fu, H., Shen, J., Shao, L.: Pranet: parallel reverse attention network for polyp segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 263–273. Springer, (2020)

  27. Kim, T., Lee, H., Kim, D., Uacanet: Uncertainty augmented context attention for polyp segmentation. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 2167–2175, (2021)

  28. Li, Xiaomeng, Chen, Hao, Qi, Xiaojuan, Dou, Qi., Chi-Wing, Fu., Heng, Pheng-Ann.: H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes. IEEE Trans Med Imag 370(12), 2663–2674 (2018)

    Article  Google Scholar 

  29. Zhang, R., Li, G., Li, Z., Cui, S., Qian, D., Yu, Y., Adaptive context selection for polyp segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 253–262. Springer, (2020)

  30. Choudhury, A., Samanta, S., Pratihar, S., Bandyopadhyay, O.: Multilevel segmentation of hippocampus images using global steered quantum inspired firefly algorithm. Appl. Intell. 52(7), 7339–7372 (2021)

    Article  Google Scholar 

  31. Arnold, M., Sierra, M.S., Laversanne, M., Soerjomataram, I., Jemal, A., Bray, F.: Freddie Global patterns and trends in colorectal cancer incidence and mortality. Gut 66(4), 683–691 (2017)

    Article  Google Scholar 

  32. Leufkens, A.M., Van Oijen, M.G.H., Vleggaar, F.P., Siersema, P.D.: Factors influencing the miss rate of polyps in a back-to-back colonoscopy study. Endoscopy 44(05), 470–475 (2012)

    Article  Google Scholar 

  33. Karkanis, Stavros A., Iakovidis, Dimitrios K., Maroulis, Dimitrios E., Karras, Dimitris A., Tzivras, M.: Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans. Infor. Technol. Biomed. 7(3), 141–152 (2003)

    Article  Google Scholar 

  34. Shin, Younghak, Qadir, Hemin Ali, Balasingham, Ilangko: Abnormal colon polyp image synthesis using conditional adversarial networks for improved detection performance. IEEE Access 6, 56007–56017 (2018)

    Article  Google Scholar 

  35. Tajbakhsh, Nima, Gurudu, Suryakanth R., Liang, Jianming: Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med. Imag. 350(2), 630–644 (2015)

    Article  Google Scholar 

  36. Bernal, Jorge, Tajkbaksh, Nima, Sanchez, Francisco Javier, Matuszewski, Bogdan J., Chen, Hao, Lequan, Yu., Angermann, Quentin, Romain, Olivier, Rustad, Bjørn., Balasingham, Ilangko, et al.: Comparative validation of polyp detection methods in video colonoscopy: results from the miccai 2015 endoscopic vision challenge. IEEE Trans Med. Imag. 360(6), 1231–1249 (2017)

    Article  Google Scholar 

  37. Park, Sun Young, Sargent, Dustin, Spofford, Inbar, Vosburgh, Kirby G., Yousif, A., et al.: A colon video analysis framework for polyp detection. IEEE Trans. Biomed. Eng. 590(5), 1408–1418 (2012)

    Article  Google Scholar 

  38. Bernal, Jorge, Sánchez, Javier, Vilarino, Fernando: Towards automatic polyp detection with a polyp appearance model. Patt. Recognit. 45(9), 3166–3182 (2012)

    Article  Google Scholar 

  39. Brandao, P., Mazomenos, E., Ciuti, G., Caliò, R., Bianchi, F., Menciassi, A., Dario, P., Koulaouzidis, A., Arezzo, A., Stoyanov, D.: Fully convolutional neural networks for polyp segmentation in colonoscopy. In Medical imaging 2017: Computer-Aided Diagnosis, 10134, pp. 101–107. SPIE, (2017)

  40. Li, Q., Yang, G., Chen, Z., Huang, B., Chen, L., Xu, D., Zhou, X., Zhong, S., Zhang, H., Wang, T.: Colorectal polyp segmentation using a fully convolutional neural network. In 2017 10th international congress on image and signal processing, biomedical engineering and informatics (CISP-BMEI), pp. 1–5. IEEE, (2017)

  41. Zhong, J., Wang, W., Huisi, W., Wen, Z., Qin, J.: Polypseg: an efficient context-aware network for polyp segmentation from colonoscopy videos. In: International conference on medical image computing and computer-assisted intervention 285–294 Springer, (2020)

  42. Huisi, Wu., Zhong, Jiafu, Wang, Wei, Wen, Zhenkun, Qin, Jing: Precise yet efficient semantic calibration and refinement in convnets for real-time polyp segmentation from colonoscopy videos. Proceed. AAAI Conf. Artif Intell. 35, 2916–2924 (2021)

    Google Scholar 

  43. Fang, Y., Chen, C., Yuan, Y., Tong, K.U.: Selective feature aggregation network with area-boundary constraints for polyp segmentation. In: International conference on medical image computing and computer-assisted intervention 302–310 Springer, (2019)

  44. Murugesan, B., Sarveswaran, K., Shankaranarayana, S.M., Ram, K., Joseph, J., Sivaprakasam, M., Psi-net: shape and boundary aware joint multi-task deep network for medical image segmentation. In 2019 41st Annual international conference of the IEEE engineering in medicine and biology society (EMBC), pages 7223–7226. IEEE, (2019)

  45. Lai, H., Luo, Y., Zhang, G., Shen, X., Li, B., Jianwei, L.: Toward accurate polyp segmentation with cascade boundary-guided attention. The Visual Computer , 1–17 (2022)

  46. Wang, P., Li, Y., Sun, Y., He, D., Wang, Z.: Multi-scale boundary neural network for gastric tumor segmentation. The Visual Computer, 1–12 (2022)

  47. Zhuang, H., Zhang, J., Liao, F.: A systematic review on application of deep learning in digestive system image processing. The Visual Computer 1–16 (2021)

  48. Mahmud, Tanvir, Paul, Bishmoy, Fattah, Shaikh Anowarul: Polypsegnet: a modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images. Comp. Biol. Med. 128, 104119 (2021)

    Article  Google Scholar 

  49. Tomar, N.K., Jha, D., Riegler, M.A., Johansen, H.D., Johansen, D., Rittscher, J., Halvorsen, P., Ali, S.: Fanet: a feedback attention network for improved biomedical image segmentation. IEEE Trans. Neur. Netw. Learn. Sys. (2022). https://doi.org/10.1109/TNNLS.2022.3159394

    Article  Google Scholar 

  50. Ji, G.P., Chou, Y.C., Fan, D.P., Chen, G., Jha, D., Fu, H., Shao, L.: Progressively normalized self-attention network for video polyp segmentation. In MICCAI, (2021)

  51. Shen, Y., Jia, X., Meng, M.Q.: Hrenet: a hard region enhancement network for polyp segmentation. In: International conference on medical image computing and computer-assisted intervention 559–568 springer, (2021)

  52. Jha, D., Ali, S., Tomar, N.K., Johansen, H.D., Johansen, D., Rittscher, J., Riegler, M.A., Halvorsen, P.: Real-time polyp detection, localization and segmentation in colonoscopy using deep learning. IEEE Access 9, 40496–40510 (2021)

    Article  Google Scholar 

  53. Huang, C.H., Wu, H.Y., Lin, Y.L.: Hardnet-mseg: a simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps. arXiv preprint arXiv:2101.07172, (2021)

  54. Chao, P., Kao, C.Y., Ruan, Y.S., Huang, C.H., Lin, Y.L.: Hardnet: a low memory traffic network. In: Proceedings of the IEEE/CVF international conference on computer vision , 3552–3561 (2019)

  55. Yeung, Michael, Sala, Evis, Schönlieb, Carola-Bibiane., Rundo, Leonardo: Focus u-net: a novel dual attention-gated cnn for polyp segmentation during colonoscopy. Comp Biol Med 137, 104815 (2021)

    Article  Google Scholar 

  56. Zhang, Y., Liu, H., Hu, Q.: Transfuse: fusing transformers and cnns for medical image segmentation. arXiv preprint arXiv:2102.08005, (2021b)

  57. Li, S., Sui, X., Luo, X., Xu, X., Liu, Y., Goh, R.: Medical image segmentation using squeeze-and-expansion transformers. arXiv preprint arXiv:2105.09511, (2021)

  58. Dong, B., Wang, W., Fan, D.P., Li, J., Fu, H., Shao, L.: Polyp-pvt: polyp segmentation with pyramid vision transformers. arXiv preprint arXiv:2108.06932, (2021)

  59. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016)

  60. Wang, Y., Zhou, Q., Xiong, J., Xiaofu, W., Jin, X.: Esnet: an efficient symmetric network for real-time semantic segmentation. In Chinese conference on pattern recognition and computer vision (PRCV) 41–52 Springer, (2019)

  61. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition 7794–7803 (2018)

  62. Sanghyun W., Jongchan P., Joon-Young L., In-So, K.: Cbam. In Proceedings of the european conference on computer vision (ECCV) , 3–19 (2018)

  63. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June (2018)

  64. Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., Lange, T.D., Johansen, D., Johansen, H.D.: Kvasir-seg: a segmented polyp dataset. In: International conference on multimedia modeling. 451–462 Springer, (2020)

  65. Vázquez D, Bernal J, Sánchez FJ, Fernández-Esparrach G, López AM, Romero A, Drozdzal M, Courville A (2017) A benchmark for endoluminal scene segmentation of colonoscopy images. J. Health. Eng., 10.1155/2017/4037190

  66. Jorge Bernal, F., Sánchez, Javier, Fernández-Esparrach, Gloria, Gil, Debora, Rodríguez, Cristina, Vilariño, Fernando: Wm-dova maps for accurate polyp highlighting in colonoscopy: validation vs saliency maps from physicians. Comput Med Imag Graph 43, 99–111 (2015)

    Article  Google Scholar 

  67. Wickstrøm, Kristoffer, Kampffmeyer, Michael, Jenssen, Robert: Uncertainty and interpretability in convolutional neural networks for semantic segmentation of colorectal polyps. Med Image Anal 60, 101619 (2020)

    Article  Google Scholar 

  68. Wei, J., Hu, Y., Zhang, R., Li, Z., Zhou, S.K., Cui, S.: Shallow attention network for polyp segmentation. In: International conference on medical image computing and computer-assisted intervention 699–708 Springer, (2021)

  69. Tan-Cong, N., Tien-Phat, N., Gia-Han, D., Anh-Huy, T.D., Tam, V.N., Minh-Triet, T.: Ccbanet. In International conference on medical image computing and computer-assisted intervention 633–643 Springer, (2021)

  70. Zhang, Y., Liu, H., Hu, Q.: Transfuse: Fusing transformers and cnns for medical image segmentation. In Marleen de Bruijne, Philippe C. Cattin, Stéphane Cotin, Nicolas Padoy, Stefanie Speidel, Yefeng Zheng, and Caroline Essert, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2021, pages 14–24, Cham, (2021c). Springer International Publishing. ISBN 978-3-030-87193-2

  71. Akbari, M., Mohrekesh, M., Nasr-Esfahani, E., Soroushmehr, S.R., Karimi, N., Samavi. S., Najarian KPolyp segmentation in colonoscopy images using fully convolutional network. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 69–72. IEEE, (2018)

  72. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2 Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520 (2018)

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Nos. 71991464/71991460 and 61877056).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, University of Science and Technology of China, Hefei, 230027, China

    Zaka-Ud-Din Muhammad, Zhangjin Huang & Naijie Gu

  2. Anhui Province Key Laboratory of Software in Computing and Communication, Hefei, 230027, China

    Zaka-Ud-Din Muhammad, Zhangjin Huang & Naijie Gu

  3. USTC-Deqing Alpha Innovation Institute, Huzhou, 313299, China

    Zhangjin Huang

  4. Department of Automation, University of Science and Technology of China, Hefei, 230027, China

    Usman Muhammad

Authors
  1. Zaka-Ud-Din Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhangjin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Naijie Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Usman Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhangjin Huang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This articles does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Muhammad, ZUD., Huang, Z., Gu, N. et al. DCANet: deep context attention network for automatic polyp segmentation. Vis Comput 39, 5513–5525 (2023). https://doi.org/10.1007/s00371-022-02677-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02677-x

Keywords

Search

Navigation