Skip to main content
SpringerLink
Log in

Toward accurate polyp segmentation with cascade boundary-guided attention

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In clinical practice, accurate polyp segmentation provides important information for the early detection of colorectal cancer. Benefiting from the advancement of deep learning techniques, various neural networks have been developed for polyp segmentation. However, most state-of-the-art methods have suffered from the challenge of precisely segmenting polyps with clear boundaries. To tackle this challenge, in this paper, we propose a novel and effective cascade boundary-guided attention network based on an encoder–decoder framework. Specifically, instead of just using the addition of shallow and deep features, the fine-grained boundary information is explicitly introduced into the skip connection of encoder and decoder layers to achieve accurate polyp segmentation. Moreover, the cascade refinement strategy is utilized into the multi-stage enhancement of boundary features to progressively produce better predictions. Extensive evaluations on five public benchmark datasets show that our method outperforms state-of-the-arts on various polyp segmentation tasks. Further experiments conducted on the cross-dataset (training on one dataset and testing on another dataset) validate the generalization ability of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ameling, S., Wirth, S., Paulus, D., Lacey, G., Vilarino, F.: Texture-based polyp detection in colonoscopy. In: Bildverarbeitung für die Medizin 2009, pp. 346–350. Springer (2009)

  2. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  3. Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., Gil, D., Rodríguez, C., Vilariño, F.: Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized Medical Imaging and Graphics 43, 99–111 (2015)

    Article  Google Scholar 

  4. Bernal, J., Sánchez, J., Vilarino, F.: Towards automatic polyp detection with a polyp appearance model. Pattern Recognition 45(9), 3166–3182 (2012)

    Article  Google Scholar 

  5. Brandao, P., Mazomenos, E., Ciuti, G., Caliò, R., Bianchi, F., Menciassi, A., Dario, P., Koulaouzidis, A., Arezzo, A., Stoyanov, D.: Fully convolutional neural networks for polyp segmentation in colonoscopy. In: Medical Imaging 2017: Computer-Aided Diagnosis, vol. 10134, p. 101340F. International Society for Optics and Photonics (2017)

  6. Chao, P., Kao, C.Y., Ruan, Y.S., Huang, C.H., Lin, Y.L.: Hardnet: A low memory traffic network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3552–3561 (2019)

  7. Chen, S., Tan, X., Wang, B., Hu, X.: Reverse attention for salient object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 234–250 (2018)

  8. Cheng, Z., Qu, A., He, X.: Contour-aware semantic segmentation network with spatial attention mechanism for medical image. The Visual Computer pp. 1–14 (2021)

  9. De Boer, P.T., Kroese, D.P., Mannor, S., Rubinstein, R.Y.: A tutorial on the cross-entropy method. Annals of operations research 134(1), 19–67 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  10. Fan, D.P., Ji, G.P., Zhou, T., Chen, G., Fu, H., Shen, J., Shao, L.: Pranet: Parallel reverse attention network for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 263–273. Springer (2020)

  11. Fang, Y., Chen, C., Yuan, Y., Tong, K.y.: Selective feature aggregation network with area-boundary constraints for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 302–310. Springer (2019)

  12. Fang, Y., Zhu, D., Yao, J., Yuan, Y., Tong, K.Y.: Abc-net: Area-boundary constraint network with dynamical feature selection for colorectal polyp segmentation. IEEE Sensors Journal 21(10), 11799–11809 (2020)

    Article  Google Scholar 

  13. Galdran, A., Carneiro, G., Ballester, M.A.G.: Double encoder-decoder networks for gastrointestinal polyp segmentation. In: International Conference on Pattern Recognition, pp. 293–307. Springer (2021)

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)

  15. Ho, J., Kalchbrenner, N., Weissenborn, D., Salimans, T.: Axial attention in multidimensional transformers. arXiv preprint arXiv:1912.12180 (2019)

  16. Huang, L., Liu, F.: Retinal vessel segmentation using simple spcnn model and line connector. The Visual Computer pp. 1–14 (2020)

  17. Huang, S.W., Zhang, A., Tian, X.L., Sun, Y.K.: Dynamic adaptive weight multi-scale and multi-structure morphological edge detection in anterior chamber oct images. In: Advanced Materials Research, vol. 340, pp. 70–75. Trans Tech Publ (2012)

  18. Hwang, S., Oh, J., Tavanapong, W., Wong, J., De Groen, P.C.: Polyp detection in colonoscopy video using elliptical shape feature. In: 2007 IEEE International Conference on Image Processing, vol. 2, pp. II–465. IEEE (2007)

  19. Jha, D., Smedsrud, P.H., Riegler, M.A., Johansen, D., De Lange, T., Halvorsen, P., Johansen, H.D.: Resunet++: An advanced architecture for medical image segmentation. In: 2019 IEEE International Symposium on Multimedia (ISM), pp. 225–2255. IEEE (2019)

  20. Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., Johansen, H.D.: Kvasir-seg: A segmented polyp dataset. In: International Conference on Multimedia Modeling, pp. 451–462. Springer (2020)

  21. Jha, D., Smedsrud, P.H., Johansen, D., de Lange, T., Johansen, H.D., Halvorsen, P., Riegler, M.A.: A comprehensive study on colorectal polyp segmentation with resunet++, conditional random field and test-time augmentation. IEEE journal of biomedical and health informatics 25(6), 2029–2040 (2021)

    Article  Google Scholar 

  22. Jiang, M., Zhai, F., Kong, J.: Sparse attention module for optimizing semantic segmentation performance combined with a multi-task feature extraction network. The Visual Computer pp. 1–16 (2021)

  23. Karkanis, S.A., Iakovidis, D.K., Maroulis, D.E., Karras, D.A., Tzivras, M.: Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE transactions on information technology in biomedicine 7(3), 141–152 (2003)

    Article  Google Scholar 

  24. Mahmud, T., Paul, B., Fattah, S.A.: Polypsegnet: A modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images. Computers in Biology and Medicine 128, 104119 (2021)

    Article  Google Scholar 

  25. Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV), pp. 565–571. IEEE (2016)

  26. Murugesan, B., Sarveswaran, K., Shankaranarayana, S.M., Ram, K., Joseph, J., Sivaprakasam, M.: Psi-net: Shape and boundary aware joint multi-task deep network for medical image segmentation. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 7223–7226. IEEE (2019)

  27. Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., et al.: Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)

  28. Pal, S., Chatterjee, S., Dey, D., Munshi, S.: Morphological operations with iterative rotation of structuring elements for segmentation of retinal vessel structures. Multidimensional Systems and Signal Processing 30(1), 373–389 (2019)

    Article  MATH  Google Scholar 

  29. Qadir, H.A., Shin, Y., Solhusvik, J., Bergsland, J., Aabakken, L., Balasingham, I.: Polyp detection and segmentation using mask r-cnn: Does a deeper feature extractor cnn always perform better? In: 2019 13th International Symposium on Medical Information and Communication Technology (ISMICT), pp. 1–6. IEEE (2019)

  30. Qian, X., Quan, H., Wu, M.: Prnet: polar regression network for medical image segmentation. The Visual Computer pp. 1–12 (2021)

  31. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28, 91–99 (2015)

    Google Scholar 

  32. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Springer (2015)

  33. Rutter, C.M., Johnson, E., Miglioretti, D.L., Mandelson, M.T., Inadomi, J., Buist, D.S.: Adverse events after screening and follow-up colonoscopy. Cancer Causes & Control 23(2), 289–296 (2012)

    Article  Google Scholar 

  34. Silva, J., Histace, A., Romain, O., Dray, X., Granado, B.: Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. International journal of computer assisted radiology and surgery 9(2), 283–293 (2014)

    Article  Google Scholar 

  35. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818–2826 (2016)

  36. Tajbakhsh, N., Gurudu, S.R., Liang, J.: Automated polyp detection in colonoscopy videos using shape and context information. IEEE transactions on medical imaging 35(2), 630–644 (2015)

    Article  Google Scholar 

  37. Vázquez, D., Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., López, A.M., Romero, A., Drozdzal, M., Courville, A.: A benchmark for endoluminal scene segmentation of colonoscopy images. Journal of healthcare engineering 2017 (2017)

  38. Wang, D., Hu, G., Lyu, C.: Frnet: an end-to-end feature refinement neural network for medical image segmentation. The Visual Computer 37, 1101–1112 (2021)

    Article  Google Scholar 

  39. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7794–7803 (2018)

  40. Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, pp. 1398–1402. Ieee (2003)

  41. Wu, Z., Su, L., Huang, Q.: Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3907–3916 (2019)

  42. Yeung, M., Sala, E., Schönlieb, C.B., Rundo, L.: Focus u-net: A novel dual attention-gated cnn for polyp segmentation during colonoscopy. Computers in Biology and Medicine 137, 104815 (2021)

    Article  Google Scholar 

  43. Yue, K., Sun, M., Yuan, Y., Zhou, F., Ding, E., Xu, F.: Compact generalized non-local network. arXiv preprint arXiv:1810.13125 (2018)

  44. Zhang, G., Shen, X., Zhang, Y., Luo, Y., Luo, J., Zhu, D., Yang, H., Wang, W., Zhao, B., Lu, J.: Cross-modal prostate cancer segmentation via self-attention distillation. IEEE Journal of Biomedical and Health Informatics pp. 1–1 (2021). https://doi.org/10.1109/JBHI.2021.3127688

  45. Zhang, R., Li, G., Li, Z., Cui, S., Qian, D., Yu, Y.: Adaptive context selection for polyp segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 253–262. Springer (2020)

  46. Zhang, Z., Liu, Q., Wang, Y.: Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters 15(5), 749–753 (2018)

    Article  Google Scholar 

  47. Zhao, Y., Gui, W., Chen, Z.: Edge detection based on multi-structure elements morphology. In: 2006 6th World Congress on Intelligent Control and Automation, vol. 2, pp. 9795–9798. IEEE (2006)

  48. Zhou, Y., Onder, O.F., Dou, Q., Tsougenis, E., Chen, H., Heng, P.A.: Cia-net: Robust nuclei instance segmentation with contour-aware information aggregation. In: International Conference on Information Processing in Medical Imaging, pp. 682–693. Springer (2019)

  49. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 3–11. Springer (2018)

  50. Zhu, L., She, Q., Li, D., Lu, Y., Kang, X., Hu, J., Wang, C.: Unifying nonlocal blocks for neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12292–12301 (2021)

Download references

Acknowledgements

This work was supported by the General Program of National Natural Science Foundation of China (NSFC) (Grant No. 61806147), the Shanghai Sailing Program (21YF1431600), the General Program of National Natural Science Foundation of China (NSFC) (Grant No. 62102259), and the State Key Laboratory of Robotics (2019-O15).

Author information

Authors and Affiliations

  1. School of Software Engineering, Tongji University, Shanghai, 200092, China

    Huilin Lai, Ye Luo, Xiaoang Shen, Bo Li & Jianwei Lu

  2. State key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, 110016, China

    Ye Luo

  3. School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Guokai Zhang

  4. College of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine, Shanghai, 201203, China

    Jianwei Lu

  5. Engineering Research Center of Traditional Chinese Medicine Intelligent Rehabilitation, Ministry of Education, Shanghai, 201203, China

    Jianwei Lu

Authors
  1. Huilin Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ye Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guokai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoang Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianwei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ye Luo or Jianwei Lu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This paper does not contain any studies with human participants or animals performed by any of the authors. The experimental samples used in this paper are all collected from the public database.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lai, H., Luo, Y., Zhang, G. et al. Toward accurate polyp segmentation with cascade boundary-guided attention. Vis Comput 39, 1453–1469 (2023). https://doi.org/10.1007/s00371-022-02422-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02422-4

Keywords

Search

Navigation