Skip to main content
SpringerLink
Log in

A novel method for image segmentation: two-stage decoding network with boundary attention

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Medical image segmentation often suffers from the challenges of class imbalance, blurred target boundaries, and small data. How to establish a framework to automatically segment medical images with these problems is an important task. Although there have been some studies on the issue, there is still a large room for improving the efficiency and quality of medical service. This paper utilizes the powerful ability of deep learning to extract features, and develops a two-stage decoding network with boundary attention (TSD-BA), which can locate the regions of interest in the target locating stage and obtain more spatial structure features in the detail refinement stage. Specifically, a deep fusion model (DFM) is used to aggregate high-level semantic features for accurately capturing the position of targets. Subsequently, a boundary attention module (BAM) is applied to further excavate the boundary features. Moreover, data augmentation and transfer learning are employed to avoid overfitting caused by small datasets. Finally, a pixel position aware (PPA) loss is introduced to focus on hard pixels and mitigate the class imbalance issues. Numerous experimental results indicate that the proposed TSD-BA achieves the best performance compared with state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  2. Andrea H, Aranguren I, Oliva D, Abd Elaziz M, Cuevas E (2021) Efficient image segmentation through 2D histograms and an improved owl search algorithm. Int J Mach Learn Cybern 12(1):131–150

    Article  Google Scholar 

  3. Dehmeshki J, Amin H, Valdivieso M, Ye X (2008) Segmentation of pulmonary nodules in thoracic CT scans: a region growing approach. IEEE Trans Med Imaging 27(4):467–480

    Article  Google Scholar 

  4. De A, Guo C (2014) An image segmentation method based on the fusion of vector quantization and edge detection with applications to medical image processing. Int J Mach Learn Cybern 5(4):543–551

    Article  Google Scholar 

  5. Fang J, Liu H, Zhang L, Liu J, Liu H (2021) Region-edge-based active contours driven by hybrid and local fuzzy region-based energy for image segmentation. Inf Sci 546:397–419

    Article  MathSciNet  Google Scholar 

  6. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  7. Wang Z, Cai W, Smith CD, Kantake N, Rosol TJ, Liu J (2019) Residual pyramid FCN for robust follicle segmentation. In: Proceedings of IEEE Conference on International Symposium on Biomedical Imaging, Venice, Italy, pp 463–467

  8. Gao Y, Huang R, Yang Y, Zhang J, Shao K, Tao C, Chen Y, Metaxas DN, Li H, Chen M (2021) FocusNetv2: Imbalanced large and small organ segmentation with adversarial shape constraint for head and neck CT images. Med Image Anal 67:101831

    Article  Google Scholar 

  9. Peng D, Xiong S, Peng W, Lu J (2021) LCP-Net: a local context-perception deep neural network for medical image segmentation. Expert Syst Appl 168:114234

    Article  Google Scholar 

  10. Calisto MB, Lai-Yuen SK (2020) AdaEn-Net: an ensemble of adaptive 2D–3D fully convolutional networks for medical image segmentation. Neural Netw 126:76–94

    Article  Google Scholar 

  11. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, pp 3431–3440

  12. Bria A, Marrocco C, Tortorella F (2020) Addressing class imbalance in deep learning for small lesion detection on medical images. Comput Biol Med 120:103735

    Article  Google Scholar 

  13. Hsiao YH, Su CT, Fu PC (2020) Integrating mts with bagging strategy for class imbalance problems. Int J Mach Learn Cybern 11(6):1217–1230

    Article  Google Scholar 

  14. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B, Glocker B, Rueckert D et al (2018) Attention U-Net: learning where to look for the pancreas, arXiv preprint https://arxiv.org/abs/1804.03999

  15. Sinha A, Dolz J (2021) Multi-scale self-guided attention for medical image segmentation. IEEE J Biomed Health Inform 25(1):121–130

    Article  Google Scholar 

  16. Zhou C, Ding C, Wang X, Lu Z, Tao D (2020) One-pass multi-task networks with cross-task guided attention for brain tumor segmentation. IEEE Trans Image Process 29:4516–4529

    Article  Google Scholar 

  17. Roth HR, Oda H, Zhou X, Shimizu N, Yang Y, Hayashi Y, Oda M, Fujiwara M, Misawa K, Mori K (2018) An application of cascaded 3D fully convolutional networks for medical image segmentation. Comput Med Imaging Graph 66:90–99

    Article  Google Scholar 

  18. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, pp 234–241

  19. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Athens, Greece, pp 424–432

  20. Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of International Conference on 3D vision , Stanford, US, pp 565–571

  21. Seo H, Huang C, Bassenne M, Xiao R, Xing L (2019) Modified U-Net (mU-Net) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images. IEEE Trans Med Imaging 39(5):1316–1325

    Article  Google Scholar 

  22. Chen X, Zhang R, Yan P (2019) Feature fusion encoder decoder network for automatic liver lesion segmentation. In: Proceedings of IEEE Conference on International Symposium on Biomedical Imaging, pp 430–433

  23. Xiao X, Lian S, Luo Z, Li S (2018) Weighted Res-UNet for high-quality retina vessel segmentation. In: Proceedings of International Conference on Information Technology in Medicine and Education, Hangzhou, China, pp 327–331

  24. Ibtehaz N, Rahman MS (2020) MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87

    Article  Google Scholar 

  25. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867

    Article  Google Scholar 

  26. Lee CY, Xie S, Gallagher P, Zhang Z, Tu Z (2015) Deeply-supervised nets. In: Proceedings of Artificial Intelligence and Statistics, pp 562–570

  27. Qamar S, Jin H, Zheng R, Ahmad P, Usama M (2020) A variant form of 3D-UNet for infant brain segmentation. Futur Gener Comp Syst 108:613–623

    Article  Google Scholar 

  28. Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention BrainLes Workshop, Quebec City, Canada, pp 178–190

  29. Li X, Chen H, Qi X, Dou Q, Fu CW, Heng PA (2018) H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging 37(12):2663–2674

    Article  Google Scholar 

  30. Yu Q, Xie L, Wang Y, Zhou Y, Fishman EK, Yuille AL (2018) Recurrent saliency transformation network: Incorporating multi-stage visual cues for small organ segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Utah, pp 8280–8289

  31. Zhu Z, Xia Y, Shen W, Fishman E, Yuille A (2018) A 3D coarse-to-fine framework for volumetric medical image segmentation. In: Proceedings of International Conference on 3d Vision, Verona, Italy, pp 682–690

  32. Zhao Y, Li P, Gao C, Liu Y, Chen Q, Yang F, Meng D (2020) Tsasnet: tooth segmentation on dental panoramic X-ray images by two-stage attention segmentation network. Knowl-Based Syst 206:106338

    Article  Google Scholar 

  33. Sudre CH, Li W, Vercauteren T, Ourselin S, Cardoso MJ (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Proceedings of Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Quebec City, Canada, pp 240–248

  34. Rahman MA, Wang Y (2016) Optimizing intersection-over-union in deep neural networks for image segmentation. In: Proceedings of IEEE International Symposium on Visual Computing, Las Vegas, USA, pp 234–244

  35. Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of IEEE International Conference on Computer Vision, Venice, Italy, pp 2980–2988

  36. Liu S, Xu D, Zhou SK, Pauly O, Grbic S, Mertelmeier T, Wicklein J, Jerebko A, Cai W, Comaniciu D (2018) 3D anisotropic hybrid network: transferring convolutional features from 2D images to 3D anisotropic volumes. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 851–858

  37. Wong KC, Moradi M, Tang H, Syeda-Mahmood T (2018) 3D segmentation with exponential logarithmic loss for highly unbalanced object sizes. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain, pp 612–619

  38. Abraham N, Khan NM (2019) A novel focal tversky loss function with improved attention U-Net for lesion segmentation. In: Proceedings of IEEE Conference of International Symposium on Biomedical Imaging, Venice, Italy, pp 683–687

  39. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp 7132–7141

  40. Kaul C, Manandhar S, Pears N (2019) Focusnet: An attention-based fully convolutional network for medical image segmentation. In: Proceedings of IEEE Conference on International Symposium on Biomedical Imaging, IEEE, Venice, Italy, pp 455–458

  41. Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Learning a discriminative feature network for semantic segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp 1857–1866

  42. Chen S, Tan X, Wang B, Hu X (2018) Reverse attention for salient object detection. In: Proceedings of the European Conference on Computer Vision, Munich, Germany, pp 234–250

  43. Roy A, Navab N, Wachinger C (2019) Recalibrating fully convolutional networks with spatial and channel squeeze and excitation blocks. IEEE Trans Med Imaging 38(2):540

    Article  Google Scholar 

  44. Zhuang D, Jiang M, Kong J, Liu T (2021) Spatiotemporal attention enhanced features fusion network for action recognition. Int J Mach Learn Cybern 12(3):823–841

    Article  Google Scholar 

  45. Zhang H, Wu C, Zhang Z, Zhu Y, Zhang Z, Lin H, Sun Y, He T, Mueller J, Manmatha R, Li M, Smola A (2020) ResNeSt: Split-attention networks, arXiv preprint https://arxiv.org/abs/2004.08955

  46. Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, USA, pp 3907–3916

  47. Zhang Q, Shi Y, Zhang X (2020) Attention and boundary guided salient object detection. Pattern Recognit 107:107484

    Article  Google Scholar 

  48. Wei J, Wang S, Huang Q (2020) \(\text{ F}^3\)Net: fusion, feedback and focus for salient object detection. Proc AAAI Conf Artif Intell 3:12321–12328

    Google Scholar 

  49. Yap MH, Pons G, Martí J, Ganau S, Sentís M, Zwiggelaar R, Davison AK, Martí R (2017) Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J Biomed Health Inform 22(4):1218–1226

    Article  Google Scholar 

  50. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F (2015) WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graph 43:99–111

    Article  Google Scholar 

  51. Bernal J, Sánchez J, Vilarino F (2012) Towards automatic polyp detection with a polyp appearance model. Pattern Recognit 45(9):3166–3182

    Article  Google Scholar 

  52. Lu Z, Carneiro G, Bradley AP (2015) An improved joint optimization of multiple level set functions for the segmentation of overlapping cervical cells. IEEE Trans Image Process 24(4):1261–1272

    Article  MathSciNet  Google Scholar 

  53. Lu Z, Carneiro G, Bradley AP, Ushizima D, Nosrati MS, Bianchi AG, Carneiro CM, Hamarneh G (2017) Evaluation of three algorithms for the segmentation of overlapping cervical cells. IEEE J Biomed Health Inform 21(2):441–450

    Article  Google Scholar 

  54. Fan DP, Ji GP, Zhou T, Chen G, Fu H, Shen J, Shao L (2020) PraNet: Parallel reverse attention network for polyp segmentation. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, pp 263–273

  55. Ni J, Wu J, Tong J, Chen Z, Zhao J (2020) GC-Net: Global context network for medical image segmentation. Comput Meth Programs Biomed 190:105121

    Article  Google Scholar 

  56. Jha D, Smedsrud PH, Riegler MA, Johansen D, De Lange T, Halvorsen P, Johansen HD (2019) ResUNet++: an advanced architecture for medical image segmentation. In: Proceedings of IEEE International Symposium on Multimedia, San Diego, USA, pp 225–2255

Download references

Acknowledgements

This research was supported the Zhejiang Provincial Natural Science Foundation of China under grant LZ20F030001.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, China Jiliang University, Hangzhou, 310018, Zhejiang, China

    Feilong Cao, Chengling Gao & Hailiang Ye

Authors
  1. Feilong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengling Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hailiang Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feilong Cao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This study did not involve human participants and animals.

Informed consent

The all authors of this paper have consented the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, F., Gao, C. & Ye, H. A novel method for image segmentation: two-stage decoding network with boundary attention. Int. J. Mach. Learn. & Cyber. 13, 1461–1473 (2022). https://doi.org/10.1007/s13042-021-01459-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01459-6

Keywords

Search

Navigation