Skip to main content
SpringerLink
Log in

Semantic image segmentation with shared decomposition convolution and boundary reinforcement structure

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep convolutional neural networks (DCNNs) have shown excellent performances in the field of computer vision. In this paper, we propose a new semantic image segmentation model, and the two hallmarks of our architecture are the usage of shared decomposition convolution (SDC) operation and boundary reinforcement (BR) structure. SDC operation can extract dense features and increase correlation of features in the same group, which can relieve the grid artifact problem. BR structure combines the spatial information from different layers in DCNNs to enhance the spatial resolution and enrich target boundary position information simultaneously. The simulation results show that the proposed model can achieve 94.6% segmentation accuracy and 76.3% mIOU on PASCAL VOC 2012 database respectively, which verifies the effectiveness of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Karpathy A, Li FF (2015) Deep visual-semantic alignments for generating image descriptions. IEEE Trans Pattern Anal Mach Intell 39(4):664–676

    Article  Google Scholar 

  2. Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhutdinov R, Zemel R, Bengio Y (2015) Show, attend and tell: neural image caption generation with visual attention. In: Proceedings of the advances in international conference on machine learning, pp 2048–2057

  3. Aneja J, Deshpande A, Schwing AG (2018) Convolutional image captioning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5561–5570

  4. Papandreou G, Kokkinos I, Savalle PA (2015) Modeling local and global deformations in deep learning: epitomic convolution, multiple instance learning, and sliding window detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 390–399

  5. Dai J, Yi L, He K, Jian S (2016) R-FCN: object detection via region-based fully convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 379–387

  6. Wang C, Bai X, Wang S, Zhou J, Ren P (2019) Multiscale visual attention networks for object detection in VHR remote sensing images. IEEE Geosci Remote Sens Lett 16(2):310–314

    Article  Google Scholar 

  7. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv:1412.7062

  8. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  9. Qin AK, Clausi DA (2010) Multivariate image segmentation using semantic region growing with adaptive edge penalty. IEEE Trans Image Process 19(8):2157–2170

    Article  MathSciNet  Google Scholar 

  10. Jiang F, Grigorev A, Rho S, Tian Z, Fu YS, Jifara W, Adil K, Liu S (2018) Medical image semantic segmentation based on deep learning. Neural Comput Appl 29(5):1257–1265

    Article  Google Scholar 

  11. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  12. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of the advances in neural information processing systems, pp 1097–1105

  13. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke D, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  14. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 818–833

  15. Papandreou G, Chen LC, Murphy K, Yuille AL (2015) Weakly-and semi-supervised learning of a DCNN for semantic image segmentation. arXiv:1502.02734

  16. Lin G, Shen C, Reid I, Hengel AVD (2016) Efficient piecewise training of deep structured models for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3194–3203

  17. Zheng S, Jayasumana S, Romeraparedes B, Vineet V, Su Z, Du D, Chang H, Torr PHS (2015) Conditional random fields as recurrent neural networks. In: International conference on computer vision, pp 1529–1537

  18. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  19. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: International conference on computer vision, pp 1520–1528

  20. Fu J, Jing L, Wang Y, Lu H (2017) Stacked deconvolutional network for semantic segmentation. arXiv:1708.04943

  21. Caponetti L, Castellano G, Basile MT, Corsini V (2014) Fuzzy mathematical morphology for biological image segmentation. Appl Intell 41(1):117–127

    Article  Google Scholar 

  22. Chakraborty S, Thounaojam DM (2019) A novel shot boundary detection system using hybrid optimization technique. Appl Intell 49(9):3207–3220

    Article  Google Scholar 

  23. Ahmed KT, Irtaza A, Iqbal MA (2017) Fusion of local and global features for effective image extraction. Appl Intell 47(2):526– 543

    Article  Google Scholar 

  24. Kuang P, Ma T, Chen ZW, Li F (2019) Image super-resolution with densely connected convolutional networks. Appl Intell 49(1):125–136

    Article  Google Scholar 

  25. Li Y, Cao G, Yu Q, Li XS (2018) Active contours driven by non-local Gaussian distribution fitting energy for image segmentation. Appl Intell 48(12):4855–4870

    Article  Google Scholar 

  26. Deng C, Yang E, Liu T, Li J, Liu W, Tao D (2019) Unsupervised semantic-preserving adversarial hashing for image search. IEEE Trans Image Process 28(8):4032–4044

    Article  MathSciNet  Google Scholar 

  27. Yang E, Deng C, Li C, Liu W, Li J, Tao D (2018) Shared predictive cross-modal deep quantization. IEEE Trans Neural Netw Learn Syst 29(11):5292–5303

    Article  Google Scholar 

  28. Deng C, Yang E, Liu T, Tao D (2019) Two-stream deep hashing with class-specific centers for supervised image search. IEEE Trans Neural Netw Learn Syst pp. 1–13. https://doi.org/10.1109/TNNLS.2019.2929068

  29. Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the advances in European conference on computer vision, pp 801–818

  30. Liu C, Chen LC, Schroff F, Adam H, Li F (2019) Auto-deepLab: hierarchical neural architecture search for semantic image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 82–92

  31. Zhou XY, Zheng JQ, Yang GZ (2019) Atrous convolutional neural network (ACNN) for semantic image segmentation with full-scale feature maps. arXiv:1901.09203

  32. Wang P, Chen P, Ye Y, Ding L, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: Proceedings of the advances in winter conference on applications of computer vision, pp 1451–1460

  33. Sermanet P, Eigen D, Zhang X, Mathieu M, Lecun Y (2013) OverFeat: integrated recognition, localization and detection using convolutional networks. arXiv:1312.6229

  34. Ren Z, Kong Q, Han J, Plumbley MD, Schuller BW (2019) Attention-based atrous convolutional neural networks: visualisation and understanding perspectives of acoustic scenes. In: Proceedings of the advances in international conference on acoustics, speech and signal processing, pp 56–60

  35. Kalchbrenner N, Espeholt L, Simonyan K, Oord AVD, Graves A, Kavukcuoglu K (2016) Neural machine translation in linear time. arXiv:1610.10099

  36. Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587

  37. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  38. Yu F, Koltun V, Funkhouser T (2017) Dilated residual networks, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 472–480

  39. Hamaguchi R, Fujita A, Nemoto K, Imaizumi T, Hikosaka S (2018) Effective use of dilated convolutions for segmenting small object instances in remote sensing imagery. In: Proceedings of the advances in winter conference on applications of computer vision, pp 1442–1450

  40. Liu Z, Li X, Luo P, Loy C, Tang X (2015) Semantic image segmentation via deep parsing network. In: International conference on computer vision, pp 1377–1385

  41. Mostajabi M, Yadollahpour P, Shakhnarovich G (2015) Feedforward semantic segmentation with zoom-out features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3376–3385

  42. Ren S, He K, Girshick R, Zhang X, Sun J (2016) Object detection networks on convolutional feature maps. IEEE Trans Pattern Anal Mach Intell 39(7):1476–1481

    Article  Google Scholar 

  43. Zhang T, Lin G, Cai J, Shen T, Shen C, Kot AC (2019) Decoupled spatial neural attention for weakly supervised semantic segmentation. IEEE Trans Multimedia PP(99):1–11

    Google Scholar 

  44. Liu Y, Yu J, Han Y (2018) Understanding the effective receptive field in semantic image segmentation. Multimed Tools Appl 77(17):22159–22171

    Article  Google Scholar 

  45. Vemulapalli R, Tuzel O, Liu MY, Chellapa R (2016) Gaussian conditional random field network for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3224–3233

  46. Vo DM, Lee SW (2018) Semantic image segmentation using fully convolutional neural networks with multi-scale images and multi-scale dilated convolutions. Multimed Tools Appl 77(17):18689–18707

    Article  Google Scholar 

  47. Dvornik N, Shmelkov K, Mairal J, Schmid C (2017) BlitzNet: a real-time deep network for scene understanding. In: International conference on computer vision, pp 4154–4162

  48. Bansal R, Staib LH, Chen Z, Rangarajan A, Knisely J, Nath R, Duncan JS (2003) Entropy-based dual-portal-to-3-DCT registration incorporating pixel correlation. IEEE Trans Med Imaging 22(1):29–49

    Article  Google Scholar 

Download references

Acknowledgements

This study was funded by the National Key R&D Program of China (No. 2017YFF0108800), the Fundamental Research Funds for the Central Universities (No. N170504019), the National Natural Science Foundation of China (No. 61772125).

Author information

Authors and Affiliations

  1. College of Sciences, Northeastern University, Shenyang, 110819, China

    Hegui Zhu, Baoyu Wang & Xiangde Zhang

  2. College of Information Science and Engineering, Northeastern University, Shenyang, 110819, China

    Jinhai Liu

Authors
  1. Hegui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baoyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangde Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinhai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hegui Zhu.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, H., Wang, B., Zhang, X. et al. Semantic image segmentation with shared decomposition convolution and boundary reinforcement structure. Appl Intell 50, 2676–2689 (2020). https://doi.org/10.1007/s10489-020-01671-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01671-x

Keywords

Search

Navigation