Skip to main content
SpringerLink
Log in

Semantic image segmentation with fused CNN features

  • Published:
Optoelectronics Letters Aims and scope Submit manuscript

Abstract

Semantic image segmentation is a task to predict a category label for every image pixel. The key challenge of it is to design a strong feature representation. In this paper, we fuse the hierarchical convolutional neural network (CNN) features and the region-based features as the feature representation. The hierarchical features contain more global information, while the region-based features contain more local information. The combination of these two kinds of features significantly enhances the feature representation. Then the fused features are used to train a softmax classifier to produce per-pixel label assignment probability. And a fully connected conditional random field (CRF) is used as a post-processing method to improve the labeling consistency. We conduct experiments on SIFT flow dataset. The pixel accuracy and class accuracy are 84.4% and 34.86%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. Bu, P. Han, Z. Liu and J. Han, Pattern Recognition 59, 188 (2016).

    Article  Google Scholar 

  2. Y. Bengio, Found. Trends Mach. Learn. 2, 1 (2009).

    Article  Google Scholar 

  3. S. Zheng, S. Jayasuamana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang and P. Torr, Conditional Random Fields as Recurrent Neural Networks, 1529 (2015).

    Google Scholar 

  4. L-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A.L. Yuille, Computer Science 4, 357 (2014).

    Google Scholar 

  5. J. Long, E. Shelhamer and T. Darrell, Fully Convolutional Networks for Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition, 2015.

    Google Scholar 

  6. H. Caesar, U. Jasper and V. Ferrari, Region-Based Semantic Segmentation with End-to-End Training, Computer Vision–ECCV 2016, Springer International Publishing, 2016.

    Google Scholar 

  7. B. Hariharan, P. Arbelaez, R. Girshick and J. Malik, Simultaneous Detection and Segmentation, European Conference on Computer Vision, 297 (2014).

    Google Scholar 

  8. J. Carreira, R. Caseiro, R. J. Batista and C. Sminchisescu, Semantic Segmentation with Second-Order Pooling, European Conference on Computer Vision, 430 (2012).

    Google Scholar 

  9. J. Uijlings, K. van de Sande, T. Gevers and A. Smeulders, International Journal of Computer Vision 104, 154 (2013).

    Article  Google Scholar 

  10. R. Girshick, Fast R-CNN, Computer Science, 2015.

    Google Scholar 

  11. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua and S. Susstrunk, IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274 (2012).

    Article  Google Scholar 

  12. P. Krahenbuhl and V. Koltun, Efficient Inference in Fully Connected CRFS with Gaussian Edge Potentials, The Neural Information Processing Systems Conference, 109 (2012).

    Google Scholar 

  13. C. Liu, J. Yuen and A. Torralba, IEEE Transactions on Pattern Analysis & Machine Intelligence 33, 2368 (2011).

    Article  Google Scholar 

  14. A. Vedaldi and K. Lenc, Matconvnet Convolutional Neural Networks for Matlab, CoRR abs/1412.4564, 689 (2014).

    Google Scholar 

  15. C. Farabet, C. Couprie, L. Najman and Y. LeCun, IEEE Transactions on Pattern Analysis & Machine Intelligence 35, 1915 (2013).

    Article  Google Scholar 

  16. G. Singh and J. Kosecka, Nonparametric Scene Parsing with Adaptive Feature Relevance and Semantic Context, IEEE Computer Vision and Pattern Recognition, 3151 (2013).

    Google Scholar 

  17. S. Gould, J. Zhao and X. He, Superpixel Graph Label Transfer with Learned Distance Metric, European Conference on Computer Vision, 632 (2014).

    Google Scholar 

  18. C. Gatta, A. Romero and J. van de Veijer, Unrolling Loopy Top-Down Semantic Feedback in Convolutional Deep Networks, IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE Computer Society, 504 (2014).

    Google Scholar 

  19. P. Pinheiro and R. Collobert, Recurrent Convolutional Neural Networks for Scene Labeling, International Conference on Machine Learning, 82 (2014).

    Google Scholar 

  20. W. Byeon, T.M. Breuel, F. Raue and M. Liwicki, Scene Labeling with LSTM Recurrent Neural Network, IEEE Computer Vision and Pattern Recognition, 3547 (2015).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Computer Vision and System of Ministry of Education, Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384, China

    Hui-qiang Geng  (耿慧强), Hua Zhang  (张桦), Yan-bing Xue  (薛彦兵), Mian Zhou  (周冕), Guang-ping Xu  (徐光平) & Zan Gao  (高赞)

Authors
  1. Hui-qiang Geng  (耿慧强)
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hua Zhang  (张桦)
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan-bing Xue  (薛彦兵)
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mian Zhou  (周冕)
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guang-ping Xu  (徐光平)
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zan Gao  (高赞)
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hua Zhang  (张桦).

Additional information

This work has been supported by the National Natural Science Foundation of China (Nos.U1509207, 61325019, 61472278, 61403281 and 61572357), and the Key Project of Natural Science Foundation of Tianjin (No.14JCZDJC31700).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Geng, Hq., Zhang, H., Xue, Yb. et al. Semantic image segmentation with fused CNN features. Optoelectron. Lett. 13, 381–385 (2017). https://doi.org/10.1007/s11801-017-7086-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11801-017-7086-6

Search

Navigation