Abstract
Semantic image segmentation is a task to predict a category label for every image pixel. The key challenge of it is to design a strong feature representation. In this paper, we fuse the hierarchical convolutional neural network (CNN) features and the region-based features as the feature representation. The hierarchical features contain more global information, while the region-based features contain more local information. The combination of these two kinds of features significantly enhances the feature representation. Then the fused features are used to train a softmax classifier to produce per-pixel label assignment probability. And a fully connected conditional random field (CRF) is used as a post-processing method to improve the labeling consistency. We conduct experiments on SIFT flow dataset. The pixel accuracy and class accuracy are 84.4% and 34.86%, respectively.
Similar content being viewed by others
References
S. Bu, P. Han, Z. Liu and J. Han, Pattern Recognition 59, 188 (2016).
Y. Bengio, Found. Trends Mach. Learn. 2, 1 (2009).
S. Zheng, S. Jayasuamana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang and P. Torr, Conditional Random Fields as Recurrent Neural Networks, 1529 (2015).
L-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A.L. Yuille, Computer Science 4, 357 (2014).
J. Long, E. Shelhamer and T. Darrell, Fully Convolutional Networks for Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition, 2015.
H. Caesar, U. Jasper and V. Ferrari, Region-Based Semantic Segmentation with End-to-End Training, Computer Vision–ECCV 2016, Springer International Publishing, 2016.
B. Hariharan, P. Arbelaez, R. Girshick and J. Malik, Simultaneous Detection and Segmentation, European Conference on Computer Vision, 297 (2014).
J. Carreira, R. Caseiro, R. J. Batista and C. Sminchisescu, Semantic Segmentation with Second-Order Pooling, European Conference on Computer Vision, 430 (2012).
J. Uijlings, K. van de Sande, T. Gevers and A. Smeulders, International Journal of Computer Vision 104, 154 (2013).
R. Girshick, Fast R-CNN, Computer Science, 2015.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua and S. Susstrunk, IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274 (2012).
P. Krahenbuhl and V. Koltun, Efficient Inference in Fully Connected CRFS with Gaussian Edge Potentials, The Neural Information Processing Systems Conference, 109 (2012).
C. Liu, J. Yuen and A. Torralba, IEEE Transactions on Pattern Analysis & Machine Intelligence 33, 2368 (2011).
A. Vedaldi and K. Lenc, Matconvnet Convolutional Neural Networks for Matlab, CoRR abs/1412.4564, 689 (2014).
C. Farabet, C. Couprie, L. Najman and Y. LeCun, IEEE Transactions on Pattern Analysis & Machine Intelligence 35, 1915 (2013).
G. Singh and J. Kosecka, Nonparametric Scene Parsing with Adaptive Feature Relevance and Semantic Context, IEEE Computer Vision and Pattern Recognition, 3151 (2013).
S. Gould, J. Zhao and X. He, Superpixel Graph Label Transfer with Learned Distance Metric, European Conference on Computer Vision, 632 (2014).
C. Gatta, A. Romero and J. van de Veijer, Unrolling Loopy Top-Down Semantic Feedback in Convolutional Deep Networks, IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE Computer Society, 504 (2014).
P. Pinheiro and R. Collobert, Recurrent Convolutional Neural Networks for Scene Labeling, International Conference on Machine Learning, 82 (2014).
W. Byeon, T.M. Breuel, F. Raue and M. Liwicki, Scene Labeling with LSTM Recurrent Neural Network, IEEE Computer Vision and Pattern Recognition, 3547 (2015).
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been supported by the National Natural Science Foundation of China (Nos.U1509207, 61325019, 61472278, 61403281 and 61572357), and the Key Project of Natural Science Foundation of Tianjin (No.14JCZDJC31700).
Rights and permissions
About this article
Cite this article
Geng, Hq., Zhang, H., Xue, Yb. et al. Semantic image segmentation with fused CNN features. Optoelectron. Lett. 13, 381–385 (2017). https://doi.org/10.1007/s11801-017-7086-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11801-017-7086-6