Abstract
The context information of images had been lost due to the low resolution of features, and due to repeated combinations of max-pooling layer and down-sampling layer. When the feature extraction process had been performed using a convolutional network, the result of semantic image segmentation loses sensitivity to the location of the object. The semantic image segmentation based on a feature fusion model with context features layer-by-layer had been proposed. Firstly, the original images had been pre-processed by the Gaussian Kernel Function to generate a series of images with different resolutions to form an image pyramid. Secondly, inputting an image pyramid into the network structure in which the plurality of fully convolutional network was been combined in parallel to obtain a set of initial features with different granularities by expanding receptive fields using Atrous Convolutions, and the initialization of feature fusion with different layer-by-layer granularities in a top-down method. Finally, the score map of feature fusion model had been calculated and sent to the conditional random field, modeling the class correlations between image pixels of the original image by the fully connected conditional random field, and the spatial position information and color vector information of image pixels were jointed to optimize and obtain results. The experiments on the PASCAL VOC 2012 and PASCAL Context datasets had achieved better mean Intersection Over Union than the state-of-the-art works. The proposed method has about 6.3% improved to the conventional methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adams A, Baek J, Davis MA (2010) Fast high-dimensional filtering using the permutohedral lattice. Comput Gr Forum 29(2):753–762. https://doi.org/10.1111/j.1467-8659.2009.01645.x
Bertasius G, Torresani L, Yu SX, Shi JB (2017) Convolutional random walk networks for semantic image segmentation. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 6137–6145. https://doi.org/10.1109/CVPR.2017.650
Bulo SR, Neuhold G, Kontschieder P (2017) Loss max-pooling for semantic image segmentation. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 7082–7091. https://doi.org/10.1109/CVPR.2017.749
Chen LC, Papandreou G, Kokkinos I, Mruphy K, Yuille AL (2016) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184
Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. ArXiv, preprint arXiv:1706.05587, https://arxiv.org/abs/1706.05587. Accessed 5 Dec 2017
Chen LC, Zhu YK, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of IEEE conference on european conference on computer vision, pp 833–851. https://doi.org/10.1007/978-3-030-01234-2_49
Chen YT, Xiong J, Xu WH, Zuo JW (2019a) A novel online incremental and decremental learning algorithm based on variable support vector machine. Clust Comput 22:7435–7445. https://doi.org/10.1007/s10586-018-1772-4
Chen YT, Wang J, Xia RL, Zhang Q, Cao ZH, Yang K (2019b) The visual object tracking algorithm research based on adaptive combination kernel. J Ambient Intell Hum Comput 10(12):4855–4867. https://doi.org/10.1007/s12652-018-01171-4
Chen YT, Wang J, Liu SJ, Chen X, Xiong J, Xie JB, Yang K (2019c) Multiscale fast correlation filtering tracking algorithm based on a feature fusion model. Concurr Comput-Pract Exp. https://doi.org/10.1002/cpe.5533
Chen YT, Wang J, Chen X, Zhu MW, Yang K, Wang Z, Xia RL (2019d) Single-image super-resolution algorithm based on structural self-similarity and deformation block features. IEEE Access 7:58791–58801. https://doi.org/10.1109/ACCESS.2019.2911892
Chen YT, Xu WH, Zuo JW, Yang K (2019e) The fire recognition algorithm using dynamic feature fusion and IV-SVM classifier. Clust Comput 22:7665–7675. https://doi.org/10.1007/s10586-018-2368-8
Chen YT, Tao JJ, Zhang Q, Yang K, Chen X, Xiong J, Xia RL, Xie JB (2020) Saliency detection via the improved hierarchical principal component analysis method. Wirel Commun Mob Comput. https://doi.org/10.1155/2020/8822777
Everingham M, Eslami SMA, Van GL, Williams CKI, Winn JM, Zisserman A (2015) The PASCAL visual object classes challenge: a retrospective. Int J Comput Vis 111(1):98–136. https://doi.org/10.1007/s11263-014-0733-5
Hariharan B, Arbelaez P, Bourdev LD, Maji S, Malik J (2011) Semantic contours from inverse detectors. In: Proceedings of IEEE international conference on computer vision, pp 991–998. https://doi.org/10.1109/ICCV.2011.6126343
He SM, Li ZZ, Tang YN, Liao ZF, Li F, Lim SJ (2020) Parameters compressing in deep learning. CMC-Comput Mater Contin 62(1):321–336. https://doi.org/10.32604/cmc.2020.06130
Li WJ, Ding Y, Yang YJ, Park YJ, Wang J (2020) Parameterized algorithms of fundamental NP-hard problems: a survey. Hum-Centric Comput Inf Sci. https://doi.org/10.1186/s13673-020-00226-w
Liao ND, Song YQ, Su S, Huang XS, Ma HL (2020) Detection of probe flow anomalies using information entropy and random forest method. J Intell Fuzzy Syst. https://doi.org/10.3233/IFS-191448
Liu ZH, Lai ZH, Ou WH, Zhang KB, Zheng RJ (2020) Structured optimal graph based sparse feature extraction for semi-supervised learning. Signal Process 170:107456. https://doi.org/10.1016/j.sigpro.2020.107456
Luo YJ, Qin JH, Xiang XY, Tan Y, Liu Q, Xiang LY (2020) Coverless real-time image information hiding based on image block matching and dense convolutional network. J Real-Time Image Process 17:125–135. https://doi.org/10.1007/s11554-019-00917-3
Mottaghi R, Chen XJ, Liu XB, Cho NG, Lee SW, Fidler S, Urtasun R, Yuille AL (2014) The role of context for object detection and semantic segmentation in the wild. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 891–898. https://doi.org/10.1109/cvpr.2014.119
Peng C, Zhang XY, Yu G, Luo JM, Sun J (2017) Large kernel matters-improve semantic segmentation by global convolutional network. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1743–1751. https://doi.org/10.1109/CVPR.2017.189
Sakai Y, Lu HM, Tan JK, Kim H (2019) Recognition of surrounding environment from electric wheelchair videos based on modified YOLOv2. Future Gener Comput Syst 92:157–161. https://doi.org/10.1016/j.future.2018.09.068
Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651. https://doi.org/10.1109/TPAMI.2016.2572683
Sheng GQ, Tang XG, Xie K, Xiong J (2019) Hydraulic fracturing microseismic first arrival picking method based on non-subsampled shearlet transform and higher-order-statistics. J Seism Explor 28(6):593–618
Wu RM, Feng MY, Guan WL, Wang D, Lu HC, Ding E (2019) A mutual learning method for salient object detection with intertwined multi-supervision. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 8150–8159. https://doi.org/10.1109/CVPR.2019.00834
Yang MK, Yu K, Zhang C, Li ZW, Yang KY (2018) DenseASPP for semantic segmentation in street scenes. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3684–3692. https://doi.org/10.1109/CVPR.2018.00388
Yu F, Qian S, Chen X, Huang YY, Liu L, Shi CQ, Cai S, Song Y, Wang C (2020a) A new 4D four-wing memristive hyperchaotic system: dynamical analysis, electronic circuit design, shape synchronization and secure communication. Int J Bifurc Chaos. https://doi.org/10.1142/S0218127420501412
Yu F, Liu L, Qian S, Li LX, Huang YY, Shi CQ, Cai S, Wu XM, Du SC, Wan QZ (2020b) Chaos-based application of a novel multistable 5D memristive hyperchaotic system with coexisting multiple attractors. Complexity. https://doi.org/10.1155/2020/8034196
Yu F, Shen H, Liu L, Zhang ZN, Huang YY, He BY, Cai S, Song Y, Yin B, Du SC, Xu Q (2020c) CCII and FPGA realization: a multistable modified four-order autonomous Chua's chaotic system with coexisting multiple attractors. Complexity. https://doi.org/10.1155/2020/5212601
Zhang PP, Wang D, Lu HC, Wang HY, Ruan X (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: Proceedings of IEEE international conference on computer vision, pp 202–211. https://doi.org/10.1109/ICCV.2017.31
Zhang JM, Xie ZP, Sun J, Zou X, Wang J (2020a) A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754. https://doi.org/10.1109/access.2020.2972338
Zhang JM, Lu CQ, Wang J, Yue XG, Lim SJ, Al-Makhadmeh Z, Tolba A (2020b) Training convolutional neural networks with multi-size images and triplet loss for remote sensing scene classification. Sensors 20(4):1188. https://doi.org/10.3390/s20041188
Zhou LY, Tang JX (2017) Fraction-order total variation blind image restoration based on L1-norm. Appl Math Model 51:469–476. https://doi.org/10.1016/j.apm.2017.07.009
Acknowledgements
This work is supported by the National Science Foundation of China (nos. 61972056, 61972212, 61402053, 61981340416), the Open Research Fund of Hunan Provincial Key Laboratory of Intelligent Processing of Big Data on Transportation (no. 2015TP1005), the Changsha Science and Technology Planning (nos. KQ1703018, KQ1706064, KQ1703018-01, KQ1703018-04), the Research Foundation of Education Bureau of Hunan Province (nos. 17A007, 19B005), Changsha Industrial Science and Technology Commissioner (no. 2017-7), the Junior Faculty Development Program Project of Changsha University of Science and Technology (no. 2019QJCZ011). Natural Science Foundation of Hunan Province (no. 2020JJ50590), the Program of Practical Innovation and Entrepreneurship Improvement (no. CSLG2020).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, Y., Tao, J., Liu, L. et al. Research of improving semantic image segmentation based on a feature fusion model. J Ambient Intell Human Comput 13, 5033–5045 (2022). https://doi.org/10.1007/s12652-020-02066-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-020-02066-z