Abstract
We propose a hybrid architecture composed of a fully convolutional network (FCN) and a Dempster-Shafer layer for image semantic segmentation. In the so-called evidential FCN (E-FCN), an encoder-decoder architecture first extracts pixel-wise feature maps from an input image. A Dempster-Shafer layer then computes mass functions at each pixel location based on distances to prototypes. Finally, a utility layer performs semantic segmentation from mass functions and allows for imprecise classification of ambiguous pixels and outliers. We propose an end-to-end learning strategy for jointly updating the network parameters, which can make use of soft (imprecise) labels. Experiments using three databases (Pascal VOC 2011, MIT-scene Parsing and SIFT Flow) show that the proposed combination improves the accuracy and calibration of semantic segmentation by assigning confusing pixels to multi-class sets.
Similar content being viewed by others
Change history
24 June 2021
The article was revised due to incorrect placement of the author photos.
References
Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495
Biggio B, Nelson B, Laskov P (2011) Support vector machines under adversarial label noise. In: Asian conference on machine learning, pp 97–112
Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848
Chen Xl, Wang Ph, Hao Ys, Zhao M (2018) Evidential KNN-based condition monitoring and early warning method with applications in power plant. Neurocomputing 315:18–32
Côme E., Oukhellou L, Denœux T, Aknin P (2009) Learning from partially supervised data using mixture models and belief functions. Pattern Recogn 42(3):334–348
Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3213–3223
Dempster AP (1967) Upper and lower probabilities induced by a multivalued mapping. Ann Math Stat 38:325–339
Denœux T (1997) Analysis of evidence-theoretic decision rules for pattern classification. Pattern Recogn 30(7):1095–1107
Denœux T (2000) A neural network classifier based on Dempster-Shafer theory. IEEE Trans Syst Man Cybern-Part A: Syst Hum 30(2):131–150
Denoeux T (2016) 40 years of Dempster-Shafer theory. Int J Approx Reason 79:1–6
Denoeux T (2019) Decision-making with belief functions: a review. Int J Approx Reason 109:87–110
Denœux T (2019) Logistic regression, neural networks and Dempster-Shafer theory: A new perspective. Knowl-Based Syst 176:54–67
Denœux T, Dubois D, Prade H (2020) Representations of uncertainty in artificial intelligence: Beyond probability and possibility. In: A guided tour of artificial intelligence research, vol 1, chap. 4, pp 119–150. Springer, Berlin
Denœux T, Kanjanatarakul O, Sriboonchitta S (2019) A new evidential k-nearest neighbor rule based on contextual discounting with partially supervised learning. Int J Approx Reason 113:287–302
Denœux T, Smets P (2006) Classification using belief functions: relationship between case-based and model-based approaches. IEEE Trans Syst Man Cybern Part B (Cybern) 36(6):1395– 1406
Ess A, Müller T, Grabner H, Van Gool L (2009) Segmentation-based urban traffic scene understanding. In: BMVC, vol 1. Citeseer, pp 2
Everingham M, Eslami SA, Van Gool L, Williams CK, Winn J, Zisserman A (2015) The pascal visual object classes challenge: a retrospective. Int J Comput Vis 111(1):98–136
Forouzanfar M, Forghani N, Teshnehlab M (2010) Parameter optimization of improved fuzzy c-means clustering algorithm for brain mr image segmentation. Eng Appl Artif Intell 23(2):160–168
Guettari N, Capelle-Laizé AS, Carré P (2016) Blind image steganalysis based on evidential K-Nearest Neighbors In: Proceedings of the 2016 IEEE International Conference on Image Processing, Phoenix, pp 2742–2746
Guo C, Pleiss G, Sun Y, Weinberger KQ (2017) On calibration of modern neural networks. arXiv:170604599
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Krähenbühl P, Koltun V (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Advances in neural information processing systems, pp 109–117
Kumar N, Berg AC, Belhumeur PN, Nayar SK (2009) Attribute and simile classifiers for face verification In: Proceedings of the 12th International Conference on Computer Vision. IEEE, Kyoto, pp 365–372
Lian C, Ruan S, Denœux T (2015) An evidential classifier based on feature selection and two-step classification strategy. Pattern Recogn 48:2318–2327
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp 3431–3440
Ma L, Denœux T (2021) Partial classification in the belief function framework. Knowl-Based Syst 214:106742
Natarajan N, Dhillon IS, Ravikumar PK, Tewari A (2013) Learning with noisy labels. In: Advances in neural information processing systems, pp 1196–1204
Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1520–1528
O’Hagan M (1988) Aggregating template or rule antecedents in real-time expert systems with fuzzy set logic. In: Twenty-second asilomar conference on signals, systems and computers, vol 2, pp 681–689
Shafer G (1976) A mathematical theory of evidence. Princeton University Press Princeton
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Smets P (1993) Belief functions: the disjunctive rule of combination and the generalized Bayesian theorem. Int J Approx Reason 9(1):1–35
Su ZG, Denoeux T, Hao YS, Zhao M (2018) Evidential k-NN classification with enhanced performance via optimizing a class of parametric conjunctive t-rules. Knowl-Based Syst 142:7–16
Tighe J, Lazebnik S (2010) Superparsing: scalable nonparametric image parsing with superpixels. In: European conference on computer vision. Springer, pp 352–365
Tong Z, Xu P, Denœux T (2019) ConvNet and Dempster-Shafer theory for object recognition. In: Processing of the 13th international conference on scalable uncertainty management. Springer International Publishing, Cham, pp 368–381
Xu P, Davoine F, Bordes JB, Zhao H, Denœux T (2016) Multimodal information fusion for urban scene understanding. Mach Vis Appl 27(3):331–349
Yager RR (1988) On ordered weighted averaging aggregation operators in multicriteria decision-making. IEEE Trans Syst Man Cybern 18(1):183–190
Yager RR, Liu L (2008) Classic works of the Dempster-Shafer theory of belief functions, vol 219. Springer, Berlin
Yoon Y, Jeon HG, Yoo D, Lee JY, So Kweon I (2015) Learning a deep convolutional network for light-field image super-resolution. In: Proceedings of the IEEE international conference on computer vision workshops, pp 24–32
Yuan B, Yue X, Lv Y, Denoeux T (2020) Evidential deep neural networks for uncertain data classification. In: International conference on knowledge science, engineering and management. Springer, pp 427–437
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, pp 818–833
Zeiler MD, Taylor GW, Fergus R (2011) Adaptive deconvolutional networks for mid and high level feature learning. In: 2011 International conference on computer vision. IEEE, pp 2018–2025
Zhou B, Zhao H, Puig X, Fidler S, Barriuso A, Torralba A (2016) Semantic understanding of scenes through the ade20k dataset. arXiv:1608.05442
Acknowledgements
This research was supported by a scholarship from the China Scholarship Council and by the Labex MS2T (reference ANR-11-IDEX-0004-02).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article belongs to the Topical Collection: 30th Anniversary Special Issue
Rights and permissions
About this article
Cite this article
Tong, Z., Xu, P. & Denœux, T. Evidential fully convolutional network for semantic segmentation. Appl Intell 51, 6376–6399 (2021). https://doi.org/10.1007/s10489-021-02327-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02327-0