Skip to main content
SpringerLink
Log in

Evidential fully convolutional network for semantic segmentation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

This article has been updated

Abstract

We propose a hybrid architecture composed of a fully convolutional network (FCN) and a Dempster-Shafer layer for image semantic segmentation. In the so-called evidential FCN (E-FCN), an encoder-decoder architecture first extracts pixel-wise feature maps from an input image. A Dempster-Shafer layer then computes mass functions at each pixel location based on distances to prototypes. Finally, a utility layer performs semantic segmentation from mass functions and allows for imprecise classification of ambiguous pixels and outliers. We propose an end-to-end learning strategy for jointly updating the network parameters, which can make use of soft (imprecise) labels. Experiments using three databases (Pascal VOC 2011, MIT-scene Parsing and SIFT Flow) show that the proposed combination improves the accuracy and calibration of semantic segmentation by assigning confusing pixels to multi-class sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Change history

  • 24 June 2021

    The article was revised due to incorrect placement of the author photos.

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  2. Biggio B, Nelson B, Laskov P (2011) Support vector machines under adversarial label noise. In: Asian conference on machine learning, pp 97–112

  3. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  4. Chen Xl, Wang Ph, Hao Ys, Zhao M (2018) Evidential KNN-based condition monitoring and early warning method with applications in power plant. Neurocomputing 315:18–32

    Article  Google Scholar 

  5. Côme E., Oukhellou L, Denœux T, Aknin P (2009) Learning from partially supervised data using mixture models and belief functions. Pattern Recogn 42(3):334–348

    Article  Google Scholar 

  6. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3213–3223

  7. Dempster AP (1967) Upper and lower probabilities induced by a multivalued mapping. Ann Math Stat 38:325–339

    Article  MathSciNet  Google Scholar 

  8. Denœux T (1997) Analysis of evidence-theoretic decision rules for pattern classification. Pattern Recogn 30(7):1095–1107

    Article  Google Scholar 

  9. Denœux T (2000) A neural network classifier based on Dempster-Shafer theory. IEEE Trans Syst Man Cybern-Part A: Syst Hum 30(2):131–150

    Article  Google Scholar 

  10. Denoeux T (2016) 40 years of Dempster-Shafer theory. Int J Approx Reason 79:1–6

    Article  MathSciNet  Google Scholar 

  11. Denoeux T (2019) Decision-making with belief functions: a review. Int J Approx Reason 109:87–110

    Article  MathSciNet  Google Scholar 

  12. Denœux T (2019) Logistic regression, neural networks and Dempster-Shafer theory: A new perspective. Knowl-Based Syst 176:54–67

    Article  Google Scholar 

  13. Denœux T, Dubois D, Prade H (2020) Representations of uncertainty in artificial intelligence: Beyond probability and possibility. In: A guided tour of artificial intelligence research, vol 1, chap. 4, pp 119–150. Springer, Berlin

  14. Denœux T, Kanjanatarakul O, Sriboonchitta S (2019) A new evidential k-nearest neighbor rule based on contextual discounting with partially supervised learning. Int J Approx Reason 113:287–302

    Article  MathSciNet  Google Scholar 

  15. Denœux T, Smets P (2006) Classification using belief functions: relationship between case-based and model-based approaches. IEEE Trans Syst Man Cybern Part B (Cybern) 36(6):1395– 1406

    Article  Google Scholar 

  16. Ess A, Müller T, Grabner H, Van Gool L (2009) Segmentation-based urban traffic scene understanding. In: BMVC, vol 1. Citeseer, pp 2

  17. Everingham M, Eslami SA, Van Gool L, Williams CK, Winn J, Zisserman A (2015) The pascal visual object classes challenge: a retrospective. Int J Comput Vis 111(1):98–136

    Article  Google Scholar 

  18. Forouzanfar M, Forghani N, Teshnehlab M (2010) Parameter optimization of improved fuzzy c-means clustering algorithm for brain mr image segmentation. Eng Appl Artif Intell 23(2):160–168

    Article  Google Scholar 

  19. Guettari N, Capelle-Laizé AS, Carré P (2016) Blind image steganalysis based on evidential K-Nearest Neighbors In: Proceedings of the 2016 IEEE International Conference on Image Processing, Phoenix, pp 2742–2746

  20. Guo C, Pleiss G, Sun Y, Weinberger KQ (2017) On calibration of modern neural networks. arXiv:170604599

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  22. Krähenbühl P, Koltun V (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Advances in neural information processing systems, pp 109–117

  23. Kumar N, Berg AC, Belhumeur PN, Nayar SK (2009) Attribute and simile classifiers for face verification In: Proceedings of the 12th International Conference on Computer Vision. IEEE, Kyoto, pp 365–372

  24. Lian C, Ruan S, Denœux T (2015) An evidential classifier based on feature selection and two-step classification strategy. Pattern Recogn 48:2318–2327

    Article  Google Scholar 

  25. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp 3431–3440

  26. Ma L, Denœux T (2021) Partial classification in the belief function framework. Knowl-Based Syst 214:106742

    Article  Google Scholar 

  27. Natarajan N, Dhillon IS, Ravikumar PK, Tewari A (2013) Learning with noisy labels. In: Advances in neural information processing systems, pp 1196–1204

  28. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1520–1528

  29. O’Hagan M (1988) Aggregating template or rule antecedents in real-time expert systems with fuzzy set logic. In: Twenty-second asilomar conference on signals, systems and computers, vol 2, pp 681–689

  30. Shafer G (1976) A mathematical theory of evidence. Princeton University Press Princeton

  31. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  32. Smets P (1993) Belief functions: the disjunctive rule of combination and the generalized Bayesian theorem. Int J Approx Reason 9(1):1–35

    Article  MathSciNet  Google Scholar 

  33. Su ZG, Denoeux T, Hao YS, Zhao M (2018) Evidential k-NN classification with enhanced performance via optimizing a class of parametric conjunctive t-rules. Knowl-Based Syst 142:7–16

    Article  Google Scholar 

  34. Tighe J, Lazebnik S (2010) Superparsing: scalable nonparametric image parsing with superpixels. In: European conference on computer vision. Springer, pp 352–365

  35. Tong Z, Xu P, Denœux T (2019) ConvNet and Dempster-Shafer theory for object recognition. In: Processing of the 13th international conference on scalable uncertainty management. Springer International Publishing, Cham, pp 368–381

  36. Xu P, Davoine F, Bordes JB, Zhao H, Denœux T (2016) Multimodal information fusion for urban scene understanding. Mach Vis Appl 27(3):331–349

    Article  Google Scholar 

  37. Yager RR (1988) On ordered weighted averaging aggregation operators in multicriteria decision-making. IEEE Trans Syst Man Cybern 18(1):183–190

    Article  Google Scholar 

  38. Yager RR, Liu L (2008) Classic works of the Dempster-Shafer theory of belief functions, vol 219. Springer, Berlin

    Book  Google Scholar 

  39. Yoon Y, Jeon HG, Yoo D, Lee JY, So Kweon I (2015) Learning a deep convolutional network for light-field image super-resolution. In: Proceedings of the IEEE international conference on computer vision workshops, pp 24–32

  40. Yuan B, Yue X, Lv Y, Denoeux T (2020) Evidential deep neural networks for uncertain data classification. In: International conference on knowledge science, engineering and management. Springer, pp 427–437

  41. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, pp 818–833

  42. Zeiler MD, Taylor GW, Fergus R (2011) Adaptive deconvolutional networks for mid and high level feature learning. In: 2011 International conference on computer vision. IEEE, pp 2018–2025

  43. Zhou B, Zhao H, Puig X, Fidler S, Barriuso A, Torralba A (2016) Semantic understanding of scenes through the ade20k dataset. arXiv:1608.05442

Download references

Acknowledgements

This research was supported by a scholarship from the China Scholarship Council and by the Labex MS2T (reference ANR-11-IDEX-0004-02).

Author information

Authors and Affiliations

  1. Université de technologie de Compiègne, CNRS, Heudiasyc, Compiègne, France

    Zheng Tong, Philippe Xu & Thierry Denœux

  2. Institut universitaire de France, Paris, France

    Thierry Denœux

Authors
  1. Zheng Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Philippe Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thierry Denœux
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thierry Denœux.

Ethics declarations

Competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: 30th Anniversary Special Issue

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tong, Z., Xu, P. & Denœux, T. Evidential fully convolutional network for semantic segmentation. Appl Intell 51, 6376–6399 (2021). https://doi.org/10.1007/s10489-021-02327-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02327-0

Keywords

Search

Navigation