Entropy of Belief Functions in the Dempster-Shafer Theory: A New Perspective

  • Radim Jiroušek
  • Prakash P. Shenoy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9861)


We propose a new definition of entropy of basic probability assignments (BPA) in the Dempster-Shafer (D-S) theory of belief functions, which is interpreted as a measure of total uncertainty in the BPA. We state a list of five desired properties of entropy for D-S belief functions theory that are motivated by Shannon’s definition of entropy of probability functions, together with the implicit requirement that any definition should be consistent with semantics of D-S belief functions theory. Three of our five desired properties are different from the five properties described by Klir and Wierman. We demonstrate that our definition satisfies all five properties in our list, and is consistent with semantics of D-S theory, whereas none of the existing definitions do. Our definition does not satisfy the sub-additivity property. Whether there exists a definition that satisfies our five properties plus sub-additivity, and that is consistent with semantics for the D-S theory, remains an open question.


Theory Semantic Probability Mass Function Combination Rule Belief Function Basic Probability Assignment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This article is a short version of [14], which has been supported in part by funds from grant GAČR 15-00215S to the first author, and from the Ronald G. Harper Distinguished Professorship at the University of Kansas to the second author. We are extremely grateful to Thierry Denoeux, Marc Pouly, Anne-Laure Jousselme, Joaquín Abellán, and Mark Wierman for their comments on earlier drafts of [14]. We are grateful to two anonymous reviewers of Belief-2016 conference for their comments. We are also grateful to Suzanna Emelio for a careful proof-reading of the text.


  1. 1.
    Abellán, J.: Combining nonspecificity measures in Dempster-Shafer theory of evidence. Int. J. Gen. Syst. 40(6), 611–622 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Abellán, J., Masegosa, A.: Requirements for total uncertainty measures in Dempster-Shafer theory of evidence. Int. J. Gen. Syst. 37(6), 733–747 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Abellán, J., Moral, S.: Completing a total uncertainty measure in Dempster-Shafer theory. Int. J. Gen. Syst. 28(4–5), 299–314 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Cobb, B.R., Shenoy, P.P.: On the plausibility transformation method for translating belief function models to probability models. Int. J. Approx. Reason. 41(3), 314–340 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Daniel, M.: On transformations of belief functions to probabilities. Int. J. Intell. Syst. 21(3), 261–282 (2006)zbMATHCrossRefGoogle Scholar
  6. 6.
    Dempster, A.P.: Upper and lower probabilities induced by a multivalued mapping. Ann. Math. Stat. 38(2), 325–339 (1967)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Dezert, J., Smarandache, F., Tchamova, A.: On the Blackman’s association problem. In: Proceedings of the 6th Annual Conference on Information Fusion, Cairns, Queensland, Australia, pp. 1349–1356. International Society for Information Fusion (2003)Google Scholar
  8. 8.
    Dubois, D., Prade, H.: Properties of measures of information in evidence and possibility theories. Fuzzy Sets Syst. 24(2), 161–182 (1987)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Fagin, R., Halpern, J.Y.: A new approach to updating beliefs. In: Bonissone, P., Henrion, M., Kanal, L., Lemmer, J. (eds.) Uncertainty in Artificial Intelligence, vol. 6, pp. 347–374. North-Holland (1991)Google Scholar
  10. 10.
    Halpern, J.Y., Fagin, R.: Two views of belief: belief as generalized probability and belief as evidence. Artif. Intell. 54(3), 275–317 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Harmanec, D., Klir, G.J.: Measuring total uncertainty in Dempster-Shafer theory: a novel approach. Int. J. Gen. Syst. 22(4), 405–419 (1994)zbMATHCrossRefGoogle Scholar
  12. 12.
    Hartley, R.V.L.: Transmission of information. Bell Syst. Tech. J. 7(3), 535–563 (1928)CrossRefGoogle Scholar
  13. 13.
    Höhle, U.: Entropy with respect to plausibility measures. In: Proceedings of the 12th IEEE Symposium on Multiple-Valued Logic, pp. 167–169 (1982)Google Scholar
  14. 14.
    Jiroušek, R., Shenoy, P.P.: A new definition of entropy of belief functions in the Dempster-Shafer theory. Working Paper 330, University of Kansas School of Business, Lawrence, KS (2016)Google Scholar
  15. 15.
    Jousselme, A.-L., Liu, C., Grenier, D., Bossé, E.: Measuring ambiguity in the evidence theory. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 36(5), 890–903 (2006)CrossRefGoogle Scholar
  16. 16.
    Klir, G.J.: Where do we stand on measures of uncertainty, ambiguity, fuzziness, and the like? Fuzzy Sets Syst. 24(2), 141–160 (1987)MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Klir, G.J., Parviz, B.: A note on the measure of discord. In: Dubois, D., Wellman, M.P., D’Ambrosio, B., Smets, P. (eds.) Uncertainty in Artificial Intelligence: Proceedings of the Eighth Conference, pp. 138–141. Morgan Kaufmann (1992)Google Scholar
  18. 18.
    Klir, G.J., Ramer, A.: Uncertainty in the Dempster-Shafer theory: a critical re-examination. Int. J. Gen. Syst. 18(2), 155–166 (1990)zbMATHCrossRefGoogle Scholar
  19. 19.
    Klir, G.J., Wierman, M.J.: Uncertainity Elements of Generalized Information Theory, 2nd edn. Springer, Berlin (1999)zbMATHGoogle Scholar
  20. 20.
    Kohlas, J., Monney, P.-A.: A Mathematical Theory of Hints: An Approach to the Dempster-Shafer Theory of Evidence. Springer, Berlin (1995)zbMATHCrossRefGoogle Scholar
  21. 21.
    Lamata, M.T., Moral, S.: Measures of entropy in the theory of evidence. Int. J. Gen. Syst. 14(4), 297–305 (1988)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Maeda, Y., Ichihashi, H.: An uncertainty measure under the random set inclusion. Int. J. Gen. Syst. 21(4), 379–392 (1993)zbMATHCrossRefGoogle Scholar
  23. 23.
    Nguyen, H.T.: On entropy of random sets and possibility distributions. In: Bezdek, J.C. (ed.) The Analysis of Fuzzy Information, pp. 145–156. CRC Press, Boca Raton (1985)Google Scholar
  24. 24.
    Pal, N.R., Bezdek, J.C., Hemasinha, R.: Uncertainty measures for evidential reasoning II: a new measure of total uncertainty. Int. J. Approx. Reason. 8(1), 1–16 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Pouly, M., Kohlas, J., Ryan, P.Y.A.: Generalized information theory for hints. Int. J. Approx. Reason. 54(1), 228–251 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Rényi, A.: On measures of information and entropy. In: Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, pp. 547–561 (1960)Google Scholar
  27. 27.
    Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)zbMATHGoogle Scholar
  28. 28.
    Shafer, G.: Constructive probability. Synthese 48(1), 1–60 (1981)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Shafer, G.: Perspectives on the theory and practice of belief functions. Int. J. Approx. Reason. 4(5–6), 323–362 (1990)MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(379–423), 623–656 (1948)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Shenoy, P.P.: Conditional independence in valuation-based systems. Int. J. Approx. Reason. 10(3), 203–234 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Smets, P.: Information content of an evidence. Int. J. Man Mach. Stud. 19, 33–43 (1983)CrossRefGoogle Scholar
  33. 33.
    Smets, P.: Constructing the pignistic probability function in a context of uncertainty. In: Henrion, M., Shachter, R., Kanal, L.N., Lemmer, J.F. (eds.) Uncertainty in Artificial Intelligence, vol. 5. pp, pp. 29–40. North-Holland, Amsterdam (1990)Google Scholar
  34. 34.
    Smets, P., Kennes, R.: The transferable belief model. Artif. Intell. 66(2), 191–234 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Vejnarová, J., Klir, G.J.: Measure of strife in Dempster-Shafer theory. Int. J. Gen. Syst. 22(1), 25–42 (1993)zbMATHCrossRefGoogle Scholar
  36. 36.
    Walley, P.: Statistical Reasoning with Imprecise Probabilities. Chapman & Hall, London (1991)zbMATHCrossRefGoogle Scholar
  37. 37.
    Wierman, M.J.: Measuring granularity in evidence theory. Int. J. Gen. Syst. 30(6), 649–660 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  38. 38.
    Yager, R.: Entropy and specificity in a mathematical theory of evidence. Int. J. Gen. Syst. 9(4), 249–260 (1983)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Faculty of ManagementUniversity of EconomicsJindřichův HradecCzech Republic
  2. 2.School of BusinessUniversity of KansasLawrenceUSA
  3. 3.Institute of Information Theory and Automation, Academy of SciencesPragueCzech Republic

Personalised recommendations