Entropy of Belief Functions in the Dempster-Shafer Theory: A New Perspective

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9861)


We propose a new definition of entropy of basic probability assignments (BPA) in the Dempster-Shafer (D-S) theory of belief functions, which is interpreted as a measure of total uncertainty in the BPA. We state a list of five desired properties of entropy for D-S belief functions theory that are motivated by Shannon’s definition of entropy of probability functions, together with the implicit requirement that any definition should be consistent with semantics of D-S belief functions theory. Three of our five desired properties are different from the five properties described by Klir and Wierman. We demonstrate that our definition satisfies all five properties in our list, and is consistent with semantics of D-S theory, whereas none of the existing definitions do. Our definition does not satisfy the sub-additivity property. Whether there exists a definition that satisfies our five properties plus sub-additivity, and that is consistent with semantics for the D-S theory, remains an open question.


Theory Semantic Probability Mass Function Combination Rule Belief Function Basic Probability Assignment 



This article is a short version of [14], which has been supported in part by funds from grant GAČR 15-00215S to the first author, and from the Ronald G. Harper Distinguished Professorship at the University of Kansas to the second author. We are extremely grateful to Thierry Denoeux, Marc Pouly, Anne-Laure Jousselme, Joaquín Abellán, and Mark Wierman for their comments on earlier drafts of [14]. We are grateful to two anonymous reviewers of Belief-2016 conference for their comments. We are also grateful to Suzanna Emelio for a careful proof-reading of the text.


  1. 1.
    Abellán, J.: Combining nonspecificity measures in Dempster-Shafer theory of evidence. Int. J. Gen. Syst. 40(6), 611–622 (2011)MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Abellán, J., Masegosa, A.: Requirements for total uncertainty measures in Dempster-Shafer theory of evidence. Int. J. Gen. Syst. 37(6), 733–747 (2008)MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Abellán, J., Moral, S.: Completing a total uncertainty measure in Dempster-Shafer theory. Int. J. Gen. Syst. 28(4–5), 299–314 (1999)MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Cobb, B.R., Shenoy, P.P.: On the plausibility transformation method for translating belief function models to probability models. Int. J. Approx. Reason. 41(3), 314–340 (2006)MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Daniel, M.: On transformations of belief functions to probabilities. Int. J. Intell. Syst. 21(3), 261–282 (2006)MATHCrossRefGoogle Scholar
  6. 6.
    Dempster, A.P.: Upper and lower probabilities induced by a multivalued mapping. Ann. Math. Stat. 38(2), 325–339 (1967)MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    Dezert, J., Smarandache, F., Tchamova, A.: On the Blackman’s association problem. In: Proceedings of the 6th Annual Conference on Information Fusion, Cairns, Queensland, Australia, pp. 1349–1356. International Society for Information Fusion (2003)Google Scholar
  8. 8.
    Dubois, D., Prade, H.: Properties of measures of information in evidence and possibility theories. Fuzzy Sets Syst. 24(2), 161–182 (1987)MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Fagin, R., Halpern, J.Y.: A new approach to updating beliefs. In: Bonissone, P., Henrion, M., Kanal, L., Lemmer, J. (eds.) Uncertainty in Artificial Intelligence, vol. 6, pp. 347–374. North-Holland (1991)Google Scholar
  10. 10.
    Halpern, J.Y., Fagin, R.: Two views of belief: belief as generalized probability and belief as evidence. Artif. Intell. 54(3), 275–317 (1992)MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Harmanec, D., Klir, G.J.: Measuring total uncertainty in Dempster-Shafer theory: a novel approach. Int. J. Gen. Syst. 22(4), 405–419 (1994)MATHCrossRefGoogle Scholar
  12. 12.
    Hartley, R.V.L.: Transmission of information. Bell Syst. Tech. J. 7(3), 535–563 (1928)CrossRefGoogle Scholar
  13. 13.
    Höhle, U.: Entropy with respect to plausibility measures. In: Proceedings of the 12th IEEE Symposium on Multiple-Valued Logic, pp. 167–169 (1982)Google Scholar
  14. 14.
    Jiroušek, R., Shenoy, P.P.: A new definition of entropy of belief functions in the Dempster-Shafer theory. Working Paper 330, University of Kansas School of Business, Lawrence, KS (2016)Google Scholar
  15. 15.
    Jousselme, A.-L., Liu, C., Grenier, D., Bossé, E.: Measuring ambiguity in the evidence theory. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 36(5), 890–903 (2006)CrossRefGoogle Scholar
  16. 16.
    Klir, G.J.: Where do we stand on measures of uncertainty, ambiguity, fuzziness, and the like? Fuzzy Sets Syst. 24(2), 141–160 (1987)MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Klir, G.J., Parviz, B.: A note on the measure of discord. In: Dubois, D., Wellman, M.P., D’Ambrosio, B., Smets, P. (eds.) Uncertainty in Artificial Intelligence: Proceedings of the Eighth Conference, pp. 138–141. Morgan Kaufmann (1992)Google Scholar
  18. 18.
    Klir, G.J., Ramer, A.: Uncertainty in the Dempster-Shafer theory: a critical re-examination. Int. J. Gen. Syst. 18(2), 155–166 (1990)MATHCrossRefGoogle Scholar
  19. 19.
    Klir, G.J., Wierman, M.J.: Uncertainity Elements of Generalized Information Theory, 2nd edn. Springer, Berlin (1999)MATHGoogle Scholar
  20. 20.
    Kohlas, J., Monney, P.-A.: A Mathematical Theory of Hints: An Approach to the Dempster-Shafer Theory of Evidence. Springer, Berlin (1995)MATHCrossRefGoogle Scholar
  21. 21.
    Lamata, M.T., Moral, S.: Measures of entropy in the theory of evidence. Int. J. Gen. Syst. 14(4), 297–305 (1988)MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Maeda, Y., Ichihashi, H.: An uncertainty measure under the random set inclusion. Int. J. Gen. Syst. 21(4), 379–392 (1993)MATHCrossRefGoogle Scholar
  23. 23.
    Nguyen, H.T.: On entropy of random sets and possibility distributions. In: Bezdek, J.C. (ed.) The Analysis of Fuzzy Information, pp. 145–156. CRC Press, Boca Raton (1985)Google Scholar
  24. 24.
    Pal, N.R., Bezdek, J.C., Hemasinha, R.: Uncertainty measures for evidential reasoning II: a new measure of total uncertainty. Int. J. Approx. Reason. 8(1), 1–16 (1993)MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Pouly, M., Kohlas, J., Ryan, P.Y.A.: Generalized information theory for hints. Int. J. Approx. Reason. 54(1), 228–251 (2013)MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Rényi, A.: On measures of information and entropy. In: Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, pp. 547–561 (1960)Google Scholar
  27. 27.
    Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)MATHGoogle Scholar
  28. 28.
    Shafer, G.: Constructive probability. Synthese 48(1), 1–60 (1981)MathSciNetMATHCrossRefGoogle Scholar
  29. 29.
    Shafer, G.: Perspectives on the theory and practice of belief functions. Int. J. Approx. Reason. 4(5–6), 323–362 (1990)MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(379–423), 623–656 (1948)MathSciNetMATHCrossRefGoogle Scholar
  31. 31.
    Shenoy, P.P.: Conditional independence in valuation-based systems. Int. J. Approx. Reason. 10(3), 203–234 (1994)MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Smets, P.: Information content of an evidence. Int. J. Man Mach. Stud. 19, 33–43 (1983)CrossRefGoogle Scholar
  33. 33.
    Smets, P.: Constructing the pignistic probability function in a context of uncertainty. In: Henrion, M., Shachter, R., Kanal, L.N., Lemmer, J.F. (eds.) Uncertainty in Artificial Intelligence, vol. 5. pp, pp. 29–40. North-Holland, Amsterdam (1990)Google Scholar
  34. 34.
    Smets, P., Kennes, R.: The transferable belief model. Artif. Intell. 66(2), 191–234 (1994)MathSciNetMATHCrossRefGoogle Scholar
  35. 35.
    Vejnarová, J., Klir, G.J.: Measure of strife in Dempster-Shafer theory. Int. J. Gen. Syst. 22(1), 25–42 (1993)MATHCrossRefGoogle Scholar
  36. 36.
    Walley, P.: Statistical Reasoning with Imprecise Probabilities. Chapman & Hall, London (1991)MATHCrossRefGoogle Scholar
  37. 37.
    Wierman, M.J.: Measuring granularity in evidence theory. Int. J. Gen. Syst. 30(6), 649–660 (2001)MathSciNetMATHCrossRefGoogle Scholar
  38. 38.
    Yager, R.: Entropy and specificity in a mathematical theory of evidence. Int. J. Gen. Syst. 9(4), 249–260 (1983)MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Faculty of ManagementUniversity of EconomicsJindřichův HradecCzech Republic
  2. 2.School of BusinessUniversity of KansasLawrenceUSA
  3. 3.Institute of Information Theory and Automation, Academy of SciencesPragueCzech Republic

Personalised recommendations