Using Maximum Entropy to compute marginal probabilities in a causal binary tree need not take exponential time

  • Paul C. Rhodes
  • Gerald R. Garside
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 946)


In a previous paper, the present authors have argued that Maximum Entropy is worth pursuing as a technique for reasoning under uncertainty when information is missing. The main drawback is that maximising Entropy has been shown to be an NP-Complete problem. However, a Maximum Entropy approach to probabilistic reasoning is not necessarily exponentially large to compute in at least one case. This paper shows that given a tree of incomplete causal information (eg. as used by Pearl but with some of the information missing), the probability of the marginals can be found in linear space and time using Maximum Entropy.


Maximum Entropy Probability Reasoningunder Uncertainty Probabilistic Inference Computational Complexity Incomplete Information Expert Systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Shortcliffe, E. H. and Buchanan, B. G. A model of Inexact Reasoning in Medicine. Mathematical Biosciences, Vol. 23 (1975) pp 351–379.CrossRefGoogle Scholar
  2. 2.
    Duda, R.O., Hart, P.E. and Nilsson, N.J. Subjective Bayesian Methods for Rule-based Inference Systems. National Computer Conference, AFIPS Conf. Proc. Vol.45 (1976) pp.1075–1082.Google Scholar
  3. 3.
    McDermott, D. and Doyle, J. Non-monotonic Logic I Artificial Intelligence, Vol 13 (1980) pp 41–72CrossRefGoogle Scholar
  4. 4.
    Reiter, R. A. A logic for Inexact Reasoning. Artificial Intelligence, Vol 13 (1980) pp 81–132CrossRefGoogle Scholar
  5. 5.
    Zadeh, L. A. A theory of approximate reasoning Machine Intelligence Vol 9 (1979) pp 149–193Google Scholar
  6. 6.
    Zadeh, L. A. The concept of a linguistic variable and its application to approximate reasoning Learning Systems and Intelligent Robots (Fu, K. S. and Tow, J. T., Eds.) Plenum Press, New York, (1974) pp 1–10Google Scholar
  7. 7.
    Zadeh, L. A. The role of fuzzy logic in the management of uncertainty in expert systems Fuzzy Sets and Systems Vol 11 (1983) pp 199–227Google Scholar
  8. 8.
    Baldwin, J. F. and Zhou, S. Q. A fuzzy rational inference language Fuzzy Sets and Systems Vol 14 (1984) pp 155–174Google Scholar
  9. 9.
    Dempster A.P. A generalisation of Bayesian inference. Journal of the Royal Statistical Society Ser. B, Vol 30 (1968) pp 205–247Google Scholar
  10. 10.
    Schafer G. A mathematical Theory of Evidence Princeton University Press, Princeton (1976)Google Scholar
  11. 11.
    Lemmer, J.F. and Barth, S.W. Efficient Minimum Information Updating for Bayesian Inferencing in Expert Systems. Proc. National Conf. on Artifical Intelligence, Pittsburgh (1982) pp.424–427Google Scholar
  12. 12.
    Cheeseman P. In Defense of Probability In Proceedings 8th International Joint Conference on AI (IJCAI-85) Los Angeles (1985) pp 1002–1009Google Scholar
  13. 13.
    Pearl J. Reverend Bayes on Inference Engines:a distributed hierachical approach Proceedings of the National Conference on AI Pittsburgh (1982) pp 133–136Google Scholar
  14. 14.
    Pearl, J. Fusion, Propagation, and Structuring in Belief Networks. Artificial Intelligence, Vol.29, No.3 (1986) pp.241–288.MathSciNetGoogle Scholar
  15. 15.
    Pearl, J.Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. (1988) Morgan Kaufmann, San Mateo, California.Google Scholar
  16. 16.
    Spiegelhalter, D.J. Probabilistic Reasoning in Predictive Expert Systems. In Uncertainty in Artificial Intelligence, Kanal, L.N. and Lemmer, F.J. (Eds.) (1986) Elsevier Science Publishers B.V. (North Holland) pp 47–68Google Scholar
  17. 17.
    Lauritzen, S.L. and Spiegelhalter, D.J. Local Computations with Probabilities on Graphical Structures and Their Applications to Expert Systems. J. Royal Statistical Society B., Vol.50, No.2 (1988) pp.157–224.Google Scholar
  18. 18.
    Lauritzen, S.L. and Spiegelhalter, D.J. Local Computations with Probabilities on Graphical Structures and Their Applications to Expert Systems. In Readings in Uncertain Reasoning, G. Shafer and J.Pearl (Eds), Morgan Kaufman, San Mateo, (1990) pp 415–433.Google Scholar
  19. 19.
    Neapolitan, R.E.Probabilistic Reasoning in Expert Systems: theory and algoritms. Wiley, New York (1990).Google Scholar
  20. 20.
    Maung, I. Measures of Information and Inference Processes. PhD Thesis, (1992) Department of Mathematics, University of Manchester.Google Scholar
  21. 21.
    Maung, I. and Paris, J.B. A Note on the Infeasibility of Some Inference Processes. International Journal of Intelligent Systems, Vol.5, No.5 (1990) pp.595–603.Google Scholar
  22. 22.
    Rhodes, P.C. and Garside G.R. The use of Maximum Entropy as a Methodology for Probabilistic Reasoning to be published in Knowledge Based SystemsGoogle Scholar
  23. 23.
    Tribus, M.Rational Descriptions, Decision and Designs. Pergamon, New York (1969).Google Scholar
  24. 24.
    Jaynes, E.T.Notes on Probability Theory in Science and Engineering. Physics Dept., Washington University, St Louis (1969).Google Scholar
  25. 25.
    Griffeath, D.S. Computer Solution of the Discrete Maximum Entropy Problem. Technometrics, Vol.14 No.4 (1972) pp.891–897.Google Scholar
  26. 26.
    Cheeseman, P. A Method of Computing Generalised Bayesian Probability Values for Expert Systems Proceeding of the 8th International Conference in Artificial Intelligence (1983) pp198–202.Google Scholar
  27. 27.
    Cooper, G. F. Probabilistic Inference Using Belief Networks is NP-Hard, Technical Report KSL-87-27, Stanford University, Stanford, California. (1988).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Paul C. Rhodes
  • Gerald R. Garside
    • 1
  1. 1.Dept. of ComputingUniversity of BradfordBradfordUK

Personalised recommendations