Maximum Entropy (Maxent) Method in Expert Systems and Intelligent Control: New Possibilities and Limitations

Abstract

To describe uncertainty of experts’ statements E₁,…, E_n that form a knowledge base, it is natural to use (subjective) probabilities p_i. Correspondingly, it is natural to use probabilities to describe uncertainty of the system’s answer to a given query Q. Since it is impossible to inquire about the expert’s probabilities for all possible (≥ 2ⁿ) propositional combinations of E_{i i}, a knowledge base is usually incomplete in the sense that there are many probability distributions consistent with this knowledge. If we want to return a single probability, we must select one of these distributions. MaxEnt is a natural selection rule, but for expert systems, computing the MaxEnt distribution often takes an unrealistically long time. In this paper, we propose computationally feasible approximate MaxEnt techniques for expert systems and intelligent control.

The authors are thankful to Peter C. Cheeseman for his interest and encouragement, and to all the participants of the workshop, especially to Professor Myron Tribus, for helpful discussion.