Computing Entropy under Interval Uncertainty. II

Formulation and Analysis of the Problem, and the Corresponding Results and Algorithms

Formulation of the problem. In most practical situations, our knowledge is incomplete: there are several (n) different states which are consistent with our knowledge. How can we gauge this uncertainty? A natural measure of uncertainty is the average number of binary (“yes”-“no”) questions that we need to ask to find the exact state. This idea is behind Shannon’s information theory: according to this theory, when we know the probabilities p₁,..., p _n of different states (for which ∑ p _i = 1), then this average number of questions is equal to S = – $\sum\limits^{n}_{i=1}$ p _i ·log₂(p _i ). In information theory, this average number of question is called the amount of information.