# On the KL Divergence of Probability Mixtures for Belief Contraction

## Abstract

Probabilistic belief change is an operation that takes a probability distribution representing a belief state along with an input sentence representing some information to be accommodated or removed, and maps it to a new probability distribution. In order to choose from many such mappings possible, techniques from information theory such as the *principle of minimum cross-entropy* have previously been used. Central to this principle is the *Kullback-Leibler (KL) divergence*. In this short study, we focus on the *contraction* of a belief state *P* by a belief *a*, which is the process of turning the belief *a* into a non-belief. The contracted belief state \(P^-_a\) can be represented as a mixture of two states: the original belief state *P*, and the resultant state \(P^*_{\lnot a}\) of revising P by \(\lnot a\). Crucial to this mixture is the mixing factor \(\epsilon \) which determines the proportion of *P* and \(P^*_{\lnot a}\) that are to be used in this process. We show that once \(\epsilon \) is determined, the KL divergence of \(P^-_a\) from *P* is given by a function whose only argument is \(\epsilon \). We suggest that \(\epsilon \) is not only a mixing factor but also captures relevant aspects of *P* and \(P^*_{\lnot a}\) required for computing the KL divergence.

## Keywords

Belief Revision Belief State Belief Change Argumentation Framework Probability Mixture## Preview

Unable to display preview. Download preview PDF.

## References

- Alchourrón, C.E., Gärdenfors, P., Makinson, D.: On the logic of theory change: Partial meet contraction and revision functions. J. Symb. Log.
**50**(2), 510–530 (1985)MathSciNetCrossRefzbMATHGoogle Scholar - Chhogyal, K., Nayak, A.C., Zhuang, Z., Sattar, A.: Probabilistic belief contraction using argumentation. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, pp. 2854–2860, July 25–31, 2015Google Scholar
- Gärdenfors, P.: Knowledge in Flux. Modelling the Dymanics of Epistemic States. MIT Press (1988)Google Scholar
- Jaynes, E.T.: Information theory and statistical mechanics. Physical Review
**106**(4), 620 (1957)MathSciNetCrossRefzbMATHGoogle Scholar - Kern-Isberner, G.: Linking iterated belief change operations to nonmonotonic reasoning. In: Principles of Knowledge Representation and Reasoning: Proceedings of the Eleventh International Conference, KR 2008, pp. 166–176 (2008)Google Scholar
- Kullback, S., Leibler, R.A.: On information and sufficiency. The Annals of Mathematical Statistics, 79–86 (1951)Google Scholar
- Potyka, N., Beierle, C., Kern-Isberner, G.: Changes of relational probabilistic belief states and their computation under optimum entropy semantics. In: Timm, I.J., Thimm, M. (eds.) KI 2013. LNCS, vol. 8077, pp. 176–187. Springer, Heidelberg (2013) CrossRefGoogle Scholar
- Ramachandran, R., Ramer, A., Nayak, A.C.: Probabilistic belief contraction. Minds and Machines
**22**(4), 325–351 (2012)CrossRefGoogle Scholar