Generalization of Concave and Convex Decomposition in Kikuchi Free Energy
Computing marginal probabilities from a high-dimensional distribution is generally an intractable task which arises in various probabilistic information processing. The minimum of Kikuchi free energy is known to yield more accurate marginals than that of Bethe free energy. Concave convex procedure (CCCP) proposed by Yuille is known to be an algorithm which guarantees to monotonically decrease both free energies.
In this paper, we generalize CCCP algorithm for Kikuchi free energy and present a new CCCP (NCCCP) algorithm which includes a high-dimensional free vector. The free vector enables NCCCP to be more stable and to reduce enormous computational cost underlying CCCP. Moreover it seems that there exists an optimal series of free vectors that can minimize the huge computational cost. This means that there exists an optimal concave and convex decomposition schedule in Kikuchi free energy.
Unable to display preview. Download preview PDF.
- 1.Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo (1988)Google Scholar
- 2.Murphy, K.P., Weiss, Y., Jordan, M.I.: Loopy Belief Propagation for Approximate Inference: An Empirical Study. In: 15th conference on Uncertainty in Artificial Intelligence, pp. 467–475 (1999)Google Scholar
- 7.Kabashima, Y., Saad, D.: The Belief in TAP. Neural Information Processing Systems 11 (1998)Google Scholar
- 8.Yedidia, J., Freeman, W., Weiss, Y.: Generalized Belief Propagation. Neural Information Processing Systems 13 (2000)Google Scholar
- 9.Welling, M., Teh, Y.W.: Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation. In: 17th conference on Uncertainty in Artificial Intelligence, pp. 554–561 (2001)Google Scholar