On the Minima of Bethe Free Energy in Gaussian Distributions
Belief propagation (BP) is effective for computing marginal probabilities of a high dimensional probability distribution. Loopy belief propagation (LBP) is known not to compute precise marginal probabilities and not to guarantee its convergence. The fixed points of LBP are known to accord with the extrema of Bethe free energy. Hence, the fixed points are analyzed by minimizing the Bethe free energy.
In this paper, we consider the Bethe free energy in Gaussian distributions and analytically clarify the extrema, equivalently, the fixed points of LBP for some particular cases. The analytical results tell us a necessary condition for LBP convergence and the quantities which determine the accuracy of LBP in Gaussian distributions. Based on the analytical results, we perform numerical experiments of LBP and compare the results with analytical solutions.
KeywordsBelief Propagation Marginal Probability Single Loop LDPC Code Neural Computation
Unable to display preview. Download preview PDF.
- 2.Weiss, Y.: Belief propagation and revision in networks with loops. Technical report 1616, MIT AI lab (1997)Google Scholar
- 7.Kabashima, Y., Saad, D.: The TAP approach to intensive and extensive connectivity systems. In: Opper, M., Saad, D. (eds.) Advanced Mean Field Methods -Theory and Practice, pp. 65–84. MIT Press, Cambridge (2001)Google Scholar
- 8.Yedidia, J., Freeman, W., Weiss, Y.: Bethe free energy, kikuchi approximations, and belief propagation algorithms. Technical Report TR2001-16, Mitsubishi Electric Research Laboratories (2001)Google Scholar
- 10.Tanaka, K., Inoue, J., Titterington, D.M.: Loopy belief propagation and probabilistic image processing. In: proceedings of 2003 IEEE International workshop on Neural Networks for Signal Processing, vol. 13, pp. 383–392 (2003)Google Scholar
- 11.Nishiyama, Y., Watanabe, S.: Theoretical Analysis of Accuracy of Belief Propagation in Gaussian Models. IEICE Technical Report 107(50), 23–28 (2007)Google Scholar
- 12.Nishiyama, Y., Watanabe, S.: Theoretical Analysis of Accuracy of Gaussian Belief Propagation. In: Proceedings of International Conference on Artificial Neural Networks 2007, pp. 29–38 (2007)Google Scholar
- 14.Tanaka, K.: Generalized belief propagation formula in probabilistic information processing based on gaussian graphical model. IEICE D-II, Vol. J88-D-II, No. 12, pp. 2368–2379 (in Japanese) (2005)Google Scholar