Belief Propagation on Partially Ordered Sets
In this paper, which is based on the important recent work of Yedidia, Freeman, and Weiss, we present a generalized form of belief propagation, viz. belief propagation on a partially ordered set (PBP). PBP is an iterative message-passing algorithm for solving, either exactly or approximately, the marginalized product density problem, which is a general computational problem of wide applicability. We will show that PBP can be thought of as an algorithm for minimizing a certain “free energy” function, and by exploiting this interpretation, we will exhibit a one-to-one correspondence between the fixed points of PBP and the stationary points of the free energy.
KeywordsFree Energy Belief Propagation Helmholtz Free Energy Hasse Diagram Cluster Variation Method
Unable to display preview. Download preview PDF.
- S.M. Aji and R.J. McEliece, “The generalized distributive law and free energy minimization,” Proc. 2001 Allerton Conf. Comm. Control and Computing (Oct. 2001).Google Scholar
- P. Pakzad and V. Anantharam, “Belief propagation and statistical physics,” Proc. 2002 Conf. Inform. Sciences and Systems, Princeton U., March 2002.Google Scholar
- J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Francisco: Morgan Kaufmann, 1988.Google Scholar
- J.S. Yedidia, “An idiosyncratic journey beyond mean field theory,” pp. 21–35 in Advanced Mean Field Methods, Theory and Practice, eds. Manfred Opper and David Saad, MIT Press, 2001.Google Scholar
- J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Generalized belief propagation,” pp. 689–695 in Advances in Neural Information Processing Systems 13 (2000) eds. Todd K. Leen, Thomas G. Dietterich, and Volker Tresp.Google Scholar
- J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Bethe free energy, Kikuchi approximations, and belief propagation algorithms,” available at http://www.merl.com/papers/TR2001-16/
- J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Constructing free energy approximations and generalized belief propagation algorithms,” available at http://www.merl.com/papers/TR2002-35/
- J.M. Yeomans, Statistical Mechanics of Phase Transitions. Oxford: Oxford University Press, 1992.Google Scholar