Mathematical Systems Theory in Biology, Communications, Computation, and Finance pp 275-299 | Cite as

# Belief Propagation on Partially Ordered Sets

## Abstract

In this paper, which is based on the important recent work of Yedidia, Freeman, and Weiss, we present a generalized form of belief propagation, viz. *belief propagation on a partially ordered set (PBP)*. PBP is an iterative message-passing algorithm for solving, either exactly or approximately, the *marginalized product density* problem, which is a general computational problem of wide applicability. We will show that PBP can be thought of as an algorithm for minimizing a certain “free energy” function, and by exploiting this interpretation, we will exhibit a one-to-one correspondence between the fixed points of PBP and the stationary points of the free energy.

## Keywords

Free Energy Belief Propagation Helmholtz Free Energy Hasse Diagram Cluster Variation Method## Preview

Unable to display preview. Download preview PDF.

## References

- [1]S.M. Aji and R.J. McEliece, “The generalized distributive law,”
*IEEE Trans. Inform. Theory*,**46**(2): 325–343 (March 2000).MathSciNetzbMATHCrossRefGoogle Scholar - [2]S.M. Aji and R.J. McEliece, “The generalized distributive law and free energy minimization,”
*Proc. 2001 Allerton Conf. Comm. Control and Computing*(Oct. 2001).Google Scholar - [3]T.M. Cover and J. A. Thomas,
*Elements of Information Theory*. New York: John Wiley and Sons, 1991.zbMATHCrossRefGoogle Scholar - [4]W.H. Fleming,
*Functions of Several Variables*. Reading, Mass.: Addison-Wesley, 1965.zbMATHGoogle Scholar - [5]R.A. Horn and C.R. Johnson,
*Matrix Analysis*. Cambridge: Cambridge University Press, 1985.zbMATHGoogle Scholar - [6]F.R. Kschichang, B.J. Frey, and H.-A. Loeliger, “Factor graphs and the sum-product algorithm,”
*IEEE Trans. Inform. Theory*,**47**(2): 498–519 (Feb. 2001).MathSciNetCrossRefGoogle Scholar - [7]P. Pakzad and V. Anantharam, “Belief propagation and statistical physics,”
*Proc. 2002 Conf. Inform. Sciences and Systems*, Princeton U., March 2002.Google Scholar - [8]J. Pearl,
*Probabilistic Reasoning in Intelligent Systems*. San Francisco: Morgan Kaufmann, 1988.Google Scholar - [9]G.R. Shafer and P.P. Shenoy, “Probability propagation,”
*Ann. Mat. Art. Intell*.,**2**: 327–352 (1990).MathSciNetzbMATHCrossRefGoogle Scholar - [10]R.P. Stanley,
*Enumerative Combinatorics, Vol. 1*. (Cambridge Studies in Advanced Mathematics 49) Cambridge: Cambridge University Press, 1997.zbMATHGoogle Scholar - [11]J.S. Yedidia, “An idiosyncratic journey beyond mean field theory,” pp. 21–35 in
*Advanced Mean Field Methods, Theory and Practice*, eds. Manfred Opper and David Saad, MIT Press, 2001.Google Scholar - [12]J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Generalized belief propagation,” pp. 689–695 in
*Advances in Neural Information Processing Systems 13*(2000) eds. Todd K. Leen, Thomas G. Dietterich, and Volker Tresp.Google Scholar - [13]J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Bethe free energy, Kikuchi approximations, and belief propagation algorithms,” available at http://www.merl.com/papers/TR2001-16/
- [14]J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Constructing free energy approximations and generalized belief propagation algorithms,” available at http://www.merl.com/papers/TR2002-35/
- [15]J.M. Yeomans,
*Statistical Mechanics of Phase Transitions*. Oxford: Oxford University Press, 1992.Google Scholar