# Region Graph Partition Function Expansion and Approximate Free Energy Landscapes: Theory and Some Numerical Results

## Abstract

Graphical models for finite-dimensional spin glasses and real-world combinatorial optimization and satisfaction problems usually have an abundant number of short loops. The cluster variation method and its extension, the region graph method, are theoretical approaches for treating the complicated short-loop-induced local correlations. For graphical models represented by non-redundant or redundant region graphs, approximate free energy landscapes are constructed in this paper through the mathematical framework of region graph partition function expansion. Several free energy functionals are obtained, each of which use a set of probability distribution functions or functionals as order parameters. These probability distribution function/functionals are required to satisfy the region graph belief-propagation equation or the region graph survey-propagation equation to ensure vanishing correction contributions of region subgraphs with dangling edges. As a simple application of the general theory, we perform region graph belief-propagation simulations on the square-lattice ferromagnetic Ising model and the Edwards-Anderson model. Considerable improvements over the conventional Bethe-Peierls approximation are achieved. Collective domains of different sizes in the disordered and frustrated square lattice are identified by the message-passing procedure. Such collective domains and the frustrations among them are responsible for the low-temperature glass-like dynamical behaviors of the system.

## Keywords

Region graph Belief propagation Bethe-Peierls free energy Edwards-Anderson spin glass Partition function expansion Graphical model## References

- 1.An, G.: A note on the cluster variation method. J. Stat. Phys.
**52**, 727–734 (1988) ADSMATHCrossRefGoogle Scholar - 2.Anderson, P.W.: Spin glass vii: Spin glass as paradigm. Phys. Today
**March**, 9–11 (1990) Google Scholar - 3.Aurell, E., Ollion, C., Roudi, Y.: Dynamics and performance of susceptibility propagation on synthetic data. Eur. Phys. J. B
**77**, 587–595 (2010) ADSCrossRefGoogle Scholar - 4.Bethe, H.A.: Statistical theory of superlattices. Proc. R. Soc. Lond. Ser. A
**150**, 552–575 (1935) ADSMATHCrossRefGoogle Scholar - 5.Braunstein, A., Zecchina, R.: Learning by message passing in networks of discrete synapses. Phys. Rev. Lett.
**96**, 030201 (2006) MathSciNetADSCrossRefGoogle Scholar - 6.Chertkov, M., Chernyak, V.Y.: Loop series for discrete statistical models on graphs. J. Stat. Mech. Theory Exp. P06009 (2006) Google Scholar
- 7.Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991) MATHCrossRefGoogle Scholar
- 8.Domínguez, E., Lage-Catellanos, A., Mulet, R., Ricci-Tersenghi, F., Rizzo, T.: Characterizing and improving generalized belief propagation algorithms on the 2d Edwards-Anderson model. J. Stat. Mech. Theory Exp. P12007 (2011) Google Scholar
- 9.Donoho, D.L., Maleki, A., Montanari, A.: Message-passing algorithms for compressed sensing. Proc. Natl. Acad. Sci. USA
**106**, 18914–18919 (2009) ADSCrossRefGoogle Scholar - 10.Ediger, M.D.: Spatially heterogeneous dynamics in supercooled liquids. Annu. Rev. Phys. Chem.
**51**, 99–128 (2000) ADSCrossRefGoogle Scholar - 11.Edwards, S.F., Anderson, P.W.: Theory of spin glasses. J. Phys. F, Met. Phys.
**5**, 965–974 (1975) ADSCrossRefGoogle Scholar - 12.Glotzer, S.C.: Spatially heterogeneous dynamics in liquids: insights from simulation. J. Non-Cryst. Solids
**274**, 342–355 (2000) ADSCrossRefGoogle Scholar - 13.Hartmann, A.K., Weigt, W.: Phase Transitions in Combinatorial Optimization Problems. Wiley-VCH, Weinheim (2005) MATHCrossRefGoogle Scholar
- 14.Kabashima, Y., Saad, D.: Statistical mechanics of error-correcting codes. Europhys. Lett.
**45**, 97–103 (1999) ADSCrossRefGoogle Scholar - 15.Kikuchi, R.: A theory of cooperative phenomena. Phys. Rev.
**81**, 988–1003 (1951) MathSciNetADSMATHCrossRefGoogle Scholar - 16.Kramers, H.A., Wannier, G.H.: Statistics of the two-dimensional ferromagnet. Part I. Phys. Rev.
**60**, 252–262 (1941) MathSciNetADSCrossRefGoogle Scholar - 17.Krzakala, F., Mézard, M., Sausset, F., Sun, Y.F., Zdeborová, L.: Statistical physics-based reconstruction in compressed sensing. arXiv:1109.4424 (2011)
- 18.Krzakala, F., Montanari, A., Ricci-Tersenghi, F., Semerjian, G., Zdeborova, L.: Gibbs states and the set of solutions of random constraint satisfaction problems. Proc. Natl. Acad. Sci. USA
**104**, 10318–10323 (2007) MathSciNetADSMATHCrossRefGoogle Scholar - 19.Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory
**47**, 498–519 (2001) MathSciNetMATHCrossRefGoogle Scholar - 20.Lage-Castellanos, A., Mulet, R., Ricci-Tersenghi, F., Rizzo, T.: Inference algorithm for finite-dimensional spin glasses: Belief propagation on the dual lattice. Phys. Rev. E
**84**, 046706 (2011) ADSCrossRefGoogle Scholar - 21.Lage-Castellanos, A., Mulet, R., Ricci-Tersenghi, F., Rizzo, T.: Replica cluster variational method: the replica symmetric solution for the 2d random bond Ising model. arXiv:1204.0439 (2012)
- 22.Mézard, M., Montanari, A.: Reconstruction on trees and spin glass transition. J. Stat. Phys.
**124**, 1317–1350 (2006) MathSciNetADSMATHCrossRefGoogle Scholar - 23.Mézard, M., Montanari, A.: Information, Physics, and Computation. Oxford University Press, New York (2009) MATHCrossRefGoogle Scholar
- 24.Mézard, M., Mora, T.: Constraint satisfaction problems and neural networks: a statistical physics perspective. J. Physiol. Paris
**103**, 107–113 (2009) CrossRefGoogle Scholar - 25.Mézard, M., Parisi, G.: The Bethe lattice spin glass revisited. Eur. Phys. J. B
**20**, 217–233 (2001) ADSCrossRefGoogle Scholar - 26.Mézard, M., Parisi, G., Virasoro, M.A.: SK model: the replica solution without replicas. Europhys. Lett.
**1**, 77–82 (1986) ADSCrossRefGoogle Scholar - 27.Mézard, M., Parisi, G., Virasoro, M.A.: Spin Glass Theory and Beyond. World Scientific, Singapore (1987) MATHGoogle Scholar
- 28.Mézard, M., Parisi, G., Zecchina, R.: Analytic and algorithmic solution of random satisfiability problems. Science
**297**, 812–815 (2002) ADSCrossRefGoogle Scholar - 29.Monasson, R.: Structural glass transition and the entropy of the metastable states. Phys. Rev. Lett.
**75**, 2847–2850 (1995) ADSCrossRefGoogle Scholar - 30.Monasson, R.: Optimization problems and replica symmetry breaking in finite connectivity spin glasses. J. Phys. A, Math. Gen.
**31**, 513–529 (1998) MathSciNetADSMATHCrossRefGoogle Scholar - 31.Montanari, A., Rizzo, T.: How to compute loop corrections to Bethe approximation. J. Stat. Mech. Theory Exp. P10011 (2005) Google Scholar
- 32.Montanari, A., Semerjian, G.: On the dynamics of the glass transition on Bethe lattices. J. Stat. Phys.
**124**, 103–189 (2006) MathSciNetADSMATHCrossRefGoogle Scholar - 33.Morgenstern, I., Binder, K.: Magnetic correlations in two-dimensional spin-glasses. Phys. Rev. B
**22**, 288–303 (1980) ADSCrossRefGoogle Scholar - 34.Morita, T., Suzuki, M., Wada, K., Kaburagi, M. (eds.): Foundations and applications of cluster variation method and path probability method. Prog. Theor. Phys., Suppl.
**115**, 1–378 (1994) CrossRefGoogle Scholar - 35.Onsager, L.: Crystal statistics I. A two-dimensional model with an order-disorder transition. Phys. Rev.
**65**, 117–149 (1944) MathSciNetADSMATHCrossRefGoogle Scholar - 36.Parisi, G.: Infinite number of order parameters for spin-glasses. Phys. Rev. Lett.
**43**, 1754–1756 (1979) ADSCrossRefGoogle Scholar - 37.Parisi, G., Slanina, F.: Loop expansion around the Bethe-Peierls approximation for lattice models. J. Stat. Mech. Theory Exp. L02003 (2006) Google Scholar
- 38.Peierls, R.: On Ising’s model of ferromagnetism. Proc. Camb. Philol. Soc.
**32**, 477–481 (1936) ADSMATHCrossRefGoogle Scholar - 39.Pelizzola, A.: Cluster variation method in statistical physics and probabilistic graphical models. J. Phys. A, Math. Gen.
**38**, R309–R339 (2005) MathSciNetADSCrossRefGoogle Scholar - 40.Rizzo, T., Lage-Castellanos, A., Mulet, R., Ricci-Tersenghi, F.: Replica cluster variational method. J. Stat. Phys.
**139**, 375–416 (2010) MathSciNetADSMATHCrossRefGoogle Scholar - 41.Rizzo, T., Wemmenhove, B., Kappen, H.J.: Cavity approximation for graphical models. Phys. Rev. E
**76**, 011102 (2007) ADSCrossRefGoogle Scholar - 42.Rota, G.C.: On the foundations of combinatorial theory I. Theory of Möbius functions. Z. Wahrscheinlichkeitstheor.
**2**, 340–368 (1964) MathSciNetMATHCrossRefGoogle Scholar - 43.Roudi, Y., Hertz, J.: Mean field theory for nonequilibrium network reconstruction. Phys. Rev. Lett.
**106**, 048702 (2011) ADSCrossRefGoogle Scholar - 44.Saul, L., Kardar, M.: Exact integer algorithm for the two-dimensional ±
*j*Ising spin glass. Phys. Rev. E**48**, R3221–R3224 (1993) ADSCrossRefGoogle Scholar - 45.Sherrington, D., Kirkpatrick, S.: Solvable model of a spin-glass. Phys. Rev. Lett.
**35**, 1792–1796 (1975) ADSCrossRefGoogle Scholar - 46.Suzuki, M., Hu, X., Hatano, N., Katori, M., Minami, K., Lipowski, A., Nonomura, Y.: Coherent Anomaly Method: Mean Field, Fluctuations and Systematics. World Scientific, Singapore (1995) Google Scholar
- 47.Tanaka, T.: Statistical mechanics of CDMA multiuser demodulation. Europhys. Lett.
**54**, 540–546 (2001) ADSCrossRefGoogle Scholar - 48.Thomas, C.K., Huse, D.A., Middleton, A.A.: Zero- and low-temperature behavior of the two-dimensional ±
*j*Ising spin glass. Phys. Rev. Lett.**107**, 047203 (2011) ADSCrossRefGoogle Scholar - 49.Toulouse, G.: Theory of the frustration effect in spin glasses: I. Commun. Phys.
**2**, 115–119 (1977) Google Scholar - 50.Viana, L., Bray, A.J.: Phase diagrams for dilute spin glasses. J. Phys. C, Solid State Phys.
**18**, 3037–3051 (1985) ADSCrossRefGoogle Scholar - 51.Weigt, M., White, R.A., Szurmant, H., Hoch, J.A., Hwa, T.: Identification of direct residue contacts in protein-protein interaction by message-passing. Proc. Natl. Acad. Sci. USA
**106**, 67–72 (2009) ADSCrossRefGoogle Scholar - 52.Xiao, J.Q., Zhou, H.: Partition function loop series for a general graphical model: free-energy corrections and message-passing equations. J. Phys. A, Math. Theor.
**44**, 425001 (2011) MathSciNetADSCrossRefGoogle Scholar - 53.Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free-energy approximations and generalized belief-propagation algorithms. IEEE Trans. Inf. Theory
**51**, 2282–2312 (2005) MathSciNetCrossRefGoogle Scholar - 54.Zdeborová, L.: Statistical physics of hard optimization problems. Acta Phys. Slovaca
**59**, 169–303 (2009) ADSCrossRefGoogle Scholar - 55.Zhou, H.: Boltzmann distribution of free energies in a finite-connectivity spin-glass system and the cavity approach. Front. Phys. China
**2**, 238–250 (2007) ADSCrossRefGoogle Scholar - 56.Zhou, H., Wang, C., Xiao, J.Q., Bi, Z.: Partition function expansion on region-graphs and message-passing equations. J. Stat. Mech. Theory Exp. L12001 (2011) Google Scholar