Journal of Statistical Physics

, Volume 148, Issue 3, pp 513–547 | Cite as

Region Graph Partition Function Expansion and Approximate Free Energy Landscapes: Theory and Some Numerical Results

  • Haijun ZhouEmail author
  • Chuang Wang


Graphical models for finite-dimensional spin glasses and real-world combinatorial optimization and satisfaction problems usually have an abundant number of short loops. The cluster variation method and its extension, the region graph method, are theoretical approaches for treating the complicated short-loop-induced local correlations. For graphical models represented by non-redundant or redundant region graphs, approximate free energy landscapes are constructed in this paper through the mathematical framework of region graph partition function expansion. Several free energy functionals are obtained, each of which use a set of probability distribution functions or functionals as order parameters. These probability distribution function/functionals are required to satisfy the region graph belief-propagation equation or the region graph survey-propagation equation to ensure vanishing correction contributions of region subgraphs with dangling edges. As a simple application of the general theory, we perform region graph belief-propagation simulations on the square-lattice ferromagnetic Ising model and the Edwards-Anderson model. Considerable improvements over the conventional Bethe-Peierls approximation are achieved. Collective domains of different sizes in the disordered and frustrated square lattice are identified by the message-passing procedure. Such collective domains and the frustrations among them are responsible for the low-temperature glass-like dynamical behaviors of the system.


Region graph Belief propagation Bethe-Peierls free energy Edwards-Anderson spin glass Partition function expansion Graphical model 


  1. 1.
    An, G.: A note on the cluster variation method. J. Stat. Phys. 52, 727–734 (1988) ADSzbMATHCrossRefGoogle Scholar
  2. 2.
    Anderson, P.W.: Spin glass vii: Spin glass as paradigm. Phys. Today March, 9–11 (1990) Google Scholar
  3. 3.
    Aurell, E., Ollion, C., Roudi, Y.: Dynamics and performance of susceptibility propagation on synthetic data. Eur. Phys. J. B 77, 587–595 (2010) ADSCrossRefGoogle Scholar
  4. 4.
    Bethe, H.A.: Statistical theory of superlattices. Proc. R. Soc. Lond. Ser. A 150, 552–575 (1935) ADSzbMATHCrossRefGoogle Scholar
  5. 5.
    Braunstein, A., Zecchina, R.: Learning by message passing in networks of discrete synapses. Phys. Rev. Lett. 96, 030201 (2006) MathSciNetADSCrossRefGoogle Scholar
  6. 6.
    Chertkov, M., Chernyak, V.Y.: Loop series for discrete statistical models on graphs. J. Stat. Mech. Theory Exp. P06009 (2006) Google Scholar
  7. 7.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991) zbMATHCrossRefGoogle Scholar
  8. 8.
    Domínguez, E., Lage-Catellanos, A., Mulet, R., Ricci-Tersenghi, F., Rizzo, T.: Characterizing and improving generalized belief propagation algorithms on the 2d Edwards-Anderson model. J. Stat. Mech. Theory Exp. P12007 (2011) Google Scholar
  9. 9.
    Donoho, D.L., Maleki, A., Montanari, A.: Message-passing algorithms for compressed sensing. Proc. Natl. Acad. Sci. USA 106, 18914–18919 (2009) ADSCrossRefGoogle Scholar
  10. 10.
    Ediger, M.D.: Spatially heterogeneous dynamics in supercooled liquids. Annu. Rev. Phys. Chem. 51, 99–128 (2000) ADSCrossRefGoogle Scholar
  11. 11.
    Edwards, S.F., Anderson, P.W.: Theory of spin glasses. J. Phys. F, Met. Phys. 5, 965–974 (1975) ADSCrossRefGoogle Scholar
  12. 12.
    Glotzer, S.C.: Spatially heterogeneous dynamics in liquids: insights from simulation. J. Non-Cryst. Solids 274, 342–355 (2000) ADSCrossRefGoogle Scholar
  13. 13.
    Hartmann, A.K., Weigt, W.: Phase Transitions in Combinatorial Optimization Problems. Wiley-VCH, Weinheim (2005) zbMATHCrossRefGoogle Scholar
  14. 14.
    Kabashima, Y., Saad, D.: Statistical mechanics of error-correcting codes. Europhys. Lett. 45, 97–103 (1999) ADSCrossRefGoogle Scholar
  15. 15.
    Kikuchi, R.: A theory of cooperative phenomena. Phys. Rev. 81, 988–1003 (1951) MathSciNetADSzbMATHCrossRefGoogle Scholar
  16. 16.
    Kramers, H.A., Wannier, G.H.: Statistics of the two-dimensional ferromagnet. Part I. Phys. Rev. 60, 252–262 (1941) MathSciNetADSCrossRefGoogle Scholar
  17. 17.
    Krzakala, F., Mézard, M., Sausset, F., Sun, Y.F., Zdeborová, L.: Statistical physics-based reconstruction in compressed sensing. arXiv:1109.4424 (2011)
  18. 18.
    Krzakala, F., Montanari, A., Ricci-Tersenghi, F., Semerjian, G., Zdeborova, L.: Gibbs states and the set of solutions of random constraint satisfaction problems. Proc. Natl. Acad. Sci. USA 104, 10318–10323 (2007) MathSciNetADSzbMATHCrossRefGoogle Scholar
  19. 19.
    Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory 47, 498–519 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Lage-Castellanos, A., Mulet, R., Ricci-Tersenghi, F., Rizzo, T.: Inference algorithm for finite-dimensional spin glasses: Belief propagation on the dual lattice. Phys. Rev. E 84, 046706 (2011) ADSCrossRefGoogle Scholar
  21. 21.
    Lage-Castellanos, A., Mulet, R., Ricci-Tersenghi, F., Rizzo, T.: Replica cluster variational method: the replica symmetric solution for the 2d random bond Ising model. arXiv:1204.0439 (2012)
  22. 22.
    Mézard, M., Montanari, A.: Reconstruction on trees and spin glass transition. J. Stat. Phys. 124, 1317–1350 (2006) MathSciNetADSzbMATHCrossRefGoogle Scholar
  23. 23.
    Mézard, M., Montanari, A.: Information, Physics, and Computation. Oxford University Press, New York (2009) zbMATHCrossRefGoogle Scholar
  24. 24.
    Mézard, M., Mora, T.: Constraint satisfaction problems and neural networks: a statistical physics perspective. J. Physiol. Paris 103, 107–113 (2009) CrossRefGoogle Scholar
  25. 25.
    Mézard, M., Parisi, G.: The Bethe lattice spin glass revisited. Eur. Phys. J. B 20, 217–233 (2001) ADSCrossRefGoogle Scholar
  26. 26.
    Mézard, M., Parisi, G., Virasoro, M.A.: SK model: the replica solution without replicas. Europhys. Lett. 1, 77–82 (1986) ADSCrossRefGoogle Scholar
  27. 27.
    Mézard, M., Parisi, G., Virasoro, M.A.: Spin Glass Theory and Beyond. World Scientific, Singapore (1987) zbMATHGoogle Scholar
  28. 28.
    Mézard, M., Parisi, G., Zecchina, R.: Analytic and algorithmic solution of random satisfiability problems. Science 297, 812–815 (2002) ADSCrossRefGoogle Scholar
  29. 29.
    Monasson, R.: Structural glass transition and the entropy of the metastable states. Phys. Rev. Lett. 75, 2847–2850 (1995) ADSCrossRefGoogle Scholar
  30. 30.
    Monasson, R.: Optimization problems and replica symmetry breaking in finite connectivity spin glasses. J. Phys. A, Math. Gen. 31, 513–529 (1998) MathSciNetADSzbMATHCrossRefGoogle Scholar
  31. 31.
    Montanari, A., Rizzo, T.: How to compute loop corrections to Bethe approximation. J. Stat. Mech. Theory Exp. P10011 (2005) Google Scholar
  32. 32.
    Montanari, A., Semerjian, G.: On the dynamics of the glass transition on Bethe lattices. J. Stat. Phys. 124, 103–189 (2006) MathSciNetADSzbMATHCrossRefGoogle Scholar
  33. 33.
    Morgenstern, I., Binder, K.: Magnetic correlations in two-dimensional spin-glasses. Phys. Rev. B 22, 288–303 (1980) ADSCrossRefGoogle Scholar
  34. 34.
    Morita, T., Suzuki, M., Wada, K., Kaburagi, M. (eds.): Foundations and applications of cluster variation method and path probability method. Prog. Theor. Phys., Suppl. 115, 1–378 (1994) CrossRefGoogle Scholar
  35. 35.
    Onsager, L.: Crystal statistics I. A two-dimensional model with an order-disorder transition. Phys. Rev. 65, 117–149 (1944) MathSciNetADSzbMATHCrossRefGoogle Scholar
  36. 36.
    Parisi, G.: Infinite number of order parameters for spin-glasses. Phys. Rev. Lett. 43, 1754–1756 (1979) ADSCrossRefGoogle Scholar
  37. 37.
    Parisi, G., Slanina, F.: Loop expansion around the Bethe-Peierls approximation for lattice models. J. Stat. Mech. Theory Exp. L02003 (2006) Google Scholar
  38. 38.
    Peierls, R.: On Ising’s model of ferromagnetism. Proc. Camb. Philol. Soc. 32, 477–481 (1936) ADSzbMATHCrossRefGoogle Scholar
  39. 39.
    Pelizzola, A.: Cluster variation method in statistical physics and probabilistic graphical models. J. Phys. A, Math. Gen. 38, R309–R339 (2005) MathSciNetADSCrossRefGoogle Scholar
  40. 40.
    Rizzo, T., Lage-Castellanos, A., Mulet, R., Ricci-Tersenghi, F.: Replica cluster variational method. J. Stat. Phys. 139, 375–416 (2010) MathSciNetADSzbMATHCrossRefGoogle Scholar
  41. 41.
    Rizzo, T., Wemmenhove, B., Kappen, H.J.: Cavity approximation for graphical models. Phys. Rev. E 76, 011102 (2007) ADSCrossRefGoogle Scholar
  42. 42.
    Rota, G.C.: On the foundations of combinatorial theory I. Theory of Möbius functions. Z. Wahrscheinlichkeitstheor. 2, 340–368 (1964) MathSciNetzbMATHCrossRefGoogle Scholar
  43. 43.
    Roudi, Y., Hertz, J.: Mean field theory for nonequilibrium network reconstruction. Phys. Rev. Lett. 106, 048702 (2011) ADSCrossRefGoogle Scholar
  44. 44.
    Saul, L., Kardar, M.: Exact integer algorithm for the two-dimensional ±j Ising spin glass. Phys. Rev. E 48, R3221–R3224 (1993) ADSCrossRefGoogle Scholar
  45. 45.
    Sherrington, D., Kirkpatrick, S.: Solvable model of a spin-glass. Phys. Rev. Lett. 35, 1792–1796 (1975) ADSCrossRefGoogle Scholar
  46. 46.
    Suzuki, M., Hu, X., Hatano, N., Katori, M., Minami, K., Lipowski, A., Nonomura, Y.: Coherent Anomaly Method: Mean Field, Fluctuations and Systematics. World Scientific, Singapore (1995) Google Scholar
  47. 47.
    Tanaka, T.: Statistical mechanics of CDMA multiuser demodulation. Europhys. Lett. 54, 540–546 (2001) ADSCrossRefGoogle Scholar
  48. 48.
    Thomas, C.K., Huse, D.A., Middleton, A.A.: Zero- and low-temperature behavior of the two-dimensional ±j Ising spin glass. Phys. Rev. Lett. 107, 047203 (2011) ADSCrossRefGoogle Scholar
  49. 49.
    Toulouse, G.: Theory of the frustration effect in spin glasses: I. Commun. Phys. 2, 115–119 (1977) Google Scholar
  50. 50.
    Viana, L., Bray, A.J.: Phase diagrams for dilute spin glasses. J. Phys. C, Solid State Phys. 18, 3037–3051 (1985) ADSCrossRefGoogle Scholar
  51. 51.
    Weigt, M., White, R.A., Szurmant, H., Hoch, J.A., Hwa, T.: Identification of direct residue contacts in protein-protein interaction by message-passing. Proc. Natl. Acad. Sci. USA 106, 67–72 (2009) ADSCrossRefGoogle Scholar
  52. 52.
    Xiao, J.Q., Zhou, H.: Partition function loop series for a general graphical model: free-energy corrections and message-passing equations. J. Phys. A, Math. Theor. 44, 425001 (2011) MathSciNetADSCrossRefGoogle Scholar
  53. 53.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free-energy approximations and generalized belief-propagation algorithms. IEEE Trans. Inf. Theory 51, 2282–2312 (2005) MathSciNetCrossRefGoogle Scholar
  54. 54.
    Zdeborová, L.: Statistical physics of hard optimization problems. Acta Phys. Slovaca 59, 169–303 (2009) ADSCrossRefGoogle Scholar
  55. 55.
    Zhou, H.: Boltzmann distribution of free energies in a finite-connectivity spin-glass system and the cavity approach. Front. Phys. China 2, 238–250 (2007) ADSCrossRefGoogle Scholar
  56. 56.
    Zhou, H., Wang, C., Xiao, J.Q., Bi, Z.: Partition function expansion on region-graphs and message-passing equations. J. Stat. Mech. Theory Exp. L12001 (2011) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.State Key Laboratory of Theoretical Physics, Institute of Theoretical PhysicsChinese Academy of SciencesBeijingChina

Personalised recommendations