Advertisement

Journal of Statistical Physics

, Volume 147, Issue 2, pp 252–314 | Cite as

Adaptive Cluster Expansion for the Inverse Ising Problem: Convergence, Algorithm and Tests

  • S. Cocco
  • R. MonassonEmail author
Article

Abstract

We present a procedure to solve the inverse Ising problem, that is, to find the interactions between a set of binary variables from the measure of their equilibrium correlations. The method consists in constructing and selecting specific clusters of spins, based on their contributions to the cross-entropy of the Ising model. Small contributions are discarded to avoid overfitting and to make the computation tractable. The properties of the cluster expansion and its performances on synthetic data are studied. To make the implementation easier we give the pseudo-code of the algorithm.

Keywords

Ising model Statistical inference Inverse problems Inverse susceptibility Cluster expansion 

Notes

Acknowledgements

We are grateful to J. Barton, J. Lebowitz, E. Speer for very useful and stimulating discussions, in particular regarding the correspondence between the inverse susceptibility and the direct correlation functions and the practical implementation of the inference algorithm. We thank E. Aurell for pointing to us the difference between P and Q, see Sect. 5.4.1.

References

  1. 1.
    Brush, S.G.: Rev. Mod. Phys. 39, 883 (1967) ADSCrossRefGoogle Scholar
  2. 2.
    Schneidman, E., Berry, M.J. II, Segev, R., Bialek, W.: Nature 440, 1007 (2006) ADSCrossRefGoogle Scholar
  3. 3.
    Tkacik, G., Schneidman, E., Berry, M.J. II, Bialek, W.: arXiv:q-Bio.NC/0611072 (2006)
  4. 4.
    Marre, O., El Boustani, S., Frégnac, Y., Destexhe, A.: Phys. Rev. Lett. 102, 138101 (2009) ADSCrossRefGoogle Scholar
  5. 5.
    Peyrache, A., et al.: Nat. Neurosci. 12, 919 (2009) CrossRefGoogle Scholar
  6. 6.
    Weigt, M., et al.: Proc. Natl. Acad. Sci. 106, 67 (2009) ADSCrossRefGoogle Scholar
  7. 7.
    Balakrishnan, S., et al.: Proteins 79, 1061 (2011) CrossRefGoogle Scholar
  8. 8.
    Bialek, W., Cavagna, A., Giardina, I., Mora, T., Silvestri, E., Viale, M., Walczak, A.M.: arXiv:1107.0604 (2011)
  9. 9.
    Cocco, S., Monasson, R.: Phys. Rev. Lett. 106, 090601 (2011) ADSCrossRefGoogle Scholar
  10. 10.
    Jaynes, E.T.: Proc. IEEE 70, 939 (1982) ADSCrossRefGoogle Scholar
  11. 11.
    Cocco, S., Leibler, S., Monasson, R.: Proc. Natl. Acad. Sci. 106, 14058 (2009) ADSCrossRefGoogle Scholar
  12. 12.
    Opper, M., Saad, D. (eds.): Advanced Mean Field Methods: Theory and Practice. MIT Press, Cambridge (2001) zbMATHGoogle Scholar
  13. 13.
    Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: Cogn. Sci. 9, 147 (1985) CrossRefGoogle Scholar
  14. 14.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Berlin (2009) zbMATHGoogle Scholar
  15. 15.
    Wallace, C.S.: Statistical and Inductive Inference by Minimum Message Length. Information Science and Statistics. Springer, Berlin (2005) zbMATHGoogle Scholar
  16. 16.
    Ravikumar, P., Wainwright, M.J., Lafferty, J.: Ann. Stat. 38, 1287 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Bento, J., Montanari, A.: Which graphical models are difficult to learn? In: NIPS, 2009 Google Scholar
  18. 18.
    Jerrum, M., Sinclair, A.: The Markov chain Monte Carlo method: an approach to approximate counting and integration. In: Hochbaum, D.S. (ed.) Approximation Algorithms for NP-Hard Problems. PWS, Boston (1996) Google Scholar
  19. 19.
    Cagliotti, E., Kuna, T., Lebowitz, J.L., Speer, E.R.: Markov Process. Relat. Fields 12, 257 (2006) Google Scholar
  20. 20.
    Kuna, T., Lebowitz, J.L., Speer, E.R.: J. Stat. Phys. 129, 417 (2007) MathSciNetADSzbMATHCrossRefGoogle Scholar
  21. 21.
    Swendsen, R.H.: Phys. Rev. Lett. 52, 1165 (1984) MathSciNetADSCrossRefGoogle Scholar
  22. 22.
    Meinshausen, N., Bühlmann, P.: Ann. Stat. 34, 1436 (2006) zbMATHCrossRefGoogle Scholar
  23. 23.
    Aurell, E., Ekeberg, M.: arXiv:1107.3536 (2011)
  24. 24.
    Plefka, T.: J. Phys. A, Math. Gen. 15, 1971 (1982) MathSciNetADSCrossRefGoogle Scholar
  25. 25.
    Georges, A., Yedidia, J.: J. Phys. A, Math. Gen. 24, 2173 (1991) ADSCrossRefGoogle Scholar
  26. 26.
    Georges, A.: In: Mancini, F., Avella, A. (eds.) Lectures on the Physics of Highly Correlated Electron Systems VIII: 8th Training Course in the Physics Correlated Electron Systems and High-Tc Superconductors, AIP Conf. Proc., vol. 715, p. 3 (2004) Google Scholar
  27. 27.
    Thouless, D.J., Anderson, P.W., Palmer, R.G.: Philos. Mag. 35, 593 (1977) ADSCrossRefGoogle Scholar
  28. 28.
    Sherrington, D., Kirkpatrick, S.: Phys. Rev. Lett. 35, 1792 (1975) ADSCrossRefGoogle Scholar
  29. 29.
    Tanaka, T.: Phys. Rev. E 58, 2302 (1998) ADSCrossRefGoogle Scholar
  30. 30.
    Roudi, Y., Aurell, E., Hertz, J.: Front. Comput. Neurosci. 3, 22 (2009) CrossRefGoogle Scholar
  31. 31.
    Sessak, V., Monasson, R.: J. Phys. A 42, 055001 (2009) MathSciNetADSCrossRefGoogle Scholar
  32. 32.
    Pelizzola, A.: J. Phys. A 38, R 309 (2005) MathSciNetADSCrossRefGoogle Scholar
  33. 33.
    Mézard, M., Mora, T.: J. Physiol. (Paris) 103, 107 (2009) CrossRefGoogle Scholar
  34. 34.
    Marinari, E., Van Kerrebroeck, V.: J. Stat. Mech. P02008 (2010) Google Scholar
  35. 35.
    de Dominicis, C.: J. Math. Phys. 3, 983 (1962) ADSCrossRefGoogle Scholar
  36. 36.
    Hansen, J.P., McDonald, I.R.: Theory of Simple Liquids. Academic Press, New York (1976) Google Scholar
  37. 37.
    Frisch, H.L., Lebowitz, J.L.: The Equilibrium Theory of Classical Fluids. Benjamin, New York (1964). (A lecture note and reprint volume) zbMATHGoogle Scholar
  38. 38.
    Flajolet, P., Gourdon, X., Dumas, P.: Theor. Comput. Sci. 144, 3 (1995) MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Schnitzer, M.J., Meister, M.: Neuron 37, 499–511 (2003) CrossRefGoogle Scholar
  40. 40.
    Gumbel, E.J.: Statistics of Extremes. Dover, New York (2004) zbMATHGoogle Scholar
  41. 41.
    MacKay, D.J.C.: Neural Comput. 4, 415 (1991) MathSciNetCrossRefGoogle Scholar
  42. 42.
    Bray, A.J., Moore, M.A.: J. Phys. A 10, 1927 (1977) MathSciNetADSCrossRefGoogle Scholar
  43. 43.
    Percus, J.K., Yevick, G.J.: Phys. Rev. 110, 1 (1958) MathSciNetADSzbMATHCrossRefGoogle Scholar
  44. 44.
    Borzi, C., Ord, G., Percus, J.K.: J. Stat. Phys. 46, 51 (1986) MathSciNetADSCrossRefGoogle Scholar
  45. 45.
    Barton, J.: private communication Google Scholar
  46. 46.
    Gori, G., Trombettoni, A.: J. Stat. Mech. P10021 (2011) Google Scholar
  47. 47.
    Fisher, M.: J. Math. Phys. (N.Y.) 5, 944 (1964) ADSCrossRefGoogle Scholar
  48. 48.
    Fisher, M.: Phys. Rev. 162, 480 (1967) ADSCrossRefGoogle Scholar
  49. 49.
    Zobin, D.: Phys. Rev. 5, 2387 (1978) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Simons Center for Systems BiologyInstitute for Advanced StudyPrincetonUSA
  2. 2.Laboratoire de Physique Statistique de l’ENSCNRS & UPMCParisFrance
  3. 3.Laboratoire de Physique Théorique de l’ENSCNRS & UPMCParisFrance

Personalised recommendations