# Adaptive Cluster Expansion for the Inverse Ising Problem: Convergence, Algorithm and Tests

- 711 Downloads
- 39 Citations

## Abstract

We present a procedure to solve the inverse Ising problem, that is, to find the interactions between a set of binary variables from the measure of their equilibrium correlations. The method consists in constructing and selecting specific clusters of spins, based on their contributions to the cross-entropy of the Ising model. Small contributions are discarded to avoid overfitting and to make the computation tractable. The properties of the cluster expansion and its performances on synthetic data are studied. To make the implementation easier we give the pseudo-code of the algorithm.

## Keywords

Ising model Statistical inference Inverse problems Inverse susceptibility Cluster expansion## Notes

### Acknowledgements

We are grateful to J. Barton, J. Lebowitz, E. Speer for very useful and stimulating discussions, in particular regarding the correspondence between the inverse susceptibility and the direct correlation functions and the practical implementation of the inference algorithm. We thank E. Aurell for pointing to us the difference between *P* and *Q*, see Sect. 5.4.1.

## References

- 1.Brush, S.G.: Rev. Mod. Phys.
**39**, 883 (1967) ADSCrossRefGoogle Scholar - 2.Schneidman, E., Berry, M.J. II, Segev, R., Bialek, W.: Nature
**440**, 1007 (2006) ADSCrossRefGoogle Scholar - 3.Tkacik, G., Schneidman, E., Berry, M.J. II, Bialek, W.: arXiv:q-Bio.NC/0611072 (2006)
- 4.Marre, O., El Boustani, S., Frégnac, Y., Destexhe, A.: Phys. Rev. Lett.
**102**, 138101 (2009) ADSCrossRefGoogle Scholar - 5.Peyrache, A., et al.: Nat. Neurosci.
**12**, 919 (2009) CrossRefGoogle Scholar - 6.Weigt, M., et al.: Proc. Natl. Acad. Sci.
**106**, 67 (2009) ADSCrossRefGoogle Scholar - 7.Balakrishnan, S., et al.: Proteins
**79**, 1061 (2011) CrossRefGoogle Scholar - 8.Bialek, W., Cavagna, A., Giardina, I., Mora, T., Silvestri, E., Viale, M., Walczak, A.M.: arXiv:1107.0604 (2011)
- 9.Cocco, S., Monasson, R.: Phys. Rev. Lett.
**106**, 090601 (2011) ADSCrossRefGoogle Scholar - 10.Jaynes, E.T.: Proc. IEEE
**70**, 939 (1982) ADSCrossRefGoogle Scholar - 11.Cocco, S., Leibler, S., Monasson, R.: Proc. Natl. Acad. Sci.
**106**, 14058 (2009) ADSCrossRefGoogle Scholar - 12.Opper, M., Saad, D. (eds.): Advanced Mean Field Methods: Theory and Practice. MIT Press, Cambridge (2001) zbMATHGoogle Scholar
- 13.Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: Cogn. Sci.
**9**, 147 (1985) CrossRefGoogle Scholar - 14.Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Berlin (2009) zbMATHGoogle Scholar
- 15.Wallace, C.S.: Statistical and Inductive Inference by Minimum Message Length. Information Science and Statistics. Springer, Berlin (2005) zbMATHGoogle Scholar
- 16.Ravikumar, P., Wainwright, M.J., Lafferty, J.: Ann. Stat.
**38**, 1287 (2010) MathSciNetzbMATHCrossRefGoogle Scholar - 17.Bento, J., Montanari, A.: Which graphical models are difficult to learn? In: NIPS, 2009 Google Scholar
- 18.Jerrum, M., Sinclair, A.: The Markov chain Monte Carlo method: an approach to approximate counting and integration. In: Hochbaum, D.S. (ed.) Approximation Algorithms for NP-Hard Problems. PWS, Boston (1996) Google Scholar
- 19.Cagliotti, E., Kuna, T., Lebowitz, J.L., Speer, E.R.: Markov Process. Relat. Fields
**12**, 257 (2006) Google Scholar - 20.Kuna, T., Lebowitz, J.L., Speer, E.R.: J. Stat. Phys.
**129**, 417 (2007) MathSciNetADSzbMATHCrossRefGoogle Scholar - 21.Swendsen, R.H.: Phys. Rev. Lett.
**52**, 1165 (1984) MathSciNetADSCrossRefGoogle Scholar - 22.Meinshausen, N., Bühlmann, P.: Ann. Stat.
**34**, 1436 (2006) zbMATHCrossRefGoogle Scholar - 23.Aurell, E., Ekeberg, M.: arXiv:1107.3536 (2011)
- 24.Plefka, T.: J. Phys. A, Math. Gen.
**15**, 1971 (1982) MathSciNetADSCrossRefGoogle Scholar - 25.Georges, A., Yedidia, J.: J. Phys. A, Math. Gen.
**24**, 2173 (1991) ADSCrossRefGoogle Scholar - 26.Georges, A.: In: Mancini, F., Avella, A. (eds.) Lectures on the Physics of Highly Correlated Electron Systems VIII: 8th Training Course in the Physics Correlated Electron Systems and High-Tc Superconductors, AIP Conf. Proc., vol. 715, p. 3 (2004) Google Scholar
- 27.Thouless, D.J., Anderson, P.W., Palmer, R.G.: Philos. Mag.
**35**, 593 (1977) ADSCrossRefGoogle Scholar - 28.Sherrington, D., Kirkpatrick, S.: Phys. Rev. Lett.
**35**, 1792 (1975) ADSCrossRefGoogle Scholar - 29.Tanaka, T.: Phys. Rev. E
**58**, 2302 (1998) ADSCrossRefGoogle Scholar - 30.Roudi, Y., Aurell, E., Hertz, J.: Front. Comput. Neurosci.
**3**, 22 (2009) CrossRefGoogle Scholar - 31.Sessak, V., Monasson, R.: J. Phys. A
**42**, 055001 (2009) MathSciNetADSCrossRefGoogle Scholar - 32.Pelizzola, A.: J. Phys. A
**38**, R 309 (2005) MathSciNetADSCrossRefGoogle Scholar - 33.Mézard, M., Mora, T.: J. Physiol. (Paris)
**103**, 107 (2009) CrossRefGoogle Scholar - 34.Marinari, E., Van Kerrebroeck, V.: J. Stat. Mech. P02008 (2010) Google Scholar
- 35.de Dominicis, C.: J. Math. Phys.
**3**, 983 (1962) ADSCrossRefGoogle Scholar - 36.Hansen, J.P., McDonald, I.R.: Theory of Simple Liquids. Academic Press, New York (1976) Google Scholar
- 37.Frisch, H.L., Lebowitz, J.L.: The Equilibrium Theory of Classical Fluids. Benjamin, New York (1964). (A lecture note and reprint volume) zbMATHGoogle Scholar
- 38.Flajolet, P., Gourdon, X., Dumas, P.: Theor. Comput. Sci.
**144**, 3 (1995) MathSciNetzbMATHCrossRefGoogle Scholar - 39.Schnitzer, M.J., Meister, M.: Neuron
**37**, 499–511 (2003) CrossRefGoogle Scholar - 40.Gumbel, E.J.: Statistics of Extremes. Dover, New York (2004) zbMATHGoogle Scholar
- 41.MacKay, D.J.C.: Neural Comput.
**4**, 415 (1991) MathSciNetCrossRefGoogle Scholar - 42.Bray, A.J., Moore, M.A.: J. Phys. A
**10**, 1927 (1977) MathSciNetADSCrossRefGoogle Scholar - 43.Percus, J.K., Yevick, G.J.: Phys. Rev.
**110**, 1 (1958) MathSciNetADSzbMATHCrossRefGoogle Scholar - 44.Borzi, C., Ord, G., Percus, J.K.: J. Stat. Phys.
**46**, 51 (1986) MathSciNetADSCrossRefGoogle Scholar - 45.Barton, J.: private communication Google Scholar
- 46.Gori, G., Trombettoni, A.: J. Stat. Mech. P10021 (2011) Google Scholar
- 47.Fisher, M.: J. Math. Phys. (N.Y.)
**5**, 944 (1964) ADSCrossRefGoogle Scholar - 48.Fisher, M.: Phys. Rev.
**162**, 480 (1967) ADSCrossRefGoogle Scholar - 49.Zobin, D.: Phys. Rev.
**5**, 2387 (1978) Google Scholar