Sparse Hopfield network reconstruction with 1 regularization

  • Haiping HuangEmail author
Regular Article


We propose an efficient strategy to infer sparse Hopfield network based on magnetizations and pairwise correlations measured through Glauber samplings. This strategy incorporates the 1 regularization into the Bethe approximation by a quadratic approximation to the log-likelihood, and is able to further reduce the inference error of the Bethe approximation without the regularization. The optimal regularization parameter is observed to be of the order of M ν where M is the number of independent samples. The value of the scaling exponent depends on the performance measure. ν ≃ 0.5001 for root mean squared error measure while ν ≃ 0.2743 for misclassification rate measure. The efficiency of this strategy is demonstrated for the sparse Hopfield model, but the method is generally applicable to other diluted mean field models. In particular, it is simple in implementation without heavy computational cost.


Statistical and Nonlinear Physics 


  1. 1.
    E. Schneidman, M.J. Berry, R. Segev, W. Bialek, Nature 440, 1007 (2006)CrossRefADSGoogle Scholar
  2. 2.
    S. Cocco, S. Leibler, R. Monasson, Proc. Natl. Acad. Sci. USA 106, 14058 (2009)CrossRefADSGoogle Scholar
  3. 3.
    F. Morcos, A. Pagnani, B. Lunt, A. Bertolino, D.S. Marks, C. Sander, R. Zecchina, J.N. Onuchic, T. Hwa, M. Weigt, Proc. Natl. Acad. Sci. USA 108, E1293 (2011)CrossRefADSGoogle Scholar
  4. 4.
    P. Ravikumar, M.J. Wainwright, J.D. Lafferty, Ann. Stat. 38, 1287 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    G. Tkacik, E. Schneidman, M.J. Berry, W. Bialek, arXiv:0912.5409 (2009)Google Scholar
  6. 6.
    H. Huang, Phys. Rev. E 81, 036104 (2010)CrossRefADSGoogle Scholar
  7. 7.
    H. Huang, Phys. Rev. E 82, 056111 (2010)CrossRefADSGoogle Scholar
  8. 8.
    S. Cocco, R. Monasson, V. Sessak, Phys. Rev. E 83, 051123 (2011)CrossRefADSGoogle Scholar
  9. 9.
    A. Braunstein, A. Ramezanpour, R. Zecchina, P. Zhang, Phys. Rev. E 83, 056114 (2011)CrossRefADSGoogle Scholar
  10. 10.
    H. Huang, Commun. Theor. Phys. 57, 169 (2012)CrossRefzbMATHADSGoogle Scholar
  11. 11.
    Y. Kabashima, T. Wadayama, T. Tanaka, J. Stat. Mech.: Theory Exp. 2009, L09003 (2009)CrossRefGoogle Scholar
  12. 12.
    M. Bayati, J. Bento, A. Montanari. The lasso risk: asymptotic results and real world examples, in Proceedings of the Neural. Inf. Process. Syst. NIPS Conference, Vancouver, 2010 Google Scholar
  13. 13.
    M. Zibulevsky, M. Elad, IEEE Signal Proc. Mag.27, 76 (2010)Google Scholar
  14. 14.
    J. Bento, A. Montanari, arXiv:1110.1769 (2011)Google Scholar
  15. 15.
    E. Aurell, M. Ekeberg, Phys. Rev. Lett. 108, 090201 (2012)CrossRefADSGoogle Scholar
  16. 16.
    A. Lage-Castellanos, A. Pagnani, M. Weigt, R. Zecchina, J. Stat. Mech.: Theory Exp. 2009, P10009 (2009)CrossRefGoogle Scholar
  17. 17.
    J. Barton, S. Cocco, J. Stat. Mech.: Theory Exp. 2013, P03002 (2013)MathSciNetGoogle Scholar
  18. 18.
    S. Cocco, R. Monasson, J. Stat. Phys. 147, 252 (2012)CrossRefzbMATHMathSciNetADSGoogle Scholar
  19. 19.
    J.J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554 (1982)CrossRefMathSciNetADSGoogle Scholar
  20. 20.
    D.J. Amit, H. Gutfreund, H. Sompolinsky, Ann. Phys. 173, 30 (1987)CrossRefADSGoogle Scholar
  21. 21.
    H. Sompolinsky, Phys. Rev. A 34, 2571 (1986)CrossRefADSGoogle Scholar
  22. 22.
    M. Okada, T. Fukai, M. Shiino, Phys. Rev. E 57, 2095 (1998)CrossRefADSGoogle Scholar
  23. 23.
    B. Wemmenhove, A.C.C. Coolen, J. Phys. A 36, 9617 (2003)CrossRefzbMATHMathSciNetADSGoogle Scholar
  24. 24.
    I.P. Castillo, N.S. Skantzos, J. Phys. A 37, 9087 (2004)CrossRefzbMATHMathSciNetADSGoogle Scholar
  25. 25.
    T.L.H. Watkin, D. Sherrington, Europhys. Lett. 14, 791 (1991)CrossRefADSGoogle Scholar
  26. 26.
    A. Canning, J.P. Naef, J. Phys. I 2, 1791 (1992)Google Scholar
  27. 27.
    M. Mézard, T. Mora, J. Physiol. Paris 103, 107 (2009)CrossRefGoogle Scholar
  28. 28.
    F. Ricci-Tersenghi, J. Stat. Mech. 2012, P08015 (2012)CrossRefGoogle Scholar
  29. 29.
    H.C. Nguyen, J. Berg, J. Stat. Mech. 2012, P03004 (2012)CrossRefGoogle Scholar
  30. 30.
    V. Sessak, R. Monasson, J. Phys. A 42, 055001 (2009)CrossRefMathSciNetADSGoogle Scholar
  31. 31.
    J. Friedman, T. Hastie, R. Tibshirani, J. Stat. Soft. 33, 1 (2010)Google Scholar
  32. 32.
    J. Hertz, Y. Roudi, J. Tyrcha, arXiv:1106.1752 (2011)Google Scholar
  33. 33.
    H. Zeng, J. Hertz, Y. Roudi, arXiv:1211.3671 (2012)Google Scholar

Copyright information

© EDP Sciences, SIF, Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Department of Computational Intelligence and Systems ScienceTokyo Institute of TechnologyYokohamaJapan

Personalised recommendations