Journal of Statistical Physics

, Volume 87, Issue 1–2, pp 333–361 | Cite as

On the free energy of the hopfield model

  • G. R. Guerberoff
  • G. A. Raggio


The general theory of inhomogeneous mean-field systems of Raggio and Werner provides a variational expression for the (almost sure) limiting free energy density of the Hopfield model
$$H_{N,p}^{\{ \xi \} } (S) = - \frac{1}{{2N}}\sum\limits_{i,j = 1}^N {\sum\limits_{\mu = 1}^N {\xi _i^\mu \xi _j^\mu S_i S_j } } $$
for Ising spinsSi andp random patterns ξμ=(ξ 1 μ 2 μ ,...,ξ N μ ) under the assumption that
$$\mathop {\lim }\limits_{N \to \gamma } N^{ - 1} \sum\limits_{i = 1}^N {\delta _{\xi _i } = \lambda ,} \xi _i = (\xi _i^1 ,\xi _i^2 ,...,\xi _i^p )$$
exists (almost surely) in the space of probability measures overp copies of {−1, 1}. Including an “external field” term −ξ μ p hμμξ i=1 N ξ i μ Si, we give a number of general properties of the free-energy density and compute it for (a)p=2 in general and (b)p arbitrary when λ is uniform and at most the two componentshμ1 andhμ2 are nonzero, obtaining the (almost sure) formula
$$f(\beta ,h) = \tfrac{1}{2}f^{ew} (\beta ,h^{\mu _1 } + h^{\mu _2 } ) + \tfrac{1}{2}f^{ew} (\beta ,h^{\mu _1 } - h^{\mu _2 } )$$
for the free energy, wherefcw denotes the limiting free energy density of the Curie-Weiss model with unit interaction constant. In both cases, we obtain explicit formulas for the limiting (almost sure) values of the so-called overlap parameters
$$m_N^\mu (\beta ,h) = N^{ - 1} \sum\limits_{i = 1}^N {\xi _i^\mu \left\langle {S_i } \right\rangle } $$
in terms of the Curie-Weiss magnetizations. For the general i.i.d. case with Prob {ξ i μ =±1}=(1/2)±ɛ, we obtain the lower bound 1+4ɛ2(p−1) for the temperatureTc separating the trivial free regime where the overlap vector is zero from the nontrivial regime where it is nonzero. This lower bound is exact forp=2, or ε=0, or ε=±1/2. Forp=2 we identify an intermediate temperature region between T*=1−4ɛ2 and Tc=1+4ɛ2 where the overlap vector is homogeneous (i.e., all its components are equal) and nonzero.T* marks the transition to the nonhomogeneous regime where the components of the overlap vector are distinct. We conjecture that the homogeneous nonzero regime exists forp≥3 and that T*=max{1−4ɛ2(p−1),0}.

Key Words

Hopfield model equilibrium statistical mechanics random mean-field models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    L. A. Pastur and A. L. Figotin, Exactly soluble model of a spin glass,Sov. J. Low Temp. Phys. 3:378–383 (1977): On the theory of disordered spin systems,Theoret. Math. Phys. 35:403–414 (1978); Infinite range limit for a class of disordered systems,Theoret. Math. Phys. 51:564–569 (1982).Google Scholar
  2. 2.
    J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities.Proc. Natl. Acad. Sci. USA 79:2554–2558 (1982).Google Scholar
  3. 3.
    D. J. Amit,Modeling Brain Function: The World of Attractor Neural Networks (Cambridge University Press, Cambridge, 1989).Google Scholar
  4. 4.
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, Spin glass models of neural networks,Phys. Rev. A 32:1007–1018 (1985).Google Scholar
  5. 5.
    J. L. van Hemmen, Spin-glass methods of a neural network,Phys. Rev. A 34:3435–3445 (1986).Google Scholar
  6. 6.
    S. Albeverio, B. Tirozzi, and B. Zegarlinski, Rigorous results for the free energy in the Hopfield model,Commun. Math. Phys. 150:337–373 (1992).Google Scholar
  7. 7.
    F. Comets, Large deviation estimates for a conditional probability distribution. Applications to random interaction Gibbs measures.Prob. Theory Related Fields 80: 407–432 (1989).Google Scholar
  8. 8.
    H. Koch and J. Piasko, Some rigorous results on the Hopfield neural network model.J. Stat. Phys. 55:903–928 (1989).Google Scholar
  9. 9.
    H. Koch, A free energy bound for the Hopfield model.J. Phys. A 26:353–355 (1993).Google Scholar
  10. 10.
    M. V. Scherbina and B. Tirozzi, The free energy for a class of Hopfield models.J. Stat. Phys. 72:113–125 (1992).Google Scholar
  11. 11.
    A. Bovier and V. Gayrard, Rigorous results on the thermodynamics of the dilute Hopfield model,J. Stat. Phys. 69:597–627 (1993).Google Scholar
  12. 12.
    A. Bovier, V. Gayrard, and P. Picco, Gibbs states of the Hopfield model in the regime of perfect memory.Prob. Theory Related Fields 100:329–363 (1994).Google Scholar
  13. 13.
    A. Bovier, V. Gayrard, and P. Picco. Large deviation principles for the Hopfield model and the Kac-Hopfield model,Prob. Theory Related Fields (1995), to appear.Google Scholar
  14. 14.
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, Storing infinite number of patterns in a spin glass model of neural networks,Phys. Rev. Lett. 55:1530–1533 (1985).Google Scholar
  15. 15.
    A. Bovier, V. Gayrard, and P. Picco, Gibbs states of the Hopfield model with extensively many patterns.J. Stat. Phys. 79:395–414 (1995).Google Scholar
  16. 16.
    A. Bovier, Self-averaging in a class of generalized Hopfield models,J. Phys. A 27:7069–7077 (1994).Google Scholar
  17. 17.
    A. Bovier and V. Gayrard, Rigorous results on the Hopfield model of neural networks. Invited lecture at the V CLAPEM, São Paulo (Brasil),Resenhas Inst. Mat. Estat. Univ. São Paulo 1:161–172 (1994).Google Scholar
  18. 18.
    G. A. Raggio and R. F. Werner, Quantum statistical mechanics of general mean field systems.Helv. Phys. Acta 62:980–1003 (1989).Google Scholar
  19. 19.
    G. A. Raggio and R. F. Werner, The Gibbs variational principle for inhomogeneous mean-field systems.Helv. Phys. Acta 64:633–667 (1991).Google Scholar
  20. 20.
    J. Dieudonné,Foundations of Modern Analysis, Vol. I(Academic Press, New York, 1969).Google Scholar
  21. 21.
    F. A. Tamarit andE. M. F. Curado, Pair-correlated patterns in the Hopfield model of neural networks.J. Stat. Phys. 62:473–480 (1991).Google Scholar
  22. 22.
    J. F. Fontanari and W. K. Theumann, On the storage of correlated patterns in Hopfield's model,J. Phys. France 51:375–386 (1990).Google Scholar

Copyright information

© Plenum Publishing Corporation 1997

Authors and Affiliations

  • G. R. Guerberoff
    • 1
  • G. A. Raggio
    • 1
  1. 1.Facultad de Matemática, Astronomía y Física, Universidad Nacional de CórdobaCiudad UniversitariaCórdobaArgentina

Personalised recommendations