Advertisement

Journal of Statistical Physics

, Volume 87, Issue 1–2, pp 333–361 | Cite as

On the free energy of the hopfield model

  • G. R. Guerberoff
  • G. A. Raggio
Articles

Abstract

The general theory of inhomogeneous mean-field systems of Raggio and Werner provides a variational expression for the (almost sure) limiting free energy density of the Hopfield model
$$H_{N,p}^{\{ \xi \} } (S) = - \frac{1}{{2N}}\sum\limits_{i,j = 1}^N {\sum\limits_{\mu = 1}^N {\xi _i^\mu \xi _j^\mu S_i S_j } } $$
for Ising spinsSi andp random patterns ξμ=(ξ 1 μ 2 μ ,...,ξ N μ ) under the assumption that
$$\mathop {\lim }\limits_{N \to \gamma } N^{ - 1} \sum\limits_{i = 1}^N {\delta _{\xi _i } = \lambda ,} \xi _i = (\xi _i^1 ,\xi _i^2 ,...,\xi _i^p )$$
exists (almost surely) in the space of probability measures overp copies of {−1, 1}. Including an “external field” term −ξ μ p hμμξ i=1 N ξ i μ Si, we give a number of general properties of the free-energy density and compute it for (a)p=2 in general and (b)p arbitrary when λ is uniform and at most the two componentshμ1 andhμ2 are nonzero, obtaining the (almost sure) formula
$$f(\beta ,h) = \tfrac{1}{2}f^{ew} (\beta ,h^{\mu _1 } + h^{\mu _2 } ) + \tfrac{1}{2}f^{ew} (\beta ,h^{\mu _1 } - h^{\mu _2 } )$$
for the free energy, wherefcw denotes the limiting free energy density of the Curie-Weiss model with unit interaction constant. In both cases, we obtain explicit formulas for the limiting (almost sure) values of the so-called overlap parameters
$$m_N^\mu (\beta ,h) = N^{ - 1} \sum\limits_{i = 1}^N {\xi _i^\mu \left\langle {S_i } \right\rangle } $$
in terms of the Curie-Weiss magnetizations. For the general i.i.d. case with Prob {ξ i μ =±1}=(1/2)±ɛ, we obtain the lower bound 1+4ɛ2(p−1) for the temperatureTc separating the trivial free regime where the overlap vector is zero from the nontrivial regime where it is nonzero. This lower bound is exact forp=2, or ε=0, or ε=±1/2. Forp=2 we identify an intermediate temperature region between T*=1−4ɛ2 and Tc=1+4ɛ2 where the overlap vector is homogeneous (i.e., all its components are equal) and nonzero.T* marks the transition to the nonhomogeneous regime where the components of the overlap vector are distinct. We conjecture that the homogeneous nonzero regime exists forp≥3 and that T*=max{1−4ɛ2(p−1),0}.

Key Words

Hopfield model equilibrium statistical mechanics random mean-field models 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    L. A. Pastur and A. L. Figotin, Exactly soluble model of a spin glass,Sov. J. Low Temp. Phys. 3:378–383 (1977): On the theory of disordered spin systems,Theoret. Math. Phys. 35:403–414 (1978); Infinite range limit for a class of disordered systems,Theoret. Math. Phys. 51:564–569 (1982).Google Scholar
  2. 2.
    J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities.Proc. Natl. Acad. Sci. USA 79:2554–2558 (1982).Google Scholar
  3. 3.
    D. J. Amit,Modeling Brain Function: The World of Attractor Neural Networks (Cambridge University Press, Cambridge, 1989).Google Scholar
  4. 4.
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, Spin glass models of neural networks,Phys. Rev. A 32:1007–1018 (1985).Google Scholar
  5. 5.
    J. L. van Hemmen, Spin-glass methods of a neural network,Phys. Rev. A 34:3435–3445 (1986).Google Scholar
  6. 6.
    S. Albeverio, B. Tirozzi, and B. Zegarlinski, Rigorous results for the free energy in the Hopfield model,Commun. Math. Phys. 150:337–373 (1992).Google Scholar
  7. 7.
    F. Comets, Large deviation estimates for a conditional probability distribution. Applications to random interaction Gibbs measures.Prob. Theory Related Fields 80: 407–432 (1989).Google Scholar
  8. 8.
    H. Koch and J. Piasko, Some rigorous results on the Hopfield neural network model.J. Stat. Phys. 55:903–928 (1989).Google Scholar
  9. 9.
    H. Koch, A free energy bound for the Hopfield model.J. Phys. A 26:353–355 (1993).Google Scholar
  10. 10.
    M. V. Scherbina and B. Tirozzi, The free energy for a class of Hopfield models.J. Stat. Phys. 72:113–125 (1992).Google Scholar
  11. 11.
    A. Bovier and V. Gayrard, Rigorous results on the thermodynamics of the dilute Hopfield model,J. Stat. Phys. 69:597–627 (1993).Google Scholar
  12. 12.
    A. Bovier, V. Gayrard, and P. Picco, Gibbs states of the Hopfield model in the regime of perfect memory.Prob. Theory Related Fields 100:329–363 (1994).Google Scholar
  13. 13.
    A. Bovier, V. Gayrard, and P. Picco. Large deviation principles for the Hopfield model and the Kac-Hopfield model,Prob. Theory Related Fields (1995), to appear.Google Scholar
  14. 14.
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, Storing infinite number of patterns in a spin glass model of neural networks,Phys. Rev. Lett. 55:1530–1533 (1985).Google Scholar
  15. 15.
    A. Bovier, V. Gayrard, and P. Picco, Gibbs states of the Hopfield model with extensively many patterns.J. Stat. Phys. 79:395–414 (1995).Google Scholar
  16. 16.
    A. Bovier, Self-averaging in a class of generalized Hopfield models,J. Phys. A 27:7069–7077 (1994).Google Scholar
  17. 17.
    A. Bovier and V. Gayrard, Rigorous results on the Hopfield model of neural networks. Invited lecture at the V CLAPEM, São Paulo (Brasil),Resenhas Inst. Mat. Estat. Univ. São Paulo 1:161–172 (1994).Google Scholar
  18. 18.
    G. A. Raggio and R. F. Werner, Quantum statistical mechanics of general mean field systems.Helv. Phys. Acta 62:980–1003 (1989).Google Scholar
  19. 19.
    G. A. Raggio and R. F. Werner, The Gibbs variational principle for inhomogeneous mean-field systems.Helv. Phys. Acta 64:633–667 (1991).Google Scholar
  20. 20.
    J. Dieudonné,Foundations of Modern Analysis, Vol. I(Academic Press, New York, 1969).Google Scholar
  21. 21.
    F. A. Tamarit andE. M. F. Curado, Pair-correlated patterns in the Hopfield model of neural networks.J. Stat. Phys. 62:473–480 (1991).Google Scholar
  22. 22.
    J. F. Fontanari and W. K. Theumann, On the storage of correlated patterns in Hopfield's model,J. Phys. France 51:375–386 (1990).Google Scholar

Copyright information

© Plenum Publishing Corporation 1997

Authors and Affiliations

  • G. R. Guerberoff
    • 1
  • G. A. Raggio
    • 1
  1. 1.Facultad de Matemática, Astronomía y Física, Universidad Nacional de CórdobaCiudad UniversitariaCórdobaArgentina

Personalised recommendations