Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
  1. Home
  2. Probability Theory and Related Fields
  3. Article
Moderate Deviations for the overlap parameter in the Hopfield model
Download PDF
Download PDF
  • Published: 09 October 2004

Moderate Deviations for the overlap parameter in the Hopfield model

  • Peter Eichelsbacher1 &
  • Matthias Löwe2 

Probability Theory and Related Fields volume 130, pages 441–472 (2004)Cite this article

  • 110 Accesses

  • 7 Citations

  • Metrics details

Abstract.

We derive moderate deviation principles for the overlap parameter in the Hopfield model of spin glasses and neural networks. If the inverse temperature β is different from the critical inverse temperature β c =1 and the number of patterns M(N) satisfies M(N)/N → 0, the overlap parameter multiplied by Nγ, 1/2 < γ < 1, obeys a moderate deviation principle with speed N1−2γ and a quadratic rate function (i.e. the Gaussian limit for γ = 1/2 remains visible on the moderate deviation scale). At the critical temperature we need to multiply the overlap parameter by Nγ, 1/4 < γ < 1. If then M(N) satisfies (M(N)6 log N ∧ M(N)2N4γ log N)/N → 0, the rescaled overlap parameter obeys a moderate deviation principle with speed N1−4γ and a rate function that is basically a fourth power. The random term occurring in the Central Limit theorem for the overlap at β c = 1 is no longer present on a moderate deviation scale. If the scaling is even closer to N1/4, e.g. if we multiply the overlap parameter by N1/4 log log N the moderate deviation principle breaks down. The case of variable temperature converging to one is also considered. If β N converges to β c fast enough, i.e. faster than the non-Gaussian rate function persists, whereas for β N converging to one slower than the moderate deviations principle is given by the Gaussian rate. At the borderline the moderate deviation rate function is the one at criticality plus an additional Gaussian term.

Download to read the full article text

Working on a manuscript?

Avoid the common mistakes

References

  1. Bovier, A., Gayrard, V.: An almost sure large deviation principle for the Hopfield model. Ann. Probab. 24 (3), 1444–1475 (1996) MR98g:60055

    MATH  Google Scholar 

  2. Bovier, A., Gayrard, V.: An almost sure central limit theorem for the Hopfield model. Markov Process. Related Fields 3 (2), 151–173 (1997) MR98j:60044

    MATH  Google Scholar 

  3. Bovier, A., Gayrard, V.: The retrieval phase of the Hopfield model: a rigorous analysis of the overlap distribution. Probab. Theory Related Fields 107 (1), 61–98 (1997) MR98b:82041

    Article  MATH  Google Scholar 

  4. Bovier, A., Gayrard, V.: Hopfield models as generalized random mean field models. Mathematical aspects of spin glasses and neural networks, Progr. Probab. 41, Birkhäuser Boston, Boston, MA, 1998, pp. 3–89. MR99a:82049

  5. Bovier, A., Gayrard, V., Picco, P.: Gibbs states of the Hopfield model in the regime of perfect memory. Probab. Theory Related Fields 100 (3), 329–363 (1994) MR96c:60115

    MATH  Google Scholar 

  6. Bovier, A., Mason, D.M.: Extreme value behavior in the Hopfield model. Ann. Appl. Probab. 11 (1), 91–120 (2001) MR2002c:82049

    MATH  Google Scholar 

  7. Bovier, A., Picco, P. (eds.): Mathematical aspects of spin glasses and neural networks. Progress in Probability, 41, Birkhäuser Boston Inc., Boston, MA, 1998 MR98i:82002

  8. Brascamp, H.J., Lieb, E.H.: On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation. J. Functional Analysis 22 (4), 366–389 (1976) MR56 #8774

    Article  MATH  Google Scholar 

  9. Dembo, A., Zeitouni, O.: Large Deviations Techniques and Applications, Springer, New York, 1998

  10. Deuschel, J.-D., Giacomin, G., Ioffe, D.: Large deviations and concentration properties for ∇φ interface models. Probab. Theory Related Fields 117 (1), 49–111 (2000) MR2001e:60206

    Article  MATH  Google Scholar 

  11. Eichelsbacher, P., Löwe, M.: Moderate deviations for a class of mean-field models. Markov Process. Related Fields 10 (2), 345–366 (2004)

    Google Scholar 

  12. Ellis, R.S., Newman, C.M.: Limit theorems for sums of dependent random variables occurring in statistical mechanics. Z. Wahrsch. Verw. Gebiete 44 (2), 117–139 (1978) MR58 20113

    MATH  Google Scholar 

  13. Gentz, B.: An almost sure central limit theorem for the overlap parameters in the Hopfield model. Stochastic Process. Appl. 62 (2), 243–262 (1996) MR97e:60038

    Article  MATH  Google Scholar 

  14. Gentz, B.: A central limit theorem for the overlap in the Hopfield model. Ann. Probab. 24 (4), 1809–1841 (1996) MR97j:60038

    MATH  Google Scholar 

  15. Gentz, B.: On the central limit theorem for the overlap in the Hopfield model. Mathematical aspects of spin glasses and neural networks, Birkhäuser Boston, Boston, MA, 1998, pp. 115–149 MR98m:82059

  16. Gentz, B., Löwe, M.: Fluctuations in the Hopfield model at the critical temperature. Markov Process. Related Fields 5 (4), 423–449 (1999) MR2001k:60140

    MATH  Google Scholar 

  17. Gentz, B., Löwe, M.: The fluctuations of the overlap in the Hopfield model with finitely many patterns at the critical temperature. Probab. Theory Related Fields 115 (3), 357–381 (1999) MR2000m:60111

    Article  MATH  Google Scholar 

  18. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982)

    Google Scholar 

  19. Komlós, J., Paturi, R.: Convergence results in an associative memory model. Neural Networks 1, 239–250 (1988)

    Article  Google Scholar 

  20. Külske, C.: Metastates in disordered mean-field models: random field and Hopfield models. J. Statist. Phys. 88 (5-6), 1257–1293 (1997) MR2000a:82038

    Google Scholar 

  21. Ledoux, M., Talagrand, M.: Probability in Banach Spaces, Springer-Verlag, Berlin, 1991

  22. Loukianova, D.: Lower bounds on the restitution error in the Hopfield model. Probab. Theory Related Fields 107, 161–176 (1997)

    Article  MATH  Google Scholar 

  23. Löwe, M.: On the storage capacity of Hopfield models with correlated patterns. Ann. Appl. Probab. 8 (4), 1216–1250 (1998) MR99k:82059

    Google Scholar 

  24. McEliece, R.J., Posner, E.C., Rodemich, E.R., Venkatesh, S.S.: The capacity of the Hopfield associative memory. IEEE Trans. Inform. Theory 33, 461–482 (1987)

    Article  MATH  Google Scholar 

  25. Newman, C.M.: Memory capacity in neural network models: Rigorous lower bounds. Neural Networks 1, 223–238 (1988)

    Article  MATH  Google Scholar 

  26. Pastur, L., Shcherbina, M., Tirozzi, B.: On the replica symmetric equations for the Hopfield model. J. Math. Phys. 40 (8), 3930–3947 (1999) MR2001a:82045

    Article  MATH  Google Scholar 

  27. Pastur, L.A., Figotin, A.L.: Exactly soluble model of a spin glass. Sov. J. Low Temp. Phys. 3 (6), 378–383 (1977)

    Google Scholar 

  28. Pastur, L.A., Figotin, A.L.: On the theory of disordered spin systems. Theor. Math. Phys. 35, 403–414 (1977)

    Google Scholar 

  29. Sherrington, D., Kirkpatrick, S.: Solvable model of a spin–glass. Phys. Rev. Lett. 35, 1796–1972 (1975) MR31 4107

    Article  Google Scholar 

  30. Talagrand, M.: Rigorous results for the Hopfield model with many patterns. Probab. Theory Related Fields 110 (2), 177–276 (1998) MR98k:82099

    Article  MATH  Google Scholar 

  31. Talagrand, M.: Rigorous low-temperature results for the mean field p-spins interaction model. Probab. Theory Related Fields 117 (3), 303–360 (2000) MR2001f:82037

    Article  MATH  Google Scholar 

  32. Talagrand, M.: On the Hopfield model at the critical temperature. Probab. Theory Related Fields 121 (2), 237–268 (2001) MR1 865 487

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ruhr-Universität Bochum, Fakultät für Mathematik, NA3/68, 44780, Bochum, Germany

    Peter Eichelsbacher

  2. Institut für Mathematische Statistik, Fachbereich Mathematik und Informatik, Universität Münster, Einsteinstr. 62, 48149, Münster, Germany

    Matthias Löwe

Authors
  1. Peter Eichelsbacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthias Löwe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Eichelsbacher.

Additional information

Research supported by the Volkswagen-Stiftung (RiP-program at Oberwolfach, Germany).

Mathematics Subject Classification (2000): 60F10 (primary), 60K35, 82B44, 82D30 (secondary)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Eichelsbacher, P., Löwe, M. Moderate Deviations for the overlap parameter in the Hopfield model. Probab. Theory Relat. Fields 130, 441–472 (2004). https://doi.org/10.1007/s00440-004-0349-8

Download citation

  • Received: 27 September 2003

  • Revised: 04 February 2004

  • Published: 09 October 2004

  • Issue Date: December 2004

  • DOI: https://doi.org/10.1007/s00440-004-0349-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Moderate deviations
  • Large deviations
  • Hopfield model
  • Neural networks
  • Spin glasses
  • Critical temperature
  • Random disorder
Download PDF

Working on a manuscript?

Avoid the common mistakes

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature