Skip to main content
Log in

Collective properties of neural networks: A statistical physics approach

  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

Among the various models proposed so far to account for the properties of neural networks, the one devised by Little and the one derived by Hopfield prove to be the most interesting because they allow the use of statistical mechanics techniques. The link between tween the Hopfield model and the statistical mechanics is provided by the existence of an extensive quantity. When the synaptic plasticity behaves according to a Hebbian procedure, the analogy with the classical spin glass models studied by Van Hemmen is complete. In particular exact solutions describing the steady states of noisy systems are found. On the other hand, the Little model introduces a Markovian dynamics. One shows that the evolution equation obeys the microreversibility principle if the synaptic efficiencies are symmetrical. Therefore, assuming that such a symmetry materializes, the Little model has to obey a Gibbs statistics. The corresponding Hamiltonian is derived accordingly. At last, using these results, both models are shown to display associative memory properties. In particular the storage capacity of neural networks working along with the Little dynamics is similar to the capacity of Hopfield neural networks. The conclusion drawn from the study of the Hopfield model can be extended to the Little model, which is certainly a more realistic description of the biological situation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Amari, S.I.: Neural theory of association and concept formation. Biol. Cybern. 26, 175–185 (1977)

    Google Scholar 

  • Anninos, P.A., Beek, B., Csermely, T.J., Harth, E.H., Pertile, G.: Dynamics of neural structures. J. Theor. Biol. 26, 121–148 (1970)

    Google Scholar 

  • Binder, K.: In: Fundamental problems in statistical mechanics. Cohen, E.O.M. (ed.). Amsterdam: North-Holland 1981

    Google Scholar 

  • Braitenberg, V.: Cell assemblies in the cerebral cortex. In: Theoretical approaches to complex systems. Heim, R., Palm, G. (eds.), p. 171. Berlin, Heidelberg, New York: Springer 1978

    Google Scholar 

  • Caianiello, E.R., de Luca, A., Ricciardi, L.M.: Reverberations and control of neural network. Kybernetik 4, 10–18 (1967)

    Google Scholar 

  • Choi, M.Y., Huberman, B.A.: Digital dynamics and the simulation of magnetic systems. Phys. Rev. B28, 2547–2554 (1983)

    Google Scholar 

  • Cooper, L.N.: A possible organization of animal memory and learning. In: Collective properties of physical systems. Nobel Symp. 24, 252–264 (1973)

    Google Scholar 

  • Feldman, J.L., Cowan, J.D.: Large scale activity in neural theory with application to motoneuron pool responses. Biol. Cybern. 17, 29–38 (1975)

    Google Scholar 

  • Fukushima, K.: A model of associative memory in the brain. Kybernetik 12, 58–63 (1973)

    Google Scholar 

  • Glauber, R.J.: Time-dependent statistics of the Ising model. Phys. Rev. 4, 294–307 (1963)

    Google Scholar 

  • Hebb, D.O.: The organization of behavior. New York: Wiley 1949

    Google Scholar 

  • Van Hemmen, J.L.: Classical spin-glass model. Phys. Rev. Lett. 49, 409–412 (1982)

    Google Scholar 

  • Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)

    Google Scholar 

  • Ill condensed matter: Les Houches summer school-session 1978. Balian, R., Maynard, R., Toulouse, G. (eds.) Amsterdam: North-Holland 1979

    Google Scholar 

  • Ingber, L.: Statistical mechanics of neocortical interactions. I. Basic formulation. Physica 5D, 83–107 (1982)

    Google Scholar 

  • Ingber, L.: Statistical mechanics of neocortical interactions. Dynamics of synaptic modifications. Phys. Rev. A8, 395–416 (1983)

    Google Scholar 

  • Katz, B., Miledi, R.: A study of synaptic transmission in the absence of nerve impulses. J. Physiol. 192, 407–436 (1967)

    Google Scholar 

  • Kirkpatrick, S., Sherrington, D.: Infinite-ranged models of spinglasses. Phys. Rev. B17, 4384–4403 (1978)

    Google Scholar 

  • Little, W.A.: The existence of persistent states in the brain. Math. Biosci. 19, 101–120 (1974)

    Google Scholar 

  • Little, W.A., Shaw, G.L.: Analytic study of the memory storage capacity of a neural network. Math. Biosci. 39, 281–290 (1978)

    Google Scholar 

  • Mattis, P.C.: Solvable spin systems with random interactions. Phys. Lett. A56, 421–422 (1976)

    Google Scholar 

  • Von Neuman, J.: The computer and the brain. New Haven, London: Yale University Press 1958

    Google Scholar 

  • Parasi, G.: Infinite number of order parameters for spin-glasses. Phys. Rev. Lett. 43, 1754–1756 (1979)

    Google Scholar 

  • Pastur, L.A., Figotin, A.L.: Theory of disordered spin systems. Teor. Mat. Fiz. 35, 193–210 (1978)

    Google Scholar 

  • Thompson, R.S., Gibson, W.G.: Neural model with probabilistic firing behavior. I. General considerations. Math. Biosci. 56, 239–253 (1981)

    Google Scholar 

  • Thompson, R.S., Gibson, W.G.: Neural model with probabilistic firing behavior. II. One- and two-neuron networks. Math. Biosci. 56, 255–285 (1981)

    Google Scholar 

  • Vannimenus, J., Maillard, J.P., De Seze, L.: Ground-state correlations in the two-dimensional Ising frustation model. J. Phys. C 12, 4523–4532 (1979)

    Google Scholar 

  • Willwacher, G.: Fahigkeiten eines assoziativen Speichersystems im Vergleich zu Gehirnfunktionen. Biol. Cybern. 24, 181–198 (1976)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Peretto, P. Collective properties of neural networks: A statistical physics approach. Biol. Cybern. 50, 51–62 (1984). https://doi.org/10.1007/BF00317939

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00317939

Keywords

Navigation