Among the various models proposed so far to account for the properties of neural networks, the one devised by Little and the one derived by Hopfield prove to be the most interesting because they allow the use of statistical mechanics techniques. The link between tween the Hopfield model and the statistical mechanics is provided by the existence of an extensive quantity. When the synaptic plasticity behaves according to a Hebbian procedure, the analogy with the classical spin glass models studied by Van Hemmen is complete. In particular exact solutions describing the steady states of noisy systems are found. On the other hand, the Little model introduces a Markovian dynamics. One shows that the evolution equation obeys the microreversibility principle if the synaptic efficiencies are symmetrical. Therefore, assuming that such a symmetry materializes, the Little model has to obey a Gibbs statistics. The corresponding Hamiltonian is derived accordingly. At last, using these results, both models are shown to display associative memory properties. In particular the storage capacity of neural networks working along with the Little dynamics is similar to the capacity of Hopfield neural networks. The conclusion drawn from the study of the Hopfield model can be extended to the Little model, which is certainly a more realistic description of the biological situation.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Amari, S.I.: Neural theory of association and concept formation. Biol. Cybern. 26, 175–185 (1977)
Anninos, P.A., Beek, B., Csermely, T.J., Harth, E.H., Pertile, G.: Dynamics of neural structures. J. Theor. Biol. 26, 121–148 (1970)
Binder, K.: In: Fundamental problems in statistical mechanics. Cohen, E.O.M. (ed.). Amsterdam: North-Holland 1981
Braitenberg, V.: Cell assemblies in the cerebral cortex. In: Theoretical approaches to complex systems. Heim, R., Palm, G. (eds.), p. 171. Berlin, Heidelberg, New York: Springer 1978
Caianiello, E.R., de Luca, A., Ricciardi, L.M.: Reverberations and control of neural network. Kybernetik 4, 10–18 (1967)
Choi, M.Y., Huberman, B.A.: Digital dynamics and the simulation of magnetic systems. Phys. Rev. B28, 2547–2554 (1983)
Cooper, L.N.: A possible organization of animal memory and learning. In: Collective properties of physical systems. Nobel Symp. 24, 252–264 (1973)
Feldman, J.L., Cowan, J.D.: Large scale activity in neural theory with application to motoneuron pool responses. Biol. Cybern. 17, 29–38 (1975)
Fukushima, K.: A model of associative memory in the brain. Kybernetik 12, 58–63 (1973)
Glauber, R.J.: Time-dependent statistics of the Ising model. Phys. Rev. 4, 294–307 (1963)
Hebb, D.O.: The organization of behavior. New York: Wiley 1949
Van Hemmen, J.L.: Classical spin-glass model. Phys. Rev. Lett. 49, 409–412 (1982)
Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)
Ill condensed matter: Les Houches summer school-session 1978. Balian, R., Maynard, R., Toulouse, G. (eds.) Amsterdam: North-Holland 1979
Ingber, L.: Statistical mechanics of neocortical interactions. I. Basic formulation. Physica 5D, 83–107 (1982)
Ingber, L.: Statistical mechanics of neocortical interactions. Dynamics of synaptic modifications. Phys. Rev. A8, 395–416 (1983)
Katz, B., Miledi, R.: A study of synaptic transmission in the absence of nerve impulses. J. Physiol. 192, 407–436 (1967)
Kirkpatrick, S., Sherrington, D.: Infinite-ranged models of spinglasses. Phys. Rev. B17, 4384–4403 (1978)
Little, W.A.: The existence of persistent states in the brain. Math. Biosci. 19, 101–120 (1974)
Little, W.A., Shaw, G.L.: Analytic study of the memory storage capacity of a neural network. Math. Biosci. 39, 281–290 (1978)
Mattis, P.C.: Solvable spin systems with random interactions. Phys. Lett. A56, 421–422 (1976)
Von Neuman, J.: The computer and the brain. New Haven, London: Yale University Press 1958
Parasi, G.: Infinite number of order parameters for spin-glasses. Phys. Rev. Lett. 43, 1754–1756 (1979)
Pastur, L.A., Figotin, A.L.: Theory of disordered spin systems. Teor. Mat. Fiz. 35, 193–210 (1978)
Thompson, R.S., Gibson, W.G.: Neural model with probabilistic firing behavior. I. General considerations. Math. Biosci. 56, 239–253 (1981)
Thompson, R.S., Gibson, W.G.: Neural model with probabilistic firing behavior. II. One- and two-neuron networks. Math. Biosci. 56, 255–285 (1981)
Vannimenus, J., Maillard, J.P., De Seze, L.: Ground-state correlations in the two-dimensional Ising frustation model. J. Phys. C 12, 4523–4532 (1979)
Willwacher, G.: Fahigkeiten eines assoziativen Speichersystems im Vergleich zu Gehirnfunktionen. Biol. Cybern. 24, 181–198 (1976)
About this article
Cite this article
Peretto, P. Collective properties of neural networks: A statistical physics approach. Biol. Cybern. 50, 51–62 (1984). https://doi.org/10.1007/BF00317939
- Neural Network
- Spin Glass
- Associative Memory
- Gibbs Statistic
- Realistic Description