Abstract
A neural network is called nonlinear if the introduction of new data into the synaptic efficacies has to be performed through anonlinear operation. The original Hopfield model is linear, whereas, for instance, clipped synapses constitute a nonlinear model. Here a general theory is presented to obtain the statistical mechanics of a neural network with finitely many patterns and arbitrary (symmetric) nonlinearity. The problem is reduced to minimizing a free energy functional over all solutions of a fixed-point equation with synaptic kernelQ. The case of clipped synapses with bimodal and Gaussian probability distribution is analyzed in detail. To this end, a simple theory is developed that gives the spectrum ofQ and thereby all the solutions that bifurcate from the high-temperature phase.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J. J. Hopfield,Proc. Natl. Acad. Sci. USA 79:2554 (1982);81:3088 (1984).
W. A. Little,Math. Biosci. 19:101 (1974); W. A. Little and G. L. Shaw,Math. Biosci. 39:281 (1978).
P. Peretto,Biol. Cybernet. 50:51 (1984).
G. Toulouse, S. Dehaene, and J.-P. Changeux,Proc. Natl. Acad. Sci. USA 83:1695 (1986).
J. J. Hopfield and D. W. Tank,Biol. Cybernet. 52:141 (1985).
W. A. McCulloch and W. Pitts,Math. Biophys. 5:115 (1943).
K. Binder, inMonte Carlo Methods in Statistical Physics, by K. Binder, ed. (Springer, New York, 1979), p. 145.
A. C. D. van Enter and J. L. van Hemmen,Phys. Rev. A 29:355 (1984).
D. O. Hebb,The Organization of Behavior (Wiley, New York, 1949).
J. L. van Hemmen and R. Kühn,Phys. Rev. Lett. 57:913 (1986); J. L. van Hemmen,Phys. Rev. A 36:1959 (1987).
H. Sompolinsky,Phys. Rev. A 34:2571 (1986).
D. J. Amit, H. Gutfreund, and H. Sompolinsky,Phys. Rev. Lett. 55:1530 (1985);Ann. Phys. 173:30 (1987).
D. J. Amit, H. Outfreund, and H. Sompolinsky,Phys. Rev. A 32:1007 (1985).
W. Kinzel,Z. Phys. B 60:205 (1985).
J. J. Hopfield, inModelling and Analysis in Biomedicine, C. Nicolini, ed. (World Scientific, Singapore, 1984), pp. 369–389, in particular p. 381.
G. Parisi,J. Phys. A: Math. Gen. 19:L617 (1986).
J. P. Nadal, G. Toulouse, J.-P. Changeux, and S. Dehaene,Europhys. Lett. 1:535 (1986).
J. L. van Hemmen, D. Grensing, A. Huber, and R. Kühn,J. Stat. Phys., this issue, following paper.
J. L. van Hemmen, D. Grensing, A. Huber, and R. Kühn,Z. Phys. B 65:53 (1986).
H. E. Stanley,Introduction to Phase Transitions and Critical Phenomena (Oxford University Press, Oxford, 1971), Section 6.5.
J. L. van Hemmen,Phys. Rev. Lett. 49:409 (1982); inLecture Notes in Physics, No. 192 (1983), pp. 203–233, in particular the Appendix.
D. Grensing and R. Kühn,J. Phys. A: Math. Gen. 19:L1153 (1986).
J. Lamperti,Probability (Benjamin, New York, 1966), Section 7.
R. B. Griffiths, Chi-Yuan Weng, and J. S. Langer,Phys. Rev. 149:301 (1966).
J. L. van Hemmen,Phys. Rev. A 34:3435 (1986).
A. Erdélyiet al., Higher Transcendental Functions, Vol. 2 (McGraw-Hill, New York, 1953), Chapter XI, in particular pp. 232–251.
J. L. van Hemmen and A. C. D. van Enter,Phys. Rev. A 34:2509 (1986), in particular the Appendix.
M. Abramowitz and I. A. Stegun (eds.),Handbook of Mathematical Functions (Dover, New York, 1965), Section 24.1.1.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
van Hemmen, J.L., Grensing, D., Huber, A. et al. Nonlinear neural networks. I. General theory. J Stat Phys 50, 231–257 (1988). https://doi.org/10.1007/BF01022994
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01022994