Abstract
Information processing in nonlinear neural networks with a finite numberq of stored patterns is studied. Each network is characterized completely by its synaptic kernelQ. At low temperatures, the nonlinearity typically results in 2q−2−q metastable, pure states in addition to theq retrieval states that are associated with theq stored patterns. These spurious states start appearing at a temperature\(\tilde T_q \), which depends onq. We give sufficient conditions to guarantee that the retrieval states bifurcate first at a critical temperatureT c and that\(\tilde T_q \)/T c → 0 asq→∞. Hence, there is a large temperature range whereonly the retrieval states and certain symmetric mixtures thereof exist. The latter are unstable, as they appear atT c . For clipped synapses, the bifurcation and stability structure is analyzed in detail and shown to approach that of the (linear) Hopfield model asq→∞. We also investigate memories that forget and indicate how forgetfulness can be explained in terms of the eigenvalue spectrum of the synaptic kernelQ.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J. L. van Hemmen, D. Grensing, A. Huber and R. Kühn,J. Stat. Phys., this issue, preceding paper.
J. L. van Hemmen and R. Kühn,Phys. Rev. Lett. 57:913 (1986).
M. H. Protter and C. B. Morrey,A First Course in Real Analysis (Springer, New York, 1977), Chapter 14.
D. H. Sattinger,Group Theoretic Methods in Bifurcation Theory (Springer, Berlin, 1979).
D. J. Amit, H. Gutfreund, and H. Sompolinsky,Phys. Rev. Lett. 55:1530 (1985);Ann. Phys. 173:30 (1987).
J. J. Hopfield,Proc. Natl. Acad. Sci. USA 79:2554 (1982).
M. Mézard, J. P. Nadal, and G. Toulouse,J. Phys. (Paris) 47:1457 (1986).
J. L. van Hemmen and V. A. Zagrebnov,J. Phys. A: Math. Gen. 20:3989 (1987).
H. Sompolinsky,Phys. Rev. A 34:2571 (1986).
D. J. Amit, H. Gutfreund, and H. Sompolinsky,Phys. Rev. A 32:1007 (1985).
M. G. Golubitsky and D. G. Schaeffer,Singularities and Groups in Bifurcation Theory, Vol. 1 (Springer, Berlin, 1985).
J. L. van Hemmen,Phys. Rev. A 34:3435 (1986), Section III.
F. R. Gantmacher,The Theory of Matrices, Vol. 1 (Chelsea, New York, 1977), Sections X.1 and 2.
D. H. Sattinger,Bull. Am. Math. Soc. 3:779–819 (1980) and references therein.
A. C. Aitken,Determinants and Matrices (Oliver & Boyd, London, 1958), p. 135.
J. J. Hopfield, inModelling in Analysis and Biomedicine, C. Nicolini, ed. (World Scientific, Singapore, 1984), pp. 369–389, in particular p. 381.
G. Toulouse, S. Dehaene, and J.-P. Changeux,Proc. Natl. Acad. Sci. USA 83:1695 (1986).
J. P. Nadal, G. Toulouse, J.-P. Changeux, and S. Dehaene,Europhys. Lett. 1:535 (1986).
G. Parisi,J. Phys. A: Math. Gen. 19:L617 (1986).
J. L. van Hemmen,Phys. Rev. A 36:1959 (1987).
J. L. van Hemmen, G. Keller, and R. Kühn, SFB Preprint No. 436 (Heidelberg, 1987).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
van Hemmen, J.L., Grensing, D., Huber, A. et al. Nonlinear neural networks. II. Information processing. J Stat Phys 50, 259–293 (1988). https://doi.org/10.1007/BF01022995
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01022995