Pattern Recognition in Nonlinear Neural Networks

  • J. L. van Hemmen
Conference paper
Part of the Springer Series in Synergetics book series (SSSYN, volume 37)


In this paper some new techniques to analyze nonlinear neural networks are reviewed. A neural network is called nonlinear if the introduction of new data into the synaptic efficacies has to be performed through a non-linear operation. The original Hopfield model is linear whereas, for instance, clipped synapses constitute a nonlinear model. We examine the statistical mechanics of a nonlinear neural network with finitely many patterns and arbitrary synaptic kernel, study the information retrieval, and show how the abundantly present spurious states which are a consequence of the nonlinearity can be eliminated.


Neural Network Model Pure State Synaptic Efficacy Retrieval State Ising Spin 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J. J. Hopfield, Proc. Natl. Acad. Sci. U.S.A. 79 (1982) 2554MathSciNetADSCrossRefGoogle Scholar
  2. 2.
    See D. J. Amit, these proceedings, for detailsGoogle Scholar
  3. 3.
    W. C. McCulloch and W. Pitts, Bull. Math. Biophys. 5 (1943) 115MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    P. Peretto, Biol. Cybernet. 50 (1984) 51zbMATHCrossRefGoogle Scholar
  5. 5.
    R. G. Palmer, Adv. Phys. 31 (1982)669ADSCrossRefGoogle Scholar
  6. 6.
    A. C. D. van Enter and J. L. van Hemmen, Phys. Rev. A 29 (1984) 355ADSGoogle Scholar
  7. 7.
    D. O. Hebb, The Organization of Behavior (Wiley, New York, 1949)Google Scholar
  8. 8.
    J. L. van Hemmen and R. Kühn, Phys. Rev. Lett. 57 (1986) 913ADSCrossRefGoogle Scholar
  9. 9.
    H. Sompolinsky, Phys. Rev. A 34 (1986) 2571ADSGoogle Scholar
  10. 10.
    J. L. van Hemmen, D. Grensing, A. Huber and R. Kühn, Nonlinear neural networks: I. General theory, II. Information processing, to be published.Google Scholar
  11. 11.
    D. Grensing, R. Kühn and J. L. van Hemmen, J. Phys. A.: Math. Gen. 20 (1987)Google Scholar
  12. 12.
    J. L. van Hemmen, Phys. Rev. Lett. 49 (1982) 409MathSciNetADSCrossRefGoogle Scholar
  13. 13.
    J. L. van Hemmen, D. Grensing, A. Huber and R. Kühn, Z. Phys. B 65 (1986) 53ADSCrossRefGoogle Scholar
  14. 14.
    J. Lamperti, Probability (Benjamin, New York, 1966) Sec. 7. More generally, one needs ergodicity.zbMATHGoogle Scholar
  15. 15.
    D. Grensing and R. Kühn, J. Phys. A: Math. Gen 19 (1986) L 1153ADSCrossRefGoogle Scholar
  16. 16.
    J. L. van Hemmen, in: Heidelberg Colloquium on Spin Glasses, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics 192 (Springer, Berlin, 1983) pp. 203–233, in particular the Appendix.Google Scholar
  17. 17.
    The intuitive justification of the Laplace argument is clear. For a proof, see: N. G. de Bruyn, Asymptotic Methods in Analysis, 2nd Ed. (North-Holland, Amsterdam, 1961) Sec. 4.2Google Scholar
  18. 17a.
    E. T. Copson, Asymptotic Expansions (Cambridge University Press, Cambridge, 1965) Chap. 5zbMATHCrossRefGoogle Scholar
  19. 18.
    J. L. van Hemmen, Phys. Rev. A 34 (1986) 3435ADSGoogle Scholar
  20. 19.
    This convention differs from the one in Ref. 8. There λ denotes an eigenvalue of QP where P = diag (pγ).Google Scholar
  21. 20.
    G. Iooss and D. D. Joseph, Elementary Stability and Bifurcation Theory (Springer, Berlin, 1980)zbMATHGoogle Scholar
  22. 21.
    J. J. Hopfield, in: Modelling and Analysis in Biomedicine, edited by C. Nicolini (World Scientific, Singapore, 1984) pp. 369–389, in particular p. 381Google Scholar
  23. 22.
    G. Parisi, J. Phys. A: Math. Gen. 19 (1986) L 617ADSGoogle Scholar
  24. 23.
    J. P. Nadal, G. Toulouse, J.-P. Changeux, and S. Dehaene, Europhys. Lett. 1 (1986) 535ADSCrossRefGoogle Scholar
  25. 24.
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, Phys. Rev. A 32 (1985) 1007MathSciNetADSGoogle Scholar
  26. 25.
    K. L. Chung, Elementary Probability Theory and Stochastic Processes, 2nd Ed. (Springer, Berlin 1975) Eqs. (7.3.5) - (7.3.7) on pp. 211–212.Google Scholar
  27. 26.
    Since sgn (x) is odd, λρ vanishes for | ρ | even; cf. (3.6). About half of the remaining eigenvalues is positive.Google Scholar
  28. 27.
    See Ref. 10. An upper bound is provided by q-1.Google Scholar
  29. 28.
    J.-P. Changeux, S. Dehaene and G. Toulouse, Proc. Natl. Acad. Sci. U.S.A. 83 (1986) 1695MathSciNetGoogle Scholar
  30. 29.
    J. L. van Hemmen and K. Rzążewski, to be publishedGoogle Scholar
  31. 30.
    J. L. van Hemmen and A. C. D. van Enter, Phys. Rev. A 34 (1986) 2509CrossRefGoogle Scholar
  32. 31.
    J. L. van Hemmen, A. C. D. van Enter, and J. Canisius, Z. Phys. B 50 (1983) 311MathSciNetCrossRefGoogle Scholar
  33. 32.
    Interestingly, states which are products of two patterns bifurcate first. This allows logical operations such as EQUIVALENCE.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1987

Authors and Affiliations

  • J. L. van Hemmen
    • 1
  1. 1.Sonderforschungsbereich 123Universität HeidelbergHeidelbergFed. Rep. of Germany

Personalised recommendations