Memory and Learning in a Class of Neural Network Models

  • D. J. Wallace
Part of the NATO ASI Series book series (NSSB, volume 140)


Neural networks are massively parallel computational models which attempt to capture the “intelligent” processing faculties of the nervous system. They have been studied extensively for more than thirty years [1]. Apart from the longer term goal of understanding the nervous system, the current upsurge of interest in such models is driven by at least three factors. First, seminal papers by Hopfield [2] and by Hinton, Rumelhardt, Sejnowski and collaborators [3] exposed many salient properties of the models and extended their richness and potential in a significant way. Second, the developments in the theory of spin-glasses [4] and the discovery of replica symmetry breaking [5] in the long-range Sherrington-Kirkpatrick model [6] have led to an understanding in some depth of the Hopfield model [7]. Finally, there is now the expectation that the implementation of neural network models using VLSI technology may lead to significant computational hardware for a number of image and signal processing applications and for optimisation problems.


Firing Pattern Synaptic Connection Finite Size Effect Training Cycle Nominal Vector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    W.S. McCulloch and W.A. Pitts, Bull. Math. Biophys. 5 (1943) 115; D.O. Webb, The organisation of behaviour (Wiley, New York, 1949 ). Useful review articles are contained in G.E. Hinton and J.A. Anderson, eds., Parallel models of associative memory, (Lawrence Erlbaum, Hillsdale, New Jersey 1981 ).Google Scholar
  2. 2.
    J.J. Hopfield, Proc. Nat. Acad. Sc. USA 79 (1982) 2554, 81 (1984) 3088. A closely related model is described by W.A. Little, Math. Biosc. 19 (1974) 101 and W.A. Little and G.L. Shaw, Math. Biosc. 39 (1978) 281.MathSciNetADSCrossRefGoogle Scholar
  3. 3.
    See for example D.H. Ackley, G.E. Hinton, and T.J. Sejnowski, Cog. Sc. 9 (1985) 147 and D.E. Rumelhardt, G.E. Hinton and R.J. Williams, to appear in Parallel distributed processing: explorations in the microstructure of cognition, Vol. 1, eds. D.E. Rumelhardt and J.L. McClelland (Bradford Books/MIT Press, Cambridge MA).CrossRefGoogle Scholar
  4. 4.
    For recent reviews see M.A. Moore in Statistical and particle physics: common problems and techniques, eds. K.C. Bowler and A.J. McKane, Proc. 26th Scottish Universities Summer School in Physics (SUSSP Publications, 1984) and C. De Dominicis in Applications of field theory to statistical mechanics, ed. L. Garrido, Proc. Sitges Conf., ( Springer Verlag, 1985 ).Google Scholar
  5. 5.
    G. Parisi, J. Phys. A13 (1980) L115; G. Parisi, Phys. Rev. Lett. 50 (1983) 1946; A. Houghton, S. Jain and A.P. Young, J. Phys. C16 (1983) L375. For reviews and further references, see [4].Google Scholar
  6. 6.
    D. Sherrington and S. Kirkpatrick, Phys. Rev. Lett. 35 (1975) 1792; S. Kirkpatrick and D. Sherrington, Phys. Rev. B17 (1978) 4384.ADSCrossRefGoogle Scholar
  7. 7.
    D.J. Amit, H. Gutfreund and H. Sompolinsky, Phys. Rev. Lett. 55 (1985) 1530; see also D. J. Amit, H. Gutfreund and H. Sompolinsky, Phys. Rev. A32 (1985) 1007.Google Scholar
  8. 8.
    Limit cycle behaviour is explored and reviewed in J.W. Clark, J. Rafelski and J.V. Winston, Phys. Rep. 123 (1985) 215.CrossRefGoogle Scholar
  9. 9.
    G. Toulouse, Commun. Phys. 2 (1977) 115.Google Scholar
  10. 10.
    S.F. Edwards and P.W. Anderson, J. Phys. F5 (1975) 965.ADSCrossRefGoogle Scholar
  11. 11.
    M. Mezard, G. Parisi, N. Sourlas, G. Toulouse and M. Virasoro, J. Physique 45 (1984) 843 and Phys. Rev. Lett. 52 (1984) 1156MathSciNetGoogle Scholar
  12. 12.
    E. Gardner, D.J. Wallace and A.D. Bruce, Memory and phase transition properties of the Hopfield model, Edinburgh preprint (1986).Google Scholar
  13. 13.
    D.J. Wallace, in Proc. Conf. Advances in Lattice Gauge Theory, eds. D.W. Duke and J.F. Owens (World Scientific, 1985 ).Google Scholar
  14. 14.
    S.F. Reddaway, D.M. Scott and K. Smith, Proc. VAPP II Conf., Comp. Phys. Comm. 37 (1985) 239 and 351.MathSciNetADSCrossRefGoogle Scholar
  15. 15.
    K.C. Bowler and G.S. Pawley, Proc. IEEE 72 (1984) 42.ADSCrossRefGoogle Scholar
  16. 16.
    A detailed discussion and further references are given in K. Binder and D.P. Landau, Phys. Rev. B30 (1984) 1477; E. Brezin and J. Zinn- Justin, Nucl. Phys. B257 [FS14] (1985) 867.ADSGoogle Scholar
  17. 17.
    D.J. Amit, H. Gutfreund and H. Sompolinsky, Statistical mechanics of neural networks near saturation, Hebrew University preprint (1986).Google Scholar
  18. 18.
    Y.S. Abu-Mostafa and J-M St. Jacques, IEEE Trans. IT-31 (1985) 461.Google Scholar
  19. 19.
    D.J. Wallace, Performance of an iterative learning algorithm for the Hopfield model, Edinburgh preprint (1986).Google Scholar
  20. 20.
    M.L. Minsky and S. Papert, Perceptrons ( MIT Press, Cambridge MA, 1969 ).MATHGoogle Scholar
  21. 21.
    W. Kinzel, IFK Jülich preprint (1985).Google Scholar
  22. 22.
    See e.g.T. Kohonen, in Parallel models of associative memory, eds. G.E. Hinton and J.A. Anderson, Ref. [1].Google Scholar
  23. 23.
    N. Parga and M. Virasoro, The ultrametric organisation of memories in a neural network, Trieste preprint (1986).Google Scholar
  24. 24.
    Vik.S. Dotsenko, J. Phys. C10 (1985) L1017.MathSciNetADSGoogle Scholar

Copyright information

© Plenum Press, New York 1986

Authors and Affiliations

  • D. J. Wallace
    • 1
  1. 1.Physics DepartmentThe University of EdinburghEdinburghUK

Personalised recommendations