Collective Phenomena in Neural Networks

  • J. Leo van Hemmen
  • Reimer Kühn
Part of the Physics of Neural Networks book series (NEURAL NETWORKS)

Synopsis and Note

In this paper we review some central notions of the theory of neural networks. In so doing we concentrate on collective aspects of the dynamics of large networks. The neurons are usually taken to be formal but this is not a necessary requirement for the central notions to be applicable. Formal neurons just make the theory simpler.

There are at least two ways of reading this review. It may be read as a self-contained introduction to the theory of neural networks. Alternatively, one may regard it as a vade mecum that goes with the other articles in the present book and may be consulted if one needs further explanation or meets an unknown idea. In order to allow the second approach as well we have tried to keep the level of redundancy much higher than is strictly necessary. So the attentive reader should not be annoyed if (s)he notices that some arguments are repeated.

Equations are labeled by (x.y.z). Referring to an equation within a subsection we only mention (z), within a section (y.z), and elsewhere in the paper (x.y.z). The chapter number is ignored.

The article also contains some new and previously unpublished results.

Keywords

Entropy Coherence Neurol mILO Suffix 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.1
    N. Wiener, Cybernetics (Wiley, New York, and Hermann, Paris, 1948)Google Scholar
  2. 1.2
    J. J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)MathSciNetADSCrossRefGoogle Scholar
  3. 1.3
    T. E. Posch, USCEE report 290 (1968)Google Scholar
  4. 1.4
    M. Minsky and S. Papert,Perceptrons: An Introduction to Computational Geometry (MIT Press, Cambridge, Mass., 1969) An expanded 2nd edition appeared in 1988. This book is a gold mine of insight.MATHGoogle Scholar
  5. 1.5
    J. J. Hopfield, Proc. Natl. Acad. Sci. USA 81, 3088–3092 (1984)CrossRefGoogle Scholar
  6. 1.6
    W. S. McCulloch and W. Pitts, Bull. Math. Biophys. 5, 115–133 (1943)MathSciNetMATHCrossRefGoogle Scholar
  7. 1.7
    K. Huang, Statistical Mechanics (Wiley, New York, 1963); a 2nd edition appeared in 1987Google Scholar
  8. 1.8
    P. Peretto, Biol. Cybem. 50, 51–62 (1984)MATHCrossRefGoogle Scholar
  9. 1.9
    K. Binder, inMonte Carlo Methods in Statistical Physics, edited by K. Binder (Springer, Berlin, Heidelberg, 1979) pp. 1–45Google Scholar
  10. 1.10
    R. J. Glauber, J. Math. Phys. 4, 294–307 (1963)MathSciNetADSMATHCrossRefGoogle Scholar
  11. 1.11
    N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, J. Chem. Phys. 21, 1087–1092 (1953)ADSCrossRefGoogle Scholar
  12. 1.12
    H. E. Stanley, Introduction to Phase Transitions and Critical Phenomena (Oxford University Press, New York, 1971)Google Scholar
  13. 1.13
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, (a) Phys. Rev. A 32, 1007–1018 (1985); (b) Phys. Rev. Lett. 55, 1530–1533 (1985); (c) Ann. Phys. (NY.) 173, 30–67 (1987)MathSciNetADSCrossRefGoogle Scholar
  14. 1.14
    J. J. Hopfield and D. W. Tank, Science 233, 625–633 (1986)ADSCrossRefGoogle Scholar
  15. 1.15
    D. O. Hebb,The Organization of Behavior (Wiley, New York, 1949) p. 62Google Scholar
  16. 1.16
    J. L. van Hemmen and R. Kühn, Phys. Rev. Lett. 57, 913–916 (1986)ADSCrossRefGoogle Scholar
  17. 1.17
    J. L. van Hemmen, D. Grensing, A. Huber, and R. Kühn, J. Stat. Phys. 50, 231–257)Google Scholar
  18. J. L. van Hemmen, D. Grensing, A. Huber, and R. Kühn, J. Stat. Phys. 50, 259–293 (1988)Google Scholar
  19. 1.18
    S. R. S. Varadhan, Large Deviations and Applications (Society for Industrial and Applied Mathematics, Philadelphia, PA, 1984). This work has become a classic.CrossRefGoogle Scholar
  20. 1.19 O. E. Lanford, “Entropy and equilibrium states in classical statistical mechanics”, in Statistical Mechanics and Mathematical Problems, edited by A. Lenard, Lecture Notes in Physics, Vol. 20 (Springer, New York, Beriin, Heidelberg, 1973) pp. 1–113. This elegant paper was seminal to, e.g., Refs. [1.20] and [1.21].Google Scholar
  21. 1.20
    R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics (Springer, New York, Berlin, Heidelberg, 1985); Ann. Prob. 12, (1984)MATHCrossRefGoogle Scholar
  22. 1.21
    J. L. van Hemmen, “Equilibrium theory of spin glasses: Mean field theory and beyond”, in Heidelberg Colloquium on Spin Glasses, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics, Vol. 192 (Springer, New York, Berlin, Heidelberg 1983), in particular, the Appendix; ’The theory of large deviation and its applications in statistical mechanics, in Mark Kac Seminar on Probability and PhysiScs, Syllabus 1985–1987, edited by F. den Hollander and R. Maassen, CWI Syllabus Series No. 17 (CWI, Amsterdam, 1988) pp. 41–47Google Scholar
  23. 1.22
    J. Lamperti, Probability (Benjamin, New York, 1966)MATHGoogle Scholar
  24. 1.23
    A. C. D. van Enter and J. L. van Hemmen, Phys. Rev. A 29, 355–365 (1984)MathSciNetADSCrossRefGoogle Scholar
  25. 1.24
    See, for instance, Ref. [1.12, Sect. 1.6.5]Google Scholar
  26. 1.25
    N. G. de Bruyn, Asymptotic Methods in Analysis, 2nd Edition (North-Holland, Amsterdam, 1961) Sect. 1.4.2; a Dover edition has been published recenüy.Google Scholar
  27. 1.26
    J. L. van Hemmen, Phys. Rev. Lett. 49, 409–412 (1982);MathSciNetADSCrossRefGoogle Scholar
  28. J. L. van Hemmen, A. C. D. van Enter, and J. Canisius, Z. Phys. B 50, 311–336 (1983)MathSciNetADSCrossRefGoogle Scholar
  29. 1.27
    A. W. Roberts and D. E. Varberg, Convex Functions (Academic, New York, 1973)MATHGoogle Scholar
  30. 1.28
    J. L. van Hemmen, Phys. Rev. A 34, 3435–3445 (1986)MathSciNetADSCrossRefGoogle Scholar
  31. 1.29
    J. L. van Hemmen, D. Grensing, A. Huber, and R. Kühn, Z. Phys. B 65, 53–63 (1986)MathSciNetADSCrossRefGoogle Scholar
  32. 1.30
    J. L. van Hemmen and R. G. Palmer, J. Phys. A: Math. Gen. 12, 3881–3890 (1986)Google Scholar
  33. 1.31
    D. Grensing and R. Kühn, J. Phys. A: Math. Gen. 19, L1153-L1157 (1986)ADSCrossRefGoogle Scholar
  34. 1.32
    J. L. van Hemmen and R. G. Palmer, J. Phys. A: Math. Gen. 12, 563–580 (1979)ADSCrossRefGoogle Scholar
  35. 1.33 In fact, as is also discussed at length in Ref. [1.32], in practical work the extension is not unique.Google Scholar
  36. 1.34
    C. M. Newman, Neural Networks 1, 223–238 (1988)CrossRefGoogle Scholar
  37. 1.35
    J. L. van Hemmen and V A. Zagrebnov, J. Phys. A: Math. Gen. 20, 3989–3999 (1987)ADSCrossRefGoogle Scholar
  38. 1.36 J. L. van Hemmen would like to thank M. Bouten (LUC, Diepenbeek) for his insistence on physical transparenceGoogle Scholar
  39. 1.37
    L. Breiman, Probability (Addison-Wesley, Reading, Mass., 1968) Sects. 11.3 and 11.4, including problem 11.6MATHGoogle Scholar
  40. 1.38
    D. Grensing, R. Kühn, and J. L. van Hemmen, J. Phys. A: Math. Gen. 20, 2935–2947 (1987)ADSCrossRefGoogle Scholar
  41. 1.39
    J. L. van Hemmen, Phys. Rev. A 36, 1959–1962 (1987)ADSCrossRefGoogle Scholar
  42. 1.40
    J. Marcinkiewicz, Sur une propriété de la loi de Gauss, Math. Z. 44, 612–618 (1939). The theorem has been rediscovered several times. For a textbook presentation, see: H. Richter, Wahrscheinlichkeitstheorie, 2nd Edition (Springer, Berlin, Heidelberg, 1966) pp. 213–214MathSciNetCrossRefGoogle Scholar
  43. 1.41
    Heidelberg Colloquium on Spin Glasses, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics, Vol. 192 (Springer, Beriin, Heidelberg, 1983)Google Scholar
  44. 1.42
    D. Sherrington and S. Kirkpatrick, Phys. Rev. Lett. 35, 1792–1796 (1975). The SK model is expected to describe a spin glass in sufficiently high dimensions (d > 8). See also: M. Mézard, G. Parisi, and M. A. Virasoro, Spin Glass Theory and Beyond (World Scientific, Singapore, 1987). It is fair to say that this book is devoted almost exclusively to the SK modelGoogle Scholar
  45. 1.43
    W. A. Litüe, Math. Biosci. 19, 101–120 (1974);CrossRefGoogle Scholar
  46. W. A. Litüe and G. L. Shaw, Math. Biosci. 39,281–290 (1978)CrossRefGoogle Scholar
  47. 1.44
    A. Crisanü, D. J. Anut, and H. Gutfreund, Europhys. Lett. 2, 337–341 (1986)ADSCrossRefGoogle Scholar
  48. 1.45
    J. A. Hertz, G. Grinstein, and S. A. Solla, in Heidelberg Colloquium on Glassy Dynamics, edited by J. L. van Hemmen and 1. Morgenstern, Lecture Notes in Physics, Vol. 275 (Springer, New York, Beriin, Heidelberg, 1987) pp. 538–546Google Scholar
  49. 1.46
    D. J. Amit, in: Ref. [1.45: pp. 466–471]; A. Treves and D. J. Amit, J. Phys. A: Math. Gen. 21,3155–3169 (1988)Google Scholar
  50. 1.47
    A. Crisanti and H. Sompolinsky, Phys. Rev. A 36, 4922–4939 (1987) and 37, 4865–4874 (1988)MathSciNetADSCrossRefGoogle Scholar
  51. 1.48
    M. V. Feigerman and L. B. loffe, Intern. J. Mod. Phys. B 1, 51–68 (1987)ADSCrossRefGoogle Scholar
  52. 1.49
    A simple and elegant proof can be found in: J. Lamperti, Stochastic Processes (Springer, New York, 1977) pp. 107–112. See also R. Kindermann and J. L. Snell, Markov Random Fields and their Applications, Conten:qx)rary Mathematics Vol. 1 (American Mathematical Society, Providence, Rhode Island, 1980) pp. 52–61Google Scholar
  53. 1.50
    J. F Fontanari and R. Koberle, Phys. Rev. A 36, 2475–2477 (1987)MathSciNetADSCrossRefGoogle Scholar
  54. 1.51
    S. Grossberg, Neural Networks 1, 17–61 (1988), in particular. Sect. 1.9Google Scholar
  55. 1.52
    R. Kühn and J. L. van Hemmen, Graded-Response Neurons (Heidelberg, 1987, unpublished); R. Kühn, S. Bös, and J.L. van Hemmen: Phys. Rev. A 43, RC (1991)Google Scholar
  56. 1.53
    Equations (2.2.8) and (2.2.9) were discovered independently by J. Jędrzejewski and A. Komoda, Z. Phys. B 63, 247–257 (1986)Google Scholar
  57. 1.54
    J.-P. Nadal, G. Toulouse, J.-P. Changeux, and S. Dehaene, Europhys. Lett. 1 (1986) 535–542 and 2, 343 (E) (1986)Google Scholar
  58. 1.55
    M. Mézard, J.-P. Nadal, and G Toulouse, J. Phys. (Paris) 47, 1457–1462 (1986)CrossRefGoogle Scholar
  59. 1.56 J. J. Hopfield, in Modelling in Analysis and Biomedicine, edited by C. Nicolini (World Scientific, Singapore, 1984) pp. 369–389, especially p. 381Google Scholar
  60. 1.57
    G. Parisi, J. Phys. A: Math. Gen. 19, L617-L620 (1986)ADSCrossRefGoogle Scholar
  61. 1.58
    J. L. van Hemmen, G Keller, and R. Kühn, Europhys. Lett. 5, 663–668 (1988)ADSCrossRefGoogle Scholar
  62. 1.59
    G Toulouse, S. Dehaene, and J.-P. Changeux, Proc. Natl. Acad. Sci. USA 83, 1695–1698 (1986)MathSciNetADSCrossRefGoogle Scholar
  63. 1.60 See Table I on p. 271 of Ref. [1.17]Google Scholar
  64. 1.61
    H. Sompolinsky, Phys. Rev.34, 2571–2574 (1986)ADSCrossRefGoogle Scholar
  65. 1.62
    H. Sompolinsky, in Heidelberg Colloquium on Glassy Dynamics, edited by J. L. van Hemmen and I. Morgenstern, Lecture Notes in Physics, Vol. 275 (Springer, New York, Berlin, Heidelberg, 1987) pp. 485–527CrossRefGoogle Scholar
  66. 1.63 There is the physiological rule “low-efficacy synapses degenerate.” See: J.-P. Changeux, T. Heidmann, and P. Patte, inThe Biology of Learning, edited by P. Marler and H. Terrace (Springer, New York, Beriin, Heidelberg 1984) pp. 115–133Google Scholar
  67. 1.64
    J. L. van Hemmen and K. Rzązewski, J. Phys. A: Math. Gen. 20, 6553–6560 (1987)ADSMATHCrossRefGoogle Scholar
  68. 1.65 G. Toulouse, in: Ref. [1.62: pp. 569–576]. Toulouse considers a slighUy different model (learning within bounds) with ore « 0.015. Estimating the connectivity Z of neurons involved in short-term memory to be of the order Z « 5(X), he finds that at most acZ « 7 items can be stored. It is known from experimental psychology that the short-term memory capacity of humans is 7 ± 2 items (a rather famous number). If more items have to be stored, none of them can be retrieved, i.e., they are all forgotten. If the Hopfield model is overloaded, no retrieval is possible either.Google Scholar
  69. 1.66
    R. Penrose, Proc. Cambridge Philos. Soc. 51, 406–413 (1955) and 52, 17–19 (1956); these papers are strongly recommended reading. The mathematics and numerics of the pseudoinverse is discussed at length in: T. N. E. Greville, SIAM Review 2, 153 (1960), and A. Albert, Regression and the Moore-Penrose Pseudoinverse (Academic, New York, 1972)Google Scholar
  70. 1.67
    T. Kohonen, IEEE Trans. Comput. C-23, 444–445 (1974); see also Kohonen’s book. Associative Memory (Springer, New York, Berlin, Heidelberg, 1977)Google Scholar
  71. 1.68 L. Personnaz, I. Guyon, and G. Dreyfus, J. Phys. (Paris) Lett. 46, L359-L365 (1985). These authors rediscovered the pseudoinverse in the nonlinear context (5.2.1), which they reduced to (5.2.2). A slightly more general, also linear, problem had been solved previously by Kohonen, see Ref. [1.67].Google Scholar
  72. 1.69 F. R. Gantmacher, The Theory ofMatnces, Vol. I (Chelsea, New York, 1977) Sects. IX. 3 and 4.Google Scholar
  73. 1.70
    S. Diederich and M Opper, Phys. Rev. Lett. 58, 949–952 (1987)MathSciNetADSCrossRefGoogle Scholar
  74. 1.71
    I. Kanter and H. Sompolinsky, Phys. Rev. A 35, 380–392 (1987)ADSCrossRefGoogle Scholar
  75. 1.72
    A. M. Odlyzko, J. Combin. Theory Ser. A 47, 124–133 (1988)MATHCrossRefGoogle Scholar
  76. 1.73 L. Personnaz, I. Guyon, and G. Dreyfus, Phys. Rev. A 34, 4217–4228 (1986). The authors use a parallel dynamics and show, for instance, that cycles cannot occur.Google Scholar
  77. 1.74
    F. Rosenblatt,Principles of Neurodynamics (Spartan Books, Washington, DC, 1961)Google Scholar
  78. 1.75 Ref. [1.4: Chap. 11] gives a lucid discussionGoogle Scholar
  79. 1.76 E. Gardner, J. Phys. A: Math. Gen. 21, 257–270 (1988)ADSCrossRefGoogle Scholar
  80. 1.77 See also Chap. 3 by Forrest and Wallace. It contains a nice appendix, which supplements well the arguments presented here; in particular, the case k > 0 in (5.3.9)Google Scholar
  81. 1.78
    T. M. Cover, IEEE Trans Electron. Comput. EC-14, 326–334 (1965);Google Scholar
  82. P. Baldi and S. Venkatesh, Phys. Rev. Utt. 58, 913–916 (1987)MathSciNetADSGoogle Scholar
  83. 1.79
    W. Krauth and M. Mézard, J. Phys. A: Math. Gen. 20, L745-L752 (1987)ADSCrossRefGoogle Scholar
  84. 1.80
    C. F. Stevens, Nature 338, 460–461 (1989) and references quoted therein; Nature 347, 16 (1990)ADSCrossRefGoogle Scholar
  85. 1.81 The signal may also be “smeared out” by the capacitance of the dendritic tree. This gives rise to an exponential delay with an RC time T 1. Since the time window associated with T 1 is rather narrow (a few milliseconds), certainly when compared with the axonal delay T, it will be neglected here. See, however, A. Herz, B. Sulzer, R. Kühn, and J. L. van Hemmen, in Neural Networks: From Models to Applications, edited by L. Personnaz and G. Dreyfus (I.D.S.E.T., Paris, 1989) pp. 307–315Google Scholar
  86. 1.82
    V. Braitenberg, in Brain Theory, edited by G. Palm and A. Aertsen (Springer, New York, Berlin, Heidelberg, 1986) pp. 81–96CrossRefGoogle Scholar
  87. 1.83
    R. Miller, Psychobiology 15, 241–247 (1987)ADSGoogle Scholar
  88. 1.84
    K. H. Lee, K. Chung, J. M. Chung, and R. E. Coggeshall, Comp. Neurol. 243, 335–346 (1986)CrossRefGoogle Scholar
  89. 1.85
    A.V.M. Herz, B. Sulzer, R. Kühn, and J. L. van Hemmen, Europhys. Lett. 7,663–669 (1988)ADSCrossRefGoogle Scholar
  90. 1.86
    A.V.M. Herz, B. Sulzer, R. Kühn, and J. L. van Hemmen, Biol. Cybem. 60,451–461 (1989)CrossRefGoogle Scholar
  91. 1.87
    S. R. Kelso, A. H. Ganong, and T. H. Brown, Proc. Natl. Acad. Sci. USA 83, 5326–5330 (1986)ADSCrossRefGoogle Scholar
  92. 1.88
    R. Malinow and J. P. Miller, Nature 320, 529–530 (1986)ADSCrossRefGoogle Scholar
  93. 1.89 The fact that Prob ξi μ = +1 ≡ p = 0.5 also allows numerical simulations at a reasonable system size N’, cf. Refs. [1.85] and [1.86]. For small p, numerical simulation is out of the question since either N is so small that the statistics is no good orN is so large that even most supercomputers have memory problemsGoogle Scholar
  94. 1.90
    L. N. Cooper, in Nobel Symposia, Vol. 24, edited by B. and S. Lundqvist (Academic, New York, 1973) pp. 252–264Google Scholar
  95. 1.91
    D. Kleinfeld, Proc. Natl. Acad. Sci. USA 83, 9469–9473 (1986)MathSciNetADSCrossRefGoogle Scholar
  96. 1.92
    H. Sompolinsky and I. Kanter, Phys. Rev. Lett. 57, 2861–2864 (1986)ADSCrossRefGoogle Scholar
  97. 1.93 R. Kühn and J. L. van Hemmen, this volume. Chap. 7Google Scholar
  98. 1.94
    U. Riedel, R. Kühn, and J. L. van Hemmen, Phys. Rev. A 38,1105–1108 (1988); U. Riedel, diploma thesis (Heidelberg, February 1988)MathSciNetADSCrossRefGoogle Scholar
  99. 1.95
    J. Hale, Theory of Functional Differential Equations (Springer, New York, Berlin, Heidelberg, 1977)MATHCrossRefGoogle Scholar
  100. 1.96
    R. Bellman and K. L. Cooke, Differential Difference Equations (Academic, New York, 1963)MATHGoogle Scholar
  101. 1.97
    N. D. Hayes, J. London Math. Soc. 25, 226–232 (1950)MathSciNetMATHCrossRefGoogle Scholar
  102. 1.98
    L. S. Pontryagin, Amer. Math. Soc. Transl. series 2,1, 95–110 (1955)Google Scholar
  103. 1.99
    P. Peretto, Neural Networks 1, 309–321 (1988)CrossRefGoogle Scholar
  104. 1.100
    L. F. Abbott and T. B. Kepler, J. Phys. A: Math. Gen. 22, L711-L717 (1989)MathSciNetADSCrossRefGoogle Scholar
  105. 1.101 B. Derrida, E. Gardner, and A. Zippelius, Europhys. Lett. 4, 167–173 (1987)Google Scholar
  106. 1.102 R. Kree and A. Zippelius, this volume. Chap. 6Google Scholar
  107. 1.103 The equations for the Hopfield case with finitely many patterns, i.e., (13) with e = 0, have been rediscovered by A. C. C. Coolen and Th. W. Ruijgrok, Phys. Rev. 38 (1988) 4253- 4255 andM. Shiino, H. Nishimori, andM. Ono, J. Phys. Soc. Jpn. 58 (1989) 763–766. Here too the notion of sublattice is instrumental.Google Scholar
  108. 1.104
    S. Amari, Neural Networks 1, 63–73 (1988)CrossRefGoogle Scholar
  109. 1.105 An illustration that should not be taken too seriously can be found on p. 561 in: Ref. [1.62]Google Scholar
  110. 1.106
    B. Forrest, J. Phys. A: Math. Gen. 21, 245–255 (1988)MathSciNetADSCrossRefGoogle Scholar
  111. 1.107
    H. Homer, D. Bormann, M. Frick, H. Kinzelbach, and A. Schmidt, Z. Phys. B 76, 381–398 (1989)ADSCrossRefGoogle Scholar
  112. 1.108
    W. Gerstner, J. L. van Hemmen, and A.V.M. Herz, manuscript in preparation; J.L. van Hemmen, W. Gerstner, A.V.M. Herz, R. Kühn, and M. Vaas, in Konnektionismus in Artificial Intelligence und Kognitionsforschung, edited by G. Dorffner (Springer, Beriin, Heidelberg, 1990) pp. 153–162Google Scholar
  113. 1.109
    S. Bös, R. Kühn, and J. L. van Hemmen, Z. Phys. B 71, 261–271 (1988); S. Bös, diploma thesis (Heidelberg, August 1988)MathSciNetADSCrossRefGoogle Scholar
  114. 1.110
    M. V. Feiger man and L. B. loffe. Int. J. Mod. Phys. B 1, 51–68 (1987)ADSCrossRefGoogle Scholar
  115. 1.111
    D. J. Amit, H. Gutfreund, and H. Sompolinsky, Phys. Rev. A 35, 2293–2303 (1987)MathSciNetADSCrossRefGoogle Scholar
  116. 1.112
    F. Crick and G. Mitchison, Nature 304, 111–114 (1983)ADSCrossRefGoogle Scholar
  117. 1.113
    E. R. Kandel and J. H. Schwartz, Principles of Neural Science, 2nd Edition (Elsevier, New York, 1985) Chap. 49Google Scholar
  118. 1.114
    J. J. Hopfield, D. I. Feinstein, and R. G. Palmer, Nature 304, 158–159 (1983)ADSCrossRefGoogle Scholar
  119. 1.115
    J. L. van Hemmen, L. B. loffe, R. Kühn, and M. Vaas, Physica A 163, 386–392 (1990); M. Vaas, diploma thesis (Heidelberg, October 1989); J.L. van Hemmen, in Neural Networks nd Spin Glasses, edited by W.K. Theumann and R. Köberle (World Scientific, Singapore 1990), pp. 91–114Google Scholar
  120. 1.116
    A.V.M. Herz, in Connectionism in Perspective, edited by R. Pfeifer, Z. Schreter, F. Fogelman-Soulié, and L. Steels (North-Holland, Amsterdam, 1989), Ph.D. thesis Heidelberg, September 1990), and work in preparationGoogle Scholar
  121. 1.117
    W. Feller, An Introduction to Probability Theory and Its Applications, Vol. I, 3rd Edition (Wiley, New York, 1970) Sect. 1.XVI.3Google Scholar
  122. 1.118
    B. Derrida and J.-P. Nadal, J. Stat. Phys. 49, 993–1009 (1987)MathSciNetADSCrossRefGoogle Scholar
  123. 1.119
    N. Burgess, M. A. Moore, and J. L. Shapiro, in Neural Networks and Spin Glasses, edited by W. K. Theumann and R. Köberie (World Scientific, Singapore, 1990) pp. 291–307Google Scholar
  124. 1.120
    J. C. Eccles,The Understanding of the Brain, 2nd Edition (McGraw-Hill, New York, 1977)Google Scholar
  125. 1.121
    C. Meunier, D. Hansel, and A. Varga, J. Stat. Phys. 55, 859–901 (1989)ADSMATHCrossRefGoogle Scholar
  126. 1.122
    V. S. Dotsenko, J. Phys. C 18, L1017-L1022; Physica A 140, 410–415 (1986)ADSCrossRefGoogle Scholar
  127. 1.123
    H. Gutfreund, Phys. Rev. A 37, 570–577 (1988)MathSciNetADSCrossRefGoogle Scholar
  128. 1.124
    N. Parga and M. A. Virasaro, J. Phys. (Paris) 47, 1857–1864 (1986)CrossRefGoogle Scholar
  129. 1.125
    1.125 J. Lamperti, Stochastic Processes (Springer, New York, Beriin, Heidelberg 1977); for the athematically minded there is a neat summary of conditioning in Appendix 2.Google Scholar
  130. 1.126
    M. V. Feiger man and L. B. loffe, this volume. Chap. 5Google Scholar
  131. 1.127
    J. Doob, Am. Math. Month. 78, 451–463 (1971)MathSciNetMATHCrossRefGoogle Scholar
  132. 1.128
    K. L. Chung, A Course in Probability Theory, 2nd Edition (Academic, New York, 1974) Chap. 9Google Scholar
  133. 1.129
    C. Cortes, A. Krogh, and J. A. Hertz, J. Phys. A: Math. Gen. 20, 4449–4455 (1987)MathSciNetADSCrossRefGoogle Scholar
  134. 1.130
    N. Parga, private conmiunicationGoogle Scholar
  135. 1.131
    A. Krogh and J. A. Hertz, J. Phys. A: Math. Gen. 21, 2211–2224 (1988)MathSciNetADSMATHCrossRefGoogle Scholar
  136. 1.132
    R. J. McEliece, E. C. Posner, E. R. Rodemich, and S. S. Venkatesh, IEEE Trans. Inf. Theory IT-33, 461–492 (1987)Google Scholar
  137. 1.133
    A. R. Damasio, H. Damasio, and G W. van Hessen, Neurology (NY) 32, 331–341 (1982)Google Scholar
  138. 1.134
    M. A. Virasoro, Europhys. Lett. 7, 293–298 (1988)ADSCrossRefGoogle Scholar
  139. 1.135
    B. Derrida, E. Gardner, and P. Mottishaw, J. Phys. (Paris) 48, 741–755 (1987)CrossRefGoogle Scholar
  140. 1.136 F. J. Dyson, Commun. Math. Phys. 12, 91–107 (1969) and 212–215. For a back-of-the- envelope discussion, see C. J. Thompson in Nonlinear Problems in the Physical Sciences and Biology, edited by I. Stakgold, D. D. Joseph, and D. H. Sattinger, Lecture Notes in Mathematics, Vol. 322 (Springer, Beriin, Heidelberg, 1973) pp. 308–342, in particular, pp. 329–330Google Scholar
  141. 1.137
    W. Kinzel. Phys. Rev. B 33, 5086–5088 (1986)ADSCrossRefGoogle Scholar
  142. 1.138 E. Domany and R. Meir, this volume. Chap. 9Google Scholar
  143. 1.139 R. Kühn, J. Lindenberg, G. Sawitzki, and J. L. van Hemmen, manuscript in preparationGoogle Scholar
  144. 1.140
    D. J. Amit, K. Y. M. Wong, and C. Campbell, J. Phys. A: Math. Gen. 22, 2039–2043 (1989)MathSciNetADSCrossRefGoogle Scholar
  145. 1.141
    A. Treves and D. J. Amit, J. Phys. A: Math. Gen. 22, 2205–2226 (1989);MathSciNetADSCrossRefGoogle Scholar
  146. H. Sompolinsky, Physics Today 41/12, 70–80 (1988)MathSciNetADSCrossRefGoogle Scholar
  147. 1.142
    J. Buhmann, preprint (USC, 1989)Google Scholar
  148. 1.143
    A. Frumkin and E. Moses, Phys. Rev. A 34, 714–716 (1986);ADSCrossRefGoogle Scholar
  149. E. Goles and G. Y. Vichniac in Neural Networks for Computing, edited by J. S. Denker, AIP Conf. Proc. 151 (American Institute of Physics, New York, 1986) pp. 165–181Google Scholar
  150. 1.144
    G.A. Kohring: J. Stat. Phys. 59, 1077–1086 (1990)ADSCrossRefGoogle Scholar
  151. 1.145
    H. Steffan and R. Kühn, Z. Phys. B 95, 249–260 (1994)ADSCrossRefGoogle Scholar
  152. 1.146
    D.J. Amit and M.V. Tsodyks, Network 2, 259–274 (1991)MATHCrossRefGoogle Scholar
  153. 1.147
    R. Kühn, Habilitationsschrift (Heidelberg, April 1991)Google Scholar
  154. 1.148 W. Gerstner and J.L. van Hemmen, in:Models of Neural Networks //, edited by E. Domany, J.L. van Hemmen and K. Schulten (Springer, New York, 1994) Ch. 1Google Scholar
  155. 1.149
    J.J. Hopfield and D.W. Tank, Biol. Cybern. 52, 141 (1985);MathSciNetMATHGoogle Scholar
  156. D.W. Tank and J.J. Hopfield, IEEE Trans, on Circuits and Systems 33, 533 (1985)CrossRefGoogle Scholar
  157. 1.150
    R. Kühn, in: Statistical Mechanics of Neural Networks, edited by L. Garrido, Springer Lecture Notes in Physics 398 (Springer, Heidelberg, 1990) pp 19–32; see also Ref. [1.52]Google Scholar
  158. 1.151
    R. Kühn and S. Bös, J. Phys. A: Math. Gen. 26, 831–857 (1993)ADSMATHCrossRefGoogle Scholar
  159. 1.152
    C.M. Marcus and R.M. Westervelt, Phys. Rev. A 40, 501–504 (1989)MathSciNetADSCrossRefGoogle Scholar
  160. 1.153
    J.R.L. de Almeida and D.J. Thouless, J. Phys. A: Math. Gen. 11, 983–990 (1978)ADSCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • J. Leo van Hemmen
  • Reimer Kühn

There are no affiliations available

Personalised recommendations