Abstract
We present a general analysis of highly connected recurrent neural networks which are able to learn and retrieve a finite number of static patterns. The arguments are based on spike trains and their interval distribution and require no specific model of a neuron. In particular, they apply to formal two-state neurons as well as to more refined models like the integrate-and-fire neuron or the Hodgkin-Huxley equations. We show that the mean firing rate defined as the inverse of the mean interval length is the only relevant parameter (apart from the synaptic weights) that determines the existence of retrieval solutions with a large overlap with one of the learnt patterns. The statistics of the spiking noise (Gaussian, Poisson or other) and hence the shape of the interval distribution does not matter. Thus our unifying approach explains why, and when, all the different associative networks which treat static patterns yield basically the same results, i.e., belong to the same universality class.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abbott LF and Kepler TB (1990) Model neurons: from Hodgkin Huxley to Hopfield. In: Luis Garrido (ed) Statistical mechanics of neural networks. Lecture Notes in Physics, vol 368. Springer, Berlin Heidelberg New York, pp 5–18
Abbott LF (1991) Realistic synaptic inputs for model neural networks. Network 2:245–258
Adrian ED (1926) The impulses produced by sensory nerve endings. J Physiol (London) 61:49–72
Amit DJ, Gutfreund H, Sompolinsky H (1985) Spin-glass models of neural networks. Phys Rev A 32:1007–1032
Amit DJ, Gutfreund H, Sompolinsky H (1987) Statistical mechanics of neural networks near saturation. Ann Phys (NY) 173:30–67
Amit DJ and Treves A (1989) Associative memory neural network with low temporal spiking rates. Proc Natl Acad Sci USA 7871–7875
Amit DJ, Evans MR, Abeles M (1991) Attractor neural networks which biological probe neurons. Network 1:381–405
Amit DJ, Tsodyks MV (1991) Quantitative study of attractor neural networks retrieving at low — spike rates, I: Substrate spike rates and neuronal gain. Network 3:259–274
Bailek W, Rieke F, Ruyter van Stevenick RR, and Warland D (1991) Reading a neural code. Science 252:1854–1857
Buhmann J, Schulten K (1986) Associative recognition and storage in a model network with physiological neurons. Biol Cybern 54:319–335
Connors B and Gutnick M (1990) Intrinsic firing patterns of diverse cortical neurons. Trends Neurosci 13:99–104
Eckhorn R, Bauer R, Jordan W, Brosch M, Kruse W, Munk M, Reitboeck HJ (1988) Coherent oscillations: A mechanism of feature linking in the visual cortex? Biol Cybern 60:121–130
Ekeberg Ö, Wallen P, Lansner A, Traven H, Brodin L, Grillner S (1991) A computer based model for realistic simulations of neural networks. Biol Cybern 65:81–90
FitzHugh R (1961) Impulses and physiological states in theoretical models of nerve membranes. Biophys J 1:445–66
Freeman WJ (1975) Mass action in the nervous system. Academic Press, New York London
Gerstner W (1990) Associative memory in a network of ‘biological’ neurons. In: Advances in Neural Information Processing Systems, vol 3. Morgan Kaufmann, San Mateo, Calif, pp 84–90
Gerstner W, van Hemmen JL (1992) Associative memory in a network of ‘spiking’ neurons Network 3:139–164
Gray CM, Singer W (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc Natl Acad Sci. USA 86:1698–1702
van Hemmen JL, Kühn R (1986) Nonlinear neural networks. Phys Rev Lett 57:913–916
van Hemmen JL, Grensing D, Huber A, Kühn R (1986) Elementary solution of classical spin glass models. Z Phys B-Condensed Mater 65:53–63
van Hemmen JL, Gerstner W, Herz AVM, Kühn R, Sulzer B, Vass M (1990) Encoding and decoding of patterns which are correlated in space and time. In: Dorffner G (ed) Konnektionismus in Artificial Intelligence und Kognitionsforschung. Springer, Berlin Heidelberg New York, pp. 153–162
Herz ABM, Sulzer B, Kühn R, van Hemmen JL (1988) The Hebb rule: Storing static and dynamic objects in an associative neural network. Europhys Lett 7:663–669 (1989). Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets. Biol Cybern 60:457–467
Hodgkin AL (1948) The local electric changes associated with repetitive action in a non-medullated axon. J Physiol (London) 107:165–181
Hodgkin AL, Huxley AF (1952) A quantitative description of ion currents and its applications to conduction and excitation in nerve membreanes. J Physiol (London) 117:500–544
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci. USA 79:2554–2558
Hopfield JJ (1984) Neurons with graded response have computational properties like those of Two-State neurons. Proc Natl Acad Sci USA 81:3088–3092
Horn D, Usher M (1989) Neural networks with dynamical thresholds. Phys Rev A 40:1036–1040
Hubel DH, Wiesel TN (1977) Functional architecture of macaque monkey visual cortex. Proc R Soc London B 198:1–59
Jahnsen H, Llinas R (1984) Electrophysiological properties of the Guinea-pig thalamic neurons: an in vitro study. J Physiol (London) 349:205–226
Koch C, Segev I (1989) Methods in neuronal modeling, from synapses to networks. MIT Press, Cambridge, Mass
Kühn R, Bös S, van Hemmen JL (1991) Statistical mechanics for networks of graded-response neurons. Phys Rev A 43:2084–2087
Llinas R, Sugimori M (1980) Electrophysiology of mammalian inferior olivary neurons in vitro. Different types of voltage dependant ionic conductances. J Physiol (London) 315:549–567
Little WA (1974) The existence of persistent states in the brain. Math Biosci 19:101–120
McCulloch WC, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133
Nagumo J, Arimoto S, Yoshizawa S (1962) An active pulse transmission line simulating nerve axon. Proc IRE 50:2061–2070
Perkel DH, Gerstein GL, Moore GP (1967) Neuronal spike trains and stochastic point processes I. The single spike train. Biophys J 7:391–418
Ritz R (1991) Kollektive Oszillationen in Neuronalen Netzwerken. Diplomarbeit, Physik-Department der Technischen Universität München
Schuster HG, Wagner P (1990) A model for neuronal oscillations in he visual cortex. Biol Cybern 64:77–82
Stein RB (1967) The frequency of nerve action potential generated by applied currents. Proc R Soc London B 167:64–86
Traub RD, Wong RKS, Miles R, Michelson H (1991) A model of a CA3 hippocampal pyramidal neuron incorporating voltageclamp data on intrinsic conductances. J Neurophysiol 66:635–650
Treves A (1990) Threshold-linear formal neurons in auto-associative nets. J Phys A 23:2631–2650
Wilson HR, Cowan JD (1972) Excitatory and inhibitory interactions in localized populations of model neurons. Biophys J 12:1–24
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Gerstner, W., van Hemmen, J.L. Universality in neural networks: the importance of the ‘mean firing rate’. Biol. Cybern. 67, 195–205 (1992). https://doi.org/10.1007/BF00204392
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00204392