Synopsis
This paper reviews some central notions of the theoretical biophysics of neural networks, viz., information coding through coherent firing of the neurons and spatio-temporal spike patterns. After an introduction to the neural coding problem we first turn to oscillator models and analyze their dynamics in terms of a Lyapunov function. The rest of the paper is devoted to spiking neurons, a pulse code. We review the current neuron models, introduce a new and more flexible one, the spike response model (SRM), and verify that it offers a realistic description of neuronal behavior. The corresponding spike statistics is considered as well. For a network of SRM neurons we present an analytic solution of its dynamics, analyze the possible asymptotic states, and check their stability. Special attention is given to coherent oscillations. Finally we show that Hebbian learning also works for low activity spatio-temporal spike patterns. The models which we study always describe globally connected networks and, thus, have a high degree of feedback. We only touch upon functional feedback, that is, feedback between areas that have different tasks. Information processing in conjunction with functional feedback is treated explicitly in a companion paper [94].
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abbott LF, Kepler TB (1990) Model neurons: From Hodgkin Huxley to Hopfield. In: Statistical Mechanics of Neural Networks, L. Garrido (Ed.), Lecture Notes in Physics 368 (Springer, Berlin) pp. 5–18
Abbott LF (1991) Realistic synaptic inputs for model neural networks. Network 2:245–258
Abbott LF, van Vreeswijk C (1993) Asynchronous states in a network of pulse-coupled oscillators. Phys. Rev. E 48:1483–1490
Abeles M, Lass Y (1975) Transmission of information by the axon. Biol. Cy-bern. 19:121–125
Abeles M (1982) Local Cortical Circuits (Springer, Berlin)
Abeles M (1991) Corticonics: Neural Circuits of the Cerebral Cortex (Cam-bridge University Press, Cambridge)
Abeles M, Prut Y, Bergman H, Vaadia E, Aertsen A (1993) Intergration, synchronicity, and periodicity. In: Brain Theory, A. Aertsen (Ed.) (Elsevier, Amsterdam)
Abeles M (1994) Firing rates and well-timed events in the cerebral cortex. This volume, Ch. 3
Abramowitz M, Stegun IA (1965) Handbook of Mathematical Functions (Dover, New York)
Adrian ED (1926) The impulses produced by sensory nerve endings. J. Physiol. (London) 61:49–72
Amit DJ, Gutfreund H, Sompolinsky H (1985) Spin-glass models of neural networks. Phys. Rev. A 32:1007–1032
Amit DJ, Gutfreund H, Sompolinsky H (1987) Statistical mechanics of neural networks near saturation. Ann. Phys. (NY) 173:30–67
Amit DJ, Tsodyks MV (1991) Quantitative study of attractor neural networks retrieving at low spike rates. I. Substrate-spike rates and neuronal gain. Network 3:259–274
Bauer HU, Pawelzik K (1993) Alternating oscillatory and stochastic dynamics in a model for a neuronal assembly. To appear in Physica D 69
Bernander Ö, Douglas RJ, Martin KAC, Koch C (1991) Synaptic background activity influences spatio-temporal integration in pyramidal cells. Proc. Natl. Acad. Sci. U.S.A. 88:11,569–11,573
Bialek W, Rieke P, de Ruyter van Stevenick RR, Warland D (1991) Reading a neural code. Science 252:1854–1857
Bindman L, Christofi G, Murphy K, Nowicky A (1991) Long-term potenti-ation (LTP) and depression (LTD) in the neocortex and hippocampus: An overview. In: Aspects of Synaptic Transmission, T.W. Stone (Ed.) (Taylor and Francis, London), Vol. 1
Brown TH, Johnston D (1983) Voltage-clamp analysis of mossy fiber synaptic input to hippocampal neurons. J. Neurophysiol. 50:487–507
Brown TH, Ganong AH, Kairiss EW, Keenan CL, Kelso SR (1989) Long-term potentation in two synaptic systems of the hippocampal brain slice. In: Neural Models of Plasticity, J.H. Byrne and W.O. Berry (Eds.) (Academic Press, San Diego, CA), pp. 266–306
Buhmann J, Schulten K (1986) Associative recognition and storage in a model network with physiological neurons. Biol. Cybern. 54:319–335
Bush P, Douglas RJ (1991) Synchronization of bursting action potential discharge. Neural Comput. 3:19–30
Choi MY (1988) Dynamic model of neural networks. Phys. Rev. Lett. 61:2809–2812
Connors BW, Gutnick MJ, Prince DA (1982) Electrophysiological properties of neocortical neurons in vitro. J. Neurophysiol. 48:1302–1320
Cox DR (1962) Renewal Theory (Methuen, London)
Davis JL, Eichenbaum H (Eds.) (1991) Olfaction. A Model System for Computational Neuroscience (MIT Press, Cambridge, Mass.)
Dinse HRO, Krüger K, Best J (1991) Temporal structure of cortical information processing: Cortical architecture, oscillations, and nonseparability of spatio-temporal receptive field organization. In: Neuronal Cooperativity, J. Krüger (Ed.) (Springer, Berlin) pp. 68–104
Eckhorn R, Grüsser OJ, Kröller J, Pellnitz K, Pöpel B (1976) Efficiency of different neural codes: Information transfer calculations for three different neural systems. Biol. Cybern. 22:49–60
Eckhorn R, Bauer R, Jordan W, Brosch M, Kruse W, Munk M, Reit-boeck HJ (1988) Coherent oscillations: A mechanism of feature linking in the visual cortex? Biol. Cybern. 60:121–130
Eckhorn R, Krause F, Nelson JI (1993) The RF cinematogram: A cross-correlation technique for mapping several visual fields at once. Biol. Cy-bern. 69:37–55
Ekeberg Ö, Wallen P, Lansner A, Traven H, Brodin L, Grillner S (1991) A computer based model for realistic simulations of neural networks. Biol. Cy-bern. 65:81–90
Eggermont JJ (1990) The Correlative Brain (Springer, Berlin)
van Enter ACD and van Hemmen JL (1984) Statistical-mechanical formalism for spin glasses. van Phys. Rev. A 29:355–365
Ermentrout GB (1985) Synchronization in a pool of mutually coupled oscillators with random frequencies. J. Math. Biol. 22:1–9
Ermentrout GB (1985) The behavior of rings of coupled oscillators. J. Math. Biol. 23:55–74
Ermentrout GB (1990) Oscillator death in populations of “all to all” coupled nonlinear oscillators. Physica D 41:219–231
Ermentrout GB (1992) Stable periodic solutions to discrete and continuum arrays of weakly coupled nonlinear oscillators. SIAM J. Appl. Math. 52:1665–1687
Ermentrout GB, Kopell N (1990) Oscillator death in systems of coupled neural oscillators. SIAM J. Appl. Math. 50:125–146, and references quoted therein
Ermentrout GB, Kopell N (1991) Multiple pulse interactions and averaging in systems of coupled neural oscillators. J. Math. Biol. 29:195–217
Eskandar EN, Richmond BJ, Hertz JA, Optican LM, Troels K (1992) Decoding of neuronal signals in visual pattern recognition. In: Advances in Neural Information Processing 4, J-E. Moody et al. (Eds.) (Morgan Kaufman, San Mateo, CA), pp. 356–363
Fitz Hugh R (1961) Impulses and physiological states in theoretical models of nerve membranes. Biophys. J. 1:445–66
Freeman WJ (1975) Mass Action in the Nervous System (Academic Press, New York)
Gardner E (1988) The space of interactions in neural network models. J. Phys. A: Math. Gen. 21:257–270
Gerstner W (1990) Associative memory in a network of “biological” neurons. In: Advances in Neural Information Processing Systems 3, R.P. Lipp-mann, J.E. Moody, D.S. Touretzky (Eds.) (Morgan Kaufmann, San Mateo, CA), pp. 84–90
Gerstner W, van Hemmen JL (1992a) Associative memory in a network of spiking neurons. Network 3:139–164
Gerstner W, van Hemmen JL (1992b) Universality in neural networks: The importance of the mean firing rate. Biol. Cybern. 67:195–205
Gerstner W, Ritz R, van Hemmen JL (1993a) A biologically motivated and analytically soluble model of collective oscillations in the cortex. I. Theory of weak locking. Biol. Cybern. 68:363–374
Gerstner W, Ritz R, van Hemmen JL (1993b) Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cy-bern. 69:503–515
Gerstner W (1993) Kodierung und Signalübertragung in neuronalen Systemen — Assoziative Netzwerke mit stochastisch feuernden Neuronen (Verlag Harri Deutsch, Frankfurt), Reihe Physik, Bd. 15
Gerstner W, van Hemmen JL (1993) Coherence and incoherence in a globally coupled ensemble of pulse-emitting units. Phys. Rev. Lett. 71:312–315
Grensing D, Kühn R (1986) Random site spin glass models J. Phys. A: Math. Gen. 19:L1153–L1157
Gray CM, Singer W (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc. Natl. Acad. Sci. U.S.A. 86:1698–1702
Gray CM, König P, Engel AK, Singer W (1989) Oscillatory responses in cat visual cortex exhibit intercolumnar synchronization which reflects global stimulus properties. Nature 338:334–337
Guckenheimer J, Holmes P (1983) Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields (Springer, Berlin)
Hassard B, Wan BY (1978) Bifurcation formulae derived from center manifold theory. J. Math. Anal. Appl. 63:297–312
Hebb DO (1949) The Organization of Behavior (Wiley, New York)
van Hemmen JL, Kühn R (1986) Nonlinear neural networks. Phys. Rev. Lett. 57:913–916
van Hemmen JL, Grensing D, Huber A, Kühn R (1986) Elementary solu tion of classical spin glass models. Z. Phys. B 65:53–63
van Hemmen JL, Grensing D, Huber A, Kühn R (1988) Nonlinear neural networks I and II. J. Stat. Phys. 50:231–257 and 259-293
van Hemmen JL, Gerstner W, Herz AVM, Kühn R, Sulzer B, Vaas M (1990) Encoding and decoding of patterns which are correlated in space and time. In: Konnektionismus in Artificial Intelligence und Kognitions-forschung, G. Dorffner (Ed.) (Springer-Verlag, Berlin), pp. 153–162
van Hemmen JL, Ioffe LB, Kühn R, Vaas M (1990) Increasing the efficiency of a neural network through unlearning. Physica A 163:386–392
van Hemmen JL, Kühn R (1991) Collective phenomena in neural networks. In: Models of Neural Networks, E. Domany, J.L. van Hemmen, K. Schulten (Eds.) (Springer-Verlag, Berlin), pp. 1–105
van Hemmen JL, Wreszinski WF (1993) Lyapunov function for the Kuramoto model of nonlinearly coupled oscillators J. Stat. Phys. 72:145–166
Hertz J, Krogh A, Palmer RG (1991) Introduction to the Theory of Neural Computation (Addison-Wesley, Redwood City, CA)
Herz AVM, Sulzer B, Kühn R, van Hemmen JL (1988) The Hebb rule: Storing static and dynamic objects in an associative neural network. Europhys. Lett. 7:663–669.
Herz AVM, Sulzer B, Kühn R, van Hemmen JL (1989) Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets. Biol. Cybern. 60:457–467
Herz AVM, Li Z, van Hemmen JL (1991) Statistical mechanics of temporal association in neural networks with transmission delays. Phys. Rev. Lett. 66:1370–1373
Hirsch MW, Smale S (1974) Differential Equations, Dynamical Systems, and Linear Algebra (Academic Press, New York), Chap. 9
Hodgkin AL, Huxley AF (1952) A quantitative description of ion currents and its applications to conduction and excitation in nerve membranes. J. Physiol. (London) 117:500–544
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U.S.A. 79:2554–2558
Hopfield JJ (1984) Neurons with graded response have computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. U.S.A. 81:3088–3092
Horn D, Usher M (1989) Neural networks with dynamical thresholds. Phys. Rev. A 40:1036–1044
Hubel DH, Wiesel TN (1977) Functional architecture of macaque monkey visual cortex. Proc. R. Soc. London Ser. B 198:1–59
Iooss G, Joseph DD (1980) Elementary Stability and Bifurcation Theory (Springer, Berlin), Chap. V; to be fair, this book is explicit but not “elementary.”
Jack JJB, Noble D, Tsien RW (1975) Electric Current Flow in Excitable Cells (Clarendon Press, Oxford)
Kandel ER, Schwartz JH (1985) Principles of Neural Science, 2nd Ed. (Elsevier, Amsterdam)
Kelso SR, Ganong AH, Brown TH (1986) Hebbian synapses in hippocampus. Proc. Natl. Acad. Sci. U.S.A. 83:5326–5330
König P, Schulen TB (1991) Stimulus-dependent assembly formation of oscillatory responses: I. Synchronization. Neural Comput. 3:155–166
Konishi M (1986) Centrally synthesized maps of sensory space. Trends in Neurosci. 9:163–168
Kopell N (1986) Phase methods for coupled oscillators and related topics: An annnotated bibliography J. Stat. Phys. 44:1035–1042
Krauth W, Mézard M (1987) Learning algorithms with optimal stability in neural networks. J. Phys. A: Math. Gen. 20:L745–L752
Krüger J (1983) Simultaneous individual recordings from many cerebral neurons: Techniques and results. Rev. Physiol. Biochem. Pharmacol. 98:177–233
Krüger J, Aiple F (1988) Multielectrode investigation of monkey striate cortex: Spike train correlations in the infragranular layers. J. Neurophysiol. 60:798–828
Krüger J, Becker JD (1991) Recognizing the visual stimulus from neuronal discharges. Trends in Neurosci. 14:282–286
Kuffler SW, Nicholls JG, Martin AR (1984) From Neuron to Brain, 2nd Ed. (Sinauer, Sunderland, Mass.)
Kuramoto Y (1975) Self-entrainment of a population of coupled nonlinear oscillators. In: International Symposium on Mathematical Problems in Theoretical Physics, H. Araki (Ed.) (Springer, Berlin), pp. 420–422
Kuramoto Y (1984) Cooperative dynamics of oscillator community. Progr. Theor. Phys. Suppl. 79:223–240
Kuramoto Y (1984) Chemical Oscillations, Waves, and Turbulence (Springer, Berlin), pp. 68–77
Kuramoto Y, Nishikawa I (1987) Statistical macrodynamics of large dynamical systems. Case of a phase transition in oscillator communities. J. Stat. Phys. 49:569–605
Lamperti J (1966) Probability (Benjamin, New York), Chap. 7
Lancaster B, Adams PR (1986) Calcium-dependent current generating the afterhyperpolarization of hippocampal neurons. J. Neurophysiol. 55:1268–1282
Larson J, Lynch G (1986) Induction of synaptic potentiation in hippocampus by patterned stimulation involves two events. Science 232:985–988
Little WA, Shaw GL (1978) Analytical study of the memory storage capacity of a neural network. Math. Biosci. 39:281–290
MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal link. Bull, of Math. Biophy. 14:127–135
von der Malsburg C, Buhmann J (1992) Sensory segmentation with coupled neural oscillators. Biol. Cybern. 67 233–242
Matthews PC, Strogatz SH (1990) Phase diagram for the collective behavior of limit-cycle oscillators. Phys. Rev. Lett. 65:1701–1704
Mirollo RE, Strogatz SH (1990) Jump bifurcation and hysteresis in an infinite-dimensional dynamical system of coupled spins. SIAM J. Appl. Math. 50:108–124
Mirollo RE, Strogatz SH (1990) Synchronization of pulse coupled biological oscillators. SIAM J. Appl. Math. 50:1645–1662
Nagumo J, Arimoto S, Yoshizawa S (1962) An active pulse transmission line simulating nerve axon. Proc. IRE 50:2061–2070
Neven H, Aertsen A (1992) Rate coherence and event coherence in the visual cortex: A neuronal model of object recognition. Biol. Cybern. 67:309–322
Niebur E, Kammen DM, Koch C, Rudermann D, Schuster HG (1991) Phase-coupling in two-dimensional networks of interacting oscillators. In: Advances in Neural Information Processing Systems 3, R.P. Lippmann, J.E. Moody, D.S. Touretzky (Eds.) (Morgan Kaufmann, San Mateo, CA), pp. 123–127
Optican LM, Richmond BJ (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior cortex: III. Information theoretic analysis. J. Neurophysiol. 57:162–178
Perkel DH, Gerstein GL, Moore GP (1967) Neuronal spike trains and stochastic point processes. I. The single spike train. Biophys. J. 7:391–418
van der Pol B (1927) Forced oscillations in a circuit with nonlinear resistance (reception with reactive triode). The London, Edinburgh, and Dublin Philos. Mag. and J. Sci. 3:65–80
Rall W (1964) Theoretical significance of dendritic trees for neuronal input-output relations. In: Neural Theory and Modeling, R.F. Reiss (Ed.) (Stanford University Press), pp. 73–97
Rapp M, Yarom Y, Segev I (1992) The impact of parallel fiber background activity on the cable properties of cerebellar Purkinje cells. Neural Comput. 4:518–533
Reitboeck HJA (1983) A multielectrode matrix for studies of temporal signal correlations within neural assemblies. In: Synergetics of the Brain, E. Basar et al. (Eds.) (Springer, Berlin), pp. 174–182
Richmond BJ, Optican LM, Podell M, Spitze H (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior cortex: I. Response characteristics. J. Neurophysiol. 57:132–146
Ritz R, Gerstner W, van Hemmen JL (1994) Associative binding and segregation in a network of spiking neurons. This volume, Ch. 5
Rotter S, Heck D, Aertsen A, Vaadia E (1993) A stochastic model for networks of spiking cortical neurons: Time-dependent description on the basis of membrane currents. In: Gene, Brain, Behavior, H. Eisner and M. Heisenberg (Eds.) (Thieme, Stuttgart), p. 491
Rudin W (1974) Real and Complex Analysis (McGraw-Hill, New York), p. 63
de Ruyter van Steveninck RR, Bialek W (1988) Real-time performance of a movement-sensitive neuron in the blowfly visual system: Coding and information transfer in short spike sequences. Proc. R. Soc. London Ser. B 234:379–414
Sakaguchi H, Shinomoto S, Kuramoto Y (1987) Local and global self-entrainments in oscillator lattices. Progr. Theor. Phys. 77:1005–1010
Sakaguchi H, Shinomoto S, Kuramoto Y (1988) Mutual entrainaient in oscillator lattices with nonvariational-type interactions. Progr. Theor. Phys. 79:1069–1079
Schillen TB, König P (1991) Stimulus-dependent assembly formation of oscillatory responses. II. Desynchronization. Neural Comput. 3:167–177
Schuster HG and Wagner P (1990a) A model for neuronal oscillations in the visual cortex: 1. Mean-field theory and derivation of the phase equations. Biol. Cybern. 64:77–82
Schuster HG and Wagner P (1990b) A model for neuronal oscillations in the visual cortex: 2. Phase description and feature dependent synchronization. Biol. Cybern. 64:83–85
Singer W (1991) The formation of cooperative cell assemblies in the visual cortex. In: Neural Cooperativity, J. Krüger (Ed.) (Springer, Berlin), pp. 165–183
Singer W (1994) The role of synchrony in neocortical processing and synap-tic plasticity. This volume, Ch. 4
Sompolinsky H, Golomb D, and Kleinfeld D (1990) Global processing of visual stimuli in a neural network of coupled oscillators. Proc. Natl. Acad. Sci. U.S.A. 87:7200–7204
Sompolinsky H, Golomb D, Kleinfeld D (1991) Cooperative dynamics in visual processing. Phys. Rev. A 43:6990–7011
Stein RB (1967) The information capacity of nerve cells using a frequency code. Biophys. J. 7:797–826
Stein RB (1967) The frequency of nerve action potentials generated by applied currents. Proc. R. Soc. London Ser. B167:64–86
Strogatz SH, Mirollo RE (1988) Collective synchronization in lattices of nonlinear oscillators with randomness. J. Phys. A: Math. Gen. 21:L699–L705
Strogatz SH, Mirollo RE (1991) Stability of incoherence in a population of coupled oscillators. J. Stat. Phys. 63:613–635
Strogatz SH, Mirollo RE, Matthews PC (1992) Coupled nonlinear oscillators below the synchronization threshold: Relaxation be generalized Landau damping. Phys. Rev. Lett. 68:2730–2733
Stuart GJ, Sakmann B (1994) Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature 367:69–72
Traub RD, Wong RKS, Miles R, Michelson H (1991) A model of a CA3 hippocampal pyramidal neuron incorporating voltage-clamp data on intrinsic conductances. J. Neurophysiol.66:635–
Tsodyks M, Mitkov I, Sompolinsky H (1993) Patterns of synchrony in inhomogeneous networks of oscillators with pulse interaction. Phys. Rev. Lett. 71:1281–1283
Usher M, Schuster HG, Niebur E (1993) Dynamics of populations of integrate-and-fire neurons, partial synchronization and memory. Neural Comput. 5:570–586
Varadhan SRS (1980) Diffusion Problems and Partial Differential Equations (Springer, Berlin), p. 266 et seq.
Wang D, Buhmann J, von der Malsburg C (1990) Pattern segmentation in associative memory. Neural Comput.2:94–
Wilson HR, Cowan JD (1972) Excitatory and inhibitory interactions in localized populations of model neurons. Biophys. J. 12:1–24
Wilson MA, Bhalla US, Uhley JD, Bower JM (1989) GENESIS: A system for simulating neural networks. In: Advances in Neural Information Processing Systems, D. Touretzky (Ed.) (Morgan Kaufmann, San Mateo, CA), pp. 485–492
Wong RKS, Prince DA, Basbaum AI (1979) Intradendritic recordings from hippocampal neurons. Proc. Natl. Acad. Sci. U.S.A. 76:986–990
Yamada WM, Koch C, Adams PR (1989) Multiple channels and calcium dynamics. In: Methods in Neuronal Modeling, from Synapses to Networks, C. Koch and I. Segev (Eds.) (MIT Press, Cambridge, Mass.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer-Verlag New York, Inc.
About this chapter
Cite this chapter
Gerstner, W., van Hemmen, J.L. (1994). Coding and Information Processing in Neural Networks. In: Domany, E., van Hemmen, J.L., Schulten, K. (eds) Models of Neural Networks. Physics of Neural Networks. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-4320-5_1
Download citation
DOI: https://doi.org/10.1007/978-1-4612-4320-5_1
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4612-8736-0
Online ISBN: 978-1-4612-4320-5
eBook Packages: Springer Book Archive