Advertisement

Biological Cybernetics

, Volume 69, Issue 5–6, pp 503–515 | Cite as

Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns

  • Wulfram Gerstner
  • Raphael Ritz
  • J. Leo van Hemmen
Article

Abstract

Hebbian learning allows a network of spiking neurons to store and retrieve spatio-temporal patterns with a time resolution of 1 ms, despite the long postsynaptic and dendritic integration times. To show this, we introduce and analyze a model of spiking neurons, the spike response model, with a realistic distribution of axonal delays and with realistic postsynaptic potentials. Learning is performed by a local Hebbian rule which is based on the synchronism of presynaptic neurotransmitter release and some short-acting postsynaptic process. The time window of this synchronism determines the temporal resolution of pattern retrieval, which can be initiated by applying a short external stimulus pattern. Furthermore, a rate quantization is found in dependence upon the threshold value of the neurons, i.e., in a given time a pattern runs n times as often as learned, where n is a positive integer (n ⩾ 0). We show that all information about the spike pattern is lost if only mean firing rates (temporal average) or ensemble activities (spatial average) are considered. An average over several retrieval runs in order to generate a post-stimulus time histogram may also deteriorate the signal. The full information on a pattern is contained in the spike raster of a single run. Our results stress the importance, and advantage, of coding by spatio-temporal spike patterns instead of firing rates and average ensemble activity. The implications regarding modelling and experimental data analysis are discussed.

Keywords

Firing Rate Hebbian Learning Excitation Pattern Spike Response Spike Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abeles M (1982) Local cortical circuits. Springer, Berlin Heidelberg New YorkGoogle Scholar
  2. Abeles M, Lass Y (1975) Transmission of information by the axon. Biol Cybern 19:121–125CrossRefPubMedGoogle Scholar
  3. Adrian ED (1926) The impulses produced by sensory nerve endings. J Physiol (Lond) 61:49–72Google Scholar
  4. Aertsen A, Gerstein G, Johannesma P (1986) From neuron to assembly: neuronal organization and stimulus representation. In: Palm G, Aertsen A (eds) Brain theory. Springr, Berlin Heidelberg New York pp 7–24Google Scholar
  5. Amit DJ, Tsodyks MV (1991) Quantitative study of attractor neural networks retrieving at low spike rates. I. Substrate spike rates and neuronal gain. Neural Computation 2:259–273Google Scholar
  6. Amit DJ, Gutfreund H, Sompolinsky H (1985) Spin-glass models of neural networks. Phys Rev A 32:1007–1032CrossRefPubMedGoogle Scholar
  7. Amit DJ, Gutfreund H, Sompolinsky H (1987) Statistical mechanics of neural networks near saturation. Ann Phys (NY) 173:30–67CrossRefGoogle Scholar
  8. Bialek W, Rieke F, Ruyter van Steveninck RR de, Warland D (1991) Reading a neural code. Science 252:1854–1857PubMedGoogle Scholar
  9. Brown TH, Ganong AH, Kairiss EW, Keenan CL, Kelso SR (1989) Long-term potentation in two synaptic systems of the hippocamrain slice. In: Byrne JH, Berry WO (eds) Neural models of plasticty. Academic Press, San Diego, pp 266–306Google Scholar
  10. Brown TH, Zador AM, Mainen ZF, Claiborne BJ (1991) Hebbian modifications in hippocampal neurons. In: Baudry M, Davis JL (eds) Long-term potentiation. MIT Press, Cambridge, Mass, pp 357–389Google Scholar
  11. Creutzfeldt OD (1983) Cortex cerebri. Springer, Berlin Heidelberg New York, pp 76–78Google Scholar
  12. Eckhorn R, Grüsser OJ, Kröller J, Pellnitz, K, Pöpel B (1976) Efficiency of different neural codes: information transfer calculations for three different neural systems. Biol Cybern 22:49–60CrossRefPubMedGoogle Scholar
  13. Eckhorn R, Bauer R, Jordan W, Brosch M, Kruse W, Munk M, Reitboeck HJ (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biol Cybern 60:121–130CrossRefPubMedGoogle Scholar
  14. Eskandar EN, Richmond BJ, Hertz JA, Optican LM, Troels K (1992) Decoding of neuronal signals in visual pattern recognition. (Advances in neural information processing 4) Morgan Kaufmann, San Mateo, CalifGoogle Scholar
  15. Gardner E (1988) The space of interactions in neural network models. J Phys A: Math Gen 21:257–270CrossRefGoogle Scholar
  16. Gerstein GL, Perkel DH (1972) Mutual temporal relations among neuronal spike trains. Biophys J 12:453–473PubMedGoogle Scholar
  17. Gerstein GL, Bloom MJ, Espinosa IE, Evanczuk S, Turner MR (1983) Design of a laboratory for multineuron studies. IEEE Trans Syst Man Cybern 13:668–676Google Scholar
  18. Gerstner W (1993) Kodierung und Signalübertragung in neuronalen Systemen: Assoziative Netzwerke mit stochastisch feuernden Neuronen Thesis, Nov (1992), TU Munich, Harri Deutsch, FrankfurtGoogle Scholar
  19. Gerstner W, Hemmen JL van (1992a) Associative memory in a network of ‘spiking’ neurons. Network 3:139–164Google Scholar
  20. Gerstner W, Hemmen JL van (1992b) Universality in neural networks: the importance of the mean firing rate. Biol Cybern 67:195–205CrossRefPubMedGoogle Scholar
  21. Gerstner W, Ritz R, Hemmen JL van (1993) A biologically motivated and analytically soluble model of collective oscillations in the cortex. I. Theory of weak locking. Biol Cybern 68:363–374CrossRefPubMedGoogle Scholar
  22. Hebb DO (1949) The organization of behavior. Wiley, New YorkGoogle Scholar
  23. Hemmen JL van, Gerstner W, Ritz R (1992) A ‘microscopic’ model of collective oscillations in the cortex. In: Taylor JG, Caianiello EK, Cotterill RNJ, Clark JW (eds) Neural network dynamics. Springer, Berlin Heidelberg New York, pp 250–257Google Scholar
  24. Herz A, Sulzer B, Kühn R, Hemmen JL van (1988) The Hebb rule: representation of static and dynamic objects in neural nets. Europhys Lett 7:663–669Google Scholar
  25. Herz A, Sulzer B, Kühn R, Hemmen JL van (1989) Hebbian learning reconsidered: representation of static and dynamic objects in associative neural nets. Biol Cybern 60:457–467CrossRefPubMedGoogle Scholar
  26. Herz AVM, Li Z, Hemmen JL van (1991) Statistical mechanics of temporal association in neural networks with transmission delays. Phys Rev Lett 66:1370–1373CrossRefPubMedGoogle Scholar
  27. Hodgkin AL (1948) The local electric changes associated with repetitive action in a non-medullated axon. J Physiol (Lond) 107:165–181Google Scholar
  28. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558PubMedGoogle Scholar
  29. Hopfield JJ (1984) Neurons with graded response have computational properties like those of two-state neurons. Proc Natl Acad Sci USA 81:3088–3092PubMedGoogle Scholar
  30. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Neurophysiol 28:215–243Google Scholar
  31. Jack JJB, Noble D, Tsien RW (1975) Electric current flow in excitable cells. Clarendon Press, OxfordGoogle Scholar
  32. Johannesma P, Aertsen A, Boogaard H van den, Eggermont J, Epping W (1986) From synchrony to harmony: ideas on the function of neural assemblies and the interpretation of neural synchrony. In: Palm G, Aertsen A (eds) Brain theory. Springer, Berlin Heidelberg New York, pp 25–48Google Scholar
  33. Kelso SR, Ganong AH, Brown TH (1986) Hebbian synapses in hippocampus. Proc Natl Acad Sci USA 83:5326–5330PubMedGoogle Scholar
  34. Krauth W, Mézard M (1987) Learning algorithms with optimal stability in neural networks. J Phys A: Math Gen 20:L745-L752CrossRefGoogle Scholar
  35. Krüger J (1983) Simultaneous individual recordings from many cerebral neurons: techniques and results. Rev Physiol Biochem Pharmacol 98:177–233PubMedGoogle Scholar
  36. Krüger J, Aiple F (1988) Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. J Neurophysiol 60:798–828PubMedGoogle Scholar
  37. Krüger J, Becker JD (1991) Recognizing the visual stimulus from neuronal discharges. Trends Neurosci 14:282–286CrossRefPubMedGoogle Scholar
  38. Kühn R, Bös S, Hemmen JL van (1991) Statistical mechanics for networks of graded-response neurons. Phys Rev A 43:2084–2087CrossRefPubMedGoogle Scholar
  39. Lancaster B, Adams PR (1986) Calcium-dependent current generating the afterhyperpolarization of hippocampal neurons. J. Neurophysiol 55:1268–1282PubMedGoogle Scholar
  40. MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal link. Bull Math Biophys 14:127–135Google Scholar
  41. Miyashita Y (1988) Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature 335:817–820CrossRefPubMedGoogle Scholar
  42. Optican LM, Richmond BJ (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. 3. Information theoretic analysis. J Neurophysiol 57:162–178PubMedGoogle Scholar
  43. Palm G, Aertsen AMHJ, Gerstein GL (1988) On the significance of correlations among neuronal spike trains. Biol Cybern 59:1–11CrossRefPubMedGoogle Scholar
  44. Perkel DH, Gerstein GL, Moore GP (1967) Neuronal spike trains and stochastic point processes. I. The single spike train. II. Simultaneous spike trains. Biophys J 7:391–418, 419–440PubMedGoogle Scholar
  45. Reitboeck HJA (1983) A multi-electrode matrix for studies of temporal signal correlations within neural assemblies. In: Basar E, Flohr H, Haken H, Mandell AJ (eds) Synergetics of the brain. (Springer series in synegetics 23) Springer, Berlin Heidelberg New York, pp 174–182Google Scholar
  46. Rumelhart DE, McClelland GL (eds) (1986) Parallel distributed processing, vol 1: Foundations. MIT Press, Cambridge, MassGoogle Scholar
  47. Stein RB (1967) The information capacity of nerve cells using a frequency code. Biophys J 7:797–826Google Scholar
  48. Wong RKS, Prince DA, Basbaum AF (1979) Intradendritic recordings from hippocampal neurons. Proc Natl Acad Sci USA 76:986–990PubMedGoogle Scholar

Copyright information

© Springer-Verlag 1993

Authors and Affiliations

  • Wulfram Gerstner
    • 1
  • Raphael Ritz
    • 1
  • J. Leo van Hemmen
    • 1
  1. 1.Physik-Department der TU MünchenGarching bei MünchenGermany

Personalised recommendations