Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns

Abstract

Hebbian learning allows a network of spiking neurons to store and retrieve spatio-temporal patterns with a time resolution of 1 ms, despite the long postsynaptic and dendritic integration times. To show this, we introduce and analyze a model of spiking neurons, the spike response model, with a realistic distribution of axonal delays and with realistic postsynaptic potentials. Learning is performed by a local Hebbian rule which is based on the synchronism of presynaptic neurotransmitter release and some short-acting postsynaptic process. The time window of this synchronism determines the temporal resolution of pattern retrieval, which can be initiated by applying a short external stimulus pattern. Furthermore, a rate quantization is found in dependence upon the threshold value of the neurons, i.e., in a given time a pattern runs n times as often as learned, where n is a positive integer (n ⩾ 0). We show that all information about the spike pattern is lost if only mean firing rates (temporal average) or ensemble activities (spatial average) are considered. An average over several retrieval runs in order to generate a post-stimulus time histogram may also deteriorate the signal. The full information on a pattern is contained in the spike raster of a single run. Our results stress the importance, and advantage, of coding by spatio-temporal spike patterns instead of firing rates and average ensemble activity. The implications regarding modelling and experimental data analysis are discussed.

This is a preview of subscription content, access via your institution.

References

  1. Abeles M (1982) Local cortical circuits. Springer, Berlin Heidelberg New York

    Google Scholar 

  2. Abeles M, Lass Y (1975) Transmission of information by the axon. Biol Cybern 19:121–125

    Article  PubMed  Google Scholar 

  3. Adrian ED (1926) The impulses produced by sensory nerve endings. J Physiol (Lond) 61:49–72

    Google Scholar 

  4. Aertsen A, Gerstein G, Johannesma P (1986) From neuron to assembly: neuronal organization and stimulus representation. In: Palm G, Aertsen A (eds) Brain theory. Springr, Berlin Heidelberg New York pp 7–24

    Google Scholar 

  5. Amit DJ, Tsodyks MV (1991) Quantitative study of attractor neural networks retrieving at low spike rates. I. Substrate spike rates and neuronal gain. Neural Computation 2:259–273

    Google Scholar 

  6. Amit DJ, Gutfreund H, Sompolinsky H (1985) Spin-glass models of neural networks. Phys Rev A 32:1007–1032

    Article  PubMed  Google Scholar 

  7. Amit DJ, Gutfreund H, Sompolinsky H (1987) Statistical mechanics of neural networks near saturation. Ann Phys (NY) 173:30–67

    Article  Google Scholar 

  8. Bialek W, Rieke F, Ruyter van Steveninck RR de, Warland D (1991) Reading a neural code. Science 252:1854–1857

    PubMed  Google Scholar 

  9. Brown TH, Ganong AH, Kairiss EW, Keenan CL, Kelso SR (1989) Long-term potentation in two synaptic systems of the hippocamrain slice. In: Byrne JH, Berry WO (eds) Neural models of plasticty. Academic Press, San Diego, pp 266–306

    Google Scholar 

  10. Brown TH, Zador AM, Mainen ZF, Claiborne BJ (1991) Hebbian modifications in hippocampal neurons. In: Baudry M, Davis JL (eds) Long-term potentiation. MIT Press, Cambridge, Mass, pp 357–389

    Google Scholar 

  11. Creutzfeldt OD (1983) Cortex cerebri. Springer, Berlin Heidelberg New York, pp 76–78

    Google Scholar 

  12. Eckhorn R, Grüsser OJ, Kröller J, Pellnitz, K, Pöpel B (1976) Efficiency of different neural codes: information transfer calculations for three different neural systems. Biol Cybern 22:49–60

    Article  PubMed  Google Scholar 

  13. Eckhorn R, Bauer R, Jordan W, Brosch M, Kruse W, Munk M, Reitboeck HJ (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biol Cybern 60:121–130

    Article  PubMed  Google Scholar 

  14. Eskandar EN, Richmond BJ, Hertz JA, Optican LM, Troels K (1992) Decoding of neuronal signals in visual pattern recognition. (Advances in neural information processing 4) Morgan Kaufmann, San Mateo, Calif

    Google Scholar 

  15. Gardner E (1988) The space of interactions in neural network models. J Phys A: Math Gen 21:257–270

    Article  Google Scholar 

  16. Gerstein GL, Perkel DH (1972) Mutual temporal relations among neuronal spike trains. Biophys J 12:453–473

    PubMed  Google Scholar 

  17. Gerstein GL, Bloom MJ, Espinosa IE, Evanczuk S, Turner MR (1983) Design of a laboratory for multineuron studies. IEEE Trans Syst Man Cybern 13:668–676

    Google Scholar 

  18. Gerstner W (1993) Kodierung und Signalübertragung in neuronalen Systemen: Assoziative Netzwerke mit stochastisch feuernden Neuronen Thesis, Nov (1992), TU Munich, Harri Deutsch, Frankfurt

    Google Scholar 

  19. Gerstner W, Hemmen JL van (1992a) Associative memory in a network of ‘spiking’ neurons. Network 3:139–164

    Google Scholar 

  20. Gerstner W, Hemmen JL van (1992b) Universality in neural networks: the importance of the mean firing rate. Biol Cybern 67:195–205

    Article  PubMed  Google Scholar 

  21. Gerstner W, Ritz R, Hemmen JL van (1993) A biologically motivated and analytically soluble model of collective oscillations in the cortex. I. Theory of weak locking. Biol Cybern 68:363–374

    Article  PubMed  Google Scholar 

  22. Hebb DO (1949) The organization of behavior. Wiley, New York

    Google Scholar 

  23. Hemmen JL van, Gerstner W, Ritz R (1992) A ‘microscopic’ model of collective oscillations in the cortex. In: Taylor JG, Caianiello EK, Cotterill RNJ, Clark JW (eds) Neural network dynamics. Springer, Berlin Heidelberg New York, pp 250–257

    Google Scholar 

  24. Herz A, Sulzer B, Kühn R, Hemmen JL van (1988) The Hebb rule: representation of static and dynamic objects in neural nets. Europhys Lett 7:663–669

    Google Scholar 

  25. Herz A, Sulzer B, Kühn R, Hemmen JL van (1989) Hebbian learning reconsidered: representation of static and dynamic objects in associative neural nets. Biol Cybern 60:457–467

    Article  PubMed  Google Scholar 

  26. Herz AVM, Li Z, Hemmen JL van (1991) Statistical mechanics of temporal association in neural networks with transmission delays. Phys Rev Lett 66:1370–1373

    Article  PubMed  Google Scholar 

  27. Hodgkin AL (1948) The local electric changes associated with repetitive action in a non-medullated axon. J Physiol (Lond) 107:165–181

    Google Scholar 

  28. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558

    PubMed  Google Scholar 

  29. Hopfield JJ (1984) Neurons with graded response have computational properties like those of two-state neurons. Proc Natl Acad Sci USA 81:3088–3092

    PubMed  Google Scholar 

  30. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Neurophysiol 28:215–243

    Google Scholar 

  31. Jack JJB, Noble D, Tsien RW (1975) Electric current flow in excitable cells. Clarendon Press, Oxford

    Google Scholar 

  32. Johannesma P, Aertsen A, Boogaard H van den, Eggermont J, Epping W (1986) From synchrony to harmony: ideas on the function of neural assemblies and the interpretation of neural synchrony. In: Palm G, Aertsen A (eds) Brain theory. Springer, Berlin Heidelberg New York, pp 25–48

    Google Scholar 

  33. Kelso SR, Ganong AH, Brown TH (1986) Hebbian synapses in hippocampus. Proc Natl Acad Sci USA 83:5326–5330

    PubMed  Google Scholar 

  34. Krauth W, Mézard M (1987) Learning algorithms with optimal stability in neural networks. J Phys A: Math Gen 20:L745-L752

    Article  Google Scholar 

  35. Krüger J (1983) Simultaneous individual recordings from many cerebral neurons: techniques and results. Rev Physiol Biochem Pharmacol 98:177–233

    PubMed  Google Scholar 

  36. Krüger J, Aiple F (1988) Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. J Neurophysiol 60:798–828

    PubMed  Google Scholar 

  37. Krüger J, Becker JD (1991) Recognizing the visual stimulus from neuronal discharges. Trends Neurosci 14:282–286

    Article  PubMed  Google Scholar 

  38. Kühn R, Bös S, Hemmen JL van (1991) Statistical mechanics for networks of graded-response neurons. Phys Rev A 43:2084–2087

    Article  PubMed  Google Scholar 

  39. Lancaster B, Adams PR (1986) Calcium-dependent current generating the afterhyperpolarization of hippocampal neurons. J. Neurophysiol 55:1268–1282

    PubMed  Google Scholar 

  40. MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal link. Bull Math Biophys 14:127–135

    Google Scholar 

  41. Miyashita Y (1988) Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature 335:817–820

    Article  PubMed  Google Scholar 

  42. Optican LM, Richmond BJ (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. 3. Information theoretic analysis. J Neurophysiol 57:162–178

    PubMed  Google Scholar 

  43. Palm G, Aertsen AMHJ, Gerstein GL (1988) On the significance of correlations among neuronal spike trains. Biol Cybern 59:1–11

    Article  PubMed  Google Scholar 

  44. Perkel DH, Gerstein GL, Moore GP (1967) Neuronal spike trains and stochastic point processes. I. The single spike train. II. Simultaneous spike trains. Biophys J 7:391–418, 419–440

    PubMed  Google Scholar 

  45. Reitboeck HJA (1983) A multi-electrode matrix for studies of temporal signal correlations within neural assemblies. In: Basar E, Flohr H, Haken H, Mandell AJ (eds) Synergetics of the brain. (Springer series in synegetics 23) Springer, Berlin Heidelberg New York, pp 174–182

    Google Scholar 

  46. Rumelhart DE, McClelland GL (eds) (1986) Parallel distributed processing, vol 1: Foundations. MIT Press, Cambridge, Mass

    Google Scholar 

  47. Stein RB (1967) The information capacity of nerve cells using a frequency code. Biophys J 7:797–826

    Google Scholar 

  48. Wong RKS, Prince DA, Basbaum AF (1979) Intradendritic recordings from hippocampal neurons. Proc Natl Acad Sci USA 76:986–990

    PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to J. Leo van Hemmen.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gerstner, W., Ritz, R. & van Hemmen, J.L. Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern. 69, 503–515 (1993). https://doi.org/10.1007/BF00199450

Download citation

Keywords

  • Firing Rate
  • Hebbian Learning
  • Excitation Pattern
  • Spike Response
  • Spike Pattern