On the relevance of time in neural computation and learning

  • Wolfgang Maass
Session 10
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1316)

Abstract

We discuss models for computation in biological neural systems that are based on the current state of knowledge in neurophysiology. Differences and similarities to traditional neural network models are highlighted. It turns out that many important questions regarding computation and learning in biological neural systems cannot be adequately addressed in traditional neural network models. In particular the role of time is quite different in biologically more realistic models, and many fundamental questions regarding computation and learning have to be rethought for this context. Simultaneously a new generation of VLSI-chips is emerging (“pulsed VLSI”) where new ideas about computing and learning with temporal coding can be tested.

Articles with details and further pointers to the literature can be found at http://www.cis.tu-graz.ac.at/igi/maass/.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    L. F. Abbott, “Decoding neuronal firing and modelling neural networks”, Quarterly review of Biophysics, vol. 27(3), pp 291–331, 1994.Google Scholar
  2. 2.
    L. F. Abbott, J. A. Varela, K. A. Sen, and S. B. Nelson, “Synaptic depression and cortical gain control” Science, vol. 275, pp 220–4, 1997.PubMedGoogle Scholar
  3. 3.
    M. Abeles, “Corticonics: Neural Circuits of the Cerebral Cortex”, Cambridge University Press, 1991.Google Scholar
  4. 4.
    H. Agmon-Snir, I. Segev, “Signal delay and input synchronization in passive dendritic structures”, Journal of Neurophysiology, vol. 70, pp 2066–2085, 1993.PubMedGoogle Scholar
  5. 5.
    C. Allen, C. F. Stevens, “An evaluation of causes for unreliability of synaptic transmission”, Proc. Natl. Acad. Sci. of the USA, vol. 91, pp 10380–10383, 1994.Google Scholar
  6. 6.
    M. A. Arbib, ed., “The Handbook of Brain Theory and Neural Networks”, MIT-Press, Cambridge, 1995.Google Scholar
  7. 7.
    W. Bair, C. Koch, “Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkey,” Neural Computation, vol. 8, pp 1185–1202, 1996.PubMedGoogle Scholar
  8. 8.
    Ö. Bernander, C. Koch, M. Usher, “The effect of synchronized inputs at the single neuron level”. Neural Computation, vol. 6, pp 622–641, 1994.Google Scholar
  9. 9.
    P. S. Churchland, T. J. Sejnowski, “The Computational Brain”, MIT-Press, 1993.Google Scholar
  10. 10.
    R. C. deCharms, M. M. Merzenich, “Primary cortical representation of sounds by the coordination of action-potential timing”, Nature, vol. 381, pp 610–613, 1996.PubMedGoogle Scholar
  11. 11.
    L. Dobrunz, C. F. Stevens, “Heterogenous release probabilities in hippocampal neurons”, Neuron, 1997, in press.Google Scholar
  12. 12.
    R. Eckhorn, H. J. Reitboeck, M. Arndt, P. Dicke, “Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex”, Neural Computation, vol. 2, pp 293–307, 1990.Google Scholar
  13. 13.
    T. J. Gawne, T. W. Kjaer, B. J. Richmond, “Latency: another potential code for feature binding in striate cortex”, J. of Neurophysiology, vol. 76(3), 1996.Google Scholar
  14. 14.
    W. Gerstner, “Time structure of the activity in neural network models”, Phys. Rev. E, vol. 51, pp 738–758, 1995.Google Scholar
  15. 15.
    W. Gerstner, J. L. van Hemmen, “How to describe neuronal activity: spikes, rates, or assemblies?”, Advances in Neural Information Processing Systems, vol. 6, Morgan Kaufmann, San Mateo, pp 463–470, 1994.Google Scholar
  16. 16.
    W. Gerstner, R. Kempter, J. L. van Hemmen, H. Wagner, “A neuronal learning rule for sub-millisecond temporal coding”, Nature, vol. 383, pp 76–78, 1996.PubMedGoogle Scholar
  17. 17.
    J. J. Hopfield, “Pattern recognition computation using action potential timing for stimulus representations”, Nature, vol. 376, pp 33–36, 1995.PubMedGoogle Scholar
  18. 18.
    D. Johnston, S. M. Wu, “Foundations of Cellular Neurophysiology”, MIT-Press, Cambridge, 1995.Google Scholar
  19. 19.
    P. Koiran, “VC-dimension in circuit complexity”, Proc. of the 11th IEEE Conference on Computational Complexity, pp 81–85, 1996.Google Scholar
  20. 20.
    C. Koch, “Biophysics of Computation: Information Processing in Single Neurons”, Oxford University Press, Oxford 1997 (to appear).Google Scholar
  21. 21.
    A. K. Kreiter, W. Singer. “Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey”. The Journal of Neuroscience, vol. 16(7), pp 2381–2396, 1996.PubMedGoogle Scholar
  22. 22.
    J. Krüger, F. Aiple, “Multielectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers”, J. Neurophysiology, vol. 60, pp 798–828, 1988.Google Scholar
  23. 23.
    M. Leshno, V. Y. Lin, A. Pinkus, S. Schocken, “Multilayer feedforward networks with a nonpolynomial activation function can approximate any function”, Neural Networks, vol. 6, pp 861–867, 1993.Google Scholar
  24. 24.
    W. Maass, “Fast sigmoidal networks via spiking neurons”, Neural Computation, vol. 9, pp 279–304, 1997.PubMedGoogle Scholar
  25. 25.
    W. Maass, “On the computational complexity of networks of spiking neurons”, Advances in Neural Information Processing Systems, vol. 7, MIT Press (Cambridge), pp 183–190, 1995.Google Scholar
  26. 26.
    W. Maass, “Lower bounds for the computational power of networks of spiking neurons”, Neural Computation, vol. 8(1), pp 1–40, 1996.Google Scholar
  27. 27.
    W. Maass, “On the computational power of noisy spiking neurons”, Advances in Neural Information Processing Systems, vol. 8, MIT-Press (Cambridge), pp 211–217, 1996.Google Scholar
  28. 28.
    W. Maass, “Networks of spiking neurons: the third generation of neural network models”, Neural Networks, 1997, to appear. FTP-host: archive.cis.ohio-state.edu, FTP-filename: /pub/neuroprose/maass.third-generation.ps.Z.Google Scholar
  29. 29.
    W. Maass, “A simple model for neural computation with firing rates and firing correlations”, submitted for publication.Google Scholar
  30. 30.
    W. Maass, T. Natschläger, “Networks of spiking neurons can emulate arbitrary Hopfield nets in temporal coding”, Proceedings of the 6th Annual Conference on Computational Neuroscience 1997, (CNS'97) in Big Sky, Montana, USA, to appear.Google Scholar
  31. 31.
    W. Maass, B. Ruf, “On the relevance of the shape of postsynaptic potentials for the computational power of networks of spiking neurons”, Proceedings of the International Conference on Artificial Neural Networks, ICANN' 95, EC2& Cie, Paris, pp. 515–520.Google Scholar
  32. 32.
    W. Maass, M. Schmitt, “On the complexity of learning for a spiking neuron” (Extendend Abstract), appears in the Proceedings of the 10th Conference on Computational Learning Theory 1997, ACM-Press (New York), 1997.Google Scholar
  33. 33.
    W. Maass, M. Schmitt, “Complexity of learning for networks of spiking neurons”, submitted for publication.Google Scholar
  34. 34.
    W. Maass, A. M. Zador, “Computing with stochastic dynamic synapses”, 1997, submitted for publication.Google Scholar
  35. 35.
    Z. F. Mainen, T. J. Sejnowski, “Reliability of spike timing in neocortical neurons”, Science, vol. 268, pp 1503–1506, 1995.PubMedGoogle Scholar
  36. 36.
    C. Mead, “Analog VLSI and Neural Systems”, Addison-Wesley (Reading), 1989.Google Scholar
  37. 37.
    P. M. Milner, “A model for visual shape recognition”, Psychological Review, vol. 81(6), pp 521–535, 1974.PubMedGoogle Scholar
  38. 38.
    A. Murray, L. Tarassenko, “Analogue Neural VLSI: A Pulse Stream Approach”, Chapman & Hall, 1994.Google Scholar
  39. 39.
    T. Natschläger, B. Ruf, “Learning Radial Basis Functions with Spiking Neurons Using Action Potential Timing”, submitted for publication.Google Scholar
  40. 40.
    W.A. Phillips, W. Singer, “In search of common foundations for cortical computation”, Behavioral and Brain Sciences, 1997, in press.Google Scholar
  41. 41.
    F. Rieke, D. Warland, R. van Stevenick, W. Bialek, “SPIKES: Exploring the Neural Code”, MIT Press, Cambridge, 1996.Google Scholar
  42. 42.
    B. Ruf, “Computing functions with spiking neurons in temporal coding”, Proc. of the Int. Work-Conference on Artificial and Natural Neural Networks IWANN'97, Lecture Notes in Computer Science, vol. 1240, pp 265–272, Springer, Berlin, 1997.Google Scholar
  43. 43.
    B. Ruf, M. Schmitt, “Hebbian learning in networks of spiking neurons using temporal coding”, Proc. of the Int. Work-Conference on Artificial and Natural Neural Networks IWANN'97, Lecture Notes in Computer Science, vol. 1240, pp 380–389, Springer, Berlin, 1997.Google Scholar
  44. 44.
    B. Ruf, M. Schmitt, “Self-organizing maps of spiking neurons using temporal coding”, Proceedings of the 6th Annual Conference on Computational Neuroscience 1997, (CNS'97) in Big Sky, Montana, USA, to appear.Google Scholar
  45. 45.
    R. R. de Ruyter van Steveninck, G.D. Lewen, S. P. Koberle, W. Bialek, “Reproducibility and Variability in Neural Spike Trains”, Science, vol. 275, pp 1805–1808, 1997.PubMedGoogle Scholar
  46. 46.
    K. Sen, J. C. Jorge-Rivera, E. Marder, L. F. Abbott, “Decoding synapses”, The Journal of Neuroscience, vol. 16(19), pp 6307–6318, 1996.PubMedGoogle Scholar
  47. 47.
    L. Shastri, V. Ajjanagadde, “From simple associations to systematic reasoning: a connectionist representation of rules, variables and dynamic bindings using temporal synchrony”, Behavioural and Brain Sciences, vol. 16, pp 417–494, 1993.Google Scholar
  48. 48.
    G. M. Shepherd, ed., “The Synaptic Organization of the Brain”, 3rd ed., Oxford University Press, New York, 1990.Google Scholar
  49. 49.
    E. D. Sontag, “Shattering all sets of κ points in ‘general position’ requires (κ-1)/2 parameters”, Neural Computation, vol. 9, pp 337–348, 1997.Google Scholar
  50. 50.
    S. Thorpe, D. Fize, C. Marlot, “Speed of processing in the human visual system”, Nature, vol. 381, pp 520–522, 1996.PubMedGoogle Scholar
  51. 51.
    S. Thorpe, J. Gautrais, “Rapid visual processing using spike asynchrony”, Advances in Neural Information Processing Systems, vol. 9, MIT-Press (Cambridge), 1997, to appear.Google Scholar
  52. 52.
    M. V. Tsodyks, H. Markram, “The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability”, Proc. Natl. Acad. Sci. of the USA, vol. 94, pp 719–723, 1997.Google Scholar
  53. 53.
    H. C. Tuckwell, “Introduction to Theoretical Neurobiology”, vol. 1 and 2, Cambridge University Press, Cambridge, 1988.Google Scholar
  54. 54.
    L. G. Valiant, “Circuits of the Mind”, Oxford University Press, 1994.Google Scholar
  55. 55.
    C. von der Malsburg, “The correlation theory of brain function”, Internal Report 81-2 of the Dept. of Neurobiology of the Max Planck Institute for Biophysical Chemistry in Göttingen, Germany, 1981.Google Scholar
  56. 56.
    M. Wehr, G. Laurent, “Odour encoding by temporal sequences of firing in oscillating neural assemblies”, Nature, vol. 384, pp 162–166, 1996.PubMedGoogle Scholar
  57. 57.
    A. M. Zador, B. A. Pearlmutter, “VC dimension of an integrate-and-fire neuron model”, Neural Computation, 8(3), pp 611–624, 1996.Google Scholar
  58. 58.
    M. L. Zaghloul, J. L. Meador, R. W. Newcomb, eds., “Silicon Implementations of Pulse Coded Neural Networks”, Kluwer Academic Publishers, 1994.Google Scholar
  59. 59.
    R. Zucker, “Short-term synaptic plasticity”, Annual Review of Neuroscience, vol. 12, pp 13–31, 1989.PubMedGoogle Scholar

Copyright information

© Springer-Verlag 1997

Authors and Affiliations

  • Wolfgang Maass
    • 1
  1. 1.Institute for Theoretical Computer ScienceTechnische Universitaet GrazGrazAustria

Personalised recommendations