Learning Temporally Precise Spiking Patterns through Reward Modulated Spike-Timing-Dependent Plasticity

  • Brian Gardner
  • André Grüning
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8131)


Precise neuronal spike timing plays an important role in many aspects of cognitive processing. Here, we explore how a spiking neural network can learn to generate temporally precise spikes in response to a spatio-temporal pattern, through spike-timing-dependent plasticity modulated by a delayed reward signal. An escape noise neuron is implemented as the readout to incorporate the effect of background noise on spike timing. We compare the performance of two different escape rate functions that drive spiking in the readout neuron: the Arrhenius & Current (A&C) and Exponential (EXP) model. Our results show that the network can learn to reproduce target spike patterns containing between 1 and 10 spikes with 10 ms temporal accuracy. We also demonstrate the superior performance of the A&C model over the EXP model for the parameters we consider, especially when reproducing a large number of target spikes.


Neuronal Plasticity Stochastic Neuron Synapses 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barto, A., Sutton, R.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)Google Scholar
  2. 2.
    Bi, G., Poo, M.: Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. The Journal of Neuroscience 18(24), 10464–10472 (1998)Google Scholar
  3. 3.
    Bohte, S.: The evidence for neural information processing with precise spike-times: A survey. Natural Computing 3(2), 195–206 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Chance, F., Abbott, L., Reyes, A.: Gain modulation from background synaptic input. Neuron 35(4), 773–782 (2002)CrossRefGoogle Scholar
  5. 5.
    El-Laithy, K., Bogdan, M.: A reinforcement learning framework for spiking networks with dynamic synapses. In: Computational Intelligence and Neuroscience 2011, vol. 4 (2011)Google Scholar
  6. 6.
    Farries, M., Fairhall, A.: Reinforcement learning with modulated spike timing dependent synaptic plasticity. Journal of Neurophysiology 98(6), 3648–3665 (2007)CrossRefGoogle Scholar
  7. 7.
    Florian, R.: Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation 19(6), 1468–1502 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Frémaux, N., Sprekeler, H., Gerstner, W.: Functional requirements for reward-modulated spike-timing-dependent plasticity. The Journal of Neuroscience 30(40), 13326–13337 (2010)CrossRefGoogle Scholar
  9. 9.
    Gerstner, W., Kistler, W.: Spiking neuron models: Single neurons, populations, plasticity. Cambridge University Press, Cambridge (2002)CrossRefGoogle Scholar
  10. 10.
    Grüning, A., Sporea, I.: Supervised learning of logical operations in layered spiking neural networks with spike train encoding. Neural Processing Letters 36(2), 117–134 (2012)CrossRefGoogle Scholar
  11. 11.
    Izhikevich, E.: Solving the distal reward problem through linkage of stdp and dopamine signaling. Cerebral Cortex 17(10), 2443–2452 (2007)CrossRefGoogle Scholar
  12. 12.
    Legenstein, R., Pecevski, D., Maass, W.: A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Computational Biology 4(10), e1000180 (2008)Google Scholar
  13. 13.
    Morrison, A., Diesmann, M., Gerstner, W.: Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics 98(6), 459–478 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Pfister, J., Toyoizumi, T., Barber, D., Gerstner, W.: Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Computation 18(6), 1318–1348 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Ponulak, F., Kasinski, A.: Supervised learning in spiking neural networks with resume: Sequence learning, classification, and spike shifting. Neural Computation 22(2), 467–510 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Rossum, M.: A novel spike distance. Neural Computation 13(4), 751–763 (2001)CrossRefzbMATHGoogle Scholar
  17. 17.
    Sporea, I., Grüning, A.: Supervised learning in multilayer spiking neural networks. Neural Computation 25(2), 473–509 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Urbanczik, R., Senn, W.: Reinforcement learning in populations of spiking neurons. Nature Neuroscience 12(3), 250–252 (2009)CrossRefGoogle Scholar
  19. 19.
    Van Rossum, M., Bi, G., Turrigiano, G.: Stable hebbian learning from spike timing-dependent plasticity. The Journal of Neuroscience 20(23), 8812–8821 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Brian Gardner
    • 1
  • André Grüning
    • 1
  1. 1.Department of ComputingUniversity of SurreySurreyUnited Kingdom

Personalised recommendations