Abstract
Spike timing dependent plasticity (STDP) likely plays an important role in forming and changing connectivity patterns between neurons in our brain. In a unidirectional synaptic connection between two neurons, it uses the causal relation between spiking activity of a presynaptic input neuron and a postsynaptic output neuron to change the strength of this connection. While the nature of STDP benefits unsupervised learning of correlated inputs, any incorporation of value into the learning process needs some form of reinforcement. Chemical neuromodulators such as Dopamine or Acetylcholine are thought to signal changes between external reward and internal expectation to many brain regions, including the basal ganglia. This effect is often modelled through a direct inclusion of the level of Dopamine as a third factor into the STDP rule. While this gives the benefit of direct control over synaptic modification, it does not account for observed instantaneous effects in neuronal activity on application of Dopamine agonists. Specifically, an instant facilitation of neuronal excitability in the striatum can not be explained by the only indirect effect that dopamine-modulated STDP has on a neuron’s firing pattern. We therefore propose a model for synaptic transmission where the level of neuromodulator does not directly influence synaptic plasticity, but instead alters the relative firing causality between pre- and postsynaptic neurons. Through the direct effect on postsynaptic activity, our rule allows indirect modulation of the learning outcome even with unmodulated, two-factor STDP. However, it also does not prohibit joint operation together with three-factor STDP rules.
Similar content being viewed by others
References
Bi G, Poo M (2001) Synaptic modification by correlated activity: Hebb’s postulate revisited. Annu Rev Neurosci 24:139–166. doi:10.1146/annurev.neuro.24.1.139
Chorley P, Seth AK (2011) Dopamine-signaled reward predictions generated by competitive excitation and inhibition in a spiking neural network model. Frontiers Comput Neurosci 5(May):21. doi:10.3389/fncom.2011.00021
Farries MA, Fairhall AL (2007) Reinforcement learning with modulated spike timing dependent synaptic plasticity. J Neurophysiol 98(6):3648–3665. doi:10.1152/jn.00364.2007
Humphries M, Khamassi M, Gurney K (2012) Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia 6(February):1–14. doi:10.3389/fnins.2012.00009
Izhikevich EM (2003) Simple model of spiking neurons. IEEE Trans Neural Netw (a publication of the IEEE Neural Networks Council) 14(6):1569–1572. doi:10.1109/TNN.2003.820440
Izhikevich EM (2004) Which model to use for cortical spiking neurons? IEEE Trans Neural Netw (a publication of the IEEE Neural Networks Council) 15(5):1063–1070. doi:10.1109/TNN.2004.832719
Izhikevich EM (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17(10):2443–2452. doi:10.1093/cercor/bhl152
Kroener S, Chandler LJ, Phillips PEM, Seamans JK (2009) Dopamine modulates persistent synaptic activity and enhances the signal-to-noise ratio in the prefrontal cortex. PloS One 4(8):e6507. doi:10.1371/journal.pone.0006507
Masquelier T, Guyonneau R, Thorpe SJ (2008) Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PloS One 3(1):e1377. doi:10.1371/journal.pone.0001377
Morrison A, Diesmann M, Gerstner W (2008) Phenomenological models of synaptic plasticity based on spike timing. Biol Cybern 98(6):459–478. doi:10.1007/s00422-008-0233-1
Pfister J-P, Gerstner W (2006) Triplets of spikes in a model of spike timing-dependent plasticity. J Neurosc 26(38):9673–9682. doi:10.1523/JNEUROSCI.1425-06.2006
Potjans W, Morrison A, Diesmann M (2009) A spiking neural network model of an actor-critic learning agent. Neural Comput 21(2):301–339. doi:10.1162/neco.2008.08-07-593
Redgrave P, Gurney K (2006) The short-latency dopamine signal: a role in discovering novel actions? Nature reviews. Neuroscience 7(12):967–975. doi:10.1038/nrn2022
Reynolds JNJ, Wickens JR (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15(4–6), 507–521
Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275(5306):1593
Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321(5890):848–851. doi:10.1126/science.1160575
Surmeier DJ, Ding J, Day M, Wang Z, Shen W (2007) D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci 30(5):228–235
Thurley K, Senn W, Lüscher HR (2008) Dopamine increases the gain of the input-output response of rat prefrontal pyramidal neurons. J Neurophysiol 99(6):2985–2997. doi:10.1152/jn.01098.2007
Trimmer PC, Houston AI, Marshall JAR, Bogacz R, Paul ES, Mendl MT, McNamara JM (2008) Mammalian choices: combining fast-but-inaccurate and slow-but-accurate decision-making systems. Proceedings. Biol Sci/R Soc 275(1649):2353–2561. doi:10.1098/rspb.2008.0417
Acknowledgments
We would like to thank A. Moser for interesting discussions on this topic.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Vogt, S.M., Hofmann, U.G. Neuromodulation of STDP through short-term changes in firing causality. Cogn Neurodyn 6, 353–366 (2012). https://doi.org/10.1007/s11571-012-9202-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11571-012-9202-4