A model of the neural mechanism of instrumentalization (spontaneous reproduction) of movements induced by reinforcing stimulation of the motor cortex is presented. Instrumentalization is based on strengthening of the excitatory connections of cortical pyramidal neurons: 1) with dopamine neurons and 2) with each other in conditions of “dopamine modulation.” Correct changes to connections between pyramidal neurons are critically dependent on weakening of modulation immediately after neuron activity reaches the target state.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anokhin, P. K. and Foreword to Konorski, J., Integrative Activity of the Brain [Russian translation], Mir, Moscow (1970).
Ashmarin, I. P., Eshchenko, N. D., and Karazeeva, E. P., Neurochemistry in Tables and Diagrams, Examen Press, Moscow (2007).
Ballard, I. C., Murty, V. P., Carter, R. M., et al., “Dorsolateral prefrontal cortex drives mesolimbic dopaminergic regions to initiate motivated behavior,” J. Neurosci., 31, No. 28, 10,340–10,346 (2011).
Berridge, K. C., “The debate over dopamine’s role in reward: the case for incentive salience,” Psychopharmacology, 191, 391–431 (2007).
Bittner, K. C., Milstein, A. D., Grienberger, C., et al., “Behavioral time scale synaptic plasticity underlies CA1 place fields,” Science, 357, 1033–1036 (2017).
Bolles, R. C., “Reinforcement, expectancy, and learning,” Psychol. Rev., 79, No. 5, 394–409 (1972).
Cepeda, C. and Levine, M. S., “Where do you think you are going? The NMDA-D1 receptor trap,” Sci. STKE, 2006, 1–5 (2006).
Chen, G., Greengard, P., and Yan, Z., “Potentiation of NMDA receptor currents by dopamine D1 receptors in prefrontal cortex,” Proc. Natl. Acad. Sci. USA, 101, No. 8, 2596–2600 (2004).
De Wit, S. and Dickinson, A., “Associative theories of goal-directed behavior: A case for animal-human translational models,” Psychol. Res., 73, No. 4, 463–476 (2009).
Fanselow, M. S. and Wassum, K. M., “The origins and organization of vertebrate Pavlovian conditioning,” Cold Spring Harb. Perspect. Biol., 8, a021717 (2016).
Fel’dman, A. G., “Central and reflex mechanisms of motor control,” Nauka, Moscow (1979).
Flagel, S. B., Clark, J. J., Robinson, T. E., et al., “A selective role for dopamine in stimulus-reward learning,” Nature, 469, No. 7328, 53–57 (2011).
Flores-Hernández, J., Cepeda, K., Hernández-Echeagaray, E., et al., “Dopamine enhancement of NMDA currents in dissociated medium-sized striatal neurons: Role of D1 receptors and DARPP-32,” J. Neurophysiol, 88, 3010–3020 (2002).
Frémaux, N. and Gerstner, W., “Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules,” Front. Neural Circuits, 9, No. 85, 1–19 (2016).
Giszter, S. F., Mussa-lvaldi, F. A., and Emilio Bizzi, E., “Convergent force fields organized in the frog’s spinal cord,” J. Neurosci., 73, No. 2, 467–491 (1993).
Glimcher, P. W., “Understanding dopamine and reinforcement learning – the dopamine reward prediction error hypothesis,” Proc. Natl. Acad. Sci. USA, 108, Suppl. 3, 15647–54 (2011).
Graziano, M. S. A., Taylor, C. S. R., and Moore, T., “Complex movements evoked by microstimulation of precentral cortex,” Neuron, 34, No. 5, 841–851 (2002).
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., et al., “Mesolimbic dopamine signals the value of work,” Nat. Neurosci., 19, No. 1, 117–126 (2016).
Heckhausen, H., Motivation and Action [Russian translation], Piter, St. Petersburg (2003).
He, K., Huertas, M., Hong, S. Z., et al., “Distinct eligibility traces for LTP and LTD in cortical synapses,” Neuron, 88, 1–11 (2015).
Hong, S. and Hikosaka, O., “Dopamine-mediated learning and switching in cortico-striatal circuit explain behavioral changes in reinforcement learning,” Front. Behav. Neurosci., 5, No. 15, 1–17 (2011).
Ilango, A., Kesner, A. J., Broker, C. J., et al., “Phasic excitation of ventral tegmental dopamine neurons potentiates the initiation of conditioned approach behavior: parametric and reinforcement-schedule analyses,” Front. Behav. Neurosci., 8, No. 155, 1–13 (2014).
Kim, H. F., Ghazizadeh, A., and Hikosaka, O., “Dopamine neurons encoding long-term memory of object value for habitual behavior,” Cell, 163, 1165–1175 (2015).
Konorski, J., Integrative Activity of the Brain [Russian translation], Mir, Moscow (1970).
Lees, A., Mentored by a Madman: The William Burroughs Experiment [Russian translation], Moscow (2020).
Lisman, J., “Glutamatergic synapses are structurally and biochemically complex because of multiple plasticity processes: long-term potentiation, long-term depression, short-term potentiation and scaling,” Philos. Trans. R. Soc. Lond. B Biol. Sci., 372, 1–11 (2017).
Lisman, J., Grace, A. A., and Duzel, E., “A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP,” Trends Neurosci., 34, No. 10, 536–547 (2011).
Lodge, D. J., “The medial prefrontal and orbitofrontal cortices differentially regulate dopamine system function,” Neuropsychopharmacology, 36, 1227–1236 (2011).
Maiorov, V. I., “Plastic properties of synaptic transmission as a possible mechanism of learning,” in: Nerve Cell Plasticity, Moscow State University Press, Moscow (1977), pp. 70–97.
Maiorov, V. I., “Functions of dopamine in operant conditioned reflexes,” Zh. Vyssh. Nerv. Deyat., 68, No. 4, 404–414 (2018).
Mohebi, A., Pettibone, J. R., Hamid, A. A., et al., “Dissociable dopamine dynamics for learning and motivation,” Nature, 570, No. 7759, 65–70 (2019).
Moreines, J. L., Owrutsky, Z. L., and Grace, A. A., “Involvement of infralimbic prefrontal cortex but not lateral habenula in dopamine attenuation after stress,” Neuropsychopharmacology, 42, 904–913 (2017).
Pavlov, I. P., “The physiological mechanisms of so-called spontaneous movements,” in: 20 Years of Experience of Objective Studies of Higher Nervous Activity (Behavior) in Animals, Nauka, Moscow (1973), pp. 481–485.
Pawlak, V., Wickens, J. R., Kirkwood, A., and Kerr, J. D., “Timing is not everything: neuromodulation opens the STDP gate,” Front. Synaptic Neurosci., 2, Art. 146 (2010).
Rescorla, R. A., “Pavlovian conditioning. It’s not what you think it is,” Am. Psychol., 43, No. 3, 151–160 (1988).
Robinson, S., Sandstrom, S. M., Denenberg, V. H., and Palmiter, R. D., “Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards,” Behav. Neurosci., 119, No. 1, 5–15 (2005).
Roitman, M. F., Stuber, G. D., Phillips, P. E., et al., “Dopamine operates as a subsecond modulator of food seeking,” J. Neurosci., 24, No. 6, 1265–1271 (2004).
Saunders, B. T., Richard, J. M., Margolis, E. B., and Janak, P. H., “Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties,” Nat. Neurosci., 21, 1072–1083 (2018).
Schultz, W., Stauffer, W. R., and Lak, A., “The phasic dopamine signal maturing: from reward via behavioural activation to formal economic utility,” Curr. Opin. Neurobiol., 43, 139–148 (2017).
Sesack, S. R. and Carr, D. B., “Selective prefrontal cortex inputs to dopamine cells: implications for schizophrenia,” Physiol. Behav., 77, 513–517 (2002).
Shahaf, G. and Marom, S., “Learning in networks of cortical neurons,” J. Neurosci., 21, No. 22, 8782–8788 (2001).
Shouval, H. Z., Wang, S. S.-H., and Wittenberg, G. M., “Spike timing dependent plasticity: a consequence of more fundamental learning rules,” Front. Comput. Neurosci., 4, Art. 19, 1–13 (2010).
Sinapayen, L., Masumori, A., and Ikegami, T., “Learning by stimulation avoidance: A principle to control spiking neural networks dynamics,” PLoS One, 12, No. 2, e0170388 (2017).
Stewart, C. V. and Plenz, D., “Inverted-U profile of dopamine-NMDA-mediated spontaneous avalanche recurrence in superficial layers of rat prefrontal cortex,” J. Neurosci., 26, No. 31, 8148–8159 (2006).
Tsai, H. C., Zhang, F., Adamantidis, A., et al., “Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning,” Science, 324, 1080–1084 (2009).
Waelti, P., Dickinson, A., and Schultz, W., “Dopamine responses comply with basic assumptions of formal learning theory,” Nature, 412, No. 6842, 43–48 (2001).
Walker, E. L., “Reinforcement – The one ring,” in: Reinforcement and Behavior, Trapp, J. T. (ed.), Academic Press, New York (1969), pp. 47–62 (cited by [Khekkhauzen, 2003]).
Wang, S., Liao, C., Meng, W., et al., “Activation of D1-like dopamine receptors increases the NMDA-induced gain modulation through a PKA-dependent pathway in the premotor nucleus of adult zebra finches,” Neurosci. Lett., 589, 37–41 (2015).
Watabe-Uchida, M., Eshel, N., and Uchida, N., “Neural circuitry of reward prediction error,” Annu. Rev. Neurosci., 40, 373–394 (2017).
Watabe-Uchida, M., Zhu, L., Ogawa, S. K., et al., “Whole-brain mapping of direct inputs to midbrain dopamine neurons,” Neuron, 74, No. 5, 858–73 (2012).
Wise, R. A. and Koob, G. F., “The development and maintenance of drug addiction,” Neuropsychopharmacology, 39, 254–262 (2014).
Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. S. R., et al., “A critical time window for dopamine actions on the structural plasticity of dendritic spines,” Science, 345, No. 6204, 1616–1620 (2014).
Yin, H. H. and Knowlton, B. J., “The role of the basal ganglia in habit formation,” Nat. Rev. Neurosci., 7, No. 6, 464–476 (2006).
Zellner, M. R. and Ranaldi, R., “How conditioned stimuli acquire the ability to activate VTA dopamine cells: A proposed neurobiological component of reward-related learning,” Neurosci. Biobehav. Rev., 34, No. 5, 769–780 (2010).
Author information
Authors and Affiliations
Corresponding author
Additional information
Translated from Zhurnal Vysshei Nervnoi Deyatel’nosti imeni I. P. Pavlova, Vol. 71, No. 2, pp. 202–212, March–April, 2021.
Rights and permissions
About this article
Cite this article
Maiorov, V.I. A Model of the Neural Mechanism of Instrumentalization of Movements Induced by Stimulation of the Motor Cortex. Neurosci Behav Physi 51, 1124–1131 (2021). https://doi.org/10.1007/s11055-021-01172-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11055-021-01172-5