Abstract
Previous studies have shown that reinforcement-based motor learning requires the brain to process feedback-related information after movement execution. However, whether reinforcement feedback changes how the brain processes motor preparation before movement execution is unclear. By using electroencephalography (EEG), this study investigates whether reinforcement feedback changes cortical preparatory activity to modulate motor learning and memory. Human subjects were divided in three groups [reward, punishment, control] to perform a visuomotor rotation task under different conditions that assess adaptation (learning) and retention (memory) during the task. Reinforcement feedback was provided in the form of points after each trial that signaled monetary gains (reward) or losses (punishment). EEG was utilized to evaluate the amplitude of movement readiness potentials (MRPs) at the beginning of each trial for each group during the adaptation and retention conditions of the task. The results show that punishment feedback significantly decreased MRPs amplitude during both task conditions compared to Reward and Control groups. Moreover, the punishment-related decrease in MRPs amplitude paralleled decreases in motor performance during the retention but not the adaptation condition. No changes in MRPs or motor performance were observed in the Reward group. These results support the idea that reinforcement feedback modulates motor preparation and suggest that changes in cortical preparatory activity contribute to the visuomotor retention deficits observed after punishment feedback.
Similar content being viewed by others
Data availability
Data are available upon reasonable request.
References
Cohen MX (2017) Where does EEG come from and what does it mean? Trends Neurosci 40:208–218
Cohen MX, Wilmes KA, van de Vijver I (2011) Cortical electrophysiological network dynamics of feedback learning. Trends Cogn Sci 15(12):558–566
Dayan E, Cohen LG (2011) Neuroplasticity subserving motor skill learning. Neuron 72(3):443–454
Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134(1):9–21
Ferdinand NK, Opitz B (2014) Different aspects of performance feedback engage different brain areas: disentangling valence and expectancy in feedback processing. Sci Rep 4(1):1–8
Galea JM, Mallia E, Rothwell J, Diedrichsen J (2015) The dissociable effects of punishment and reward on motor learning. Nat Neurosci 18(4):597–602
Hadipour-Niktarash A, Lee CK, Desmond JE, Shadmehr R (2007) Impairment of retention but not acquisition of a visuomotor skill through time-dependent disruption of primary motor cortex. J Neurosci 27(49):13413–13419
Hill CM, Stringer M, Waddell DE, Del Arco A (2020) Punishment feedback impairs memory and changes cortical feedback-related potentials during motor learning. Front Hum Neurosci 14(294):1–14
Hosp JA, Luft AR (2013) Dopaminergic meso-cortical projections to M1: role in motor learning and motor cortex plasticity. Front Neurol 4(145):1–7
Huang VS, Haith A, Mazzoni P, Krakauer JW (2011) Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70(4):787–801
Huang J, Hegele M, Billino J (2018) Motivational modulation of age-related effects on reaching adaptation. Front Psychol 9(11):1–13
Ikeda A, Shibasaki H, Nagamine T, Terada K, Kaji R, Fukuyama H, Kimura J (1994) Dissociation between contingent negative variation and Bereitschaftspotential in a patient with cerebellar efferent lesion. Electroencephalogr clin neurophysiol 90(5):359–364
Izawa J, Shadmehr R (2011) Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput Biol 7(3):1–11
Jo HG, Wittmann M, Hinterberger T, Schmidt S (2014) The readiness potential reflects intentional binding. Front Hum Neurosci 8(6):1–9
Jochumsen M, Rovsing C, Rovsing H, Cremoux S, Signal N, Allen K, Niazi IK (2017) Quantification of movement-related EEG correlates associated with motor training: a study on movement-related cortical potentials and sensorimotor rhythms. Front Hum Neurosci 11(12):1–12
Kappenman ES, Luck SJ (eds) (2011) ERP components: the ups and downs of brainwave recordings. In: The Oxford handbook of event-related potential components, Oxford University Press, New York, pp 3–30. https://doi.org/10.1093/oxfordhb/9780195374148.013.0014
Kawai R, Markman T, Poddar R, Ko R, Fantana AL, Dhawale AK, Ölveczky BP (2015) Motor cortex is required for learning but not for executing a motor skill. Neuron 86(3):800–812
Kirsch W, Hennighausen E, Rösler F (2010) ERP correlates of linear hand movements in a motor reproduction task. Psychophysiology 47(3):486–500
Kitamura JI, Shabasaki H, Terashi A, Tashima K (1999) Cortical potentials preceding voluntary finger movement in patients with focal cerebellar lesion. Clin neurophysiol 110(1):126–132
Krakauer JW, Hadjiosif AM, Xu J, Wong AL, Haith AM (2019) Motor learning. Comprehensive. Physiology 9(2):613–663
Krigolson O, Bell J, Kent CM, Heath M, Holroyd CB (2012) Reduced cortical motor potentials underlie reductions in memory-guided reaching performance. Mot Control 16(3):353–370
Libet B, Gleason CA, Wright EW, Pearl DK (1983) Time of conscious intention to act in relation to onset of cerebral activities (readiness-potential): the unconscious initiation of a freely voluntary act. Brain 106(3):623–642
Libet B, Wright EW, Cleason CA (1982) Readiness-potentials preceding unrestricted “spontaneous” vs. pre-planned voluntary acts. Electroencephalogr Clin Neurophysiol 54(3):322–355
Marinelli L, Crupi D, Di Rocco A, Bove M, Eidelberg D, Abbruzzese G, Ghilardi MF (2009) Learning and consolidation of visuo-motor adaptation in Parkinson’s disease. Parkinsonism Relat Disord 15(1):6–11
Marsh BT, Tarigoppula VSA, Chen C, Francis JT (2015) Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning. J Neurosci 35(19):7374–7387
McNamee D, Wolpert DM (2019) Internal models in biological control. Annu Rev Control Robot Auton Syst 2(5):339–364
Miall RC, Galea J (2015) Cerebellar damage limits reinforcement learning. Brain 139(1):4–7
Monosov IE (2017) Anterior cingulate is a source of valence-specific information about value and uncertainty. Nat Commun 8(1):1–12
Mothes H, Enge S, Strobel A (2016) The interplay between feedback-related negativity and individual differences in altruistic punishment: an EEG study. Cogn Affect Behav Neurosci 16(2):276–288
Nieuwenhuis S, Holroyd CB, Mol N, Coles MG (2004) Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance. Neurosci Biobehav Rev 28(4):441–448
Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9(1):97–113
Palidis DJ, Cashaback JG, Gribble PL (2019) Neural signatures of reward and sensory error feedback processing in motor learning. J Neurophysiol 121(4):1561–1574
Paus T (2001) Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nature rev neurosci 2(6):417–424. https://doi.org/10.1007/s00221-021-06200-x
Paz R, Natan C, Boraud T, Bergman H, Vaadia E (2005) Emerging patterns of neuronal responses in supplementary and primary motor areas during sensorimotor adaptation. J Neurosci 25(47):10941–10951
Pinheiro AP, Schwartze M, Gutierrez F, Kotz SA (2020) Real and imagined sensory feedback have comparable effects on action anticipation. Cortex 130:290–301
Quattrocchi G, Monaco J, Ho A, Irmen F, Strube W, Ruge D, Bestmann S, Galea JM (2018) Pharmacological dopamine manipulation does not alter reward-based improvements in memory retention during a visuomotor adaptation task. Eneuro 5(3):1–12
Ramakrishnan A, Byun YW, Rand K, Pedersen CE, Lebedev MA, Nicolelis MA (2017) Cortical neurons multiplex reward-related signals along with sensory and motor information. Proc Natl Acad Sci 114(24):4841–4850
Ramkumar P, Dekleva B, Cooler S, Miller L, Kording K (2016) Premotor and motor cortices encode reward. PLoS ONE 11(8):1–13
Reznik D, Simon S, Mukamel R (2018) Predicted sensory consequences of voluntary actions modulate amplitude of preceding readiness potentials. Neuropsychologia 119:302–307
Shadmehr R, Krakauer JW (2008) A computational neuroanatomy for motor control. Exp Brain Res 185(3):359–381
Sheahan HR, Franklin DW, Wolpert DM (2016) Motor planning, not execution, separates motor memories. Neuron 92(4):773–779
Shibasaki H, Hallett M (2006) What is the Bereitschafts potential? Clin Neurophysiol 117(11):2341–2356
Smith AL, Staines WR (2006) Cortical adaptations and motor performance improvements associated with short-term bimanual training. Brain Res 1071(1):165–174
Smith AL, Staines WR (2010) Cortical and behavioral adaptations in response to short-term inphase versus antiphase bimanual movement training. Exp Brain Res 205(4):465–477
Smith AL, Staines WR (2012) Externally cued inphase bimanual training enhances preparatory premotor activity. Clin Neurophysiol 123(9):1846–1857
Song Y, Smiley-Oyen AL (2017) Probability differently modulating the effects of reward and punishment on visuomotor adaptation. Exp Brain Res 235(12):3605–3618
Song Y, Lu S, Smiley-Oyen AL (2019) Differential motor learning via reward and punishment. Q J Exp Psychol 73(2):249–259
Spampinato DA, Satar Z, Rothwell JC (2019) Combining reward and M1 transcranial direct current stimulation enhances the retention of newly learnt sensorimotor mappings. Brain Stimul 12(5):1205–1212
Steel A, Silson EH, Stagg CJ, Baker CI (2016) The impact of reward and punishment on skill learning depends on task demands. Sci Rep 6(10):1–10
Steel A, Baker CI, Stagg CJ (2020) Intention to learn modulates the impact of reward and punishment on sequence learning. Sci Rep 10(1):1–13
Tanaka S, Honda M, Hanakawa T, Cohen LG (2010) Differential contribution of the supplementary motor area to stabilization of a procedural motor skill acquired through different practice schedules. Cerebral cortex. 20(9):2114–2121
Tanji J, Shima K (1994) Role for supplementary motor area cells in planning several movements ahead. Nature 371(6496):413–416
Therrien AS, Wolpert DM, Bastian AJ (2016) Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise. Brain 139(1):101–114
Torrecillos F, Albouy P, Brochier T, Malfait N (2014) Does the processing of sensory and reward-prediction errors involve common neural resources? Evidence from a frontocentral negative potential modulated by movement execution errors. J Neurosci 34(14):4845–4856
Traver E, Friedemann M, Haggard P (2021) The Readiness Potential reflects planning-based expectation, not uncertainty, in the timing of action. Cogn Neurosci 12(1):14–27
Vercillo T, O’Neil S, Jiang F (2018) Action–effect contingency modulates the readiness potential. Neuroimage 183(12):273–279
Vyas S, O’Shea DJ, Ryu SI, Shenoy KV (2020) Causal role of motor preparation during error-driven learning. Neuron 2(4):329–339
Wang C, Ulbert I, Schomer DL, Marinkovic K, Halgren E (2005) Responses of human anterior cingulate cortex microdomains to error detection, conflict monitoring, stimulus-response mapping, familiarity, and orienting. J Neurosci 25(3):604–613
Wright DJ, Holmes PS, Di Russo F, Loporto M, Smith D (2012a) Differences in cortical activity related to motor planning between experienced guitarists and nonmusicians during guitar playing. Hum Mov Sci 31(3):567–577
Wright DJ, Holmes P, Di Russo F, Loporto M, Smith D (2012b) Reduced motor cortex activity during movement preparation following a period of motor skill practice. PLoS ONE 13(1):1–17
Funding
This research was supported by the School of Applied Science at The University of Mississippi.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethics approval
All procedures of this study was approved by the University of Mississippi Institutional Review Board.
Consent to participate
Informed consent was obtained from all individual participants included in the study.
Additional information
Communicated by Melvyn A. Goodale.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hill, C.M., Waddell, D.E. & Del Arco, A. Cortical preparatory activity during motor learning reflects visuomotor retention deficits after punishment feedback. Exp Brain Res 239, 3243–3254 (2021). https://doi.org/10.1007/s00221-021-06200-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-021-06200-x