Abstract
Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.
Similar content being viewed by others
Main
The ability to learn from past outcomes, often referred to as reinforcement learning, is fundamental for complex biological systems1. Reinforcement learning has been classically studied in the context of decision-making, when agents have to decide among a discrete number of potential options2. There is increasing recognition that reinforcement learning processes are also at play in other contexts, including during practice of a new motor skill3,4,5. For instance, the addition of reinforcement feedback during motor training can improve motor learning, presumably by boosting the retention of newly acquired motor memories6,7. Interestingly, reinforcement feedback also appears to be relevant for the rehabilitation of patients suffering from motor impairments8,9,10. Yet, despite these promising results, there is currently a limited understanding of the brain mechanisms that are critical to implement this behaviour.
A prominent hypothesis in the field is that the striatum, a structure that is particularly active during both reinforcement11 and motor learning12, may be causally involved in the beneficial effects of reinforcement on motor learning. The striatum shares dense connections with dopaminergic structures of the midbrain as well as with prefrontal and motor cortical regions13 and is therefore well positioned to mediate reinforcement motor learning14,15,16. This idea is supported by neuroimaging studies showing reward-related activation of the striatum during motor learning17,18. More specifically, within the striatum, oscillatory activity in specific frequency bands is suggested to be involved in aspects of reinforcement processing. Previous rodent studies have shown that striatal high gamma oscillations (~80 Hz) transiently increase following reward delivery19,20,21,22,23, but not when reward is withheld19. Hence, dynamic changes of high gamma activity in the striatum19,24,25 and in other parts of the basal ganglia26,27 may encode the outcome of previous movements (that is, success or failure) and support learning. Consistent with a role of such oscillatory activity in reinforcement learning, high gamma activity in the striatum shows coherence with frontal cortex oscillations and is upregulated by dopaminergic agonists19. This body of work thus suggests that reinforcement-related modulation of striatal oscillatory activity, especially in the gamma range, may be crucial for reinforcement learning of motor skills. Conversely, striatal beta oscillations (~20 Hz) have been largely associated with sensorimotor functions28. For instance, beta oscillations in the striatum are exacerbated in Parkinson’s disease and associated with the severity of motor symptoms29,30,31. Consistently, excessive beta connectivity is reduced by anti-Parkinsonian treatment in proportion to the related motor improvement32. Taken together, these elements suggest that striatal high gamma and beta activity may have different functional roles preferentially associated with reinforcement and sensorimotor functions, respectively.
The studies mentioned above provide associative evidence linking the presence of reinforcement with changes of neural activity in the striatum determined through neuroimaging17,18, but they do not allow us to draw conclusions regarding its causal role in reinforcement motor learning in humans. The only causal evidence available to date comes from animal work showing modulation of reinforcement-based decision-making with striatal stimulation33,34. A reason for the current absence of investigations of the causal role of the striatum in human behaviour is related to its deep localization in the brain. Current non-invasive brain stimulation techniques, such as transcranial magnetic stimulation and classical transcranial electric stimulation, do not allow the selective targeting of deep brain regions, because these techniques exhibit a steep depth–focality trade-off35,36. Studies of patients with striatal lesions37,38 or invasive deep brain stimulation of connected nuclei39,40 have provided insights into the role of the basal ganglia in reinforcement learning. However, their conclusions are partially limited by the fact that the studied patients also exhibit altered network properties resulting from the underlying pathology (for example, neurodegeneration or lesions) or from the respective compensatory mechanisms. Here we address these challenges by exploiting transcranial temporal interference stimulation (tTIS), a recently introduced non-invasive electric brain stimulation approach allowing us to target deep brain regions in a frequency-specific and focal manner in the physiological state41,42.
The concept of tTIS was initially proposed and validated on the hippocampus of rodents41 and was then further tested through computational modelling43,44,45,46,47 and in first applications on cortical areas in humans48,49. tTIS requires two pairs of electrodes to be placed on the head, each pair delivering a high-frequency alternating current. One key element is that this frequency has to be high enough (that is, in the kHz range) to avoid direct neuronal entrainment, owing to the low-pass-filtering properties of neuronal membranes50. The second key element is the application of a small difference of frequency between the two alternating currents. The superposition of the electric fields creates an envelope oscillating at this low-frequency difference, which can be steered towards individual deep brain structures (for example, by optimizing electrodes’ placement) and is in a range able to influence neuronal activity41,51,52,53. An interesting feature of tTIS is the ability to stimulate at a particular frequency of interest to preferentially interact with specific neuronal processes41,42. Despite these exciting opportunities, current evidence for tTIS-related neuromodulation of deep brain structures, such as the striatum, remains sparse in humans52,53.
Here we combine tTIS with electric field modelling for target localization, behavioural data and functional magnetic resonance imaging (fMRI) to evaluate the causal role of specific patterns of striatal activity in reinforcement learning of motor skills. On the basis of the studies mentioned above, we hypothesized that striatal tTIS at high gamma frequency (tTIS80Hz) would disturb the fine-tuning of high gamma oscillatory activity in the striatum and thereby would perturb reinforcement motor learning, in contrast to beta (tTIS20Hz) or sham (tTISSham) stimulation. More specifically, we reasoned that applying a constant high gamma rhythm in the striatum would disturb the temporally precise and reinforcement-specific modulation of high gamma activity. Moreover, given that the stimulation protocol was not individualized to endogenous high gamma activity and not synchronized to ongoing activity in other hubs of the reinforcement learning network (for example, the frontal cortex), we anticipated disruptive rather than beneficial effects of tTIS80Hz.
In line with our prediction, we report that tTIS80Hz disrupted motor learning compared with the controls, but only in the presence of reinforcement. To evaluate the potential neural correlates of these behavioural effects, we measured blood-oxygen-level-dependent (BOLD) activity in the striatum and effective connectivity between the striatum and frontal cortical areas involved in reinforcement motor learning. We found that the disruptive effect of tTIS80Hz on reinforcement learning was associated with a specific modulation of BOLD activity in the putamen and caudate, but not in the cortex, supporting the ability of tTIS to selectively modulate striatal activity without affecting overlying cortical areas. Moreover, tTIS80Hz also increased the neuromodulatory influence of the striatum on frontal cortical areas involved in reinforcement motor learning. Overall, the present study shows that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning.
Results
A total of 24 healthy participants (15 women, 25.3 ± 0.1 years old (mean ± s.e.)) performed a force-tracking task in the MRI scanner with concurrent tTIS of the striatum. The task required the participants to modulate the force applied on a hand-grip force sensor to track a moving target with a cursor with the right, dominant hand54,55 (Fig. 1a). In each block, the participants had to learn a new pattern of motion of the target (Supplementary Fig. 1a and Methods). In ReinfON blocks, the participants were provided with online reinforcement feedback during training, giving them real-time information about success or failure throughout the trial, indicated as a green or red target, respectively (please see Supplementary Video 1 for the task). The reinforcement feedback was delivered according to a closed-loop schedule8, in which the success criterion to consider a force sample as successful was updated on the basis of the median performance over the four previous trials (see Methods for more details). In ReinfOFF blocks, the participants practised with visually matched random feedback (cyan/magenta). Importantly, in both types of blocks, training was performed with partial visual feedback of the cursor, a condition that has been shown to maximize reinforcement effects in various motor learning paradigms5,56,57,58 and that yielded significant effects of reinforcement on motor learning, as also demonstrated in an additional behavioural study testing another group of healthy participants on the same task (n = 24; Supplementary Fig. 1b–e). Before and after training, the participants performed pre- and post-training assessments with full visual feedback, no reinforcement and no tTIS, allowing us to evaluate motor learning. To assess the effect of tTIS on reinforcement-related benefits in motor learning and the associated neural changes, the participants performed six blocks of 36 trials in the MRI machine, with concurrent tTIS during training, delivered with a Δf of 20 Hz (tTIS20Hz) or 80 Hz (tTIS80Hz) or as a sham (tTISSham; 3 tTISTYPE × 2 ReinfTYPE conditions; Fig. 1b,c). The order of the conditions was balanced among the 24 participants, ensuring that any potential carry-over effect would have the same impact on each experimental condition. To determine the best electrode montage to stimulate the human striatum (putamen, caudate and nucleus accumbens (NAc) bilaterally), computational modelling with a realistic head model was conducted with Sim4Life59 (Methods). The selected montage (F3–F4 and TP7–TP8) generated a theoretical temporal interference electric field that was ~30–40% stronger in the striatum than in the overlying cortex, reaching magnitudes of 0.5 to 0.6 V m−1 (Fig. 1d,e).
tTIS80Hz disrupts reinforcement learning of motor skills
Task performance was evaluated by means of the Error, which was defined as the absolute difference between the applied and target force averaged across samples for each trial, as done previously5,55,58 (Fig. 2a). Across conditions, the post-training Error was lower than the pre-training Error (single-sample two-sided t-test on the normalized post-training data: t24 = −2.69; P = 0.013; Cohen’s d = −0.53; 95% confidence interval (CI), (−0.99, −0.09)), indicating significant motor learning during the task (Fig. 2b). Such improvement was greater when participants had trained with reinforcement (ReinfTYPE effect in the linear mixed model (LMM): F1,1062.2 = 5.17; P = 0.023; partial eta-squared (ηp2), 0.005; 95% CI, (0.00, 0.02)), confirming the beneficial effect of reinforcement on motor learning7,57. Crucially, though, this effect depended on the type of stimulation applied during training (ReinfTYPE × tTISTYPE interaction: F2,1063.5 = 2.11; P = 0.034; ηp2 = 0.006; 95% CI, (0.00, 0.02); Fig. 2c). While reinforcement significantly improved learning when training was performed with tTISSham (two-sided Tukey-corrected pairwise comparison: P = 0.036; d = −0.22; 95% CI, (−0.46, 0.01)) and tTIS20Hz (P = 0.0089; d = −0.27; 95% CI, (−0.51, −0.04)), this was not the case with tTIS80Hz (P = 0.43; d = 0.083; 95% CI, (−0.14, 0.31)). Consistently, direct between-condition comparisons showed that in the ReinfON condition, learning was reduced with tTIS80Hz compared with tTIS20Hz (P = 0.039; d = 0.26; 95% CI, (0.02, 0.49)) and tTISSham (P < 0.001; d = 0.45; 95% CI, (0.19, 0.72)), while there was no evidence for a difference between tTIS20Hz and tTISSham (P = 0.15; d = 0.20; 95% CI, (−0.04, 0.43)). This disruption of motor learning with tTIS80Hz was not observed in the absence of reinforcement (tTIS80Hz versus tTIS20Hz: P = 0.59; d = −0.10; 95% CI, (−0.33, 0.12); tTIS80Hz versus tTISSham: P = 0.34; d = 0.15; 95% CI, (−0.08, 0.38)). These results point to the fact that high gamma striatal tTIS specifically disrupts the benefits of reinforcement for motor learning and not motor learning in general.
Although training with tTIS20Hz did not alter the benefits of reinforcement for motor learning, we found that learning without reinforcement was significantly impaired in this condition (tTIS20Hz versus tTISSham: P = 0.046; d = 0.25; 95% CI, (0.01, 0.49); Fig. 2c). This suggests that tTIS20Hz may disrupt a qualitatively different mechanism involved in motor learning from sensory feedback60, in line with the role of striatal beta oscillations in sensorimotor function28.
Next, we evaluated the effect of tTIS on motor performance during training itself. As shown in Fig. 2a, the Error was generally higher during training than in test trials due to the presence of visual uncertainty during this phase. The extent of this disruption was reduced in the presence of reinforcement (ReinfTYPE: F1,3262.4 = 339.89; P < 0.001; ηp2 = 0.09; 95% CI, (0.08, 0.11)), demonstrating the ability of participants to exploit real-time reinforcement information to improve tracking (Fig. 2d). Notably, this effect was not modulated by tTISTYPE (ReinfTYPE × tTISTYPE: F2,3265.8 = 0.91; P = 0.40; ηp2 = 6 × 10−4), indicating that tTIS did not directly influence reinforcement gains during tracking. However, striatal stimulation did impact general tracking performance independently of reinforcement, as indicated by a significant tTISTYPE effect (tTISTYPE: F2,3262.4 = 42.85; P < 0.001; ηp2 = 0.03; 95% CI, (0.02, 0.04)). This effect was due to an increase in the Error when tTIS20Hz was applied (P < 0.001; d = 0.28; 95% CI, (0.16, 0.39) when compared with tTISSham), which was even larger during tTIS80Hz (P < 0.001; d = 0.38; 95% CI, (0.25, 0.52) and P = 0.031; d = 0.11; 95% CI, (0.02, 0.20) when compared with tTISSham and tTIS20Hz, respectively). An additional analysis showed that the detrimental effect of tTIS on motor performance was actually due to an impaired ability to improve performance during training (LMM with continuous fixed effect Trial: tTISTYPE × Trial interaction: F2,3399 = 4.46; P = 0.012; ηp2 = 0.003; 95% CI, (0.00, 0.01); post hoc tests: tTISSham versus tTIS20Hz: P = 0.013; d = −0.02; 95% CI, (−0.03, 0.00); tTISSham versus tTIS80Hz: P = 0.068; d = −0.01; 95% CI, (−0.03, 0.00); tTIS20Hz versus tTIS80Hz: P = 0.81; d = 0.004; 95% CI, (−0.01, 0.02); Supplementary Fig. 1f). However, again, this effect did not depend on the presence of reinforcement (ReinfTYPE × tTISTYPE × Trial: F2,3399 = 0.51; P = 0.60; ηp2 = 3 × 10−4). We also found that the detrimental effect of striatal tTIS did not depend on the availability of visual information on the cursor, but rather that tTIS had a general effect on motor performance irrespective of visual and reinforcement feedback (Supplementary Information). This analysis also confirmed that reinforcement gains in motor performance were stronger when visual information was not available (Supplementary Fig. 1g), in line with the behavioural data mentioned above (Supplementary Fig. 1b) and previous studies56,61. Overall, these results suggest that striatal tTIS altered motor performance in a frequency-dependent manner but did not influence the ability to rapidly adjust motor commands on the basis of reinforcement feedback during training. Hence, tTIS80Hz may not disrupt real-time processing of reinforcement feedback but may instead impair the beneficial effect of reinforcements on the retention of motor memories6,7.
To further understand this dissociation, we ran additional analyses exploring the relationship between reinforcement gains in the training (performed with partial visual feedback and ReinfON or ReinfOFF) and post-training phases (performed with full visual feedback and no reinforcement). We found consistent positive associations between individual reinforcement gains at the end of training (T6) and at the beginning of post-training (Post1) in the tTISSham (robust linear regression: R2 = 0.45, P < 0.001) and tTIS20Hz (R2 = 0.36, P = 0.003) conditions and in the additional behavioural dataset (R2 = 0.31, P = 0.009, Supplementary Fig. 2). This association was abolished specifically in the tTIS80Hz condition (R2 = 0.028, P = 0.39): participants who benefited from reinforcement during training did not exhibit gains in learning at post-training (see Supplementary Information for more details on this analysis). This suggests that the disruption of reinforcement motor learning with tTIS80Hz did not concern all participants (in this case, we would still have found a correlation but an upward shift in the intercept) but primarily affected participants who actually benefited from reinforcement during training, further supporting the idea of a specific disruption of reinforcement motor learning with tTIS80Hz.
These effects could not be explained by potential differences in initial performance between conditions (ReinfTYPE × tTISTYPE: F2,519.99 = 1.08; P = 0.34; ηp2 = 0.004; 95% CI, (0.00, 0.02)), by changes in the flashing properties of the reinforcement feedback (that is, the frequency of colour change during tracking; ReinfTYPE × tTISTYPE: F2,3283 = 0.19; P = 0.82; ηp2 = 1 × 10−4) or by differences in success rate in the ReinfON blocks (that is, the proportion of success feedback during tracking; tTISTYPE: F2,1702 = 0.17; P = 0.84; ηp2 = 2 × 10−4). There was also no evidence that the ReinfTYPE × tTISTYPE effect on learning was influenced by the order of the reinforcement conditions (analysis on sub-groups based on whether participants experienced ReinfON or ReinfOFF first; no ReinfTYPE × tTISTYPE × GroupTYPE interaction: F2,1105.06 = 1.75; P = 0.17; ηp2 = 0.003; 95% CI, (0.00, 0.01); see Supplementary Information for more details on these analyses).
Finally, we confirmed that these results were not a consequence of inefficient blinding. During debriefing after the experiment, only 6/24 participants were able to successfully identify the order of the stimulation applied (for example, real–real–placebo; chance level, 4/24; Fisher exact test on proportions, P = 0.74). Consistently, the magnitude (Supplementary Fig. 3a) and type (Supplementary Fig. 3b) of tTIS-evoked sensations evaluated before the experiment were qualitatively similar across conditions, and tTIS was generally well tolerated in all participants (no adverse events reported). This suggests that blinding was successful and is unlikely to explain our findings. More generally, this indicates that tTIS evokes very limited sensations (for example, only 2/24 and 1/24 participants rated sensations evoked at 2 mA as “strong” for tTIS20Hz and tTIS80Hz, respectively; Supplementary Fig. 3a) that are compatible with efficient blinding.
Behavioural effect of tTIS80Hz is linked to striatal modulation
As mentioned above, task-based fMRI was acquired during training with concomitant tTIS. This allowed us to evaluate the neural effects of tTIS and their potential relationship to the behavioural effects reported above. As a first qualitative evaluation of the data, we performed a whole-brain analysis in the tTISSham condition to assess the network activated during reinforcement motor learning (ReinfON condition). Consistent with previous neuroimaging studies employing similar tasks62,63, we found prominent BOLD activations in a motor network including the putamen, thalamus, cerebellum and sensorimotor cortex, particularly in the left hemisphere, contralateral to the trained hand (Supplementary Fig. 4 and Supplementary Table 1). However, contrasting ReinfON and ReinfOFF conditions did not reveal any significant cluster at the whole-brain level. This first analysis thus did not reveal any region specifically activated in the presence of reinforcement, but rather confirms the involvement of a motor network engaged in this type of task irrespective of the reinforcement feedback.
As a second step, we evaluated the effect of tTIS on striatal activity, as a function of the type of reinforcement feedback and focusing on the same regions of interest (ROIs) that were used to optimize tTIS exposure in the modelling. We extracted averaged BOLD activity within the bilateral putamen, caudate and NAc based on the Brainnetome Atlas (BNA)64 in the different experimental conditions and considered these six striatal ROIs (ROISTR) as fixed effects in the LMM. This model revealed a significant enhancement of striatal activity with ReinfON with respect to ReinfOFF (F1,800.01 = 13.23; P < 0.001; ηp2 = 0.02; 95% CI, (0.00, 0.04)), consistent with previous literature11, but no tTISTYPE effect (F2,800.01 = 0.46; P = 0.63; ηp2 = 0.001; 95% CI, (0.00, 0.01)) and no interaction (all P > 0.65; Fig. 3a). Despite the absence of effects of tTIS on averaged striatal activity, we then asked whether the behavioural effects of tTIS80Hz on reinforcement motor learning (that is, tTIS80Hz versus tTIS20Hz and tTISSham with ReinfON) could be linked to the modulation of activity in core brain regions. To do so, we ran a whole-brain analysis focusing on the main behavioural effects mentioned above. The results revealed that the effect of tTIS80Hz (with respect to tTIS20Hz) on motor learning in the ReinfON condition was specifically related to the modulation of activity in two clusters encompassing the left putamen and bilateral caudate (Fig. 3b and Supplementary Table 2). The presence of the high-frequency carrier (kHz) in both stimulation conditions rules out the possibility that the correlation was due to putative neuromodulatory effects of high-frequency stimulation. No significant clusters were found for the tTIS80Hz–tTISSham contrast or for the control tTIS20Hz–tTISSham contrast, indicating that the reported correlation is not due to a general link between striatal activity and reinforcement motor learning. Overall, these results provide evidence that the detrimental effect of tTIS80Hz on reinforcement learning of motor skills is related to the modulation of neural activity specifically in the striatum.
tTIS80Hz enhances striatum-to-frontal-cortex connectivity
Interactions between the striatum and frontal cortex are crucial for a variety of behaviours, including motor and reinforcement learning13. In particular, reinforcement motor learning requires the use of information about task success to guide future motor commands5, a process in which the striatum may play an integrative role at the interface between fronto-striatal loops involved in reward processing and motor control13,65. In a subsequent analysis, we asked whether striatal tTIS modulates striatum-to-frontal-cortex communication during reinforcement motor learning. More specifically, we computed effective connectivity (using the generalized psychophysiological interactions (gPPI) method66) between striatal and frontal regions classically associated with motor and reward-related functions, and thought to be involved in reinforcement motor learning67,68. For the motor network, we evaluated effective connectivity between motor parts of the striatum (that is, dorso-lateral putamen and dorsal caudate) and two regions strongly implicated in motor learning: the medial part of the supplementary motor area (SMA) and the part of the primary motor cortex (M1) associated with upper limb functions (Fig. 4a). For the reward network, we assessed connectivity between parts of the striatum classically associated with limbic functions (that is, the NAc, the ventro-medial putamen and two frontal areas involved in reward processing: the anterior cingulate cortex (ACC) and the ventro-medial prefrontal cortex (vmPFC); Fig. 4b)11. The LMM run with the fixed effects ReinfTYPE, tTISTYPE and NetworkTYPE showed a significant effect of tTISTYPE (F2,2264.0 = 5.42; P = 0.005; ηp2 = 0.005; 95% CI, (0.00, 0.01)) that was due to higher connectivity in the tTIS80Hz condition than in tTISSham (Tukey-corrected P = 0.004; d = 0.16; 95% CI, (0.05, 0.28)). There was no significant difference in connectivity between tTIS80Hz and tTIS20Hz (P = 0.069; d = 0.11; 95% CI, (0.00, 0.22)) or between tTIS20Hz and tTISSham (P = 0.58; d = 0.051; 95% CI, (−0.05, 0.16)). Hence, tTIS80Hz, but not tTIS20Hz, enhanced effective connectivity between the striatum and frontal cortex during motor training. This increase in effective connectivity with tTIS80Hz actually led to a connectivity closer to the resting state (values closer to 0; Methods). Put differently, while the task induced a reduction in effective connectivity between striatum and frontal cortex, tTIS80Hz disrupted this modulation by bringing connectivity back to the resting state.
The LMM did not reveal any effect of ReinfTYPE (F1,2264.0 = 0.010; P = 0.92; ηp2 = 5 × 10−6), NetworkTYPE (F1,2264.0 = 3.16; P = 0.076; ηp2 = 0.001; 95% CI, (0.00, 0.01)) or a double interaction (ReinfTYPE × NetworkTYPE: F1,2264.0 = 3.52; P = 0.061; ηp2 = 0.002; 95% CI, (0.00, 0.01)). Yet, we did find a significant ReinfTYPE × tTISTYPE × NetworkTYPE interaction (F2,2264.0 = 4.87; P = 0.008; ηp2 = 0.004; 95% CI, (0.00, 0.01)). This triple interaction was related to the fact that tTIS80Hz increased connectivity in the ReinfON condition in the motor network (ReinfON versus ReinfOFF: P = 0.001; d = 0.33; 95% CI, (0.11, 0.55); Fig. 4a), while this effect was not observed in the reward network (P = 0.063; d = −0.19; 95% CI, (−0.40, 0.02); Fig. 4b). There was no evidence for such an increase in either of the two networks when either tTISSham or tTIS20Hz was applied (all P > 0.40, all d [−0.09, −0.02]). Moreover, in the motor network, connectivity in the ReinfON condition was higher with tTIS80Hz than with tTISSham (P < 0.001; d = 0.42; 95% CI, (0.19, 0.65)). This effect did not reach significance when contrasting tTIS80Hz with tTIS20Hz (P = 0.059; d = 0.23; 95% CI, (0.02, 0.44); Fig. 4a). These data suggest that tTIS80Hz enhanced the neuromodulatory influence of the striatum on motor cortex during task performance, but only in the presence of reinforcement. In the reward network, post hoc tests revealed that connectivity in the ReinfOFF condition was significantly higher with tTIS80Hz than with tTIS20Hz (P = 0.045; d = 0.24; 95% CI, (0.03, 0.46); Fig. 4b), in line with the general effect of tTISTYPE on connectivity reported above. This pattern of results suggests that the increase of connectivity from striatum to frontal cortex observed with tTIS80Hz depends on the presence of reinforcement, in particular in the motor network. This reinforcement-dependent increase of connectivity may reflect the preferential effect of tTIS80Hz on striatal gamma oscillations69 in a situation where these oscillations are already boosted by the presence of reinforcement19 (Discussion).
In a subsequent analysis, we verified that these results did not depend on the specific frontal ROIs considered in the analysis (ROITYPE: M1 and SMA in the motor network and ACC and vmPFC in the reward network). Importantly, we did not find a tTISTYPE × ReinfTYPE × ROITYPE interaction in the motor network (F2,1112 = 0.83; P = 0.44; ηp2 = 0.001; 95% CI, (0.00, 0.01)) or in the reward network (F2,1112 = 0.61; P = 0.54; ηp2 = 0.001; 95% CI, (0.00, 0.01)), suggesting that the main connectivity results were consistent within a network and were not influenced by the specific frontal ROI included in the analysis (see Supplementary Information for more details on this analysis). As an additional control, we verified that the effects of tTISTYPE on connectivity could not be observed in a control network associated with language (as defined by ref. 70), which was unlikely to be involved in the present task and did not include the striatum (Methods). As expected, there was no evidence for a modulation of effective connectivity within the language network by ReinfTYPE (F1,547 = 0.81; P = 0.37; ηp2 = 0.001; 95% CI, (0.00, 0.01)) or by tTISTYPE (F2,547 = 0.58; P = 0.56; ηp2 = 0.002; 95% CI, (0.00, 0.01)) or by ReinfTYPE × tTISTYPE (F2,547 = 0.45; P = 0.64; ηp2 = 0.002; 95% CI, (0.00, 0.01)). Hence, tTIS-related and reinforcement-related changes in connectivity were consistent within the considered fronto-striatal networks and not observed in a control network unrelated to the task.
Contrary to the BOLD results presented above, we did not find any correlations between the effects of tTIS80Hz on connectivity and motor learning, in either the motor network (robust linear regression: tTIS80Hz–tTISSham: R2 = 0.019, P = 0.48; tTIS80Hz–tTIS20Hz: R2 = 0.034, P = 0.54) or the reward network (tTIS80Hz–tTISSham: R2 = 0.037, P = 0.46; tTIS80Hz–tTIS20Hz: R2 < 0.001, P = 0.75), suggesting some degree of independence between the effect of tTIS80Hz on reinforcement motor learning and that on effective connectivity.
Overall, these results highlight the ability of tTIS80Hz, but not tTIS20Hz, to modulate striatum-to-frontal-cortex connectivity, depending on the presence of reinforcement. However, the absence of a correlation with behaviour suggests that this effect may not be directly associated with the detrimental effect of tTIS80Hz on reinforcement motor learning or that tTIS80Hz-related changes in striato-frontal communication were linked to other aspects of reinforcement learning not captured by our task.
Neural effects of tTIS80Hz depend on impulsivity
Determining individual factors that shape responsiveness to non-invasive brain stimulation approaches is a crucial step towards better understanding the mechanisms of action as well as envisioning the stratification of patients in future clinical interventions71. A potential factor that could explain inter-individual differences in responsiveness to tTIS80Hz is the level of impulsivity. Impulsivity has been associated with changes of gamma oscillatory activity in the striatum of rats72 and with the activity of fast-spiking interneurons in the striatum73,74, a neuronal population that is strongly entrained to gamma rhythms19,21 and may therefore be particularly sensitive to tTIS80Hz. In a subsequent exploratory analysis, we asked whether the neural effects of tTIS80Hz were associated with impulsivity levels, as evaluated by a well-established independent delay-discounting questionnaire performed at the beginning of the experiment75,76. A whole-brain analysis revealed that impulsivity was associated with the effect of tTIS80Hz on BOLD activity (with respect to tTIS20Hz) specifically in the left caudate nucleus (Supplementary Fig. 5a,b and Supplementary Table 3). Moreover, the effect of tTIS80Hz on striatum-to-motor-cortex connectivity reported above was negatively correlated with impulsivity when contrasting tTIS80Hz with both tTISSham (Supplementary Fig. 5c, left) and tTIS20Hz (Supplementary Fig. 5c, middle). Such correlations were absent when contrasting tTIS20Hz with tTISSham (Supplementary Fig. 5c, right), as well as when considering the same contrasts in the reward instead of the motor network (see Supplementary Information for more details). Taken together, these results suggest that inter-individual variability in impulsivity might influence the neural responses to striatal tTIS80Hz.
Discussion
In this study, we combined striatal tTIS with electric field modelling, behavioural and fMRI analyses to evaluate the causal role of the striatum in reinforcement learning of motor skills in healthy humans. tTIS80Hz, but not tTIS20Hz, disrupted the ability to learn from reinforcement feedback. This behavioural effect was associated with modulation of neural activity specifically in the striatum. We also show that tTIS80Hz, but not tTIS20Hz, increased the neuromodulatory influence of the striatum on connected frontal cortical areas involved in reinforcement motor learning. Finally, inter-individual variability in the neural effects of tTIS80Hz could be partially explained by impulsivity, suggesting that this trait may constitute a determinant of responsiveness to high gamma striatal tTIS. Overall, the present study shows that striatal tTIS can non-invasively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.
We investigated the causal role of the human striatum in reinforcement learning of motor skills in healthy humans, a question that cannot be addressed with conventional non-invasive brain stimulation techniques. In particular, by stimulating at different frequencies, we aimed to dissociate striatal mechanisms involved in reinforcement and sensorimotor learning. In line with our main hypothesis, we found that striatal tTIS80Hz altered reinforcement learning of a motor skill. Such disruption was frequency- and reinforcement-specific: learning was not altered with striatal tTIS20Hz in the presence of reinforcement, or when striatal tTIS80Hz was delivered in the absence of reinforcement. The rationale to stimulate at high gamma frequency was based on previous work showing reinforcement-related modulation of gamma oscillations in the striatum19,20,21,24,26,72,77 and in the frontal cortex77,78,79,80. Several neuronal mechanisms may contribute to the detrimental effect of tTIS80Hz on reinforcement motor learning. First, as tTIS80Hz consisted of a constant high gamma oscillating field applied on the striatum, it may have perturbed the encoding of reinforcement information into high gamma oscillations19,20,21,25,26,27, preventing participants from learning the motor skill on the basis of different outcomes. Put differently, tTIS80Hz may specifically saturate high gamma activity in the striatum, preventing reinforcement-related modulations81. Moreover, because reinforcement motor learning probably engages synchronized activity in a network of regions including fronto-striatal loops, neuromodulation of a single node of the circuit may alter the synchronization of activity in the network81 and the temporal coordination with interacting rhythms25. Finally, because we did not have access to electrophysiological recordings of oscillatory activity in the striatum, the applied stimulation was not personalized, as it did not take into account the individual high gamma frequency peak associated with reward processing and the potential heterogeneity of gamma activity within the striatum24. Hence, tTIS80Hz may have resulted in a frequency mismatch between the endogenous high gamma activity and the externally imposed rhythm, which could paradoxically result in a reduction of neuronal entrainment, in particular when the frequency mismatch is relatively low82. Importantly, in contrast to striatal tTIS80Hz, we found that tTIS20Hz reduced learning, but only in the absence of reinforcement. This result fits well with the literature linking striatal beta oscillations to sensorimotor functions28,29,31,83,84,85. Taken together, an interpretation of these results is that different oscillations in the striatum support qualitatively distinct motor learning mechanisms, with beta activity contributing mostly to sensory-based learning and high gamma activity being particularly important for reinforcement learning. This being said, it is important to note that because we do not have concurrent electrophysiological recordings in the striatum, we cannot be sure whether the effects of tTIS20Hz and tTIS80Hz were related to frequency-specific interactions with beta or high gamma rhythms, respectively, or rather resulted from different broadband responses when stimulating at these frequencies. Yet, these results still suggest that sensory- and reinforcement-based motor learning rely on partially different neural mechanisms, in line with previous literature8,9,60,68,86,87.
Striatal tTIS also impaired tracking performance during training, irrespective of the presence of reinforcement. This frequency-dependent reduction of motor performance may be due to altered neuronal processing in the sensorimotor striatum that may lead to less fine-tuned motor control abilities88. Importantly, though, tTIS did not modulate the ability of participants to benefit from real-time reinforcement feedback during motor performance. This suggests that striatal tTIS80Hz altered the beneficial effects of reinforcement on learning (as evaluated in test conditions at post-training), but not on motor performance (as evaluated during training). This dissociation between the effects of striatal tTIS80Hz on reinforcement-related gains in motor performance and in learning may be explained by the fact that these two phases of the protocol probe different processes7,55,58,89,90,91. While the improvement of motor performance with reinforcement relies on rapid feedback corrections based on expected outcomes67,92,93,94,95,96, reinforcement gains in learning (that is, probed in test conditions without reinforcement) may rather reflect the beneficial effect of reinforcement on the retention of motor memories4,7,55,90. This idea that the mechanisms underlying performance changes in training and retention phases are partially different is well supported by previous motor learning literature6,8,97. For instance, in sensorimotor adaptation paradigms, the presence of reward boosts motor memory retention but not the adaptation process itself7,86,90,91,98,99, and M1 transcranial direct current stimulation modulates the effect of reward on retention but has no effect on the training phase90. Such dissociation also appears to generalize to other motor learning tasks18,100, including force-tracking paradigms55 (see also Supplementary Fig. 1b). Importantly, while reinforcement gains in motor performance and learning seem to reflect the operation of partially dissociable mechanisms, it is no surprise that these processes are correlated at the group level (Supplementary Fig. 2), as they may be influenced by common individual factors (for example, sensitivity to reward)101. In contrast, the absence of correlation in the striatal tTIS80Hz condition suggests that the stimulation particularly impaired reinforcement gains in learning in the participants who initially benefited from reinforcement during training (Supplementary Fig. 2a). Hence, a potential explanation for the present results is that striatal tTIS80Hz did not disrupt rapid motor corrections based on recent outcomes during training, but may rather have altered the strengthening of the memory trace based on reinforcements6,7. Overall, these results are compatible with the view that specific patterns of oscillatory activity in the striatum are involved in motor control and learning processes31 and can be modulated with electrical stimulation69,102,103.
To better understand the neural effects and frequency specificity of tTIS, we coupled striatal tTIS and task performance with simultaneous fMRI acquisition. The imaging results support the view that the effect of tTIS80Hz on reinforcement learning of motor skills was indeed related to neuromodulation of the striatum. When considering averaged BOLD activity, we found a general increase of striatal activity when reinforcement was provided11, but no effect of tTIS. Crucially, though, the detrimental effect of tTIS80Hz on reinforcement learning was related to a specific modulation of activity in the caudate and putamen, providing evidence that the present behavioural effects were indeed driven by focal neuromodulation of the striatum (Fig. 3). Interestingly, participants with stronger disruption of reinforcement learning at the behavioural level were also the ones exhibiting stronger suppression of striatal activity with tTIS80Hz (than with tTIS20Hz), suggesting that tTIS-induced reduction of striatal activity is detrimental for reinforcement motor learning. Further analyses showed that tTIS80Hz, but not tTIS20Hz, increased the neuromodulatory influence of the striatum on frontal areas known to be important for motor learning and reinforcement processing97,104. More specifically, tTIS80Hz disrupted the task-related decrease in connectivity observed with tTISSham and tTIS20Hz, bringing connectivity closer to resting-state values. This effect depended on the type of network considered (reward versus motor) and on the presence of reinforcement. Striatal tTIS80Hz coupled with reinforcement increased connectivity between the motor striatum and the motor cortex, while this effect was not observed when considering the connectivity between limbic parts of the striatum and prefrontal areas involved in reward processing (Fig. 4). This result may reflect the differential influence of striatal tTIS on distinct subparts of the striatum, depending on their pattern of activity during the task53. A recent study in non-human primates showed that transcranial alternating current stimulation can have opposite effects on neuronal activity depending on the initial entrainment of neurons to the target frequency82. Hence, the present differential effects of tTIS80Hz on motor and reward striato-frontal pathways may be due to different initial patterns of activity in these networks in the presence of reinforcement. Electrophysiological recordings with higher temporal resolution than fMRI are required to confirm or infirm this hypothesis. Overall, the present neuroimaging results support the idea that the behavioural effects of striatal tTIS80Hz on reinforcement learning are associated with a selective modulation of striatal activity that influences striato-frontal communication.
The fact that we observed increased connectivity with tTIS80Hz and at the same time a disruption of behaviour may appear contradictory at first glance. Yet, multiple lines of evidence indicate that increases in connectivity are not necessarily beneficial for behaviour. For instance, the severity of motor symptoms in Parkinson’s disease is associated with excessive connectivity in the beta band, and the reduction of such connectivity with treatment is associated with clinical improvement29,32. Moreover, there is evidence that excessive functional105,106 as well as structural107,108 connectivity in fronto-striatal circuits is associated with impulsivity. Hence, the increase in connectivity observed with tTIS80Hz appears to be compatible with the behavioural findings. This being said, contrary to the BOLD results, we did not find any correlation between the effects of tTIS80Hz on connectivity and on reinforcement motor learning, suggesting some degree of independence between these two effects. Future studies could aim at determining whether tTIS80Hz-related changes in striato-frontal communication are linked to other aspects of reward processing that are not captured by our reinforcement motor learning task.
From a methodological point of view, the present results provide experimental support for the idea that the effects of tTIS are related to amplitude modulation of electric fields deep in the brain and not to the high-frequency fields themselves, in line with recent work41,42,52,53. The different behavioural and neural effects of striatal tTIS80Hz and tTIS20Hz despite comparable carrier frequencies (centred on 2 kHz) indicate that temporal interference was indeed the driving force of the present effects. Moreover, the disruption of reinforcement motor learning with tTIS80Hz (relative to tTIS20Hz) was specifically related to neuromodulation of the striatum, where the amplitude of the tTIS field was highest according to our simulations (see refs. 51,52 for recent validations of comparable simulations in cadaver experiments). Hence, we believe that the frequency- and reinforcement-dependent tTIS effects reported here cannot be explained by direct modulation of neural activity by the high-frequency fields. Yet, disentangling the neural effects of the low-frequency envelope and the high-frequency carrier appears to be an important next step to better characterize the mechanisms underlying tTIS47. We also note that the tTIS field strengths achieved according to our simulations (in the range of 0.5–0.6 V m−1) were sufficient to induce behavioural and neural effects, in line with recent data52,53 (see also ref. 48). Determining the minimum effective dose for tTIS is an important line of future research given recent simulation results suggesting that stimulation via amplitude modulation with high-frequency carrier signals (such as those arising during tTIS) may require higher dosages than conventional electrical stimulation with low frequencies (such as during transcranial alternating current stimulation), probably due to the low-pass-filtering properties of neurons43,109.
Finally, the strength of the behavioural effects of tTIS can be considered small to medium110 (d = 0.2–0.5). We note that these effect sizes are consistent with studies applying other types of non-invasive brain stimulation in healthy young adults, in the context of both motor learning (see ref. 111 for a meta-analysis) and reward tasks (for example, refs. 112,113), despite the much longer stimulation time used in these studies (between 3 and 20 times longer). Moreover, when expressed relative to the plateau of performance in the task (Supplementary Fig. 1d), the effect of tTIS80Hz represents a complete disruption of a ~24% reinforcement-related learning gain (Supplementary Fig. 1e). Overall, although they are moderate, we believe that the present effect sizes are relevant and consistent with what can be expected from the non-invasive brain stimulation literature.
Limitations
The present study includes some limitations that we would like to acknowledge. First, at the imaging level, we did not find a significant effect of reinforcement at the whole-brain level. This might be due to the short duration of the task (6 × 40 s), combined with the fact that we did not couple reinforcement to monetary incentives, a manipulation known to boost striatal activity in the context of motor learning18. Yet, when considering BOLD activity in the striatal ROIs, we did find a significant effect of reinforcement, suggesting that our experimental manipulation did increase striatal activity but that the strength of the effect was insufficient to survive at the whole-brain level. Second, we did not find any effect of tTIS when considering averaged BOLD activity. Again, the short duration of the blocks may contribute to this non-significant effect. Another possible interpretation is that the effect of tTIS on BOLD activity is not uniform across participants, as it probably depends on individual anatomy and function of the targeted brain region, as observed for other non-invasive brain stimulation techniques114. Consistently, we found a correlation between levels of impulsivity and the neural effects of tTIS80Hz (both BOLD and connectivity; Supplementary Fig. 5). Importantly, though, when including learning as a behavioural regressor, we did find significant clusters of correlation specifically in the striatum (Fig. 3), suggesting that the behavioural effects were indeed related to modulation of activity in the target region. This result was significant when contrasting tTIS80Hz to the active control (tTIS20Hz), but not to tTISSham. Overall, we believe that the fMRI data do support the idea that the behavioural effects of the stimulation were indeed related to modulation of neural activity in the striatum, also in line with the present simulations on realistic head models (Fig. 1) and the connectivity results (Fig. 4). This idea is also in agreement with another recent study investigating the effects of tTIS on motor sequence learning53. However, a limitation of the present dataset is the very short duration of stimulation and imaging for each experimental condition, which may explain some inconsistencies in the results. Hence, following this proof-of-concept study showing robust behavioural effects and related neural changes, future studies including longer fMRI and stimulation sessions are required to further confirm these results.
Finally, in the present study, the computational modelling was performed on a realistic, detailed head model (that is, the MIDA model59; Methods). One limitation of this approach is that the electric field simulations do not take individual structural information into account. Such individual modelling would require information on brain anisotropy, an aspect that is likely to significantly influence tTIS exposure44,115. However, in the present study, diffusion MRI to evaluate fractional anisotropy was not acquired. Future studies including diffusion MRI data will allow for personalized modelling, paving the way for individualized tTIS informed by brain structure52.
Conclusion
The present findings show the ability of non-invasive striatal tTIS to interfere with reinforcement learning in humans through selective modulation of striatal activity and support the causal functional role of the human striatum in reinforcement motor learning. This deep brain stimulation was well tolerated and compatible with efficient blinding, suggesting that tTIS provides the option to circumvent the steep depth–focality trade-off of current non-invasive brain stimulation approaches in a safe and effective way. Overall, tTIS opens possibilities for the study of causal brain–behaviour relationships and for the treatment of neuropsychiatric disorders associated with alterations of deep brain structures.
Methods
Participants
All participants gave their written informed consent in accordance with the Declaration of Helsinki and with the approval of the Cantonal Ethics Committee Vaud, Switzerland (project number 2020-00127). A total of 48 right-handed healthy volunteers participated in the study. Of these, 24 participants were enrolled for the main tTIS study (15 women, 25.3 ± 0.7 years old (mean ± s.e.)). Another group of 24 volunteers participated in the behavioural control experiment (Supplementary Fig. 1b–d; 14 women, 24.2 ± 0.5 years old). Handedness was determined via a shortened version of the Edinburgh Handedness Inventory116 (laterality index, 89.3 ± 2.14% for the main study and 86.4 ± 2.51% for the control experiment). None of the participants had any neurological or psychiatric disorder or were taking any centrally acting medication (see Supplementary Information for a complete list of exclusion criteria). Finally, all participants were asked to fill out a delay-discounting monetary choice questionnaire117, which evaluates the propensity of participants to choose smaller, sooner rewards over larger, later rewards, a preference commonly associated with choice impulsivity75,118. The participants were financially compensated at a standard rate of 20 CHF per hour.
Experimental procedures
The study employed a randomized, double-blind, sham-controlled design. Following screening and inclusion, the participants were invited to a single experimental session including the performance of a motor learning task with concurrent tTIS of the striatum and fMRI. Overall, the participants practised six blocks of trials, which resulted from the combination of two reinforcement feedback conditions (ReinfTYPE: ReinfON or ReinfOFF) with three types of striatal stimulation (tTISTYPE: tTISSham, tTIS20Hz or tTIS80Hz).
Motor learning task
General aspects
The participants practised an adaptation of a widely used force-tracking motor task54,55 with an fMRI-compatible fibre-optic grip-force sensor (Current Designs) positioned in their right hand. This task has the advantage of evaluating learning in a context in which movements have to be dynamically adjusted in response to constantly evolving sensory information. Such careful, continuous force control represents a situation that is relevant in many daily-life activities, including situations involving limited visual feedback such as when learning to drive or to manipulate fragile objects119. In these situations, the learner has to use somatosensory information in combination with information about task success to improve future motor commands, a process that might be particularly relevant in tasks where visual information is limited as well as in early stages of motor learning, when the desired sensory state is unknown5,58,68,120. In addition to these elements, force-modulation tasks are relevant for rehabilitation as they can be used to evaluate motor function in clinical populations121,122.
The task was developed in MATLAB 2018 (MathWorks) exploiting the Psychophysics Toolbox extensions123,124 and was displayed on a computer screen with a refresh rate of 60 Hz. The task required the participants to squeeze the force sensor to control a cursor displayed on the screen. Increasing the exerted force resulted in the cursor moving vertically and upward in a linear way. Each trial started with a preparatory period in which a sidebar appeared at the bottom of the screen (Fig. 1a). After a variable time interval (0.9 to 1.1 s), a cursor (a black circle) popped up in the sidebar, and simultaneously a target (a grey larger circle with a cross in the middle) appeared, indicating the start of the movement period. The participants were asked to modulate the force applied on the transducer to keep the cursor as close as possible to the centre of the target. The target moved in a sequential way along a single vertical axis for 7 s. The maximum force required (that is, the force required to reach the target when it was in the uppermost part of the screen; MaxTargetForce) was set at 4% of MVC evaluated at the beginning of the experiment. This low force level was chosen on the basis of pilot experiments to limit muscular fatigue. Finally, each trial ended with a blank screen displayed for 2 s before the beginning of the next trial.
Trial types and reinforcement manipulation
During the experiment, the participants were exposed to different types of trials (Fig. 1a and Supplementary Video 1). In test trials, the cursor remained on the screen, and the target was consistently displayed in grey for the whole duration of the trial. These trials served to evaluate pre- and post-training performance for each block, without any disturbance. In ReinfON and ReinfOFF trials (used during training only), we provided only partial visual feedback to the participants to increase the impact of reinforcement on learning5,56,57,58. The cursor was only intermittently displayed during the trial: it was always displayed in the first second of the trial and then disappeared for a total of 4.5 s randomly split in the remaining time by bits of 0.5 s. The cursor was therefore displayed 35.7% of the time during these trials (2.5 s over the 7 s trial). Importantly, unlike the cursor, the target always remained on the screen for the whole trial, and the participants were instructed to continue to track the target even when the cursor was away.
In addition to this visual manipulation, in ReinfON trials, the participants also trained with reinforcement feedback indicating success or failure of the tracking in real time. The participants were informed that, during these trials, the colour of the target would vary as a function of their performance: the target was displayed in green when tracking was considered successful and in red when it was considered a failure. Online success on the task was determined on the basis of the Error, defined as the absolute force difference between the force required to be in the centre of the target and the exerted force5,54,55,58. The Error, expressed as a percentage of MVC, was computed for each frame refresh and allowed to classify a sample as successful or not on the basis of a closed-loop reinforcement schedule8. More specifically, for each training trial, a force sample (recorded at 60 Hz, corresponding to the refresh rate of the monitor) was considered successful if the computed Error was below the median Error over the four previous trials at this specific sample. Put differently, to be successful, the participants had to constantly beat their previous performance. This closed-loop reinforcement schedule allowed us to deliver consistent reinforcement feedback across individuals and conditions (see the control analysis on success rates in the Supplementary Information), while maximizing uncertainty on the presence of reinforcement, an aspect that is crucial for efficient reinforcement motor learning125. In addition to this closed-loop design, samples were also considered successful if the cursor was very close to the centre of the target (that is, within one radius around the centre, corresponding to an Error below 0.2% of MVC). This was done to prevent any conflict between visual information (provided by the position of the cursor relative to the target) and reinforcement feedback (provided by the colour of the target), which could occur in situations of extremely good performance (when the closed-loop Error cut-off is below 0.2% of MVC).
As a control, ReinfOFF trials were similar to ReinfON trials, with the only difference being that the displayed colours were either cyan or magenta and were generated randomly. The participants were explicitly told that, in this condition, the colours were displayed randomly and could be ignored. The visual properties of the target in the ReinfOFF condition were designed to match the ReinfON condition in terms of relative luminance (cyan (RGB, (127.5, 242.1, 255)) matched to green (127.5, 255, 127.5) and magenta (211.7, 127.5, 255) to red (255, 127.5, 127.5)) and average frequency of change in colours (that is, the average number of changes in colours divided by the total duration of a trial; Supplementary Information).
In this task, training trials differed from test trials regarding not only the colour of the target (red/green or cyan/magenta in training trials and grey in test trials) but also the visual feedback experienced (partial and full visual feedback in training and test trials, respectively). This choice was motivated by several reasons. First, we wanted to evaluate learning in the classical, unperturbed, version of the force-tracking task54,55, which is compatible with clinical translation. Second, on the basis of additional behavioural data on another group of participants (n = 24; Supplementary Fig. 1b–d), we found that significant effects of reinforcement on learning were observed only when training was performed with partial visual feedback (displayed for 35.7% of the trial time, as in the present study), in line with previous results56,61. However, this additional study also revealed very limited improvement of performance during training with partial visual feedback, potentially due to ceiling effects on performance in this condition. Yet, the improvement of performance when comparing the pre- and post-training assessments suggested that practising the task with partial visual feedback still induced significant learning of the skill. Finally, the change in visual feedback between training and post-training was the same in all experimental conditions; this aspect of the task is therefore unlikely to explain the reinforcement as well as the stimulation effects reported here.
Even though our study focused on reinforcement motor learning, it is worth mentioning that other learning mechanisms such as error-based or strategic processes are likely to be also engaged during the force-tracking task and may have recruited other brain regions beyond the striatum4. Notably, though, our protocol was specifically designed to compare learning in the ReinfON and ReinfOFF conditions in the same individuals while keeping the other parameters of the task constant, to specifically isolate the contribution of reinforcement processes in motor learning.
Motor learning protocol
After receiving standardized instructions about the force-tracking task, the participants practised five blocks of familiarization (total of 75 trials) without tTIS. The first block of familiarization included 20 trials with the target moving in a regular fashion (0.5 Hz sinuoid). Then, in a second block of familiarization, the participants performed 35 trials of practice with an irregular pattern, with the same properties as the training patterns (see below). Finally, we introduced the reinforcement manipulation and let the participants perform two short blocks (eight trials each) including ReinfON and ReinfOFF trials. These first four blocks of familiarization were performed outside the MRI environment. A final familiarization block (four trials) was performed after installation in the scanner, to allow the participants to get used to performing the task in the MRI machine. This long familiarization allowed the participants to get acquainted with the use of the force sensor before the beginning of the experiment.
During the main part of the experiment, the participants performed six blocks of trials in the MRI machine with concurrent striatal tTIS (Fig. 1b). Each block was composed of 4 pre-training trials followed by 24 training and 8 post-training trials. Pre- and post-training trials were performed in test conditions, without tTIS, and were used to evaluate motor learning. Training trials were performed with or without reinforcement feedback and with concomitant striatal tTIS and were used as a proxy of motor performance. During training, trials were interspersed with 25 s resting periods every four trials (used for fMRI contrasts; see below). The order of the six experimental conditions was pseudo-randomized across participants: the six blocks were divided into three pairs of blocks with the same tTIS condition, and each pair was then composed of one ReinfON and one ReinfOFF block. Within this structure, the order of the tTISTYPE and ReinfTYPE conditions were balanced among the 24 participants. Hence, this randomization allowed us to ensure that any order effect that may arise from the repetition of the learning blocks would have the same impact on each experimental condition (for example, four participants experienced tTIS80Hz–ReinfON in the first block, four other participants in the second block, four in the third block and so on).
As mentioned above, the protocol involved multiple evaluations of motor learning within the same experimental session. To limit carry-over effects from one block to the next, each experimental block was associated with a different pattern of movement of the target (Supplementary Fig. 1a). Put differently, in each block, the participants had to generate a new pattern of force to successfully track the target. To balance the patterns’ difficulty, they all consisted of the summation of five sinusoids of variable frequency (range, 0.1–1.5 Hz) that presented the following properties: the average force was between 45% and 55% of MaxTargetForce, the absolute average derivative was between 54% and 66% of MaxTargetForce per second and the number of peaks was 14 (defined as an absolute change of force of at least 1% of MaxTargetForce). These parameters were determined on the basis of pilot experiments to obtain a relevant level of difficulty for young healthy adults and consistent learning across the different patterns.
Striatal tTIS
General concept
tTIS is an innovative non-invasive brain stimulation approach, in which two or more independent stimulation channels deliver high-frequency currents in the kHz range (oscillating at f1 and f1 + Δf; Fig. 1c). These high-frequency currents are assumed to be too high to effectively modulate neuronal activity41,50,126. Still, by applying a small shift in frequency, they result in a modulated electric field with the envelope oscillating at the low-frequency Δf (target frequency) where the two currents overlap. The peak of the modulated envelope amplitude can be steered towards specific areas located deep in the brain, by tuning the positions of the electrodes and the current ratio across stimulation channels41 (Fig. 1c,d). On the basis of these properties, tTIS has been shown to be able to focally target the activity of deep structures in rodents, without engaging overlying tissues41. Here we applied tTIS via surface electrodes applying a low-intensity, sub-threshold protocol following the currently accepted cut-offs and safety guidelines for low-intensity transcranial electric stimulation in humans127.
Stimulators
The currents for tTIS were delivered by two independent DS5 isolated bipolar constant current stimulators (Digitimer Ltd). The stimulation patterns were generated using a custom-based MATLAB graphical user interface and transmitted to the current sources using a standard digital–analogue converter (DAQ USB-6216, National Instruments). Finally, an audio transformer was added between stimulators and participants to avoid possible direct current accumulation.
Stimulation protocols
During the six training blocks, we applied three different types of striatal tTIS (two blocks each): a stimulation with a tTIS envelope modulated at 20 Hz (tTIS20Hz), a stimulation with a tTIS envelope modulated at 80 Hz (tTIS80Hz) and a sham stimulation (tTISSham). For tTIS20Hz, the posterior stimulation channel (TP7–TP8; see below) delivered a 1.99 kHz stimulation, while the anterior one delivered a 2.01 kHz stimulation (Δf = 20 Hz). For tTIS80Hz, the posterior and anterior channels delivered 1.96 kHz and 2.04 kHz, respectively (Δf = 80 Hz). Hence, in both conditions, the high-frequency component was comparable, and the only difference was Δf. During each block, tTIS was applied for five minutes (6 × 50 s) during training. Each stimulation period started and ended with currents ramping up and down, respectively, for 5 s. tTIS was applied only while the participants were performing the motor task and not during resting periods or pre- and post-training assessments. Finally, tTISSham consisted of a ramping-up (5 s) immediately followed by a ramping-down (5 s) of 2 kHz currents delivered without any shift in frequency. This condition allowed us to mimic the sensations experienced during the active conditions tTIS20Hz and tTIS80Hz, while delivering minimal brain stimulation (Supplementary Fig. 3). A trigger was sent 5 s before the beginning of each trial to align the beginning of the task and the beginning of the frequency shift after the ramp-up. Other tTIS parameters were set as follows: current intensity per stimulation channel, 2 mA (baseline-to-peak); electrode type, round, conductive rubber with conductive cream/paste; electrode size, 3 cm2 (see the ContES checklist in the Supplementary Information for more details).
The stimulation was applied in the MRI environment (Siemens 3T MAGNETOM Prisma; Siemens Healthcare) using a standard RF filter module and MRI-compatible cables (neuroConn GmbH). The technological, safety and noise tests and the methodological factors can be found in the Supplementary Information (Supplementary Table 4) and are based on the ContES checklist128.
Modelling
Electromagnetic simulations were carried out to identify optimized electrode placement and current steering parameters. The simulations were performed using the MIDA head model59, a detailed anatomical head model featuring >100 distinguished tissues and regions that was derived from multi-modal image data of a healthy female volunteer. Importantly, for brain stimulation modelling, the model differentiates different scalp layers, skull layers, grey and white matter, cerebrospinal fluid, and the dura and accounts for electrical conductivity anisotropy and neural orientation on the basis of diffusion tensor imaging data. Circular electrodes (radius, 0.7 cm) were positioned on the skin according to the 10–10 system, and the electromagnetic exposure was computed using the ohmic-current-dominated electro-quasistatic solver from Sim4Life v.5.0 (ZMT Zurich MedTech AG), which is suitable due to the dominance of ohmic currents over displacement currents and the long wavelength compared with the simulation domain129. Dielectric properties were assigned on the basis of the IT’IS Tissue Properties Database v.4.0 (ref. 130). Rectilinear discretization was performed, and grid convergence as well as solver convergence analyses were used to ensure negligible numerical uncertainty, resulting in a grid that included more than 54 million voxels. Dirichlet voltage boundary conditions and then current normalization were applied. The electrode–head interface contact was treated as ideal. tTIS exposure was quantified according to the maximum modulation envelope magnitude formula from Grossman et al.41. A sweep over 960 permutations of the four electrode positions was then performed, considering symmetric and asymmetric montages with parallel (sagittal and coronal) or crossing current paths, while quantifying bilateral striatum (putamen (BNA regions 225, 226, 229 and 230), caudate (BNA regions 219, 220, 227 and 228) and NAc (BNA regions 223 and 224)) exposure performance according to three metrics: (1) target exposure strength, (2) focality ratio (the ratio of target tissue volume above the threshold compared to the whole-brain tissue volume above the threshold, a measure of stimulation selectivity) and (3) activation ratio (the percentage of target volume above the threshold with respect to the total target volume, a measure of target coverage). We defined the threshold as the 98th volumetric iso-percentile level of the tTIS. From the resulting Pareto-optimal front, two configurations stood out particularly: one that maximized focality and activation (AF3–AF4 and P7–P8) and one that accepted a reduction of these two metrics by a quarter, while increasing the target exposure strength by more than 50% (F3–F4 and TP7–TP8). This last montage was selected to ensure sufficient tTIS exposure in the striatum53 (Fig. 1c,d).
Electrode positioning and stimulation-related sensations
On the basis of the modelling approach described above, we defined the stimulation electrode positions in the framework of the EEG 10–10 system131. The optimal montage leading in terms of target (that is, bilateral striatum) exposure strength and selectivity was composed of the following electrodes: F3, F4, TP7 and TP8. Their locations were marked with a pen on the scalp, and, after skin preparation (cleaning with alcohol), round conductive rubber electrodes of 3 cm2 were placed, adding a conductive paste (Ten20, Weaver; or Abralyt HiCl, Easycap GmbH) as an interface to the skin. The electrodes were held in position with tape, and the cables were oriented towards the top to allow good positioning inside the scanner. Impedances were checked and optimized until they were below 20 kΩ (ref. 48). Once good contact was obtained, we tested different intensities of stimulation for each stimulation protocol to familiarize the participants with the perceived sensations and to systematically document them. tTISSham, tTIS20Hz and tTIS80Hz were applied for 20 seconds with the following increasing current amplitudes per channel: 0.5 mA, 1 mA, 1.5 mA and 2 mA. The participants were asked to report any kind of sensation, and, if a sensation was felt, they were asked to grade the intensity from 1 to 3 (light to strong) as well as give at least one adjective to describe it (Supplementary Fig. 3). Following this step, the cables were removed and replaced by MRI-compatible cables, and a bandage was added to apply pressure on the electrodes and keep them in place. An impedance check was repeated in the MRI machine right before the training and then again at the end of all recordings.
MRI data acquisition
Structural and functional images were acquired using a 3T MAGNETOM PRISMA scanner (Siemens). T1-weighted images were acquired via the 3D MPRAGE sequence with the following parameters: repetition time (TR), 2.3 s; echo time (TE), 2.96 ms; flip angle, 9°; slices, 192; voxel size, 1 × 1 × 1 mm3; field-of-view (FOV), 256 mm. Anatomical T2 images were also acquired with the following parameters: TR, 3 s; TE, 409 ms; flip angle, 120°; slices, 208; voxel size, 0.8 × 0.8 × 0.8 mm3; FOV, 320 mm. Finally, functional images were recorded using echo-planar imaging sequences with the following parameters: TR, 1.25 s; TE, 32 ms; flip angle, 58°; slices, 75; voxel size, 2 × 2 × 2 mm3; FOV, 112 mm.
Data and statistical analyses
Data and statistical analyses were carried out with MATLAB 2018a (MathWorks) and the R software environment (version 2021) for statistical computing and graphics132. Robust linear regressions were fitted with the MATLAB function robustfit. LMMs were fitted using the lmer function of the lme4 package in R133. As random effects, we added intercepts for participants and block. The normality of residuals and the homoscedasticity of the data were systematically checked, and logarithmic transformations were applied when necessary (that is, when the skewness of the residuals’ distribution was not between −2 and 2 (ref. 134) or when homoscedasticity was violated on the basis of visual inspection). To mitigate the impact of isolated influential data points on the outcome of the final model, we used tools of the influence.ME package to detect and remove influential cases on the basis of the following criterion: distance > 4 × mean distance135. Statistical significance was determined using the anova function with Satterthwaite’s approximations of the lmerTest package136. For specific post hoc comparisons, we conducted two-sided pairwise tests by computing estimated marginal means with the emmeans package with Tukey adjustment of P values to correct for multiple comparisons137. Standardized effect size measures were obtained using the eff_size function of the emmeans package138 and the eta_squared function of the effectsize package139. The level of significance was set at P < 0.05.
Behavioural data
Evaluation of motor learning
The main goal of the present study was to evaluate the influence of striatal tTIS on reinforcement motor learning. To do so, we first removed trials in which participants did not react within 1 s after the appearance of the cursor and target, considering that these extremely long preparation times may reflect substantial fluctuations in attention140. This occurred extremely rarely (0.52% of the whole dataset). For each participant and each trial, we then quantified the tracking Error as the absolute force difference between the applied and required force, as done previously5,55,58. Tracking performance during training and post-training trials was then normalized according to each participant’s initial level by expressing the Error data as a percentage of the average pre-training Error for each block. To test our main hypothesis predicting specific effects of striatal tTIS on reinforcement motor learning, we performed an LMM on the post-training data with tTISTYPE and ReinfTYPE as fixed effects. We then ran the same analysis on the training data, to evaluate whether striatal tTIS also impacted motor performance while stimulation was being delivered.
As a control, we checked that initial performance at pre-training was not different between conditions with an LMM on the Error data obtained at pre-training. Again, tTISTYPE and ReinfTYPE were considered as fixed effects. Finally, another LMM was fitted with the fixed effect tTISTYPE to verify that the amount of positive reinforcement (as indicated by a green target) in the ReinfON blocks was similar across tTISTYPES.
fMRI data
Imaging preprocessing
We analysed the functional imaging data using Statistical Parametric Mapping v.12 (Wellcome Department of Cognitive Neurology) implemented in MATLAB R2018a (MathWorks). All functional images underwent a common preprocessing procedure including the following steps: slice time correction, spatial realignment to the first image, normalization to the standard MNI space and smoothing with a 6 mm full-width half-maximal Gaussian kernel. T1 anatomical images were then co-registered to the mean functional image and segmented. This allowed us to obtain bias-corrected grey and white matter images by normalizing the functional images via the forward deformation field. To select participants with acceptable levels of head movement, framewise displacement was calculated for each run. A visual check of both non-normalized and normalized images was performed to ensure good preprocessing quality. Finally, possible tTIS-related artefacts were investigated on the basis of signal-to-noise ratio maps (see below).
Signal-to-noise ratio
Total signal-to-noise ratio maps were computed to check the presence of possible artefacts induced by the electrical stimulation. The values were calculated for each voxel by dividing the mean of the voxel time series by its standard deviation. Spherical ROIs were then defined both underneath the tTIS electrodes and at four different locations distant from the electrodes as a control. The centre of each spherical ROI was obtained by projecting the standard MNI coordinates of each electrode on the scalp141 towards the centre of the brain. After visual inspection of the ROIs, average total signal-to-noise ratio maps were extracted within each sphere. An LMM was used to compare the average signal-to-noise ratio underneath the electrodes versus the control regions and between stimulation protocols. The results of this analysis are presented in the Supplementary Information (Supplementary Fig. 6).
Task-based BOLD activity analysis
A general linear model was implemented at the single-participant level to estimate signal amplitude. Eight regressors were included in the model: six head motion parameters (displacement and rotation) and normalized time series within the white matter and the cerebrospinal fluid. Linear contrasts were then computed to estimate specific activity during the motor task with respect to resting periods. Functional activation was also extracted within specific ROIs individually defined on the basis of structural images. More specifically, the Freesurfer recon-all function was run on the basis of the structural T1w and T2w images (Freesurfer v.7.1.1, https://surfer.nmr.mgh.harvard.edu/; coded in Bash (v.4.4.20(1)-release) and Python (v.3.8.3)). The BNA parcellation was derived on the individual participant space, and the selected ROIs were then co-registered to the functional images and normalized to the MNI space. BOLD activity within the individual striatal masks was averaged and compared between different striatal nuclei—namely, the putamen (BNA regions 225, 226, 229 and 230), caudate (BNA regions 219, 220, 227 and 228) and NAc (BNA regions 223 and 224). Comparisons between conditions were presented for uncorrected P = 0.001 at the voxel level and multiple comparison corrected at the cluster level to reduce the false discovery rate, P = 0.05.
Effective connectivity analyses
As an additional investigation, we computed task-modulated effective functional connectivity by means of the CONN toolbox 2021a (www.nitrc.org/projects/conn, RRID:SCR_009550) running in MATLAB R2018a (MathWorks). An additional denoising step was added by applying band-pass filtering from 0.01 to 0.1 Hz and by regressing potential confounders (white matter, cerebrospinal fluid and realignment parameters). After that, gPPI connectivity was extracted within specific pre-defined customized sub-networks: a reward network and a motor network. gPPI evaluates condition-specific changes in effective connectivity, defined as the directed effect that one brain region has on another under some model of neuronal coupling142. In particular, gPPI considers a series of equations in which activity in a ROI (pre-defined frontal areas in our case) depends on a specific condition (the ‘psychological’ factor) and on activity in the seed region (striatum here, the ‘physiological’ factor). By solving these equations, it is possible to determine a coefficient that represents task modulation of effective connectivity143. Importantly, task-related changes in effective connectivity are expressed relative to rest, and therefore values closer to 0 reflect a connectivity similar to the resting state.
The reward network was defined as follows: two regions within the striatum, the NAc (BNA regions 223 and 224) and the ventro-medial putamen (BNA regions 225 for left and 226 for right), and two frontal areas, the ACC (BNA regions 177, 179 and 183 for left and 178, 180 and 184 for right) and the orbitofrontal cortex within the vmPFC (BNA regions 41, 45, 47, 49 and 187 for left and 42, 46, 48, 50 and 188 for right). The motor network included the following areas: the dorso-lateral putamen (BNA 229 for left and 230 for right), the dorsal caudate (BNA regions 227 for left and 228 for right), the medial part of the SMA (BNA regions 9 for left and 10 for right) and the part of the M1 associated with upper limb function (BNA regions 57 for left and 58 for right). We considered connectivity in the left and right motor and reward networks regardless of laterality. These ROIs were selected on the basis of the following rationale. First, they are consistent with previous literature on reinforcement learning of motor skills68,90,144,145. Second, there is structural and functional evidence for these fronto-striatal connections146,147. Third, the frontal areas included in the analyses are well-established hubs of the motor learning (M1 and SMA; see ref. 12 for a meta-analysis) and reward networks (vmPFC and ACC; see ref. 11 for a meta-analysis). Finally, gPPI was also extracted within a control language network, defined on the basis of the functional atlas described by Shirer et al.70.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
All data necessary to generate the main results and figures are available in the Zenodo repository (https://doi.org/10.5281/zenodo.10458885)148. The BNA was used and can be downloaded from http://atlas.brainnetome.org/. The tissue properties used for the modelling of electric fields were based on the IT’IS Tissue Properties Database v.4.0 and can be downloaded here: https://itis.swiss/virtual-population/tissue-properties/overview/.
Code availability
The scripts necessary to generate the main results are available in the Zenodo repository (https://doi.org/10.5281/zenodo.10458885)148.
References
Neftci, E. O. & Averbeck, B. B. Reinforcement learning in artificial and biological systems. Nat. Mach. Intell. 1, 133–143 (2019).
Schultz, W. Neuronal reward and decision signals: from theories to data. Physiol. Rev. 95, 853–951 (2015).
Dhawale, A. K., Smith, M. A. & Ölveczky, B. P. The role of variability in motor learning. Annu. Rev. Neurosci. 40, 479–498 (2017).
Spampinato, D. & Celnik, P. Multiple motor learning processes in humans: defining their neurophysiological bases. Neuroscientist 27, 246–267 (2021).
Vassiliadis, P. et al. Reward boosts reinforcement-based motor learning. iScience 24, 102821 (2021).
Huang, V. S., Haith, A., Mazzoni, P. & Krakauer, J. W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801 (2011).
Galea, J. M., Mallia, E., Rothwell, J. & Diedrichsen, J. The dissociable effects of punishment and reward on motor learning. Nat. Neurosci. 18, 597–602 (2015).
Therrien, A. S., Wolpert, D. M. & Bastian, A. J. Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise. Brain 139, 101–114 (2016).
Vassiliadis, P., Derosiere, G. & Duque, J. Beyond motor noise: considering other causes of impaired reinforcement learning in cerebellar patients. eNeuro 6, ENEURO.0458-18.2019 (2019).
Widmer, M. et al. Reward during arm training improves impairment and activity after stroke: a randomized controlled trial. Neurorehabil. Neural Repair 36, 140–150 (2022).
Bartra, O., McGuire, J. T. & Kable, J. W. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage 76, 412–427 (2013).
Hardwick, R. M., Rottschy, C., Miall, R. C. & Eickhoff, S. B. A quantitative meta-analysis and review of motor learning in the human brain. NeuroImage 67, 283–297 (2013).
Haber, S. N. Corticostriatal circuitry. Dialogues Clin. Neurosci. 18, 7–21 (2016).
Balleine, B. W., Delgado, M. R. & Hikosaka, O. The role of the dorsal striatum in reward and decision-making. J. Neurosci. 27, 8161–8165 (2007).
Piray, P., den Ouden, H. E. M., van der Schaaf, M. E., Toni, I. & Cools, R. Dopaminergic modulation of the functional ventrodorsal architecture of the human striatum. Cereb. Cortex 27, 485–495 (2017).
Hori, Y. et al. Ventral striatum links motivational and motor networks during operant-conditioned movement in rats. NeuroImage 184, 943–953 (2019).
Wachter, T., Lungu, O. V., Liu, T., Willingham, D. T. & Ashe, J. Differential effect of reward and punishment on procedural learning. J. Neurosci. 29, 436–443 (2009).
Widmer, M., Ziegler, N., Held, J., Luft, A. & Lutz, K. Rewarding feedback promotes motor skill consolidation via striatal activity. Prog. Brain Res. 229, 303–323 (2016).
Berke, J. D. Fast oscillations in cortical–striatal networks switch frequency following rewarding events and stimulant drugs. Eur. J. Neurosci. 30, 848–859 (2009).
van der Meer, M. A. A. et al. Integrating early results on ventral striatal gamma oscillations in the rat. Front. Neurosci. 4, 300 (2010).
van der Meer, M. A. A. & Redish, A. D. Low and high gamma oscillations in rat ventral striatum have distinct relationships to behavior, reward, and spiking activity on a learned spatial decision task. Front. Integr. Neurosci. 3, 9 (2009).
Dwiel, L. L., Khokhar, J. Y., Connerney, M. A., Green, A. I. & Doucette, W. T. Finding the balance between model complexity and performance: using ventral striatal oscillations to classify feeding behavior in rats. PLoS Comput. Biol. 15, 4 (2019).
Matsumoto, J. et al. Neuronal responses in the nucleus accumbens shell during sexual behavior in male rats. J. Neurosci. 32, 1672–1686 (2012).
Kalenscher, T., Lansink, C. S., Lankelma, J. V. & Pennartz, C. M. A. Reward-associated gamma oscillations in ventral striatum are regionally differentiated and modulate local firing activity. J. Neurophysiol. 103, 1658–1672 (2010).
Cohen, M. X. et al. Good vibrations: cross-frequency coupling in the human nucleus accumbens during reward processing. J. Cogn. Neurosci. 21, 875–889 (2009).
Sepe-Forrest, L., Carver, F. W., Quentin, R., Holroyd, T. & Nugent, A. C. Basal ganglia activation localized in MEG using a reward task. NeuroImage Rep. 1, 100034 (2021).
Herrojo-Ruiz, M. et al. Involvement of human internal globus pallidus in the early modulation of cortical error-related activity. Cereb. Cortex 24, 1502–1517 (2014).
Jenkinson, N. & Brown, P. New insights into the relationship between dopamine, beta oscillations and motor function. Trends Neurosci. 34, 611–618 (2011).
Brown, P. Abnormal oscillatory synchronisation in the motor system leads to impaired movement. Curr. Opin. Neurobiol. 17, 656–664 (2007).
McCarthy, M. M. et al. Striatal origin of the pathologic beta oscillations in Parkinson’s disease. Proc. Natl Acad. Sci. USA 108, 11620–11625 (2011).
Kondabolu, K. et al. Striatal cholinergic interneurons generate beta and gamma oscillations in the corticostriatal circuit and produce motor deficits. Proc. Natl Acad. Sci. USA 113, 3159–3168 (2016).
Silberstein, P. et al. Cortico-cortical coupling in Parkinson’s disease and its modulation by therapy. Brain 128, 1277–1291 (2005).
Williams, Z. M. & Eskandar, E. N. Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat. Neurosci. 9, 562–568 (2006).
Nakamura, K. & Hikosaka, O. Facilitation of saccadic eye movements by postsaccadic electrical stimulation in the primate caudate. J. Neurosci. 26, 12885–12895 (2006).
Deng, Z. D., Lisanby, S. H. & Peterchev, A. V. Electric field depth–focality tradeoff in transcranial magnetic stimulation: simulation comparison of 50 coil designs. Brain Stimul. 6, 1–13 (2013).
Wagner, T. et al. Transcranial direct current stimulation: a computer-based human model study. NeuroImage 35, 1113–1124 (2007).
Nickchen, K. et al. Reversal learning reveals cognitive deficits and altered prediction error encoding in the ventral striatum in Huntington’s disease. Brain Imaging Behav. 11, 1862–1872 (2017).
Schmidt, L. et al. Disconnecting force from money: effects of basal ganglia damage on incentive motivation. Brain 131, 1303–1310 (2008).
Seymour, B. et al. Deep brain stimulation of the subthalamic nucleus modulates sensitivity to decision outcome value in Parkinson’s disease. Sci. Rep. 6, 32509 (2016).
Atkinson-Clement, C. et al. Effects of subthalamic nucleus stimulation and levodopa on decision-making in Parkinson’s disease. Mov. Disord. 34, 377–385 (2019).
Grossman, N. et al. Noninvasive deep brain stimulation via temporally interfering electric fields. Cell 169, 1029–1041.e16 (2017).
Song, S., Zhang, J., Tian, Y., Wang, L. & Wei, P. Temporal interference stimulation regulates eye movements and neural activity in the mice superior colliculus. Proc. Annual International Conference of the IEEE Engineering in Medicine and Biology Society 6231–6234 (IEEE, 2021); https://doi.org/10.1109/EMBC46164.2021.9629968
Esmaeilpour, Z., Kronberg, G., Reato, D., Parra, L. C. & Bikson, M. Temporal interference stimulation targets deep brain regions by modulating neural oscillations. Brain Stimul. 14, 55–65 (2021).
Rampersad, S. et al. Prospects for transcranial temporal interference stimulation in humans: a computational study. NeuroImage 202, 116124 (2019).
von Conta, J. et al. Interindividual variability of electric fields during transcranial temporal interference stimulation (tTIS). Sci. Rep. 11, 20357 (2021).
Cao, J., Doiron, B., Goswami, C. & Grover, P. The mechanics of temporal interference stimulation. Preprint at bioRxiv https://doi.org/10.1101/2020.04.23.051870 (2020).
Mirzakhalili, E., Barra, B., Capogrosso, M. & Lempka, S. F. Biophysics of temporal interference stimulation. Cell Syst. 11, 557–572.e5 (2020).
von Conta, J. et al. Benchmarking the effects of transcranial temporal interference stimulation (tTIS) in humans. Cortex 154, 299–310 (2022).
Ma, R. et al. High gamma and beta temporal interference stimulation in the human motor cortex improves motor functions. Front. Neurosci. 15, 800436 (2022).
Hutcheon, B. & Yarom, Y. Resonance, oscillation and the intrinsic frequency preferences of neurons. Trends Neurosci. 23, 216–222 (2000).
Acerbo, E. et al. Focal non-invasive deep-brain stimulation with temporal interference for the suppression of epileptic biomarkers. Front. Neurosci. 16, 1–12 (2022).
Violante, I. R. et al. Non-invasive temporal interference electrical stimulation of the human hippocampus. Nat. Neurosci. 26, 1994–2004 (2023).
Wessel, M. J. et al. Noninvasive theta-burst stimulation of the human striatum enhances striatal activity and motor skill learning. Nat. Neurosci. 26, 2005–2016 (2023).
Steel, A., Silson, E. H., Stagg, C. J. & Baker, C. I. The impact of reward and punishment on skill learning depends on task demands. Sci. Rep. 6, 36056 (2016).
Abe, M. et al. Reward improves long-term retention of a motor memory through induction of offline memory gains. Curr. Biol. 21, 557–562 (2011).
Izawa, J. & Shadmehr, R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput. Biol. 7, 3 (2011).
Mawase, F., Uehara, S., Bastian, A. J. & Celnik, P. Motor learning enhances use-dependent plasticity. J. Neurosci. 37, 2673–2685 (2017).
Vassiliadis, P., Lete, A., Duque, J. & Derosiere, G. Reward timing matters in motor learning. iScience 25, 104290 (2022).
Iacono, M. I. et al. MIDA: a multimodal imaging-based detailed anatomical model of the human head and neck. PLoS ONE 10, e0124126 (2015).
Areshenkoff, C. N., de Brouwer, A. J., Gale, D. J., Nashed, J. Y. & Gallivan, J. P. Separate and shared low-dimensional neural architectures for error-based and reinforcement motor learning. Preprint at biorXiv https://doi.org/10.1101/2022.08.16.504134 (2022).
Cashaback, J. G. A., McGregor, H. R., Mohatarem, A. & Gribble, P. L. Dissociating error-based and reinforcement-based loss functions during sensorimotor learning. PLoS Comput. Biol. 13, 7 (2017).
Floyer-Lea, A. & Matthews, P. M. Changing brain networks for visuomotor control with increased movement automaticity. J. Neurophysiol. 92, 2405–2412 (2004).
Floyer-Lea, A. & Matthews, P. M. Distinguishable brain activation networks for short- and long-term motor skill learning. J. Neurophysiol. 94, 512–518 (2005).
Jiang, T. Brainnetome: a new -ome to understand the brain and its disorders. NeuroImage 80, 263–272 (2013).
Graybiel, A. M. & Grafton, S. T. The striatum: where skills and habits meet. Cold Spring Harb. Perspect. Biol. 7, 3 (2015).
McLaren, D. G., Ries, M. L., Xu, G. & Johnson, S. C. A generalized form of context-dependent psychophysiological interactions (gPPI): a comparison to standard approaches. NeuroImage 61, 1277–1286 (2012).
Codol, O., Holland, P. J., Manohar, S. G. & Galea, J. M. Reward-based improvements in motor control are driven by multiple error-reducing mechanisms. J. Neurosci. 40, 3604–3620 (2020).
Sidarta, A., Vahdat, S., Bernardi, N. F. & Ostry, D. J. Somatic and reinforcement-based plasticity in the initial stages of human motor learning. J. Neurosci. 36, 11682–11692 (2016).
Krause, M. R., Vieira, P. G., Csorba, B. A., Pilly, P. K. & Pack, C. C. Transcranial alternating current stimulation entrains single-neuron activity in the primate brain. Proc. Natl Acad. Sci. USA 116, 5747–5755 (2019).
Shirer, W. R., Ryali, S., Rykhlevskaia, E., Menon, V. & Greicius, M. D. Decoding subject-driven cognitive states with whole-brain connectivity patterns. Cereb. Cortex 22, 158–165 (2012).
Morishita, T. & Hummel, F. C. Non-invasive Brain Stimulation (NIBS) in motor recovery after stroke: concepts to increase efficacy. Curr. Behav. Neurosci. Rep. 4, 280–289 (2017).
Donnelly, N. A. et al. Oscillatory activity in the medial prefrontal cortex and nucleus accumbens correlates with impulsivity and reward outcome. PLoS ONE 9, 14–17 (2014).
Schall, T. A., Wright, W. J. & Dong, Y. Nucleus accumbens fast-spiking interneurons in motivational and addictive behaviors. Mol. Psychiatry 26, 234–246 (2021).
Pisansky, M. T. et al. Nucleus accumbens fast-spiking interneurons constrain impulsive action. Biol. Psychiatry 86, 836–847 (2019).
Kirby, K. N., Petry, N. M. & Bickel, W. K. Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. J. Exp. Psychol. Gen. 128, 78–87 (1999).
Mitchell, J. M., Fields, H. L., D’Esposito, M. & Boettiger, C. A. Impulsive responding in alcoholics. Alcohol. Clin. Exp. Res. 29, 2158–2169 (2005).
Catanese, J., Carmichael, J. E. & van der Meer, M. A. A. Low- and high-gamma oscillations deviate in opposite directions from zero-phase synchrony in the limbic corticostriatal loop. J. Neurophysiol. 116, 5–17 (2016).
Rothé, M., Quilodran, R., Sallet, J. & Procyk, E. Coordination of high gamma activity in anterior cingulate and lateral prefrontal cortical areas during adaptation. J. Neurosci. 31, 11110–11117 (2011).
Del Arco, A., Park, J., Wood, J., Kim, Y. & Moghaddam, B. Adaptive encoding of outcome prediction by prefrontal cortex ensembles supports behavioral flexibility. J. Neurosci. 37, 8363–8373 (2017).
Yoshimoto, A., Shibata, Y., Kudara, M., Ikegaya, Y. & Matsumoto, N. Enhancement of motor cortical gamma oscillations and sniffing activity by medial forebrain bundle stimulation precedes locomotion. eNeuro 9, ENEURO.0521-21.2022 (2022).
Grover, S., Nguyen, J. A., Viswanathan, V. & Reinhart, R. M. G. High-frequency neuromodulation improves obsessive–compulsive behavior. Nat. Med. 27, 232–238 (2021).
Krause, M. R., Vieira, P. G., Thivierge, J. P. & Pack, C. C. Brain stimulation competes with ongoing oscillations for control of spike timing in the primate brain. PLoS Biol. 20, 5 (2022).
Courtemanche, R., Fujii, N. & Graybiel, A. M. Synchronous, focally modulated β-band oscillations characterize local field potential activity in the striatum of awake behaving monkeys. J. Neurosci. 23, 11741–11752 (2003).
Costa, R. M. et al. Rapid alterations in corticostriatal ensemble coordination during acute dopamine-dependent motor dysfunction. Neuron 52, 359–369 (2006).
Engel, A. K. & Fries, P. Beta-band oscillations—signalling the status quo? Curr. Opin. Neurobiol. 20, 156–165 (2010).
Uehara, S., Mawase, F. & Celnik, P. Learning similar actions by reinforcement or sensory-prediction errors rely on distinct physiological mechanisms. Cereb. Cortex 28, 3478–3490 (2018).
Mathis, M. W., Mathis, A. & Uchida, N. Somatosensory cortex plays an essential role in forelimb motor adaptation in mice. Neuron 93, 1493–1503.e6 (2017).
Brücke, C. et al. Scaling of movement is related to pallidal γ oscillations in patients with dystonia. J. Neurosci. 32, 1008–1019 (2012).
Soderstrom, N. C. & Bjork, R. A. Learning versus performance: an integrative review. Perspect. Psychol. Sci. 10, 176–199 (2015).
Spampinato, D. A., Satar, Z. & Rothwell, J. C. Combining reward and M1 transcranial direct current stimulation enhances the retention of newly learnt sensorimotor mappings. Brain Stimul. https://doi.org/10.1016/j.brs.2019.05.015 (2019).
Shmuelof, L. et al. Overcoming motor ‘forgetting’ through reinforcement of learned actions. J. Neurosci. 32, 14617–14621a (2012).
Dhawale, A. K., Miyamoto, Y. R., Smith, M. A. & Ölveczky, B. P. Adaptive regulation of motor variability. Curr. Biol. 29, 3551–3562.e7 (2019).
Carroll, T. J., McNamee, D., Ingram, J. N. & Wolpert, D. M. Rapid visuomotor responses reflect value-based decisions. J. Neurosci. 39, 3906–3920 (2019).
De Comite, A., Crevecoeur, F. & Lefèvre, P. Reward-dependent selection of feedback gains impacts rapid motor decisions. eNeuro 9, ENEURO.0439-21.2022 (2022).
Codol, O. et al. Sensorimotor feedback loops are selectively sensitive to reward. eLife 12, e81325 (2023).
Vassiliadis, P. & Derosiere, G. Selecting and executing actions for rewards. J. Neurosci. 40, 6474–6476 (2020).
Krakauer, J. W., Hadjiosif, A. M., Xu, J., Wong, A. L. & Haith, A. M. Motor learning. Compr. Physiol. 9, 613–663 (2019).
Quattrocchi, G. et al. Pharmacological dopamine manipulation does not alter reward-based improvements in memory retention during a visuomotor adaptation task. eNeuro 5, ENEURO.0453-17.2018 (2018).
Codol, O., Holland, P. J. & Galea, J. M. The relationship between reinforcement and explicit control during visuomotor adaptation. Sci. Rep. 8, 9121 (2018).
Truong, C. et al. Error-based and reinforcement learning in basketball free throw shooting. Sci. Rep. 13, 499 (2023).
Dayan, E., Hamann, J. M., Averbeck, B. B. & Cohen, L. G. Brain structural substrates of reward dependence during behavioral performance. J. Neurosci. 34, 16433–16441 (2014).
Johnson, L. et al. Dose-dependent effects of transcranial alternating current stimulation on spike timing in awake nonhuman primates. Sci. Adv. 6, 36 (2020).
Beliaeva, V., Savvateev, I., Zerbi, V. & Polania, R. Toward integrative approaches to study the causal role of neural oscillations via transcranial electrical stimulation. Nat. Commun. 12, 2243 (2021).
Averbeck, B. & O’Doherty, J. P. Reinforcement-learning in fronto-striatal circuits. Neuropsychopharmacology 47, 147–162 (2022).
Ma, I. et al. Ventral striatal hyperconnectivity during rewarded interference control in adolescents with ADHD. Cortex 82, 225–236 (2016).
Wang, Q. et al. Dissociated neural substrates underlying impulsive choice and impulsive action. NeuroImage 134, 540–549 (2016).
Mosley, P. E. et al. The structural connectivity of discrete networks underlies impulsivity and gambling in Parkinson’s disease. Brain 142, 3917–3935 (2019).
Hampton, W. H., Alm, K. H., Venkatraman, V., Nugiel, T. & Olson, I. R. Dissociable frontostriatal white matter connectivity underlies reward and motor impulsivity. NeuroImage 150, 336–343 (2017).
Negahbani, E., Kasten, F. H., Herrmann, C. S. & Fröhlich, F. Targeting alpha-band oscillations in a cortical model with amplitude-modulated high-frequency transcranial electric stimulation. NeuroImage 173, 3–12 (2018).
Cohen, J. Statistical Power Analysis for the Behavioral Sciences 13 (L. Erlbaum Associates, 1988).
Hashemirad, F., Zoghi, M., Fitzgerald, P. B. & Jaberzadeh, S. The effect of anodal transcranial direct current stimulation on motor sequence learning in healthy individuals: a systematic review and meta-analysis. Brain Cogn. 102, 1–12 (2016).
Soutschek, A., Kang, P., Ruff, C. C., Hare, T. A. & Tobler, P. N. Brain stimulation over the frontopolar cortex enhances motivation to exert effort for reward. Biol. Psychiatry 84, 38–45 (2018).
Wischnewski, M., Zerr, P. & Schutter, D. J. L. G. Effects of theta transcranial alternating current stimulation over the frontal cortex on reversal learning. Brain Stimul. 9, 705–711 (2016).
Guerra, A., López-Alonso, V., Cheeran, B. & Suppa, A. Variability in non-invasive brain stimulation studies: reasons and results. Neurosci. Lett. 719, 133330 (2020).
Cassarà, A. M. et al. Safety recommendations for temporal interference stimulation in the brain. Preprint at bioRxiv https://doi.org/10.1101/2022.12.15.520077 (2022).
Oldfield, R. C. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97–113 (1971).
Kaplan, B. A. et al. Automating scoring of delay discounting for the 21- and 27-item Monetary Choice Questionnaires. Behav. Anal. 39, 293–304 (2016).
Mitchell, M. R. & Potenza, M. N. Recent insights into the neurobiology of impulsivity. Curr. Addict. Rep. 1, 309–319 (2014).
Clemente, F., D’Alonzo, M., Controzzi, M., Edin, B. B. & Cipriani, C. Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 24, 1314–1322 (2016).
Bernardi, N. F., Darainy, M. & Ostry, D. J. Somatosensory contribution to the initial stages of human motor learning. J. Neurosci. 35, 14316–14326 (2015).
Hardwick, R. M., Rajan, V. A., Bastian, A. J., Krakauer, J. W. & Celnik, P. A. Motor learning in stroke. Neurorehabil. Neural Repair 31, 178–189 (2017).
Raspopovic, S. et al. Restoring natural sensory feedback in real-time bidirectional hand prostheses. Sci. Transl. Med. 6, 222ra19 (2014).
Brainard, D. H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).
Pelli, D. G. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 10, 437–442 (1997).
Dayan, E., Averbeck, B. B., Richmond, B. J. & Cohen, L. G. Stochastic reinforcement benefits skill acquisition. Learn. Mem. 21, 140–142 (2014).
Grossman, N. Modulation without surgical intervention. Science 361, 461–462 (2018).
Antal, A. et al. Low intensity transcranial electric stimulation: safety, ethical, legal regulatory and application guidelines. Clin. Neurophysiol. 128, 1774–1809 (2017).
Ekhtiari, H. et al. A checklist for assessing the methodological quality of concurrent tES–fMRI studies (ContES checklist): a consensus study and statement. Nat. Protoc. 17, 596–617 (2022).
Bossetti, C. A., Birdno, M. J. & Grill, W. M. Analysis of the quasi-static approximation for calculating potentials generated by neural stimulation. J. Neural Eng. 5, 44–53 (2008).
Hasgall, P. et al. IT’IS Database for Thermal and Electromagnetic Parameters of Biological Tissues v.4.1 (IT’IS Foundation, 2022); https://doi.org/10.13099/VIP21000-04-1
Seeck, M. et al. The standardized EEG electrode array of the IFCN. Clin. Neurophysiol. 128, 2070–2077 (2017).
R Core Team (2021). R: A language and environment for statistical computing. (R Foundation for Statistical Computing, 2021). https://www.R-project.org/
Bates, D., Mächler, M., Bolker, B. M. & Walker, S. C. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015).
Ryu, E. Effects of skewness and kurtosis on normal-theory based maximum likelihood test statistic in multilevel structural equation modeling. Behav. Res. Methods 43, 1066–1074 (2011).
Nieuwenhuis, R., te Grotenhuis, M. & Pelzer, B. Influence.ME: tools for detecting influential data in mixed effects models. R J. 4, 38–47 (2012).
Luke, S. G. Evaluating significance in linear mixed-effects models in R. Behav. Res. Methods 49, 1494–1502 (2017).
Searle, S. R., Speed, F. M. & Milliken, G. A. Population marginal means in the linear model: an alternative to least squares means. Am. Stat. 34, 216–221 (1980).
Lenth, R. emmeans: Estimated marginal means, aka least-squares means. R package version 1.10.1 https://CRAN.R-project.org/package=emmeans (2023).
Ben-Shachar, M., Lüdecke, D. & Makowski, D. effectsize: estimation of effect size indices and standardized parameters. J. Open Source Softw. 5, 2815 (2020).
Derosière, G., Billot, M., Ward, E. T. & Perrey, S. Adaptations of motor neural structures’ activity to lapses in attention. Cereb. Cortex 25, 66–74 (2015).
Okamoto, M. et al. Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10–20 system oriented for transcranial functional brain mapping. NeuroImage 21, 99–111 (2004).
Friston, K. J. Functional and effective connectivity in neuroimaging: A synthesis. Hum. Brain. Mapp. 2, 56–78 (1994).
Di, X., Zhang, Z. & Biswal, B. B. Understanding psychophysiological interaction and its relations to beta series correlation. Brain Imaging Behav. 15, 958–973 (2021).
Bowles, S. et al. Vagus nerve stimulation drives selective circuit modulation through cholinergic reinforcement. Neuron 110, 2867–286 (2022).
Codol, O., Galea, J. M., Jalali, R. & Holland, P. J. Reward-driven enhancements in motor control are robust to TMS manipulation. Exp. Brain Res. 238, 1781–1793 (2020).
Draganski, B. et al. Evidence for segregated and integrative connectivity patterns in the human basal ganglia. J. Neurosci. 28, 7143–7152 (2008).
Morris, L. S. et al. Fronto-striatal organization: defining functional and microstructural substrates of behavioural flexibility. Cortex 74, 118–133 (2016).
Vassiliadis, P. et al. Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills—UPHUMMEL—EPFL. Zenodo https://doi.org/10.5281/zenodo.10458885 (2024).
Acknowledgements
P.V. was a PhD student supported by the Fund for Research training in Industry and Agriculture (FRIA/FNRS; FC29690) and grants by the Platform for Education and Talent (Gustave Boël—Sofina Fellowships) and Wallonie-Bruxelles International. J.D. was supported by the Belgian FNRS (T008219F). G.D. was a post-doctoral fellow supported by the Belgian FNRS (1B134.18). The research was partially funded by grants from the Novartis Research Foundation—FreeNovation (Basel, Switzerland) to M.J.W. and E.N., the Bertarelli Foundation (Catalyst ‘Deep-MCI-T’, Gstaad, Switzerland) to F.C.H., the SNSF Lead Agency (NiBS-iCog 320030L_197899) to F.C.H., the ‘Personalized Health and Related Technologies’ (PHRT#2017-205) of the ETH Domain to F.C.H. and the Defitech Foundation (Morges, Switzerland) to F.C.H. This work has also been supported by the Social and hUman ceNtered XR (SUN) project, which has received funding from the Horizon Europe Research and Innovation Programme under grant agreement no. 101092612. The views and opinions expressed in this paper are those of the authors only and do not necessarily reflect those of the European Union. Neither the European Union nor the European Commission can be held responsible for them. We acknowledge access to and expertise of the Neuromodulation and the MRI facilities of the Human Neuroscience Platform of the Campus Biotech Geneva and of the MRI Platform of the HVS (Sion). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript We also thank L. Cohen for insightful feedback on a previous version of this manuscript.
Author information
Authors and Affiliations
Contributions
Conceptualization: P.V., E.B., T.P., J.D., G.D., M.J.W. and F.C.H. Methodology: P.V., E.B., T.P., F.W., E.N., M.J.W. and F.C.H. Formal analysis: P.V. and E.B. Investigation: P.V., E.B., F.W. and T.M. Resources: F.C.H. Data curation: P.V., E.B. and F.W. Writing original draft/visualization: P.V. and E.B. Review and editing: all authors. Supervision: F.C.H. Funding: P.V., T.P., E.N., J.D., G.D., M.J.W. and F.C.H.
Corresponding author
Ethics declarations
Competing interests
E.N. is co-founder of TI Solutions AG, a company committed to producing hardware and software solutions to support tTIS research. The remaining authors declare no competing interests.
Peer review
Peer review information
Nature Human Behaviour thanks Pablo Celnik, Florian Kasten and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Figs. 1–6 and Tables 1–4.
Supplementary Video 1
Example trials of the motor learning task in the ReinfON condition. Note that only 4 trials of training are shown here (compared with 24 trials per block in the experiment) and 4 trials of post-training (compared with 8 trials per block in the experiment). Here the video was shot outside the MRI machine, similarly as during familiarization.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Vassiliadis, P., Beanato, E., Popa, T. et al. Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills. Nat Hum Behav (2024). https://doi.org/10.1038/s41562-024-01901-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41562-024-01901-z
- Springer Nature Limited
This article is cited by
-
Non-invasive deep brain stimulation: interventional targeting of deep brain areas in neurological disorders
Nature Reviews Neurology (2024)