Independent contributions of noradrenaline to behavioural flexibility and motivation

Among neuromodulatory systems, the noradrenergic system remains one of the least understood. Several theories have pointed out its implication in behavioural flexibility and more recently in motivation, with a strong role in effort processing. Here, we designed a sequential cost/benefit decision task to test the causal role of noradrenaline in these two functions. We manipulated noradrenaline using clonidine, an alpha-2 noradrenergic receptor agonist, which reduces central noradrenaline levels. Clonidine had two distinct effects: it decreased choice volatility (without affecting the cost/benefit trade off) and reduced force production. Because the effects were independent, they cannot be accounted for by a non-specific effect on arousal. Altogether, these results support the global implication of noradrenaline in facing challenging situations in two complementary ways: by modulating behavioural volatility, which would facilitate adaptation depending on the lability of the environment, and by modulating the mobilization of resources to face immediate challenges.


Introduction 30
Noradrenaline is among the most widespread neuromodulator in the brain and it has 31 been involved in many cognitive processes, but its specific function remains unclear. 32 The noradrenergic system was initially associated with vigilance and arousal (Kety, 33 1972;Harley, 1987

Effect of clonidine on reaction times 186
We next evaluated the effects of clonidine on reaction time across task conditions. 187 We separated conditions where monkeys had to make a choice between two options 188 from conditions where they only squeezed the grip to progress through the trial. As 189 frequently observed, monkeys were slower to respond in choice than no-choice trials 190 4A). We examined the influence of clonidine on reaction times in these two types 191 of trials and a multi-level linear regression taking into account variability across 192 p<0.001) and dose (β=0.070±0.007, t(17)=2.90, p<0.01), but no significant interaction 194 (t(16)=0.13, p=0.89). Hence, clonidine significantly slowed down reaction times, but 195 its effects were undistinguishable between choice and non-choice conditions. We 196 also separated choice reaction times according to two levels of choice difficulty 197 (depending on whether the dimensions to integrate to make the choice were 198 congruent or not: hard and easy choices, respectively) and found a significant linear 199 After assessing the implication of noradrenaline in behavioural flexibility, we 224 examined the causal role of noradrenaline in motivation in this task. For that, we 225 examined the influence of clonidine on two behavioural measures that are classically 226 used to assess motivation, willingness to work and physical force production ( fig 5). 227 We measured monkeys' willingness to work by counting the proportion of accepted 228 squeezes. Since the action is very easy, monkeys never failed to complete a 229 squeeze if they tried to, the number of squeezes that they accept to perform directly 230 reflects their motivation to complete the trial. First, the willingness to work during 1-231 hour-long sessions was not significantly affected by dose (linear regression taking clonidine did not have a global effect on the monkeys' engagement in the task. 234 We also examined the effect of clonidine on the animals' ability to adjust their 235 behaviour across levels of progression through the sequences. We first examined 236 willingness to work for the first squeeze of all sequences. We included every trial, 237 since there was no way for monkeys to predict at the start of a sequence if a choice 238 was going to be offered later in that sequence. We again examined the influence of 239 reward size and sequence length on the willingness to perform the first squeeze 240 In short, monkeys displayed robust adjustments of their willingness to produce 266 the action across conditions, defined by the distance to reward and the amount of 267 expected reward at the end of the sequence, but this was unaffected by clonidine. 268 This implies that clonidine did not affect the monkeys' general willingness to work, 269 ruling out a non-specific effect on arousal. had a global, non-specific effect on arousal or vigilance, which would be responsible for both longer reaction times and smaller force peaks. Such a scenario implies a 279 strong relation between the effects of clonidine on force peak and reaction time. We 280 first compared the effect on force peak and reaction time in a 2-way ANOVA with 281 factors dose and measure (force peak vs. reaction time). We found a significant main 282 effect of dose (F(3, 16)=5.25, p=0.01), measure (F(1,16)=4.72, p=0.04), but 283 importantly also a significant interaction between the two (F(3, 16)=8.59, p=0.001), 284 indicating that the effect of clonidine differs significantly between the two measures. 285 We furthered this analysis using a linear regression ( fig 4D): there was no reliable 286 correlation between the two measures (r=-0.4, p=0.19). Moreover, we ran separate 287 linear regressions without the highest dose, we found that the effect on force peak 288 was still significant (t(7)=-3.93, p=0.005) whereas it was not the case for reaction 289 times (t(7)=0.16, p=0.88), implying that the linear effect was mostly due to the last 290 dose. We also found a greater linear effect of dose on (β=-0.222±0.068) than on 291 reaction times (β=0.106±0.040). Overall, the analyses imply that clonidine had a 292 greater impact of effort production than willingness to work and reaction times. 293 Finally we found no significant correlation between the consistency parameter 294 and the force peak, (t(10)=-1.39, p=0.15). This shows that the effect of clonidine on 295 force production and decision-making ("speed-consistency trade-off") are 296 independent and probably not due to a global effect on arousal/vigilance.

Figure 5: Effects of clonidine on motivation 299
In the present work, we used a novel decision-making task, which required monkeys 301 to make sequential actions for reward and, on a majority of trials, to choose whether 302 to stick with the current sequence or to switch to an alternative based on the costs 303 and benefits of the options. This task allowed us to independently evaluate two Optimal economic theory stipulates that the behaviour is optimal when there is 337 no noise, meaning that the choices follow exactly the values of the options. In that 338 sense, clonidine would seem to make monkeys more optimal since it increases 339 choice consistency. But this absence of noise is only optimal in a constant  Thus, the effect of clonidine causing reduced force production seems to be 382 relatively specific to the action requirements, but did not affect the overall cost-benefit 383 analysis. Indeed, neither the initial choice to engage with the sequence, nor the 384 binary choice in the middle of the sequence to stick with the initial option or to switch 385 to the novel alternative were affected by the treatment as shown in the model-based 386 analysis. At first, this might appear surprising since squeezing the grip multiple times 387 could be taken as an effort. But since the minimal force to validate a squeeze was 388 very small and monkeys always succeeded to reach it if they initiated the action, 389 physical effort is unlikely to be a major component of the cost in this task. Moreover, 390 previous studies using a similar task suggest that some monkeys could treat 391 sequences as delay, and neglect the motor cost relative to the temporal discounting A key question for future studies will be to directly contrast the precise roles these 408 neurotransmitters play in effort-based decision-making. 409 Last, we found that the effects on force production were not correlated with 410 choice consistency. Hence the two effects were independent, further ruling out an 411 interpretation in terms of global, low level process such as arousal or vigilance. This 412 is probably due to the action of clonidine on different networks. It has recently been 413 shown different populations of LC neurons project to the prefrontal cortex and the 414 motor cortex (Chandler et al., 2014). These two distinct networks could underlie the the underlying neurobiological mechanisms, this work demonstrates that these two 417 facets of noradrenergic functions are relatively independent, and specific. In other 418 words, the implication of noradrenaline in cognition and behavior cannot be reduced 419 to arousal or vigilance, even if LC activity strongly correlates with autonomic arousal. 420 One of the upcoming challenges will be to understand the neuronal mechanisms 421 underlying these specific operations, and how they articulate with functions of other 422 neuromodulatory systems such as dopamine and serotonine. what is its precise action on its target networks.  The task consisted of performing sequences of squeezes on a grip to obtain appeared simultaneously with a red dot on either the left or right side of the screen 462 (counterbalanced across trials) (Fig 1). There were nine initial options defined by 463 three initial sequence lengths (6, 8 and 10 squeezes) and three reward sizes (small, 464 medium and big). After a fixed delay of 2s, the red dot turned green and to initiate a 465 trial, monkeys had 2s to perform a squeeze above the minimum force threshold with 466 the grip corresponding to the side of the screen where stimuli were displayed. The In 30% of trials, monkeys had no option other than to complete the initial 481 sequence. However, in 70% of trials, monkeys were given the choice during the 482 sequence to take an alternative option (Fig 1A). The alternative option was presented 483 on the opposite side of the screen and occurred at least three squeezes after the alpha-2 noradrenergic receptor agonist that suppresses LC firing and consequently 508 noradrenaline release at the doses that we used ( Motivational changes with treatment were assessed by two variables: force 558 peak and willingness to work. 559 To calculate force peak, force time series for both grips were low-pass filtered 560 at 15 Hz (zero-phase second-order Butterworth filter) and we took the maximal value 561 of the force signal between two crossings of the minimal force threshold. 562 Willingness to work corresponds to the proportion of accepted squeezes per 563 session. Willingness to work at the beginning of each sequence was computed by 564 taking the first squeeze of all trials. Willingness to work across sequences of given 565 length and reward size were estimated by taking all trials where no choice was 566 offered in a given drug condition for each monkey and fitted using the following 567 model: 568 Where n is the number of squeezes done in the sequence. Parameters k(intercept) 569 and k(slope) were estimated for each dose and each monkey using the same procedure as for the parameters of equations 1 and 2. 571 572

Statistical Analysis 573
Data are plotted as mean +/-standard error to the mean. Statistics used are indicated 574 in the Results sections. Comparisons between means were performed using 575 parametric tests (ANOVA and T-test). We performed linear regression on z-scored 576 distributions (reaction times and force peaks) using the function glmfit in Matlab. In 577 cases when distributions were not z-scored, we fitted an intercept for each subject 578 hence taking into account the variability in mean across subjects using the function 579 fitlme in Matlab. The general equation was: 580 where y is the data, ! a constant, ! a constant fitted for each subject, ! 581 the experimental factors and ! their weights in the linear regression. T-tests were 582 performed on weights distributions. In all cases t-values and degrees of freedom are 583 given according to statistical analysis reports of glmfit and fitlme. All statistical tests 584 were two-sided. P > 0.05 was considered to be not statistically significant. We 585 evaluated the quality of our models' fit using balanced accuracy (between 0 and 1) 586 computed as: which was between 0.80 and 0.86 for all fits (Brodersen et al., 2010). 588 We thank Morgan Weissenburger and the animal housing facility staff for their 590 assistance and care of the animals. We also thank Nicolas Borderies for discussion. 591 corresponding to the side where stimuli are displayed, wait when the dot is red, 849 squeeze when it is green and a blue dot indicates a correct squeeze. All squeezes of 850 a sequence must be performed correctly to obtain the reward. A squeeze is incorrect 851 if monkeys do not squeeze above the minimum force threshold when the green dot is 852 displayed, squeeze when the red dot is displayed, or use the wrong grip. After an 853 squeezes to obtain the small reward (bottom). After all squeezes of a sequence are 867 performed correctly, the gauge indicating the remaining number of squeezes in the 868 sequence appears as empty and monkeys receive the fluid reward. After an inter-trial 869 interval of 1 to 1.5s, another trial starts. 870 B) Task structure. Initial sequences start with 6, 8 to 10 squeezes and lead to 3 sizes 871 of reward (small, medium and big). In 30% of trials, no choice is offered and 872 monkeys must perform the initial sequence to be rewarded. In 70% of trials one 873 choice is offered during one of the squeeze in light grey (at least 3 trials after the 874 beginning of a sequence and 3 trials before the end) on the figure. There was no effect of treatment condition. Size of average reward chosen (1 for 879 small, 2 for medium and 3 for big) in each treatment condition. Same as A. There 880 was no effect of treatment condition. Symbols correspond to each subject (circle: 881 monkey A, square: monkey D, triangle: monkey E). There was no significant bias 882 toward staying or switching across all treatment conditions..

B) Correlation between choice reaction times and consistency parameter estimates. 920
Reaction times corresponds to the time between the display of the green dot and the 921 crossing of the minimum force threshold for correct squeezes where monkeys had to 922 make a choice and did not change grip. The consistency parameter was computed 923 by fitting the choice model. They were computed for each subject and treatment 924 condition. Same as A. The correlation between these two parameters was significant