Skip to main content

Effects of average reward rate on vigor as a function of individual variation in striatal dopamine

Abstract

Rationale

We constantly need to decide not only which actions to perform, but also how vigorously to perform them. In agreement with an earlier theoretical model, it has been shown that a significant portion of the variance in our action vigor can be explained by the average rate of rewards received for that action. Moreover, this invigorating effect of average reward rate was shown to vary with within-subject changes in dopamine, both in human individuals and experimental rodents.

Objectives

Here, we assessed whether individual differences in the effect of average reward rate on vigor are related to individual variation in a stable measure of striatal dopamine function in healthy, unmedicated participants.

Methods

Forty-four participants performed a discrimination task to test the effect of average reward rate on response times to index vigor and completed an [18F]-DOPA PET scan to index striatal dopamine synthesis capacity.

Results

We did not find an interaction between dopamine synthesis capacity and average reward rate across the entire group. However, a post hoc analysis revealed that participants with higher striatal dopamine synthesis capacity, particularly in the nucleus accumbens, exhibited a stronger invigorating effect of average reward rate among the 30 slowest participants.

Conclusions

Our findings provide converging evidence for a role of striatal dopamine in average reward rate signaling, thereby extending the current literature on the mechanistic link between average reward rate, vigor, and dopamine.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Data availability

The data and analysis scripts used in this article will be made publicly available after manuscript acceptance at the following web address: https://doi.org/10.34973/kb2p-j456. Prior to accessing and downloading the shared data, users must create an account. It is possible to use an institutional account or a social ID from Google, Facebook, Twitter, LinkedIn, or Microsoft. After authentication, users must accept the Data Use Agreement (DUA); after which, they are automatically authorized to download the shared data. The DUA specifies whether there are any restrictions on how the data may be used. The Radboud University and the Donders Institute for Brain, Cognition and Behaviour will keep these shared data available for at least 10 years.

Notes

  1. Inferences were identical regardless of the exact procedure, including retaining only the 50% or 75% slowest participants, or retaining only the subsample of slower participants which most closely resembled that of the placebo group in Beierholm et al. (2013) in terms of response times (N = 31).

References

  • Barrett SP, Pihl RO, Benkelfat C et al (2008) The role of dopamine in alcohol self-administration in humans: individual differences. Eur Neuropsychopharmacol 18:439–447

    CAS  PubMed  Google Scholar 

  • Beierholm U, Guitart-Masip M, Economides M et al (2013) Dopamine modulates reward-related vigor. Neuropsychopharmacology 38:1495–1503

    CAS  PubMed  PubMed Central  Google Scholar 

  • Berke JD (2018) What does dopamine mean?. Nat Neurosci. Springer, US 21:787–793

    CAS  Google Scholar 

  • Berry AS, Shah VD, Baker SL et al (2016) Aging affects dopaminergic neural mechanisms of cognitive flexibility. J Neurosci 36:12559–12569

    CAS  PubMed  PubMed Central  Google Scholar 

  • Berry AS, Shah VD, Furman DJ et al (2017) Dopamine synthesis capacity is associated with D2/3 receptor binding but not dopamine release. Nat Publ Gr Nature Publishing Group 43:1201–1211

    Google Scholar 

  • Bürkner P-C (2017). Brms: an R package for bayesian multilevel models using Stan. J Stat Softw 80: 1–28. https://doi.org/10.18637/jss.v080.i01.

  • Collins AGE, Frank MJ (2014) Opponent actor learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychol Rev 121:337–366

    PubMed  Google Scholar 

  • Cools R, Nakamura K, Daw ND (2011) Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology Nature Publishing Group 36:98–113

    CAS  Google Scholar 

  • Dardenne K, McClure SM, Nystrom LE et al (2008) BOLD Responses reflecting Dopaminergic signals in the human ventral tegmental area. Science (80-) 319:1264–1267

    CAS  Google Scholar 

  • DeJesus OT, Endres CJ, Shelton SE et al (2001) Noninvasive assessment of aromatic L-amino acid decarboxylase activity in aging rhesus monkey brain in vivo. Synapse 39:58–63

    CAS  PubMed  Google Scholar 

  • Deserno L, Huys QJMQ, Boehme R et al (2015) Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making. Proc Natl Acad Sci 112:1595–1600

    CAS  PubMed  PubMed Central  Google Scholar 

  • Doll BB, Jacobs WJ, Sanfey AG et al (2009) Instructional control of reinforcement learning: a behavioral and neurocomputational investigation. Brain Res 1299:74–94

    CAS  PubMed  PubMed Central  Google Scholar 

  • Egerton A, Demjaha A, McGuire P, et al. (2010). The test-retest reliability of 18F-DOPA PET in assessing striatal and extrastriatal presynaptic dopaminergic function. Neuroimage. Elsevier Inc. 50: 524–531.

  • Grogan JP, Sandhu TR, Hu MT et al (2020) Dopamine promotes instrumental motivation, but reduces reward-related vigour. Elife 9:e58321

    CAS  PubMed  PubMed Central  Google Scholar 

  • Guitart-Masip M, Beierholm UR, Dolan R et al (2011) Vigor in the face of fluctuating rates of reward: an experimental examination. J Cogn Neurosci 23:3933–3938

    PubMed  Google Scholar 

  • Hamid AA, Pettibone JR, Mabrouk OS et al (2016) Mesolimbic dopamine signals the value of work. Nat Neurosci 19:117–126

    CAS  PubMed  Google Scholar 

  • Hayes SC (ed) (1989) Rule-governed behavior: cognition, contingencies, and instructional control. Springer, US

    Google Scholar 

  • du Hoffmann J, Nicola SM (2014) Dopamine invigorates reward seeking by promoting Cue-evoked excitation in the nucleus accumbens. J Neurosci 34:14349–14364

    PubMed  PubMed Central  Google Scholar 

  • Hofmans L, Papadopetraki D, van den Bosch R, et al. (2020) Methylphenidate boosts choices of mental labor over leisure depending on baseline striatal dopamine. Neuropsychopharmacology 45:2170-2179

  • van Holst RJ, Sescousse G, Janssen LK, et al. (2018). Increased striatal dopamine synthesis capacity in gambling addiction. Biol Psychiatry. Elsevier Inc 83: 1036–1043.

  • Howes OD, Egerton A, Allan V et al (2009) Mechanisms underlying psychosis and antipsychotic treatment response in schizophrenia: insights from PET and SPECT imaging. Curr Pharm Des 15:2550–2559

    CAS  PubMed  PubMed Central  Google Scholar 

  • Hübner R, Schlösser J (2010) Monetary reward increases attentional effort in the flanker task. Psychon Bull Rev 17:821–826

    PubMed  Google Scholar 

  • Huys QJM, Browning M, Paulus MP et al (2021) Advances in the computational understanding of mental illness. Neuropsychopharmacology. Springer, US 46:3–19

    Google Scholar 

  • Ikemoto S and Panksepp J (1999). The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Rev 6–41.

  • Ito H, Kodaka F, Takahashi H et al (2011) Relation between presynaptic and postsynaptic dopaminergic functions measured by positron emission tomography: implication of dopaminergic tone. J Neurosci 31:7886–7890

    CAS  PubMed  PubMed Central  Google Scholar 

  • Khalighinejad N, Bongioanni A, Verhagen L, et al. (2020). A basal forebrain-cingulate circuit in macaques decides it is time to act. Neuron. Elsevier Inc. 105: 370–384.e8.

  • Koerts J, Leenders KL, Koning M et al (2007) Striatal dopaminergic activity (FDOPA-PET) associated with cognitive items of a depression scale (MADRS) in Parkinson’s disease. Eur J Neurosci 25:3132–3136

    PubMed  Google Scholar 

  • Kumakura Y, Vernaleken I, Buchholz HG et al (2010) Age-dependent decline of steady state dopamine storage capacity of human brain: an FDOPA PET study. Neurobiol Aging 31:447–463

    CAS  PubMed  Google Scholar 

  • Laakso A, Vilkman H, Bergman J et al (2002) Sex differences in striatal presynaptic dopamine synthesis capacity in healthy subjects. Biol Psychiatry 52:759–763

    CAS  PubMed  Google Scholar 

  • Maia TV, Frank MJ (2011) From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 14:154–162

    CAS  PubMed  PubMed Central  Google Scholar 

  • Manohar SG, Chong TTJ, Apps MAJ et al (2015) Reward pays the cost of noise reduction in motor and cognitive control. Curr Biol Elsevier 25:1707–1716

    CAS  Google Scholar 

  • Mikhael JG, Lai L, Gershman SJ (2021) Rational inattention and tonic dopamine. PLoS Comput Biol 17:e1008659

    CAS  PubMed  PubMed Central  Google Scholar 

  • Mohebi A, Pettibone JR, Hamid AA et al (2019) Dissociable dopamine dynamics for learning and motivation. Nature. Springer, US 570:65–70

    CAS  Google Scholar 

  • Nagano-Saito A, Kato T, Arahata Y et al (2004) Cognitive- and motor-related regions in Parkinson’s disease: FDOPA and FDG PET studies. Neuroimage 22:553–561

    PubMed  Google Scholar 

  • Niv Y, Daw ND, Joel D et al (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191:507–520

    CAS  PubMed  Google Scholar 

  • O’Hara CB, Keyes A, Renwick B et al (2016) The effects of acute dopamine precursor depletion on the reinforcing value of exercise in anorexia nervosa. PLoS ONE 11:1–16

    Google Scholar 

  • Ota M, Yasuno F, Ito H et al (2006) Age-related decline of dopamine synthesis in the living human brain measured by positron emission tomography with l-[β-11C]DOPA. Life Sci 79:730–736

    CAS  PubMed  Google Scholar 

  • Otto AR, Daw ND (2019) The opportunity cost of time modulates cognitive effort. Neuropsychologia Elsevier Ltd 123:92–105

    Google Scholar 

  • Patlak CS, Blasberg RG, Fenstermacher JD (1983) Graphical evaluation of blood-to-brain transfer constants from multiple-time uptake data. Generalizations J Cereb Blood Flow Metab 5:584–590

    Google Scholar 

  • Pavese N, Rivero-Bosch M, Lewis SJ, et al. (2011). Progression of monoaminergic dysfunction in Parkinson’s disease: a longitudinal 18F-dopa PET study. Neuroimage. Elsevier Inc. 56: 1463–1468.

  • Piray P, Den OHEM, Van Der SME et al (2017) Dopaminergic modulation of the Functional ventrodorsal architecture of the human striatum. Cereb Cortex 27:485–495

    PubMed  Google Scholar 

  • R Core Team (2018). R: a language and environment for statistical computing. Vienna, Austria. Available at: https://www.r-project.org/.

  • Rigoli F, Chew B, Dayan P et al (2016) The dopaminergic midbrain mediates an effect of average reward on Pavlovian vigor. J Cogn Neurosci 28:1303–1317

    PubMed  Google Scholar 

  • Robbins TW, Everitt BJ (2007) A role for mesencephalic dopamine in activation: commentary on Berridge (2006). Psychopharmacology 191:433–437

    CAS  PubMed  Google Scholar 

  • Salamone JD, Yohn SE, López-Cruz L, et al. (2016). Activational and effort-related aspects of motivation: neural mechanisms and implications for psychopathology. Brain 1325–1347.

  • Salamone JD, Correa M (2002) Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 137:3–25

    CAS  PubMed  Google Scholar 

  • Sayalı C, Badre D (2019) Neural systems of cognitive demand avoidance Ceyda. Neuropsychologia 123176:41–54

    Google Scholar 

  • Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science (80-) 275:1593–1599

    CAS  Google Scholar 

  • Sokolowski JD, Salamone JD (1998) The role of accumbens dopamine in lever pressing and response allocation: effects of 6-OHDA injected into core and dorsomedial shell. Pharmacol Biochem Behav 59:557–566

    CAS  PubMed  Google Scholar 

  • Sossi V, De F-fernández R, Holden JE et al (2002) Increase in Dopamine turnover occurs early in Parkinson ’ s disease : evidence from a new modeling approach to PET F-fluorodopa data. J Cereb Blood Flow Metab 22:232–239

    CAS  PubMed  Google Scholar 

  • Sossi V, Doudet DJ, Holden JE (2001) A reversible tracer analysis approach to the study of effective dopamine turnover. J Cereb Blood Flow Metab 21:469–476

    CAS  PubMed  Google Scholar 

  • Takikawa Y, Kawagoe R, Itoh H et al (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142:284–291

    PubMed  Google Scholar 

  • Taylor JR, Robbins TW (1984) Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens. Psychopharmacology 84:405–412

    CAS  PubMed  Google Scholar 

  • Taylor JR, Robbins TW (1986) 6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine. Psychopharmacology 90:390–397

    CAS  PubMed  Google Scholar 

  • Venugopalan V V, Casey KF, Hara CO, et al. (2011). Acute phenylalanine/tyrosine depletion reduces motivation to smoke cigarettes across stages of addiction. 2469–2476.

  • Veronese M, Santangelo B, Jauhar S et al (2021) A potential biomarker for treatment stratification in psychosis: evaluation of an [18F] FDOPA PET imaging approach. Neuropsychopharmacology Springer, US 46:1122–1132

    CAS  Google Scholar 

  • Wang Y, Toyoshima O, Kunimatsu J et al (2021) Tonic firing mode of midbrain dopamine neurons continuously tracks reward values changing moment-by-moment. Elife 10:e63166

    CAS  PubMed  PubMed Central  Google Scholar 

  • Westbrook A, van den Bosch R, Määttä JI et al (2020) Dopamine promotes cognitive effort by biasing the benefits versus costs of cognitive work. Science (80-) 367:1362–1366

    CAS  Google Scholar 

  • Westbrook A, Frank MJ, Cools R (2021) A mosaic of cost–benefit control over cortico-striatal circuitry. Trends Cogn Sci Elsevier Ltd 25:710–721

    Google Scholar 

  • Zandbelt B (2017). Slice display. figshare 10.%0A6084/m9.figshare.4742866.

  • Zénon A, Devesse S, Olivier E (2016) Dopamine manipulation affects response vigor independently of opportunity cost. J Neurosci 36:9516–9525

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Jessica Määttä for assistance with project administration and Lieke van Lieshout and Felix Linsen for the assistance during data collection. We thank Ulrik Beierholm and Marc Guitart-Masip for providing code specifying the reward function.

Funding

The work was funded by a Vici grant from the Netherlands Organization for Scientific Research (NWO; Grant No. 453–14-015), an Ammodo Science Award (KNAW), and a James McDonnell scholar award from the James S McDonnell Foundation, all awarded to RC. AW is funded by a National Institute of Mental Health (NIMH) grant (F32MH115600).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lieke Hofmans.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hofmans, L., Westbrook, A., van den Bosch, R. et al. Effects of average reward rate on vigor as a function of individual variation in striatal dopamine. Psychopharmacology 239, 465–478 (2022). https://doi.org/10.1007/s00213-021-06017-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00213-021-06017-0

Keywords

  • Dopamine
  • Average reward rate
  • Opportunity cost
  • Vigor
  • PET
  • Individual differences