Abstract
Anhedonia – a common feature of depression and other neuropsychiatric disorders – encompasses a reduction in the subjective experience and anticipation of rewarding events, and a reduction in the motivation to seek out such events. The presence of anhedonia often predicts or accompanies treatment resistance, and as such better interventions and treatments are important. Yet the mechanisms giving rise to anhedonia are not well understood. In this chapter, we briefly review existing computational conceptualisations of anhedonia. We argue that they are mostly descriptive and fail to provide an explanatory account of why anhedonia may occur. Working within the framework of reinforcement learning, we examine two potential computational mechanisms that could give rise to anhedonic phenomena. First, we show how anhedonia can arise in multi-dimensional drive-reduction settings through a trade-off between different rewards or needs. We then generalise this in terms of model-based value inference and identify a key role for associational belief structure. We close with a brief discussion of treatment implications of both of these conceptualisations. In summary, computational accounts of anhedonia have provided a useful descriptive framework. Recent advances in reinforcement learning suggest promising avenues by which the mechanisms underlying anhedonia may be teased apart, potentially motivating novel approaches to treatment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alloy LB, Abramson LY, Whitehouse WG, Hogan ME, Tashman NA, Steinberg DL, Rose DT, Donovan P (1999) Depressogenic cognitive styles: predictive validity, information processing and personality characteristics, and developmental origins. Behav Res Ther 37(6):503–531
American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders: DSM-5, 5th edn. Author, Washington
Amsterdam JD, Settle RG, Doty RL, Abelman E, Winokur A (1987) Taste and smell perception in depression. Biol Psychiatry 22(12):1481–1485
Atanasova B, El-Hage W, Chabanet C, Gaillard P, Belzung C, Camus V (2010) Olfactory anhedonia and negative olfactory alliesthesia in depressed patients. Psychiatry Res 176(2–3):190–196. https://doi.org/10.1016/j.psychres.2008.11.016
Beevers CG, Worthy DA, Gorlick MA, Nix B, Chotibut T, Maddox WT (2013) Influence of depression symptoms on history-independent reward and punishment processing. Psychiatry Res 207:53–60. https://doi.org/10.1016/j.psychres.2012.09.054
Bellman RE (1957) Dynamic programming. Princeton University Press
Berlin I, Givry-Steiner L, Lecrubier Y, Puech AJ (1998) Measures of anhedonia and hedonic responses to sucrose in depressive and schizophrenic patients in comparison with healthy subjects. Eur Psychiatry 13(6):303–309. https://doi.org/10.1016/S0924-9338(98)80048-5
Berridge KC (2004) Motivation concepts in behavioral neuroscience. Physiol Behav 81(2):179–209. https://doi.org/10.1016/j.physbeh.2004.02.004. https://www.sciencedirect.com/science/article/pii/S0031938404000435
Berridge KC (2012) From prediction error to incentive salience: mesolimbic computation of reward motivation. Eur J Neurosci 35(7):1124–1143. https://doi.org/10.1111/j.1460-9568.2012.07990.x
Blanco NJ, Otto AR, Maddox WT, Beevers CG, Love BC (2013) The influence of depression symptoms on exploratory decision-making. Cognition 129:563–568. https://doi.org/10.1016/j.cognition.2013.08.018
Bolles RC (1980) Some functionalist thoughts about regulation. In: Analysis of motivational processes. Academic Press, New York, pp 63–75
Bylsma LM, Morris BH, Rottenberg J (2008) A meta-analysis of emotional reactivity in major depressive disorder. Clin Psychol Rev 28(4):676–691. https://doi.org/10.1016/j.cpr.2007.10.001
Chase HW, Michael A, Bullmore ET, Sahakian BJ, Robbins TW (2010) Paradoxical enhancement of choice reaction time performance in patients with major depression. J Psychopharmacol 24(4):471–479. https://doi.org/10.1177/0269881109104883
Clepce M, Gossler A, Reich K, Kornhuber J, Thuerauf N (2010) The relation between depression, anhedonia and olfactory hedonic estimates – a pilot study in major depression. Neurosci Lett 471(3):139–143. https://doi.org/10.1016/j.neulet.2010.01.027
Colle R, El Asmar K, Verstuyft C, Lledo PM, Lazarini F, Chappell K, Deflesselle E, Ait Tayeb AEK, Falissard B, Duron E, Rotenberg S, Costemale-Lacoste JF, David DJ, Gressier F, Gardier AM, Hummel T, Becquemont L, Corruble E (2020) The olfactory deficits of depressed patients are restored after remission with venlafaxine treatment. Psychol Med:1–9. https://doi.org/10.1017/S0033291720003918
Craske MG, Meuret AE, Ritz T, Treanor M, Dour H, Rosenfield D (2019) Positive affect treatment for depression and anxiety: a randomized clinical trial for a core feature of anhedonia. J Consult Clin Psychol 87(5):457–471. https://doi.org/10.1037/ccp0000396
Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8(12):1704–1711. https://doi.org/10.1038/nn1560
Dayan P (1993) Improving generalization for temporal difference learning: the successor representation. Neural Comput 5(4):613–624
Dichter GS, Smoski MJ, Kampov-Polevoy AB, Gallop R, Garbutt JC (2010) Unipolar depression does not moderate responses to the sweet taste test. Depress Anxiety 27(9):859–863. https://doi.org/10.1002/da.20690
Dunn BD, Widnall E, Reed N, Owens C, Campbell J, Kuyken W (2019) Bringing light into darkness: a multiple baseline mixed methods case series evaluation of augmented depression therapy (ADepT). Behav Res Ther 120(103):418. https://doi.org/10.1016/j.brat.2019.103418
Eckstrand KL, Forbes EE, Bertocci MA, Chase HW, Greenberg T, Lockovich J, Stiffler R, Aslam HA, Graur S, Bebko G, Phillips ML (2019) Anhedonia reduction and the association between left ventral striatal reward response and 6-month improvement in life satisfaction among young adults. JAMA Psychiatry 76:958–965. https://doi.org/10.1001/jamapsychiatry.2019.0864
Geschwind N, Arntz A, Bannink F, Peeters F (2019) Positive cognitive behavior therapy in the treatment of depression: a randomized order within-subject comparison with traditional cognitive behavior therapy. Behav Res Ther 116:119–130. https://doi.org/10.1016/j.brat.2019.03.005. https://www.sciencedirect.com/science/article/pii/S0005796719300464
Geugies H, Mocking RJT, Figueroa CA, Groot PFC, Marsman JBC, Servaas MN, Steele JD, Schene AH, Ruh’e HG (2019) Impaired reward-related learning signals in remitted unmedicated patients with recurrent depression. Brain 142:2510–2522, https://doi.org/10.1093/brain/awz167
Greenberg T, Chase HW, Almeida JR, Stiffler R, Zevallos CR, Aslam HA, Deckersbach T, Weyandt S, Cooper C, Toups M, Carmody T, Kurian B, Peltier S, Adams P, McInnis MG, Oquendo MA, McGrath PJ, Fava M, Weissman M, Parsey R, Trivedi MH, Phillips ML (2015) Moderation of the relationship between reward expectancy and prediction error-related ventral striatal reactivity by anhedonia in unmedicated major depressive disorder: findings from the embarc study. Am J Psychiatry 172:881–891. https://doi.org/10.1176/appi.ajp.2015.14050594
Herzallah MM, Moustafa AA, Natsheh JY, Abdellatif SM, Taha MB, Tayem YI, Sehwail MA, Amleh I, Petrides G, Myers CE, Gluck MA (2013) Learning from negative feedback in patients with major depressive disorder is attenuated by SSRI antidepressants. Front Integr Neurosci 7. https://doi.org/10.3389/fnint.2013.00067. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3779792/
Hull CL (1943) Principles of behavior: an introduction to behavior theory. Principles of behavior: an introduction to behavior theory. Appleton-Century, Oxford. x-422
Husain M, Roiser JP (2018) Neuroscience of apathy and anhedonia: a transdiagnostic approach. Nat Rev Neurosci 19(8):470–484. https://doi.org/10.1038/s41583-018-0029-9. https://www.nature.com/articles/s41583-018-0029-9
Huys QJM, Dayan P (2009) A Bayesian formulation of behavioral control. Cognition 113(3):314–328. https://doi.org/10.1016/j.cognition.2009.01.008
Huys QJM, Renz D (2017) A formal valuation framework for emotions and their control. Biol Psychiatry 82:413–420. https://doi.org/10.1016/j.biopsych.2017.07.003
Huys QJM, Eshel N, O’Nions E, Sheridan L, Dayan P, Roiser JP (2012) Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput Biol 8(3):e1002410. https://doi.org/10.1371/journal.pcbi.1002410
Huys QJM, Pizzagalli DA, Bogdan R, Dayan P (2013) Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol Mood Anx Dis 3(1):12. https://doi.org/10.1186/2045-5380-3-12
Jackson JC, Watts J, Henry TR, List JM, Forkel R, Mucha PJ, Greenhill SJ, Gray RD, Lindquist KA (2019) Emotion semantics show both cultural variation and universal structure. Science 366(6472):1517–1522. https://doi.org/10.1126/science.aaw8160
Juechems K, Summerfield C (2019) Where does value come from? Trends Cogn Sci 23(10):836–850. https://doi.org/10.1016/j.tics.2019.07.012
Juechems K, Balaguer J, Castañón SH, Ruz M, O’Reilly JX, Summerfield C (2019) A network for computing value equilibrium in the human medial prefrontal cortex. Neuron 101(5):977–987.e3. https://doi.org/10.1016/j.neuron.2018.12.029. https://www.sciencedirect.com/science/article/pii/S0896627318311498
Keramati M, Gutkin B (2014) Homeostatic reinforcement learning for integrating reward collection and physiological stability. Elife 3:e04811. https://doi.org/10.7554/eLife.04811
Knutson B, Bhanji JP, Cooney RE, Atlas LY, Gotlib IH (2008) Neural responses to monetary incentives in major depression. Biol Psychiatry 63(7):686–692. https://doi.org/10.1016/j.biopsych.2007.07.023
Kohli P, Soler ZM, Nguyen SA, Muus JS, Schlosser RJ (2016) The association between olfaction and depression: a systematic review. Chem Senses 41(6):479–486. https://doi.org/10.1093/chemse/bjw061. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4918728/
Krajbich I (2019) Accounting for attention in sequential sampling models of decision making. Curr Opin Psychol 29:6–11. https://doi.org/10.1016/j.copsyc.2018.10.008
Kumar P, Waiter G, Ahearn T, Milders M, Reid I, Steele JD (2008) Abnormal temporal difference rewardlearning signals in major depression. Brain 131(Pt 8):2084–2093. https://doi.org/10.1093/brain/awn136
Kunisato Y, Okamoto Y, Ueda K, Onoda K, Okada G, Yoshimura S, Si S, Samejima K, Yamawaki S (2012) Effects of depression on reward-based decision making and variability of action in probabilistic learning. J Behav Ther Exp Psychiatry 43:1088–1094. https://doi.org/10.1016/j.jbtep.2012.05.007
Lombion-Pouthier S, Vandel P, Nezelof S, Haffen E, Millot JL (2006) Odor perception in patients with mood disorders. J Affect Disord 90:187–191. https://doi.org/10.1016/j.jad.2005.11.012
Maier SF, Watkins LR (2005) Stressor controllability and learned helplessness: the roles of the dorsal raphe nucleus, serotonin, and corticotropin-releasing factor. Neurosci Biobehav Rev 29(4–5):829–841. https://doi.org/10.1016/j.neubiorev.2005.03.021
Martell CR, Dimidjian S, Herman-Dunn R (2010) Behavioral activation for depression: a clinician’s guide. Guilford Press, p xvi-220
McMakin DL, Olino TM, Porta G, Dietz LJ, Emslie G, Clarke G, Wagner KD, Asarnow JR, Ryan ND, Birmaher B, Shamseddeen W, Mayes T, Kennard B, Spirito A, Keller M, Lynch FL, Dickerson JF, Brent DA (2012) Anhedonia predicts poorer recovery among youth with selective serotonin reuptake inhibitor treatment-resistant depression. J Am Acad Child Adolesc Psychiatry 51(4):404–411. https://doi.org/10.1016/j.jaac.2012.01.011
Nair A, Niyogi RK, Shang F, Tabrizi SJ, Rees G, Rutledge R (2020) Opportunity cost determines action initiation latency and predicts apathy. psyArxiv. https://psyarxiv.com/d9pgz
Nassar MR, Frank MJ (2016) Taming the beast: extracting generalizable knowledge from computational models of cognition. Curr Opin Behav Sci 11:49–54. https://doi.org/10.1016/j.cobeha.2016.04.003
Naudin M, El-Hage W, Gomes M, Gaillard P, Belzung C, Atanasova B (2012) State and trait olfactory markers of major depression. PLoS One 7(10):e46938. https://doi.org/10.1371/journal.pone.0046938. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0046938
Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) 191(3):507–520. https://doi.org/10.1007/s00213-006-0502-4
Pause BM, Miranda A, Goder R, Aldenhoff JB, Ferstl R (2001) Reduced olfactory performance in patients with major depression. J Psychiatr Res 35:271–277. https://doi.org/10.1016/s0022-3956(01)00029-2
Pizzagalli DA, Holmes AJ, Dillon DG, Goetz EL, Birk JL, Bogdan R, Dougherty DD, Iosifescu DV, Rauch SL, Fava M (2009) Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with major depressive disorder. Am J Psychiatry 166(6):702–710. https://doi.org/10.1176/appi.ajp.2008.08081201
Pizzagalli DA, Smoski M, Ang YS, Whitton AE, Sanacora G, Mathew SJ, Nurnberger J, Lisanby SH, Iosifescu DV, Murrough JW, Yang H, Weiner RD, Calabrese JR, Goodman W, Potter WZ, Krystal AD (2020) Selective kappa-opioid antagonism ameliorates anhedonic behavior: evidence from the fast-fail trial in mood and anxiety spectrum disorders (fast-mas). Neuropsychopharmacology 45(10):1656–1663. https://doi.org/10.1038/s41386-020-0738-4
Rescorla R, Wagner A (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99
Rizvi SJ, Pizzagalli DA, Sproule BA, Kennedy SH (2016) Assessing anhedonia in depression: potentials and pitfalls. Neurosci Biobehav Rev 65:21–35. https://doi.org/10.1016/j.neubiorev.2016.03.004. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4856554/
Robinson OJ, Chase HW (2017) Learning and choice in mood disorders: searching for the computational parameters of anhedonia. Comput Psychiatr 1:208–233. https://doi.org/10.1162/CPSY_a_00009
Rupprechter S, Stankevicius A, Huys QJM, Series P, Steele JD (2020) Abnormal reward valuation and event-related connectivity in unmedicated major depressive disorder. Psychol Med:1–9. https://doi.org/10.1017/s0033291719003799
Russek EM, Momennejad I, Botvinick MM, Gershman SJ, Daw ND (2017) Predictive representations can link model-based reinforcement learning to model-free mechanisms. PLoS Comput Biol 13:e1005768. https://doi.org/10.1371/journal.pcbi.1005768
Rutledge RB, Moutoussis M, Smittenaar P, Zeidman P, Taylor T, Hrynkiewicz L, Lam J, Skandali N, Siegel JZ, Ousdal OT, Prabhu G, Dayan P, Fonagy P, Dolan RJ (2017) Association of neural and emotional impacts of reward prediction errors with major depression. JAMA Psychiatry 74:790–797. https://doi.org/10.1001/jamapsychiatry.2017.1713
Seligman ME, Maier SF (1967) Failure to escape traumatic shock. J Exp Psychol 74(1):1–9
Spijker J, Bijl RV, de Graaf R, Nolen WA (2001) Determinants of poor 1-year outcome of DSM-III-R major depression in the general population: results of the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Acta Psychiatr Scand 103(2):122–130. https://doi.org/10.1034/j.1600-0447.2001.103002122.x
Steele JD, Kumar P, Ebmeier KP (2007) Blunted response to feedback information in depressive illness. Brain 130(Pt 9):2367–2374. https://doi.org/10.1093/brain/awm150
Sutton RS, Barto AG (2017) Reinforcement learning, 2nd edn. MIT Press, Cambridge. http://www.cs.ualberta.ca/sutton/book/the-book.html
Swiecicki L, Zatorski P, Bzinkowska D, Sienkiewicz-Jarosz H, Szyndler J, Scinska A (2009) Gustatory and olfactory function in patients with unipolar and bipolar depression. Prog Neuropsychopharmacol Biol Psychiatry 33(5):827–834. https://doi.org/10.1016/j.pnpbp.2009.03.030
Treadway MT, Zald DH (2011) Reconsidering anhedonia in depression: lessons from translational neuroscience. Neurosci Biobehav Rev 35(3):537–555. https://doi.org/10.1016/j.neubiorev.2010.06.006
Trøstheim M, Eikemo M, Meir R, Hansen I, Paul E, Kroll SL, Garland EL, Leknes S (2020) Assessment of anhedonia in adults with and without mental illness: a systematic review and meta-analysis. JAMA Netw Open 3(8):e2013233. https://doi.org/10.1001/jamanetworkopen.2020.13233
Uher R, Perlis RH, Henigsberg N, Zobel A, Rietschel M, Mors O, Hauser J, Dernovsek MZ, Souery D, Bajs M, Maier W, Aitchison KJ, Farmer A, McGuffin P (2012) Depression symptom dimensions as predictors of antidepressant treatment outcome: replicable evidence for interest-activity symptoms. Psychol Med 42(5):967–980. https://doi.org/10.1017/S0033291711001905. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3787526/
Watson R, Harvey K, McCabe C, Reynolds S (2020) Understanding anhedonia: a qualitative study exploring loss of interest and pleasure in adolescent depression. Eur Child Adolesc Psychiatry 29(4):489–499. https://doi.org/10.1007/s00787-019-01364-y
WHO (1992) The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines. World Health Organization, Geneva
Zhang J, Berridge KC, Tindell AJ, Smith KS, Aldridge JW (2009) A neural computational model of incentive salience. PLoS Comput Biol 5(7):e1000437. https://doi.org/10.1371/journal.pcbi.1000437
Acknowledgements
QH is supported by the UCLH Biomedical Research Centre. MB is supported by the NIHR Oxford Health Biomedical Research Centre and the NIHR Oxford cognitive health Clinical Research Facility. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the UK Department of Health.
Disclosures
QH has received options and consultancy fees from Aya Health, and a research grant from Koa Health. MB has acted as a consultant for Janssen Research, P1vital Ltd. and CHDR, owns shares in P1vital Products Ltd. and was previously a paid employee of P1vital Ltd.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Huys, Q.J.M., Browning, M. (2022). A Computational View on the Nature of Reward and Value in Anhedonia. In: Pizzagalli, D.A. (eds) Anhedonia: Preclinical, Translational, and Clinical Integration. Current Topics in Behavioral Neurosciences, vol 58. Springer, Cham. https://doi.org/10.1007/7854_2021_290
Download citation
DOI: https://doi.org/10.1007/7854_2021_290
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09682-2
Online ISBN: 978-3-031-09683-9
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)