Abstract
Rationale
Depression is a disorder characterized by sustained negative affect and blunted positive affect, suggesting potential abnormalities in reward learning and its interaction with episodic memory.
Objectives
This study investigated how reward prediction errors experienced during learning modulate memory for rewarding events in individuals with depressive and non-depressive symptoms.
Methods
Across three experiments, participants learned the average values of two scene categories in two learning contexts. Each learning context had either high or low outcome variance, allowing us to test the effects of small and large prediction errors on learning and memory. Participants were later tested for their memory of trial-unique scenes that appeared alongside outcomes. We compared learning and memory performance of individuals with self-reported depressive symptoms (N = 101) to those without (N = 184).
Results
Although there were no overall differences in reward learning between the depressive and non-depressive group, depression severity within the depressive group predicted greater error in estimating the values of the scene categories. Similarly, there were no overall differences in memory performance. However, in depressive participants, negative prediction errors enhanced episodic memory more so than did positive prediction errors, and vice versa for non-depressive participants who showed a larger effect of positive prediction errors on memory. These results reflected differences in memory both within group and across groups.
Conclusions
Individuals with self-reported depressive symptoms showed relatively intact reinforcement learning, but demonstrated a bias for encoding events that accompanied surprising negative outcomes versus surprising positive ones. We discuss a potential neural mechanism supporting these effects, which may underlie or contribute to the excessive negative affect observed in depression.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Admon R, Pizzagalli DA (2015) Dysfunctional reward processing in depression. Curr Opin Psychol 4:114–118. https://doi.org/10.1016/j.copsyc.2014.12.011
Admon R, Nickerson LD, Dillon DG, Holmes AJ, Bogdan R, Kumar P, Dougherty DD, Iosifescu DV, Mischoulon D, Fava M, Pizzagalli DA (2015) Dissociable cortico-striatal connectivity abnormalities in major depression in response to monetary gains and penalties. Psychol Med 45(1):121–131. https://doi.org/10.1017/S0033291714001123
Barto AG (1995) Adaptive critic and the basal ganglia. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT press, p 215
Bates D, Maechler M, Bolker B, Walker S, Christensen RHB, Singmann H, … Grothendieck G (2015) Fitting linear mixed-effects models using lme4. J Stat Softw 67(1):1–48. https://doi.org/10.18637/jss.v067.i01
Bornstein AM, Khaw MW, Shohamy D, Daw ND (2017) Reminders of past choices bias decisions for reward in humans. Nat Commun 8:15958. https://doi.org/10.1038/ncomms15958
Burt DB, Zembar MJ, Niederehe G (1995) Depression and memory impairment: a meta-analysis of the association, its pattern, and specificity. Psychol Bull 117(2):285–305. https://doi.org/10.1037/0033-2909.117.2.285
Chase HW, Frank MJ, Michael A, Bullmore ET, Sahakian BJ, Robbins TW (2010) Approach and avoidance learning in patients with major depression and healthy controls: relation to anhedonia. Psychol Med 40(3):433–440. https://doi.org/10.1017/S0033291709990468
Clewett D, Huang R, Velasco R, Lee T-H, Mather M (2018) Locus coeruleus activity strengthens prioritized memories under arousal. J Neurosci 2097–17. https://doi.org/10.1523/JNEUROSCI.2097-17.2017
Dillon DG, Pizzagalli DA (2018) Mechanisms of memory disruption in depression. Trends Neurosci 41:137–149. https://doi.org/10.1016/j.tins.2017.12.006
Dillon DG, Dobbins IG, Pizzagalli DA (2014) Weak reward source memory in depression reflects blunted activation of VTA/SN and parahippocampus. Soc Cogn Affect Neurosci 9(10):1576–1583 Retrieved from http://scan.oxfordjournals.org/. Accessed 31 Oct 2018
Duncan KD, Shohamy D (2016) Memory states influence value-based decisions. J Exp Psychol Gen 145(9):3–9
Duszkiewicz AJ, Mcnamara CG, Takeuchi T, Genzel L (2018) Novelty and dopaminergic modulation of memory persistence: a tale of two systems. Trends Neurosci. https://doi.org/10.1016/j.tins.2018.10.002
Gaddy MA, Ingram RE (2014) A meta-analytic review of mood-congruent implicit memory in depressed mood. Clin Psychol Rev 34:402–416. https://doi.org/10.1016/j.cpr.2014.06.001
Gershman SJ, Daw ND (2017) Reinforcement learning and episodic memory in humans and animals: an integrative framework. Annu Rev Psychol 68:101–128. https://doi.org/10.1146/annurev-psych-122414-033625
Gotlib IH, Joormann J (2010) Cognition and depression: current status and future directions. SSRN. https://doi.org/10.1146/annurev.clinpsy.121208.131305
Gradin VB, Kumar P, Waiter G, Ahearn T, Stickle C, Milders M, Reid I, Hall J, Steele JD (2011) Expected value and prediction error abnormalities in depression and schizophrenia. Brain 134(6):1751–1764. https://doi.org/10.1093/brain/awr059
Huys QJ, Pizzagalli DA, Bogdan R, Dayan P (2013) Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol Mood Anxiety Disord 3(1):12. https://doi.org/10.1186/2045-5380-3-12
Joormann J, Quinn ME (2014) Cognitive processes and emotion regulation in depression. Depress Anxiety 31(4):308–315. https://doi.org/10.1002/da.22264
Kempadoo KA, Mosharov EV, Choi SJ, Sulzer D, Kandel ER (2016) Dopamine release from the locus coeruleus to the dorsal hippocampus promotes spatial learning and memory. Proc Natl Acad Sci 113(51):14835–14840. https://doi.org/10.1073/pnas.1616515114
Knutson B, Bhanji JP, Cooney RE, Atlas LY, Gotlib IH (2008) Neural responses to monetary incentives in major depression. Biol Psychiatry 63(7):686–692. https://doi.org/10.1016/j.biopsych.2007.07.023
Kumar P, Goer F, Murray L, Dillon DG, Beltzer ML, Cohen AL, Brooks NH, Pizzagalli DA (2018) Impaired reward prediction error encoding and striatal-midbrain connectivity in depression. Neuropsychopharmacology 43(7):1581–1588. https://doi.org/10.1038/s41386-018-0032-x
Leal SL, Tighe SK, Jones CK, Yassa MA (2014) Pattern separation of emotional information in hippocampal dentate and CA3. Hippocampus 24(9):1146–1155. https://doi.org/10.1002/hipo.22298
Matsumoto M, Hikosaka O (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447(7148):1111–1115. https://doi.org/10.1038/nature05860
Matt GE, Vázquez C, Campbell WK (1992) Mood-congruent recall of affectively toned stimuli: a meta-analytic review. Clin Psychol Rev 12(2):227–255. https://doi.org/10.1016/0272-7358(92)90116-P
Mueller EM, Pechtel P, Cohen AL, Douglas SR, Pizzagalli DA (2015) Potentiated processing of negative feedback in depression is attenuated by anhedonia. Depress Anxiety 32(4):296–305. https://doi.org/10.1002/da.22338
Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI (2012) Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15(7):1040–1046. https://doi.org/10.1038/nn.3130
Pizzagalli DA (2014) Depression, stress, and anhedonia: toward a synthesis and integrated model MDD: major depressive disorder. Annu Rev Clin Psychol 10:393–423. https://doi.org/10.1146/annurev-clinpsy-050212-185606
Pollack I, Norman DA (1964) A non-parametric analysis of recognition experiments. Psychon Sci 1(1–12):125–126. https://doi.org/10.3758/BF03342823
Read Montague P, Dayan P, Sejnowskw TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 76. Retrieved from http://www.jneurosci.org/content/jneuro/16/5/1936.full.pdf. Accessed 31 Oct 2018
Roesch MR, Esber GR, Li J, Daw ND, Schoenbaum G (2012) Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain. Eur J Neurosci 35(7):1190–1200. https://doi.org/10.1111/j.1460-9568.2011.07986.x
Rouhani N, Norman KA, Niv Y (2018) Dissociable effects of surprising rewards on learning and memory. J Exp Psychol Learn Mem Cogn. https://doi.org/10.1037/xlm0000518
Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH (1996) The inventory of depressive symptomatology (IDS): psychometric properties. Psychol Med 26(03):477–486. https://doi.org/10.1017/S0033291700035558
Rutledge RB, Moutoussis M, Smittenaar P, Zeidman P, Taylor T, Hrynkiewicz L, Lam J, Skandali N, Siegel JZ, Ousdal OT, Prabhu G, Dayan P, Fonagy P, Dolan RJ (2017) Association of neural and emotional impacts of reward prediction errors with major depression. JAMA Psychiatry 74:10–12. https://doi.org/10.1001/jamapsychiatry.2017.1713
Sacchet MD, Levy BJ, Hamilton JP, Maksimovskiy A, Hertel PT, Joormann J, Anderson MC, Wagner AD, Gotlib IH (2017) Cognitive and neural consequences of memory suppression in major depressive disorder. Cogn Affect Behav Neurosci 17(1):77–93. https://doi.org/10.3758/s13415-016-0464-x
Sara SJ (2009) The locus coeruleus and noradrenergic modulation of cognition. Nat Rev Neurosci 10:211–223. https://doi.org/10.1038/nrn2573
Shohamy D, Adcock RA (2010) Dopamine and adaptive memory. Trends Cogn Sci 14:464–472. https://doi.org/10.1016/j.tics.2010.08.002
Smoski MJ, Felder J, Bizzell J, Green SR, Ernst M, Lynch TR, Dichter GS (2009) fMRI of alterations in reward selection, anticipation, and feedback in major depressive disorder. J Affect Disord 118:69–78. https://doi.org/10.1016/j.jad.2009.01.034
Takeuchi T, Duszkiewicz AJ, Sonneborn A, Spooner PA, Yamasaki M, Watanabe M, Smith CC, Fernández G, Deisseroth K, Greene RW, Morris RGM (2016) Locus coeruleus and dopaminergic consolidation of everyday memory. Nature 537(7620):1–18. https://doi.org/10.1038/nature19325
Ubl B, Kuehner C, Kirsch P, Ruttorf M, Diener C, Flor H (2015) Altered neural reward and loss processing and prediction error signalling in depression. Soc Cogn Affect Neurosci 10(8):1102–1112. https://doi.org/10.1093/scan/nsu158
Walters K, Christakis DA, Wright DR (2018) Are mechanical Turk worker samples representative of health status and health behaviors in the U.S.? PLoS ONE 13(6). https://doi.org/10.1371/journal.pone.0198835
Weber EU, Blais A-R, Betz NE (2002) A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. J Behav Decis Mak 15(4):263–290. https://doi.org/10.1002/bdm.414
Whitton AE, Kakani P, Foti D, Van’t Veer A, Haile A, Crowley DJ, Pizzagalli DA (2016) Blunted neural responses to reward in remitted major depression: a high-density event-related potential study. Biol Psychiatry Cogn Neurosci Neuroimaging 1(1):87–95. https://doi.org/10.1016/j.bpsc
Wimmer GE, Shohamy D (2012) Preference by association: how memory mechanisms in the hippocampus bias decisions. Science 338:270–273. https://doi.org/10.1126/science.1223252
Yang Y, Cui Y, Sang K, Dong Y, Ni Z, Ma S, Hu H (2018) Ketamine blocks bursting in the lateral habenula to rapidly relieve depression. Nature 554(7692):317–322. https://doi.org/10.1038/nature25509
Young KD, Siegle GJ, Bodurka J, Drevets WC (2016) Amygdala activity during autobiographical memory recall in depressed and vulnerable individuals: association with symptom severity and autobiographical overgenerality. Am J Psychiatry 173:78–89. https://doi.org/10.1176/appi.ajp.2015.15010119
Young KD, Siegle GJ, Zotev V, Phillips R, Misaki M, Yuan H, Drevets WC, Bodurka J (2017) Randomized clinical trial of real-time fMRI amygdala neurofeedback for major depressive disorder: effects on symptoms and autobiographical memory recall. Am J Psychiatry 174:748–755. https://doi.org/10.1176/appi.ajp.2017.16060637
Funding
This work was supported by grant W911NF-14-1-0101 from the Army Research Office (Y.N.), the Ellison Foundation (Y.N.), grant R01MH098861 from the National Institute for Mental Health (Y.N.), and the National Science Foundation’s Graduate Research Fellowship Program (N.R.).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Participants completed informed consent online and were required to correctly answer questions checking for their understanding of the task before proceeding; procedures were approved by Princeton University’s Institutional Review Board.
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article belongs to a Special Issue on Translational Computational Psychopharmacology
Rights and permissions
About this article
Cite this article
Rouhani, N., Niv, Y. Depressive symptoms bias the prediction-error enhancement of memory towards negative events in reinforcement learning. Psychopharmacology 236, 2425–2435 (2019). https://doi.org/10.1007/s00213-019-05322-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00213-019-05322-z