Cognitive, Affective, & Behavioral Neuroscience

, Volume 13, Issue 4, pp 737–746 | Cite as

Feedback-related negativity codes outcome valence, but not outcome expectancy, during reversal learning

  • A. K. L. von Borries
  • R. J. Verkes
  • B. H. Bulten
  • R. Cools
  • E. R. A. de Bruijn
Article

Abstract

Optimal behavior depends on the ability to assess the predictive value of events and to adjust behavior accordingly. Outcome processing can be studied by using its electrophysiological signatures—that is, the feedback-related negativity (FRN) and the P300. A prominent reinforcement-learning model predicts an FRN on negative prediction errors, as well as implying a role for the FRN in learning and the adaptation of behavior. However, these predictions have recently been challenged. Notably, studies so far have used tasks in which the outcomes have been contingent on the response. In these paradigms, the need to adapt behavioral responses is present only for negative, not for positive feedback. The goal of the present study was to investigate the effects of positive as well as negative violations of expectancy on FRN amplitudes, without the usual confound of behavioral adjustments. A reversal-learning task was employed in which outcome value and outcome expectancy were orthogonalized; that is, both positive and negative outcomes were equally unexpected. The results revealed a double dissociation, with effects of valence but not expectancy on the FRN and, conversely, effects of expectancy but not valence on the P300. While FRN amplitudes were largest for negative-outcome trials, irrespective of outcome expectancy, P300 amplitudes were largest for unexpected-outcome trials, irrespective of outcome valence. These FRN effects were interpreted to reflect an evaluation along a good–bad dimension, rather than reflecting a negative prediction error or a role in behavioral adaptation. By contrast, the P300 reflects the updating of information relevant for behavior in a changing context.

Keywords

Learning Feedback-related negativity Psychopathy 

References

  1. Adams, R. B., Jr., Ambady, N., Macrae, C. N., & Kleck, R. E. (2006). Emotional expressions forecast approach-avoidance behavior. Motivation and Emotion, 30, 179–188.CrossRefGoogle Scholar
  2. Bellebaum, C., Polezzi, D., & Daum, I. (2010). It is less than you expected: The feedback-related negativity reflects violations of reward magnitude expectations. Neuropsychologia, 48, 3343–3350.PubMedCrossRefGoogle Scholar
  3. Chase, H. W., Swainson, R., Durham, L., Benham, L., & Cools, R. (2010). Feedback-related negativity codes prediction error but not behavioural adjustment during probabilistic reversal learning. Journal of Cognitive Neuroscience, 23, 936–946.PubMedCrossRefGoogle Scholar
  4. Cohen, M. X., Elger, C. E., & Ranganath, C. (2007). Reward expectation modulates feedback-related negativity and EEG spectra. NeuroImage, 35, 968–978.PubMedCentralPubMedCrossRefGoogle Scholar
  5. Cohen, M. X., & Ranganath, C. (2007). Reinforcement learning signals predict future decision. Journal of Neuroscience, 27, 371–378.PubMedCrossRefGoogle Scholar
  6. Cools, R., Altamirano, L., & D’Esposito, M. (2006). Reversal learning in Parkinson’s disease depends on medication status and outcome valence. Neuropsychologia, 44, 1663–1673.PubMedCrossRefGoogle Scholar
  7. Cools, R., Robinson, O. J., & Sahakian, B. J. (2008). Acute tryptophan depletion in healthy volunteers enhances punishment prediction but does not affect reward prediction. Neuropsychopharmacology, 33, 2291–2299.PubMedCrossRefGoogle Scholar
  8. de Bruijn, E. R. A., Mars, R. B., & Hulstijn, W. (2004). It wasn’t me. .. or was it? How false feedback affects performance. In M. Ullsperger & M. Falkenstein (Eds.), Errors, conflict, and the brain: Current opinions on performance monitoring (pp. 118–124). Leipzig: Max Planck Institute of Cognitive Neuroscience.Google Scholar
  9. Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D. Y., & Engel, A. K. (2005). Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. Journal of Neuroscience, 25, 11730–11737.PubMedCrossRefGoogle Scholar
  10. Donchin, E., & Coles, M. G. H. (1988). Is the P300 component a manifestation of cognitive updating? Behavioural and Brain Sciences, 11, 357–427.CrossRefGoogle Scholar
  11. Donchin, E., & Coles, M. G. H. (1998). Context updating and the P300. The Behavioral and Brain Sciences, 21, 149–168.Google Scholar
  12. Duncan-Johnson, C. C., & Donchin, E. (1977). On quantifying surprise: The variation of event-related potentials with subjective probability. Psychophysiology, 14, 456–467.PubMedCrossRefGoogle Scholar
  13. Ekman, P., & Friesen, W. V. (1976). Pictures of facial affect. Palo Alto: Consulting Psychologists Press.Google Scholar
  14. Gentsch, A., Ullsperger, P., & Ullsperger, M. (2009). Dissociable medial frontal negativities from a common monitoring system for self- and externally caused failure of goal achievement. NeuroImage, 47, 2023–2030.PubMedCrossRefGoogle Scholar
  15. Hajcak, G., Holroyd, C. B., Moser, J. S., & Simons, R. F. (2005). Brain potentials associated with expected and unexpected good and bad outcomes. Psychophysiology, 42, 161–170.PubMedCrossRefGoogle Scholar
  16. Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2006). The feedback-related negativity reflects the binary evaluation of good versus bad outcomes. Biological Psychology, 71, 148–154. doi:10.1016/j.biopsycho.2005.04.001 PubMedCrossRefGoogle Scholar
  17. Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2007). It’s worse than you thought: The feedback negativity and violations of reward prediction in gambling tasks. Psychophysiology, 44, 905–912. doi:10.1111/j.1469-8986.2007.00567.x PubMedCrossRefGoogle Scholar
  18. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709. doi:10.1037/0033-295X.109.4.679 PubMedCrossRefGoogle Scholar
  19. Holroyd, C. B., & Krigolson, O. E. (2007). Reward prediction error signals associated with a modified time estimation task. Psychophysiology, 44, 913–917. doi:10.1111/j.1469-8986.2007.00561.x PubMedCrossRefGoogle Scholar
  20. Holroyd, C. B., Larsen, J. T., & Cohen, J. D. (2004a). Context dependence of the event-related brain potential associated with reward and punishment. Psychophysiology, 41, 245–253. doi:10.1111/j.1469-8986.2004.00152.x PubMedCrossRefGoogle Scholar
  21. Holroyd, C. B., Nieuwenhuis, S., Yeung, N., & Cohen, J. D. (2003). Errors in reward prediction are reflected in the event-related brain potential. NeuroReport, 14, 2481–2484.PubMedCrossRefGoogle Scholar
  22. Holroyd, C. B., Nieuwenhuis, S., Yeung, N., Nystrom, L., Mars, R. B., Coles, M. G. H., et al. (2004b). Dorsal anterior cingulated cortex shows fMRI response to internal and external error signals. Nature Neuroscience, 7, 497–498.PubMedCrossRefGoogle Scholar
  23. Ito, T. A., Larsen, J. T., Smith, N. K., & Cacioppo, J. T. (1998). Negative information weighs more heavily on the brain: The negativity bias in evaluative categorization. Journal of Personality and Social Psychology, 75, 887–900.PubMedCrossRefGoogle Scholar
  24. Jessup, R. K., Busemeyer, J. R., & Brown, J. W. (2010). Error effects in anterior cingulate cortex reverse when error likelihood is high. Journal of Neuroscience, 30, 3467–3472.PubMedCentralPubMedCrossRefGoogle Scholar
  25. Liao, Y., Gramann, K., Feng, W., Deák, G. O., & Li, H. (2011). This ought to be good: Brain activity accompanying positive and negative expectations and outcomes. Psychophysiology, 48, 1412–1419.PubMedCrossRefGoogle Scholar
  26. Mars, R. B., Coles, M. G. H., Grol, M. J., Holroyd, C. B., Nieuwenhuis, S., Hulstijn, W., et al. (2005). Neural dynamics of error processing in medial frontal cortex. NeuroImage, 28, 1007–1013.PubMedCrossRefGoogle Scholar
  27. Mars, R. B., de Bruijn, E. R. A., Hulstijn, W., Miltner, W. H. R., & Coles, M. G. H. (2004). What if I told you: “You were wrong”? Brain potentials and behavioral adjustments elicited by feedback in a time-estimation task. In M. Ullsperger & M. Falkenstein (Eds.), Errors, conflicts, and the brain. Current opinions on performance monitoring (pp. 129–134). Leipzig: Max Planck Institute of Cognitive Neuroscience.Google Scholar
  28. Moser, J. S., & Simons, R. F. (2009). The neural consequences of flip‐flopping: The feedback‐related negativity and salience of reward prediction. Psychophysiology, 46(2), 313–320.Google Scholar
  29. Nieuwenhuis, S., Holroyd, C. B., Mol, N., & Coles, M. G. H. (2004a). Reinforcement-related brain potentials from medial frontal cortex: Origins, and functional significance. Neuroscience and Biobehavioural Reviews, 28, 441–448.CrossRefGoogle Scholar
  30. Nieuwenhuis, S., Slagter, H. A., von Geusau, N. J., Heslenfeld, D. J., & Holroyd, C. B. (2005). Knowing good from bad: Differential activation of human cortical areas by positive and negative outcomes. European Journal of Neuroscience, 21, 3161–3168.PubMedCrossRefGoogle Scholar
  31. Nieuwenhuis, S., Yeung, N., Holroyd, C. B., Schurger, A., & Cohen, J. D. (2004b). Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback. Cerebral Cortex, 14, 741–747.PubMedCrossRefGoogle Scholar
  32. Oliveira, F. T., McDonald, J. J., & Goodman, D. (2007). Performance monitoring in the anterior cingulate is not all error related: Expectancy deviation and the representation of action-outcome associations. Journal of Cognitive Neuroscience, 19, 1994–2004.PubMedCrossRefGoogle Scholar
  33. Pfabigan, D. M., Alexopoulos, J., Bauer, H., & Sailer, U. (2010). Manipulation of feedback expectancy and valence induces negative and positive reward prediction error signals manifest in event-related brain potentials. Psychophysiology, 48, 656–664.PubMedCrossRefGoogle Scholar
  34. Ridderinkhof, K. R., Ullsperger, M., Crone, E. A., & Nieuwenhuis, S. (2004). The role of the medial frontal cortex in cognitive control. Science, 306, 443–447. doi:10.1126/science.1100301 PubMedCrossRefGoogle Scholar
  35. Robinson, O. J., Cools, R., & Sahakian, B. J. (2012). Tryptophan depletion disinhibits punishment but not reward prediction: Implications for resilience. Psychopharmacology, 219, 599–605.PubMedCentralPubMedCrossRefGoogle Scholar
  36. Rushworth, M. F. S. (2008). Intention, choice, and the medial frontal cortex. Annals of the New York Academy of Sciences, 1124, 181–207.PubMedCrossRefGoogle Scholar
  37. Rushworth, M. F. S., Walton, M. E., Kennerley, S. W., & Bannerman, D. M. (2004). Action sets and decisions in the medial frontal cortex. Trends in Cognitive Sciences, 8, 410–417.PubMedCrossRefGoogle Scholar
  38. Sato, A., Yasuda, A., Ohira, H., Miyawaki, K., Nishikawa, M., Kumano, H., et al. (2005). Effects of value and reward magnitude on feedback negativity and P300. NeuroReport, 16, 407–411.PubMedCrossRefGoogle Scholar
  39. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron, 36, 241–263.PubMedCrossRefGoogle Scholar
  40. van der Helden, J., Boksem, M. A. S., & Blom, J. H. G. (2010). The importance of failure: Feedback-related negativity predicts motor learning efficiency. Cerebral Cortex, 20, 1596–1603. doi:10.1093/cercor/bhp224 PubMedCrossRefGoogle Scholar
  41. van Veen, V., Holroyd, C. B., Cohen, J. D., Stenger, V. A., & Carter, C. S. (2004). Errors without conflict: Implications for performance monitoring theories of anterior cingulated cortex. Brain and Cognition, 56, 267–276.PubMedCrossRefGoogle Scholar
  42. Walsh, M. M., & Anderson, J. R. (2011). Modulation of the feedback-related negativity by instruction and experience. Proceedings of the National Academy of Science, 108, 19048–19053.CrossRefGoogle Scholar
  43. Yeung, N., Holroyd, C. B., & Cohen, J. D. (2005). ERP correlates of feedback and reward processing in the presence and absence of response choice. Cerebral Cortex, 15, 535–544.PubMedCrossRefGoogle Scholar
  44. Yeung, N., & Sanfey, A. G. (2004). Independent coding of reward magnitude and valence in the human brain. Journal of Neuroscience, 24, 6258–6264.PubMedCrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2013

Authors and Affiliations

  • A. K. L. von Borries
    • 1
    • 2
    • 3
  • R. J. Verkes
    • 1
    • 2
    • 3
  • B. H. Bulten
    • 2
  • R. Cools
    • 1
    • 3
  • E. R. A. de Bruijn
    • 3
    • 4
  1. 1.Department of PsychiatryRadboud University Nijmegen Medical CentreNijmegenThe Netherlands
  2. 2.PompestichtingNijmegenThe Netherlands
  3. 3.Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenThe Netherlands
  4. 4.Institute of Psychology, Unit of Clinical Psychology and Leiden Institute for Brain and CognitionLeiden UniversityLeidenThe Netherlands

Personalised recommendations