Abstract
Recent advances in computational behavioral modeling can help rigorously quantify differences in how individuals learn behaviors that affect both themselves and others. But social learning remains understudied in the context of understanding individual variation in social phenomena like aggression, which is defined by persistent engagement in behaviors that harm others. We adapted a go/no-go reinforcement learning task across social and non-social contexts such that monetary gains and losses explicitly impacted the subject, a study partner, or no one. We then quantified participants’ (n = 61) sensitivity to others’ rewards, sensitivity to others’ losses, and the Pavlovian influence of expected outcomes on approach and avoidance behavior. Results showed that subjects learned in response to punishments and rewards that affected their partner in a way that was computationally similar to how they learned for themselves, consistent with the possibility that social learning engages empathic processes. Further supporting this interpretation, an individualized model parameter that indexed sensitivity to others’ punishments was inversely associated with trait antisociality. Modeled sensitivity to others’ losses also mapped onto post-task motivation ratings, but was not associated with self-reported trait empathy. This work is the first to apply a social reinforcement learning task that spans affect and action requirement (go/no-go) to measure multiple facets of empathic sensitivity.
Similar content being viewed by others
References
Archer, J. (2004). Sex differences in aggression in real-world settings: A meta-analytic review. Review of General Psychology, 8(4), 291–322. https://doi.org/10.1037/1089-2680.8.4.291
Archer, J., & Coyne, S. M. (2005). An integrated review of indirect, relational, and social aggression. Personality and Social Psychology Review, 9(3), 212–230. https://doi.org/10.1207/s15327957pspr0903_2
Arnett, P. A. (1997). Autonomic responsivity in psychopaths: A critical review and theoretical proposal. Clinical Psychology Review, 17(8), 903–936. https://doi.org/10.1016/S0272-7358(97)00045-7
Bernardo, M. O., Cecílio-Fernandes, D., Costa, P., Quince, T. A., Costa, M. J., & Carvalho-Filho, M. A. (2018). Physicians’ self-assessed empathy levels do not correlate with patients’ assessments. PLoS ONE, 13(5), e0198488.
Björkqvist, K., Österman, K., & Lagerspetz, K. M. J. (1994). Sex differences in covert aggression among adults. Aggressive Behavior, 20(1), 27–33. https://doi.org/10.1002/1098-2337(1994)20:1<27::AID-AB2480200105>3.0.CO;2-Q
Blair, R. J. R. (1995). A cognitive developmental approach to morality: Investigating the psychopath. Cognition, 57(1), 1–29.
Blair, R. J. R. (2003). Neurobiological basis of psychopathy. The British Journal of Psychiatry, 182(1), 5–7.
Brethel-Haurwitz, K. M., Cardinale, E. M., Vekaria, K. M., Robertson, E. L., Walitt, B., VanMeter, J. W., & Marsh, A. A. (2018). Extraordinary altruists exhibit enhanced self–other overlap in neural responses to distress. Psychological Science, 29(10), 1631–1641.
Buckholtz, J. W., Treadway, M. T., Cowan, R. L., Woodward, N. D., Benning, S. D., Li, R., Ansari, M. S., Baldwin, R. M., Schwartzman, A. N., Shelby, E. S., Smith, C. E., Cole, D., Kessler, R. M., & Zald, D. H. (2010). Mesolimbic dopamine reward system hypersensitivity in individuals with psychopathic traits. Nature Neuroscience., 13, 419–421. https://doi.org/10.1038/nn.2510
Buckholtz, J. W., Karmarkar, U., Ye, S., Brennan, G. M., & Baskin-Sommers, A. (2017). Blunted ambiguity aversion during cost-benefit decisions in antisocial individuals. Scientific Reports, 7, 2030. https://doi.org/10.1038/s41598-017-02149-6
Budhani, S., Richell, R. A., & Blair, R. J. R. (2006). Impaired reversal but intact acquisition: Probabilistic response reversal deficits in adult individuals with psychopathy. Journal of Abnormal Psychology, 115(3), 552–558.
Burt, S. A., & Donnellan, M. B. (2009). Development and validation of the Subtypes of Antisocial Behavior questionnaire. Aggressive Behavior, 35(5), 376–398. https://doi.org/10.1002/ab.20314
Burt, S. A., & Donnellan, M. B. (2010). Evidence that the Subtypes of Antisocial Behavior questionnaire (STAB) predicts momentary reports of acting-out behaviors. Personality and Individual Differences, 48(8), 917–920. https://doi.org/10.1016/j.paid.2010.02.021
Burt, S. A., Donnellan, M. B., & Tackett, J. L. (2012). Should social aggression be considered “antisocial”? Journal of Psychopathology and Behavioral Assessment, 34(2), 153–163. https://doi.org/10.1007/s10862-011-9267-0
Chen, M., & Bargh, J. A. (1999). Consequences of automatic evaluation: Immediate behavioral predispositions to approach or avoid the stimulus. Personality and Social Psychology Bulletin, 25(2), 215–224. https://doi.org/10.1177/0146167299025002007
Chrysikou, E. G., & Thompson, W. J. (2016). Assessing cognitive and affective empathy through the interpersonal reactivity index: An argument against a two-factor model. Assessment, 23(6), 769–777.
Collins, A. G., & Frank, M. J. (2018). Within-and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory. Proceedings of the National Academy of Sciences, 115(10), 2502–2507.
Czar, K. A., Dahlen, E. R., Bullock, E. E., & Nicholson, B. C. (2011). Psychopathic personality traits in relational aggression among young adults. Aggressive Behavior, 37(2), 207–214. https://doi.org/10.1002/ab.20381
Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44(1), 113–126. https://doi.org/10.1037/0022-3514.44.1.113
de Berker, A. O., Tirole, M., Rutledge, R. B., Cross, G. F., Dolan, R. J., & Bestmann, S. (2016). Acute stress selectively impairs learning to act. Scientific Reports, 6(1), 1–12.
De Brito, S. A., Viding, E., Kumari, V., Blackwood, N., & Hodgins, S. (2013). Cool and hot executive function impairments in violent offenders with antisocial personality disorder with and without psychopathy. PLoS ONE, 8(6), e65566. https://doi.org/10.1371/journal.pone.0065566
Decety, J. (2010). The neurodevelopment of empathy in humans. Developmental Neuroscience, 32(4), 257–267. https://doi.org/10.1159/000317771
Eagly, A. H., & Steffen, V. J. (1986). Gender and aggressive behavior. A meta-analytic review of the social psychological literature. Psychological Bulletin, 100(3), 309–330. https://doi.org/10.1037/0033-2909.100.3.309
Eisenberg, N., Eggum, N. D., & Di Giunta, L. (2010). Empathy-related responding: Associations with prosocial behavior, aggression, and intergroup relations. Social Issues and Policy Review, 4(1), 143–180.
Estes, W. K., & Skinner, B. F. (1941). Some quantitative properties of anxiety. Journal of Experimental Psychology, 29(5), 390–400. https://doi.org/10.1037/h0062283
Fultz, A. A., & Bernieri, F. J. (2021). Observer descriptions of the empathic person: A look at the Davis IRI and Hogan empathy scales. The Journal of Social Psychology, 1–15.
Gandhi, A. U., Dawood, S., & Schroder, H. S. (2021). Empathy mind-set moderates the association between low empathy and social aggression. Journal of Interpersonal Violence, 36(3–4), NP1679–1697NP.
Gray, J. A., & McNaughton, N. (2000). The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system. Oxford University Press.
Guitart-Masip, M., Fuentemilla, L., Bach, D. R., Huys, Q. J. M., Dayan, P., Dolan, R. J., & Duzel, E. (2011). Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. The Journal of Neuroscience, 31(21), 7867–7875. https://doi.org/10.1523/JNEUROSCI.6376-10.2011
Guitart-Masip, M., Huys, Q. J. M., Fuentemilla, L., Dayan, P., Duzel, E., & Dolan, R. J. (2012). Go and no-go learning in reward and punishment: Interactions between affect and effect. NeuroImage, 62(1), 154–166. https://doi.org/10.1016/j.neuroimage.2012.04.024
Guitart-Masip, M., Duzel, E., Dolan, R., & Dayan, P. (2014). Action versus valence in decision making. Trends in Cognitive Sciences, 18(4), 194–202. https://doi.org/10.1016/j.tics.2014.01.003
Hall, J. A., Andrzejewski, S. A., & Yopchick, J. E. (2009). Psychosocial correlates of interpersonal sensitivity: A meta-analysis. Journal of Nonverbal behavior, 33(3), 149–180.
Hershberger, W. A. (1986). An approach through the looking-glass. Animal Learning & Behavior, 14(4), 443–451. https://doi.org/10.3758/BF03200092
Huys, Q. J., Pizzagalli, D. A., Bogdan, R., & Dayan, P. (2013). Mapping anhedonia onto reinforcement learning: A behavioural meta-analysis. Biology of Mood & Anxiety Disorders, 3(1), 1–16.
Jolliffe, D., & Farrington, D. P. (2006). Examining the relationship between low empathy and bullying. Aggressive Behavior: Official Journal of the International Society for Research on Aggression, 32(6), 540–550.
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1990). Emotion, attention, and the startle reflex. Psychological Review, 97(3), 377–395. https://doi.org/10.1037/0033-295X.97.3.377
Lengersdorff, L., Wagner, I., Lockwood, P. L., & Lamm, C. (2020). When implicit prosociality trumps selfishness: The neural valuation system underpins more optimal choices when learning to avoid harm to others than to oneself. The Journal of Neuroscience, 40(38), 7286–7299. https://doi.org/10.1523/JNEUROSCI.0842-20.2020
Lockwood, P. L., Apps, M. A. J., Valton, V., Viding, E., & Roiser, J. P. (2016). Neurocomputational mechanisms of prosocial learning and links to empathy. Proceedings of the National Academy of Sciences of the United States of America, 113(35), 9763–9768. https://doi.org/10.1073/pnas.1603198113
Lockwood, P. L., Apps, M. A. J., & Chang, S. W. C. (2020). Is there a ‘social’ brain? Implementations and algorithms. Trends in Cognitive Sciences, 24(10), P802–P813. https://doi.org/10.1016/j.tics.2020.06.011
Lorber, M. F. (2004). Psychophysiology of aggression, psychopathy, and conduct problems: A meta-analysis. Psychological Bulletin, 130(4), 531–552. https://doi.org/10.1037/0033-2909.130.4.531
Ly, V., von Borries, A. K. L., Brazil, I. A., Bulten, B. H., Cools, R., & Roelofs, K. (2016). Reduced transfer of affective value to instrumental behavior in violent offenders. Journal of Abnormal Psychology, 125(5), 657–663. https://doi.org/10.1037/abn0000166
Marsh, A. A. (2016). Understanding amygdala responsiveness to fearful expressions through the lens of psychopathy and altruism. Journal of Neuroscience Research, 94(6), 513–525. https://doi.org/10.1002/jnr.23668
Marsh, A. A. (2019). The caring continuum: Evolved hormonal and proximal mechanisms explain prosocial and antisocial extremes. Annual Review of Psychology, 70(September 2018), 347–371. https://doi.org/10.1146/annurev-psych-010418-103010
Marsh, A. A., Stoycos, S. A., Brethel-Haurwitz, K. M., Robinson, P., VanMeter, J. W., & Cardinale, E. M. (2014). Neural and cognitive characteristics of extraordinary altruists. Proceedings of the National Academy of Sciences, 111(42), 15036–15041.
Miller, P. A., & Eisenberg, N. (1988). The relation of empathy to aggressive and externalizing/antisocial behavior. Psychological Bulletin, 103(3), 324–344. https://doi.org/10.1037/0033-2909.103.3.324
Mkrtchian, A., Aylward, J., Dayan, P., Roiser, J. P., & Robinson, O. J. (2017). Modeling avoidance in mood and anxiety disorders using reinforcement learning. Biological Psychiatry, 82(7), 532–539. https://doi.org/10.1016/j.biopsych.2017.01.017
Moul, C., Robinson, O. J., & Livesey, E. J. (2021). Antisocial learning: Using learning window width to model callous-unemotional traits? Computational Psychiatry, 5(1).
Murphy, B. A., & Lilienfeld, S. O. (2019). Are self-report cognitive empathy ratings valid proxies for cognitive empathy ability? Negligible meta-analytic relations with behavioral task performance. Psychological Assessment, 31(8), 1062–1072. https://doi.org/10.1037/pas0000732
Murphy, B. A., Costello, T. H., Watts, A. L., Cheong, Y. F., Berg, J. M., & Lilienfeld, S. O. (2020). Strengths and weaknesses of two empathy measures: A comparison of the measurement precision, construct validity, and incremental validity of two multidimensional indices. Assessment, 27(2), 246–260.
Murray, L., Shaw, D. S., Forbes, E. E., & Hyde, L. W. (2017). Reward-related neural correlates of antisocial behavior and callous–unemotional traits in young men. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 2(4), 346–354. https://doi.org/10.1016/j.bpsc.2017.01.009
Murray, L., Waller, R., & Hyde, L. W. (2018). A systematic review examining the link between psychopathic personality traits, antisocial behavior, and neural reactivity during reward and loss processing. Personality Disorders: Theory, Research, and Treatment, 9(6), 497–509. https://doi.org/10.1037/per0000308
Oba, T., Katahira, K., & Ohira, H. (2019). The effect of reduced learning ability on avoidance in psychopathy: A computational approach. Frontiers in Psychology, 10, 2432.
Piray, P., Dezfouli, A., Heskes, T., Frank, M. J., & Daw, N. D. (2019). Hierarchical Bayesian inference for concurrent model fitting and comparison for group studies. PLoS Computational Biology, 15(6), e1007043.
Reniers, R. L. E. P., Corcoran, R., Drake, R., Shryane, N. M., & Völlm, B. A. (2011). The QCAE: A questionnaire of cognitive and affective empathy. Journal of Personality Assessment, 93(1), 84–95. https://doi.org/10.1080/00223891.2010.528484
Rescorla, R. A., & Solomon, R. L. (1967). Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. Psychological Review, 74(3), 151–182. https://doi.org/10.1037/h0024475
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II Current research and theory (pp. 64–99). Appleton-Century-Crofts.
Slawinski, B. L., Klump, K. L., & Burt, A. S. (2018). No sex differences in the origins of covariation between social and physical aggression. Psychological Medicine, 49(15), 1–9. https://doi.org/10.1017/s0033291718003392
Solarz, A. K. (1960). Latency of instrumental responses as a function of compatibility with the meaning of eliciting verbal signs. Journal of Experimental Psychology, 59(4), 239–245. https://doi.org/10.1037/h0047274
Spielberger, C. D. (1983). State-trait anxiety inventory for adults (STAI-AD). APA PsycTests. https://doi.org/10.1037/t06496-000
Spielberger, C. D. (2010). State-trait anxiety inventory. In The Corsini encyclopedia of psychology. John Wiley & Sons, Inc. https://doi.org/10.1002/9780470479216.corpsy0943
Spielberger, C. D., & Sydeman, S. J. (1994). State-Trait Anxiety Inventory and State-Trait Anger Expression Inventory. In M. E. Maurish (Ed.), The use of psychological testing for treatment planning and outcome assessment (pp. 292–321). Lawrence Erlbaum.
Sutton, R. S., & Barto, A. G. (1998). Introduction to reinforcement learning. MIT Press.
Toyama, A., Katahira, K., & Ohira, H. (2019). Reinforcement learning with parsimonious computation and a forgetting process. Frontiers in Human Neuroscience, 13, 153.
Vachon, D. D., Lynam, D. R., & Johnson, J. A. (2014). The (non)relation between empathy and aggression: Surprising results from a meta-analysis. Psychological Bulletin, 140(3), 751–773. https://doi.org/10.1037/a0035236
Vekaria, K. M., O’Connell, K., Rhoads, S. A., Brethel-Haurwitz, K. M., Cardinale, E. M., Robertson, E. L., Walitt, B., VanMeter, J. W., & Marsh, A. A. (2020). Activation in bed nucleus of the stria terminalis (BNST) corresponds to everyday helping. Cortex, 127, 67–77.
White, S. F., Fowler, K. A., Sinclair, S., Schechter, J. C., Majestic, C., Pine, D. S., & Blair, R. J. R. (2014). Disrupted expected value signaling in youth with disruptive behavior disorders to environmental reinforcers. Journal of the American Academy of Child and Adolescent Psychiatry, 53(5), 579–588. https://doi.org/10.1016/j.jaac.2013.12.023
White, S. F., Tyler, P. M., Erway, A. K., Botkin, M. L., Kolli, V., Meffert, H., Pope, K., & Blair, J. R. (2016). Dysfunctional representation of expected value is associated with reinforcement-based decision-making deficits in adolescents with conduct problems. Journal of Child Psychology and Psychiatry, 57(8), 938–946. https://doi.org/10.1111/jcpp.12557
Yechiam, E., Busemeyer, J. R., Stout, J. C., & Bechara, A. (2005). Using cognitive models to map relations between neuropsychological disorders and human decision-making deficits. Psychological Science, 16(12), 973–978.
Zhou, Q., Valiente, C., & Eisenberg, N. (2003). Empathy and its measurement. In S. J. Lopez & C. R. Snyder (Eds.), Positive psychological assessment: A handbook of models and measures (pp. 269–284). American Psychological Association.
Acknowledgements
The authors thank Lydia Meena for help with data collection and Shawn Rhoads for helpful feedback.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
This work was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number TL1TR001431 to KO and by internal Georgetown University funding provided to AAM.
Data Availability
The datasets generated and analyzed in the current study are available in the Open Science Framework Repository, https://osf.io/u9fx3/.
Ethical Approval
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted approved by the Institutional Review Board at Georgetown University in Washington, DC.
Conflict of Interest
The authors declare no competing interests.
Informed Consent
Written informed consent was obtained from all subjects included in the study.
Code Availability
Code specific to this study is made available osf.io/u9fx3/. The emfit_toolbox code was provided by Q.J.M. Huys and is available at https://github.com/mpc-ucl/emfit.
Consent for Publication
Not applicable.
Author Contribution
KO conceptualized study, programmed tasks, collected data, analyzed data, and wrote manuscript. MW, BP, and SC collected data. AAM conceptualized study, acquired funding, and provided critical revisions to the manuscript. All authors approved the final version of the manuscript.
Additional information
Handling Editor: Jonathan Gratch
Supplementary Information
ESM 1
(PDF 1259 kb)
Rights and permissions
About this article
Cite this article
O’Connell, K., Walsh, M., Padgett, B. et al. Modeling Variation in Empathic Sensitivity Using Go/No-Go Social Reinforcement Learning. Affec Sci 3, 603–615 (2022). https://doi.org/10.1007/s42761-022-00119-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42761-022-00119-4