Skip to main content
SpringerLink
Log in

Machine and human agents in moral dilemmas: automation–autonomic and EEG effect

  • Main Paper
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

Automation is inherently tied to ethical challenges because of its potential involvement in morally loaded decisions. In the present research, participants (n = 34) took part in a moral multi-trial dilemma-based task where the agent (human vs. machine) and the behavior (action vs. inaction) factors were randomized. Self-report measures, in terms of morality, consciousness, responsibility, intentionality, and emotional impact evaluation were gathered, together with electroencephalography (delta, theta, beta, upper and lower alpha, and gamma powers) and peripheral autonomic (electrodermal activity, heart rate variability) data. Data showed that moral schemata vary as a function of the involved decider, and when the agent and behavior factors are crossed. Subjects did not consider machines full moral deciders to the same degree as humans and tend to morally better accept human active behavior and machine inaction. Moreover, the autonomic physiological activity might support the a-posteriori moral evaluation. In the evaluation of the agent’s consciousness, a beta ventrolateral prefrontal synchronization was detected for human action and machine inaction, while a generalized gamma synchronization occurred in artificial agent trials while rating the emotional impact of the decider’s behavior. The detected differences might point to a potential explicit and implicit asymmetry in moral reasoning toward artificial and human agents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data that support the findings of this research are available from the corresponding author, upon reasonable request.

References

  • Allen C, Varner G, Zinser J (2000) Prolegomena to any future artificial moral agent. J Exp Theor Artif Intell 12(3):251–261

    Article  MATH  Google Scholar 

  • Apperly IA (2012) What is “theory of mind”? concepts, cognitive processes and individual differences. Quart J Exp Psychol 65(5):825–839

    Article  Google Scholar 

  • Armbruster D, Strobel A (2022) The heart as judge: association of heart rate variability with moral judgement—a replication study. Biol Psychol 169:108284

    Article  Google Scholar 

  • Balconi M, Fronda G (2019) Physiological correlates of moral decision-making in the professional domain. Brain Sci 9(9):229

    Article  Google Scholar 

  • Balconi M, Fronda G (2020) Morality and management: an oxymoron? fNIRS and neuromanagement perspective explain us why things are not like this. Cogn Affect Behav Neurosci 20(6):1336–1348

    Article  Google Scholar 

  • Balconi M, Lucchiari C (2006) EEG correlates (event-related desynchronization) of emotional face elaboration: a temporal analysis. Neurosci Lett 392:118–123

    Article  Google Scholar 

  • Banks CL (2019) Criminal justice ethics: theory and practice, 5th edn. SAGE Publications, London

    Google Scholar 

  • Bigman YE, Gray K (2018) People are averse to machines making moral decisions. Cognition 181:21–34

    Article  Google Scholar 

  • Briggs G, Scheutz M (2014) How robots can affect human behavior: investigating the effects of robotic displays of protest and distress. Int J Soc Robot 6(3):343–355

    Article  Google Scholar 

  • Bruers S, Braeckman J (2013) A review and systematization of the trolley problem. Philosophia 42(2):251–269

    Article  Google Scholar 

  • Coffman BA, Torrence N, Murphy T, Bebko G, Graur S, Chase HW, Salisbury DF, Phillips ML (2021) Trait sensation seeking is associated with heightened beta-band oscillatory dynamics over left ventrolateral prefrontal cortex during reward expectancy. J Affect Disord 292:67–74

    Article  Google Scholar 

  • Cohen S, Kamarck T, Mermelstein R (1983) A Global Measure of Perceived Stress. J Health Soc Behav 24(4):385

    Article  Google Scholar 

  • Crivelli D, Balconi M (2017) The agent brain: a review of non-invasive brain stimulation studies on sensing agency. Front Behav Neurosci. https://doi.org/10.3389/fnbeh.2017.00229

    Article  Google Scholar 

  • Cushman F (2013) Action, outcome, and value. Pers Soc Psychol Rev 17(3):273–292

    Article  Google Scholar 

  • Cushman F, Young L, Hauser M (2006) The role of conscious reasoning and intuition in moral judgment. Psychol Sci 17:1082–1089

    Article  Google Scholar 

  • Damasio AR (1996) The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philos Trans R Soc Lond Ser B 351(1346):1413–1420

    Article  Google Scholar 

  • Dan Glauser ES, Scherer KR (2008) Neuronal processes involved in subjective feeling emergence: oscillatory activity during an emotional monitoring task. Brain Topogr 20(4):224–231

    Article  Google Scholar 

  • Davis N (1984) The doctrine of double effect: problems of interpretation. Pac Philos Q 65(2):107–123

    Article  Google Scholar 

  • Etkin A, Prater KE, Hoeft F, Menon V, Schatzberg AF (2010) Failure of anterior cingulate activation and connectivity with the amygdala during implicit regulation of emotional processing in generalized anxiety disorder. Am J Psychiatr 167(5):545–554

    Article  Google Scholar 

  • Eyssel FA, Pfundmair M (2015). Predictors of psychological anthropomorphization, mind perception, and the fulfillment of social needs: A case study with a zoomorphic robot. 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)

  • Foot P (2002) The Problem of Abortion and the Doctrine of the Double Effect. Virtues and Vices: and other essays in moral philosophy, online edn. Oxford Academic. https://doi.org/10.1093/0199252866.003.0002

  • Forte G, Morelli M, Grässler B, Casagrande M (2022) Decision making and heart rate variability: a systematic review. Appl Cogn Psychol 36(1):100–110

    Article  Google Scholar 

  • Gazzola V, Rizzolatti G, Wicker B, Keysers C (2007) The anthropomorphic brain: the mirror neuron system responds to human and robotic actions. Neuroimage 35(4):1674–1684

    Article  Google Scholar 

  • Ghisi M, Flebus GB, Montano A, Sanavio E, Sica C (2006) Beck Depression Inventory, II, Italian Version. Firenze: Editore Giunti OS

  • Giroux M, Kim J, Lee JC, Park J (2022) Artificial intelligence and declined guilt: retailing morality comparison between human and AI. J Bus Ethics 178:1027–1041

    Article  Google Scholar 

  • Goel V, Stollstorff M, Nakic M, Knutson K, Grafman J (2009) A role for right ventrolateral prefrontal cortex in reasoning about indeterminate relations. Neuropsychologia 47(13):2790–2797

    Article  Google Scholar 

  • Greene JD, Sommerville RB, Nystrom LE, Darley JM, Cohen JD (2001) An fMRI investigation of emotional engagement in moral judgment. Science 293(5537):2105–2108

    Article  Google Scholar 

  • Greene JD, Morelli SA, Lowenberg K, Nystrom LE, Cohen JD (2008) Cognitive load selectively interferes with utilitarian moral judgment. Cognition 107(3):1144–1154

    Article  Google Scholar 

  • Grippo AJ, Lamb DG, Carter CS, Porges SW (2007) Social isolation disrupts autonomic regulation of the heart and influences negative affective behaviors. Biol Psychiat 62(10):1162–1170

    Article  Google Scholar 

  • Gui DY, Gan T, Liu C (2015) Neural evidence for moral intuition and the temporal dynamics of interactions between emotional processes and moral cognition. Soc Neurosci 11(4):380–394

    Article  Google Scholar 

  • Güntekin B, Emek-Savaş DD, Kurt P, Yener GG, Başar E (2013) Beta oscillatory responses in healthy subjects and subjects with mild cognitive impairment. Neuroimage 3:39–46

    Article  Google Scholar 

  • HajiHosseini A, Holroyd C (2015) Reward feedback stimuli elicit high-beta EEG oscillations in human dorsolateral prefrontal cortex. Sci Rep 5:13021. https://doi.org/10.1038/srep13021

  • Hew PC (2014) Artificial moral agents are infeasible with foreseeable technologies. Ethics Inf Technol 16(3):197–206

    Article  Google Scholar 

  • Huettel SA (2005) Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices. J Neurosci 25(13):3304–3311

    Article  Google Scholar 

  • Jensen O, Mazaheri A (2010) Shaping functional architecture by oscillatory alpha activity: gating by inhibition. Front Hum Neurosci 4:186. https://doi.org/10.3389/fnhum.2010.00186

  • Jeneson A, Squire LR (2011) Working memory, long-term memory, and medial temporal lobe function. Learn Mem 19(1):15–25

    Article  Google Scholar 

  • Johansson L (2010) The functional morality of robots. Int J Tech 1(4):65–73

    Google Scholar 

  • Kalvin CB, Bierman KL, Gatzke-Kopp LM (2016) Emotional reactivity, behavior problems, and social adjustment at school entry in a high-risk sample. J Abnorm Child Psychol 44(8):1527–1541

    Article  Google Scholar 

  • Kawasaki M, Yamaguchi Y (2012) Frontal theta and beta synchronizations for monetary reward increase visual working memory capacity. Soc Cogn Affect Neurosci 8(5):523–530

    Article  Google Scholar 

  • Kegeles L, Frankle W, Gil R, Narendran R, Slifstein M, Hwang DR, Laruelle M (2006) Schizophrenia is associated with increased synaptic dopamine in associative rather than limbic regions of the striatum: implications for mechanisms of action of antipsychotic drugs. J Nuclear Med 47:139–179

    Google Scholar 

  • Keil A, Mussweiler T, Epstude K (2006) Alpha-band activity reflects reduction of mental effort in a comparison task: a source space analysis. Brain Res 1121(1):117–127

    Article  Google Scholar 

  • Knyazev G, Slobodskoj-Plusnin J, Bocharov A (2009) Event-related delta and theta synchronization during explicit and implicit emotion processing. Neuroscience 164(4):1588–1600

    Article  Google Scholar 

  • Krall SC, Rottschy C, Oberwelland E, Bzdok D, Fox PT, Eickhoff SB, Fink GR, Konrad K (2014) The role of the right temporoparietal junction in attention and social interaction as revealed by ALE meta-analysis. Brain Struct Funct 220(2):587–604

    Article  Google Scholar 

  • Kreibig SD (2010) Autonomic nervous system activity in emotion: a review. Biol Psychol 84(3):394–421

    Article  Google Scholar 

  • Kropotov J (2008) Quantitative EEG, event-related potentials and neurotherapy, 1st edn. Academic Press, Cambridge

    Google Scholar 

  • Leicht G, Troschütz S, Andreou C, Karamatskos E, Ertl M, Naber D, Mulert C (2013) Relationship between oscillatory neuronal activity during reward processing and trait impulsivity and sensation seeking. PLoS One 8(12):e83414

    Article  Google Scholar 

  • Luo Q, Holroyd T, Jones M, Hendler T, Blair J (2007) Neural dynamics for facial threat processing as revealed by gamma band synchronization using MEG. Neuroimage 34(2):839–847

    Article  Google Scholar 

  • Malle BF, Magar ST, Scheutz M (2019) AI in the sky: how people morally evaluate human and machine decisions in a lethal strike dilemma. Robot Well Being 3:111–133

    Article  Google Scholar 

  • Malle BF, Scheutz M, Arnold T, Voiklis J, Cusimano C (2015). Sacrifice one for the good of many? Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction

  • Marco-Pallarés J, Münte TF, Rodríguez-Fornells A (2015) The role of high-frequency oscillatory activity in reward processing and learning. Neurosci Biobehav Rev 49:1–7

    Article  Google Scholar 

  • Mendes WB (2009) Assessing the autonomic nervous system. In: Harmon-Jones E, Beer J (eds) Methods in the neurobiology of social and personality psychology. Guilford Press, New York

    Google Scholar 

  • Mendez MF, Anderson E, Shapira JS (2005) An investigation of moral judgement in frontotemporal dementia. Cogn Behav Neurol 18(4):193–197

    Article  Google Scholar 

  • Millar J (2014) Proxy prudence: rethinking models of responsibility for semi‐autonomous robots. Available at SSRN 2442273.

  • Monroe AE, Dillon KD, Malle BF (2014) Bringing free will down to earth: People’s psychological concept of free will and its role in moral judgment. Conscious Cogn 27:100–108

    Article  Google Scholar 

  • Müller MM, Keil A, Gruber T, Elbert T (1999) Processing of affective pictures modulates right-hemispheric gamma band EEG activity. Clin Neurophysiol 110(11):1913–1920

    Article  Google Scholar 

  • Nácher V, Ledberg A, Deco G, Romo R (2013) Coherent delta-band oscillations between cortical areas correlate with decision making. Proc Natl Acad Sci 110(37):15085–15090

    Article  Google Scholar 

  • Naqvi N, Shiv B, Bechara A (2006) The role of emotion in decision making. Curr Dir Psychol Sci 15(5):260–264

    Article  Google Scholar 

  • Oathes DJ, Ray WJ, Yamasaki AS, Borkovec TD, Castonguay LG, Newman MG, Nitschke J (2008) Worry, generalized anxiety disorder, and emotion: evidence from the EEG gamma band. Biol Psychol 79(2):165–170

    Article  Google Scholar 

  • Park G, Kappes A, Rho Y, van Bavel JJ (2016) At the heart of morality lies neuro-visceral integration: lower cardiac vagal tone predicts utilitarian moral judgment. Social Cogn Affect Neurosci 11(10):1588–1596

    Article  Google Scholar 

  • Patil I, Cogoni C, Zangrando N, Chittaro L, Silani G (2013) Affective basis of judgment-behavior discrepancy in virtual experiences of moral dilemmas. Soc Neurosci 9(1):94–107

    Article  Google Scholar 

  • Petras K, ten Oever S, Jansma BM (2016) The effect of distance on moral engagement: event related potentials and alpha power are sensitive to perspective in a virtual shooting task. Front Psychol 6:2008. https://doi.org/10.3389/fpsyg.2015.02008

  • Petrides M (1994) Frontal lobes and behaviour. Curr Opin Neurobiol 4(2):207–211

    Article  Google Scholar 

  • Porges SW (1995) Cardiac vagal tone: a physiological index of stress. Neurosci Biobehav Rev 19(2):225–233

  • Rosas A, Bermúdez JP, MartínezCotrina J, Aguilar-Pardo D, CaicedoMera JC, Aponte-Canencio DM (2021) Perceiving utilitarian gradients: heart rate variability and self-regulatory effort in the moral dilemma task. Soc Neurosci 16(4):391–405

    Article  Google Scholar 

  • Ryan M (2020) In AI we trust: ethics, artificial intelligence, and reliability. Sci Eng Ethics 26:2749–2767

    Article  Google Scholar 

  • Sakagami M, Pan X (2007) Functional role of the ventrolateral prefrontal cortex in decision making. Curr Opin Neurobiol 17(2):228–233

    Article  Google Scholar 

  • Saltik I (2022). Explicit and implicit measurement of mind perception in social robots through individual differences modulation (Doctoral disseration, Bilkent University). http://repository.bilkent.edu.tr/bitstream/handle/11693/105463/B161035.pdf?sequence=1

  • Sarlo M, Lotto L, Manfrinati A, Rumiati R, Gallicchio G, Palomba D (2012) Temporal dynamics of cognitive-emotional interplay in moral decision-making. J Cogn Neurosci 24(4):1018–1029

    Article  Google Scholar 

  • Sarlo M, Lotto L, Rumiati R, Palomba D (2014) If it makes you feel bad, don’t do it! Egoistic rather than altruistic empathy modulates neural and behavioral responses in moral dilemmas. Physiol Behav 130:127–134

    Article  Google Scholar 

  • Schilbach L, Wohlschlaeger AM, Kraemer NC, Newen A, Shah NJ, Fink GR, Vogeley K (2006) Being with virtual others: neural correlates of social interaction. Neuropsychologia 44(5):718–730

    Article  Google Scholar 

  • Sebastiani L, Simoni A, Gemignani A, Ghelarducci B, Santarcangelo E (2003) Human hypnosis: autonomic and electroencephalographic correlates of a guided multimodal cognitive-emotional imagery. Neurosci Lett 338(1):41–44

    Article  Google Scholar 

  • Sparrow R (2007) Killer robots. J Appl Philos 24(1):62–77

    Article  Google Scholar 

  • Swiatek MS (2012) Intending to err: the ethical challenge of lethal, autonomous systems. Ethics Inf Technol 14(4):241–254

    Article  Google Scholar 

  • Thayer JF, Lane RD (2000) A model of neurovisceral integration in emotion regulation and dysregulation. J Affect Disord 61(3):201–216

    Article  Google Scholar 

  • Thayer JF, Hansen AL, Saus-Rose E, Johnsen BH (2009) Heart rate variability, prefrontal neural function, and cognitive performance: the neurovisceral integration perspective on self-regulation, adaptation, and health. Ann Behav Med 37(2):141–153

    Article  Google Scholar 

  • Thayer JF, Åhs F, Fredrikson M, Sollers III JJ, Wager TD (2012) A meta-analysis of heart rate variability and neuroimaging studies: implications for heart rate variability as a marker of stress and health. Neurosci Biobehav Rev 36(2):747–756

  • Thomson JJ (1993) Reply to commentators. Philos Phenomenol Res 53(1):187

    Article  Google Scholar 

  • Turing AM (1950) Computing Machinery and Intelligence. Mind 59(236):433–460. http://www.jstor.org/stable/2251299

  • Vanutelli M, Meroni F, Fronda G, Balconi M, Lucchiari C (2020) Gender differences and unfairness processing during economic and moral decision-making: a fNIRS study. Brain Sci 10(9):647

    Article  Google Scholar 

  • Wolff A, Gomez-Pilar J, Nakao T, Northoff G (2019). Interindividual neural differences in moral decision-making are mediated by alpha power and delta/theta phase coherence. Sci Rep 9:4432. https://doi.org/10.1038/s41598-019-40743-y

  • Zhan Y, Xiao X, Li J, Liu L, Chen J, Fan W, Zhong Y (2018) Interpersonal relationship modulates the behavioral and neural responses during moral decision-making. Neurosci Lett 672:15–21

    Article  Google Scholar 

  • Zhi-Jiang YAN, Jin-Long SU, Pan-Cha SU (2019) From human empathy to artificial empathy. J Psychol Sci 1:299

    Google Scholar 

  • Zumer JM, Scheeringa R, Schoffelen JM, Norris DG, Jensen O (2014) Occipital alpha activity during stimulus processing gates the information flow to object-selective cortex. PLoS Biol 12(10):e1001965

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International research center for Cognitive Applied Neuroscience (IrcCAN), Università Cattolica del Sacro Cuore, Largo Gemelli, 1, 20123, Milan, Italy

    Federico Cassioli, Laura Angioletti & Michela Balconi

  2. Research Unit in Affective and Social Neuroscience, Department of Psychology, Università Cattolica del Sacro Cuore, Largo Gemelli, 1, 20123, Milan, Italy

    Federico Cassioli, Laura Angioletti & Michela Balconi

  3. International research center for Cognitive Applied Neuroscience (IrcCAN), Research Unit in Affective and Social Neuroscience, Department of Psychology, Catholic University of Milan, Largo Gemelli, 1, 20123, Milan, Italy

    Laura Angioletti

Authors
  1. Federico Cassioli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura Angioletti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michela Balconi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

FC and MB designed the research paradigm, carried out the data collection and analysis, and wrote the article. LA edited the manuscript, checked the statistical data analysis, and supported data visualization. All the authors read and approved the submission of the final version of the article.

Corresponding author

Correspondence to Laura Angioletti.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Ethical approval

Research was conducted following the principles and guidelines of the Helsinki Declaration (2013) and was approved by the Ethical Committee of the Department of Psychology, Catholic University of the Sacred Heart, Milan, Italy. All experiment participants read, understood, and signed the written informed consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cassioli, F., Angioletti, L. & Balconi, M. Machine and human agents in moral dilemmas: automation–autonomic and EEG effect. AI & Soc (2023). https://doi.org/10.1007/s00146-023-01772-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00146-023-01772-4

Keywords

Search

Navigation