Abstract
We motivate the desire for self-driving and explain its potential and limitations, and explore the need for—and potential implementation of—morals, ethics, and other value systems as complementary “capabilities” to the Deep Technologies behind self-driving. We consider how the incorporation of such systems may drive or slow adoption of high automation within vehicles. First, we explore the role for morals, ethics, and other value systems in self-driving through a representative hypothetical dilemma faced by a self-driving car. Through the lens of engineering, we explain in simple terms common moral and ethical frameworks including utilitarianism, deontology, and virtue ethics before characterizing their relationship to the fundamental algorithms enabling self-driving. The concepts of behavior cloning, state-based modeling, and reinforcement learning are introduced, with some algorithms being more suitable for the implementation of value systems than others. We touch upon the contemporary cross-disciplinary landscape of morals and ethics in self-driving systems from a joint philosophical and technical perspective, and close with considerations for practitioners and the public, particularly as individuals may not appreciate the nuance and complexity of using imperfect information to navigate diverse scenarios and tough-to-quantify value systems, while “typical” software development reduces complex problems to black and white decision-making.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Not applicable.
Code availability
Not applicable.
References
Ackerman E (2016) People want driverless cars with utilitarian ethics, unless they’re a passenger. IEEE Spectrum
Asimov I (1942) Runaround. Astounding Science Fiction 29(1):94–103
Awad E, Dsouza S, Kim R, Schulz J, Henrich J, Shariff A, Bonnefon J-F, Rahwan I (2018) The moral machine experiment. Nature 563(7729):59–64
Baram M (2019) Why the trolley dilemma is a terrible model for trying to make self-driving cars safer. Fast Company
Bertoncello M, Wee D (2015) Ten ways autonomous driving could redefine the automotive world. Available at https://www.mckinsey.com/industries/automotive-and-assembly/our-insights/ten-ways-autonomous-driving-couldredefine-the-automotive-world. Accessed 15 July 2019
Borenstein J, Herkert JR, Miller KW (2019) Self-driving cars and engineering ethics: the need for a system level analysis. Sci Eng Ethics 25(2):383–398. https://doi.org/10.1007/s11948-017-0006-0
Davnall R (2020) Solving the single-vehicle self-driving car trolley problem using risk theory and vehicle dynamics. Sci Eng Ethics 26(1):431–449. https://doi.org/10.1007/s11948-019-00102-6
Dubljević V (2020) Toward implementing the ADC model of moral judgment in autonomous vehicles. Sci Eng Ethics. https://doi.org/10.1007/s11948-020-00242-0
Dunn J (2012) Virtual worlds and moral evaluation. Ethics Inf Technol 14(4):255–265. https://doi.org/10.1007/s10676-012-9298-6
Etienne H (2020) When AI ethics goes astray: a case study of autonomous vehicles. Soc Sci Comput Rev. https://doi.org/10.1177/0894439320906508
Faulhaber AK, Dittmer A, Blind F, Wächter MA, Timm S, Sütfeld LR, Stephan A, Pipa G, König P (2019) Human decisions in moral dilemmas are largely described by utilitarianism: virtual car driving study provides guidelines for autonomous driving vehicles. Sci Eng Ethics 25(2):399–418. https://doi.org/10.1007/s11948-018-0020-x
Flipse SM, Puylaert S (2018) Organizing a collaborative development of technological design requirements using a constructive dialogue on value profiles: a case in automated vehicle development. Sci Eng Ethics 24(1):49–72. https://doi.org/10.1007/s11948-017-9877-3
Foot P (1967) The problem of abortion and the doctrine of double effect. Oxford Review 5, pp 5–15
Gentzel M (2020) Classical liberalism, discrimination, and the problem of autonomous cars. Sci Eng Ethics 26(2):931–946. https://doi.org/10.1007/s11948-019-00155-7
Gerdes JC, Thornton SM (2015) Implementable ethics for autonomous vehicles. In: Autonomes fahren. Springer Vieweg, Berlin, Heidelberg, pp 87–102
Gogoll J, Müller JF (2017) Autonomous cars: in favor of a mandatory ethics setting. Sci Eng Ethics 23(3):681–700. https://doi.org/10.1007/s11948-016-9806-x
Gupta P, Coleman D, Siegel JE (2020) Towards safer self-driving through great pain (Physically Adversarial Intelligent Networks)
Hevelke A, Nida-Rümelin J (2015) Responsibility for crashes of autonomous vehicles: an ethical analysis. Sci Eng Ethics 21(3):619–630. https://doi.org/10.1007/s11948-014-9565-5
Jessie S (2019) The trolley problem isn’t theoretical anymore. Available at https://towardsdatascience.com/trolley-problem-isnt-theoretical-2fa92be4b050. Accessed 1 Aug 2020
Kamm FM (1989) Harming some to save others. Philos Stud 57(3):227–260
Keeling G (2020) Why trolley problems matter for the ethics of automated vehicles. Sci Eng Ethics 26(1):293–307. https://doi.org/10.1007/s11948-019-00096-1
Kenwright B (2018) Virtual reality: ethical challenges and dangers [Opinion]. IEEE Technol Soc Mag 37(4):20–25. https://doi.org/10.1109/MTS.2018.2876104
Lawlor R (2021) The ethics of automated vehicles: why self-driving cars should not swerve in dilemma cases. Res Publica. https://doi.org/10.1007/s11158-021-09519-y
Martin D (2017) Who should decide how machines make morally laden decisions? Sci Eng Ethics 23(4):951–967. https://doi.org/10.1007/s11948-016-9833-7
McMillan J, King M (2017) Why be moral in a virtual world? J Pract Ethics 5(2):30–48. https://ssrn.com/abstract=3087197
Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans. Info Syst E77-D:1321–1329
Millar J (2014) Should your robot driver kill you to save a child’s life?. The Conversation. https://theconversation.com/should-your-robot-driver-kill-you-to-save-a-childs-life-29926
Müller JF, Gogoll J (2020) Should manual driving be (eventually) outlawed? Sci Eng Ethics 26(3):1549–1567. https://doi.org/10.1007/s11948-020-00190-9
Noothigattu R, Gaikwad S, Awad E, Dsouza S, Rahwan I, Ravikumar P, Procaccia AD (2017) A voting-based system for ethical decision making. ArXiv Preprint. ArXiv:1709.06692
Otsuka M (2008) Double effect, triple effect and the trolley problem: squaring the circle in looping cases. Utilitas 20(1):92–110
Pappas G, Siegel Joshua E, Politopoulos K, Sun Y (2021) A gamified simulator and physical platform for self-driving algorithm training and validation. Electronics 10(9):2021
Pan X, Slater M (2011) Confronting a moral dilemma in virtual reality: a pilot study. In Proceedings of the 25th BCS Conference on Human-Computer Interaction. https://doi.org/10.14236/ewic/HCI2011.26
Parfit D (2011a) On what matters, vol 2. Oxford University Press. https://doi.org/10.1093/acprof:osobl/9780199572816.001.0001
Parfit D (2011b) On what matters, vol 1. Oxford University Press. https://doi.org/10.1093/acprof:osobl/9780199572809.001.0001
Philosophy Prof Wins NSF Grant on Ethics of Self-Driving Cars (2017). Available at https://www.uml.edu/news/stories/2017/selfdrivingcars.aspx. Accessed 1 Aug 2020
Ramirez EJ, LaBarge S (2018) Real moral problems in the use of virtual reality. Ethics Inf Technol 20(4):249–263. https://doi.org/10.1007/s10676-018-9473-5
Ramsey M (2015) Self-driving cars could cut down on accidents, study says. The Wall Street Journal, 1425567905
RAND Corporation (2016) Driving to safety
Renda A (2018) Ethics, algorithms and self-driving cars—a CSI of the ‘trolley problem.’ CEPS Policy Insight, 2018/02
Roff H (2018) The folly of trolleys: ethical challenges and autonomous vehicles. Brookings
Ryan M (2020) The future of transportation: ethical, legal, social and economic impacts of self-driving vehicles in the year 2025. Sci Eng Ethics 26(3):1185–1208. https://doi.org/10.1007/s11948-019-00130-2
Siegel JE (2016) Data proxies, the cognitive layer, and application locality: enablers of cloud-connected vehicles and next-generation internet of things. Massachusetts Institute of Technology
Siegel JE, Krishnan S (2020) Cultivating invisible impact with deep technology and creative destruction. J Innovation Manag 8(3):6–19
Siegel JE, Erb DC, Sarma SE (2017) A survey of the connected vehicle landscape—architectures, enabling technologies, applications, and development areas. IEEE Trans Intell Transp Syst 19(8):2391–2406
Siegel JE, Kumar S, Sarma SE (2018) The future internet of things: secure, efficient, and model-based. IEEE Internet of Things J. https://doi.org/10.1109/JIOT.2017.2755620
Soltanzadeh S, Galliott J, Jevglevskaja N (2020) Customizable ethics settings for building resilience and narrowing the responsibility gap: case studies in the socio-ethical engineering of autonomous systems. Sci Eng Ethics. https://doi.org/10.1007/s11948-020-00221-5
Terwilliger A, Brazil G, Liu X (2019) Recurrent flow-guided semantic forecasting. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). https://doi.org/10.1109/wacv.2019.00186
Thomson JJ (1976) Killing, letting die, and the trolley problem. Monist 59(2):204–217
Thrun S (2000) Probabilistic algorithms in robotics. AI Mag 21(4):93
Unger PK et al (1996) Living high and letting die: our illusion of innocence. Oxford University Press, USA
U.S. Department of Transportation (2008) National motor vehicle crash causation survey—report to congress. DOT HS 811:059
Wallach W, Allen C (2009) Moral machines: teaching robots right from wrong. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195374049.001.0001
Waymo (2021) Waymo safety report (February 2021). https://waymo.com/safety/safety-report. Accessed 10 Aug 2021
Funding
No funds, grants, or other support was received.
Author information
Authors and Affiliations
Contributions
The authors contributed equally to the preparation and submission of this manuscript. JS was responsible for the research concept and initial manuscript preparation; GP edited, formatted, and substantially revised the manuscript. Both co-authors approved the versions of the manuscript submitted for review and accepted for publication.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Siegel, J., Pappas, G. Morals, ethics, and the technology capabilities and limitations of automated and self-driving vehicles. AI & Soc 38, 213–226 (2023). https://doi.org/10.1007/s00146-021-01277-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-021-01277-y