Abstract
The development of artificial intelligence will require systems of ethical decision making to be adapted for automatic computation. However, projects to implement moral reasoning in artificial moral agents so far have failed to satisfactorily address the widespread disagreement between competing approaches to moral philosophy. In this paper I argue that the proper response to this situation is to design machines to be fundamentally uncertain about morality. I describe a computational framework for doing so and show that it efficiently resolves common obstacles to the implementation of moral philosophy in intelligent machines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allen, C., Smit, I., & Wallach, W. (2005). Artificial morality: Top-down, bottom-up, and hybrid approaches. Ethics and Information Technology. doi:10.1007/s10676-006-0004-4.
Archard, D. (2011). Why moral philosophers are not and should not be moral experts. Bioethics, 25(3), 119–127.
Arkoudas, K., Bringsjord, S., & Bello, P. (2005). Toward ethical robots via mechanized deontic logic. In Machine ethics: Papers from the 2005 AAAI fall symposium (pp. 17–23). Retrieved from http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Toward+ethical+robots+via+mechanized+deontic+logic#0.
Bello, P., & Bringsjord, S. (2012). On how to build a moral machine. Topoi, 32(2), 251–266. doi:10.1007/s11245-012-9129-8.
Bostrom, N. (2009). Moral uncertainty—towards a solution? http://www.overcomingbias.com/2009/01/moral-uncertainty-towards-a-solution.html.
Bostrom, N. (2012). The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds and Machines, 22(2), 71–85. doi:10.1007/s11023-012-9281-3.
Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford: Oxford University Press.
Brundage, M. (2014). Limitations and risks of machine ethics. Journal of Experimental & Theoretical Artificial Intelligence, 26(3), 355–372. doi:10.1080/0952813X.2014.895108.
Cotton-Barratt, O. (2013). Geometric reasons for normalising variance to aggregate preferences. http://users.ox.ac.uk/~ball1714/Variance%20normalisation.pdf.
Cross, B. (2016). Moral philosophy, moral expertise, and the argument from disagreement. Bioethics, 30(3), 188–194.
Driver, J. (2014). Moral expertise: Judgement, practice, and analysis. Social Philosophy and Policy, 30(1–2), 280–296.
Gloor, L. (2016). Suffering-focused AI safety: Why “fail-safe” measures might be our top intervention. In Foundational Research Institute, Report FRI-16-1. https://foundational-research.org/files/suffering-focused-ai-safety.pdf.
Greene, J. D., Sommerville, R. B., Nystrom, L., Darley, J., & Cohen, J. (2001). An fMRI investigation of emotional engagement in moral judgment. Science, 293(5537), 2105–2108. doi:10.1126/science.1062872.
Jones, K., & Schroeter, F. (2012). Moral expertise. Analyse & Kritik, 34(2), 217–230.
Lockhart, T. (2000). Moral uncertainty and its consequences. Oxford: Oxford University Press. ISBN 978-0195126105.
MacAskill, W. (2013). The infectiousness of nihilism. Ethics, 123(3), 508–520.
MacAskill, W. (2014). Normative uncertainty. http://commonsenseatheism.com/wp-content/uploads/2014/03/MacAskill-Normative-Uncertainty.pdf.
MacAskill, W. (2016). Normative uncertainty as a voting problem. Mind, 125(500), 967–1004. doi:10.1093/mind/fzv169.
Nissan-Rozen, I. (2015). Against moral hedging. Economics and Philosophy, 3, 1–21. doi:10.1017/S0266267115000206.
Oesterheld, C. (2015). Formalizing preference utilitarianism in physical world models. Synthese. doi:10.1007/s11229-015-0883-1.
Oesterheld, C. (2016). Backup utility functions as a fail-safe AI technique. In Foundational Research Institute, Report FRI-16-2. https://foundational-research.org/files/backup-utility-functions.pdf.
PhilPapers Foundation (2009). Preliminary survey results. http://philpapers.org/surveys/results.pl.
Schultz, E., Cokely, E. T., & Feltz, A. (2011). Persistent bias in expert judgments about free will and moral responsibility: A test of the expertise defense. Consciousness and Cognition, 20(4), 1722–1731.
Schwitzgebel, E., & Cushman, F. (2012). Expertise in moral reasoning? Order effects on moral judgment in professional philosophers and non-philosophers. Mind and Language, 27(2), 135–153.
Shulman, C., Tarleton, N., & Jonsson, H. (2009). Which consequentialism? Machine ethics and moral divergence. In AP-CAP 2009: The 5th Asia-Pacific computing and philosophy conference. https://intelligence.org/files/WhichConsequentialism.pdf.
Williams, E. (2015). The possibility of an ongoing moral catastrophe. Ethical Theory and Moral Practice, 18(5), 971–982. doi:10.1007/s10677-015-9567-7.
Wiltshire, T. J. (2015). A prospective framework for the design of ideal artificial moral agents: Insights from the science of heroism in humans. Minds and Machines. doi:10.1007/s11023-015-9361-2.
Żuradzki, T. (2016). Meta-reasoning in making moral decisions under normative uncertainty. In Argumentation and reasoned action: Proceedings of the 1st European conference on argumentation, Lisbon, 2015 (Vol. 2, pp. 1093–1104).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bogosian, K. Implementation of Moral Uncertainty in Intelligent Machines. Minds & Machines 27, 591–608 (2017). https://doi.org/10.1007/s11023-017-9448-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11023-017-9448-z
Keywords
- Moral uncertainty
- Normative uncertainty
- Metanormativity
- Metanormative theory
- Macaskill
- Machine ethics
- AI ethics
- Moral disagreement
- Moral divergence
- Metaethics
- Machine intelligence
- Top-down
- Bottom-up
- Value differences
- Moral voting
- Moral trade
- Moral trading
- Ethical trade
- Ethical trading
- Value specification
- Value alignment