Advertisement

Minds and Machines

, Volume 27, Issue 4, pp 591–608 | Cite as

Implementation of Moral Uncertainty in Intelligent Machines

  • Kyle BogosianEmail author
Article
First Online:

Abstract

The development of artificial intelligence will require systems of ethical decision making to be adapted for automatic computation. However, projects to implement moral reasoning in artificial moral agents so far have failed to satisfactorily address the widespread disagreement between competing approaches to moral philosophy. In this paper I argue that the proper response to this situation is to design machines to be fundamentally uncertain about morality. I describe a computational framework for doing so and show that it efficiently resolves common obstacles to the implementation of moral philosophy in intelligent machines.

Keywords

Moral uncertainty Normative uncertainty Metanormativity Metanormative theory Macaskill Machine ethics AI ethics Moral disagreement Moral divergence Metaethics Machine intelligence Top-down Bottom-up Value differences Moral voting Moral trade Moral trading Ethical trade Ethical trading Value specification Value alignment 
This is a preview of subscription content, log in to check access.

References

  1. Allen, C., Smit, I., & Wallach, W. (2005). Artificial morality: Top-down, bottom-up, and hybrid approaches. Ethics and Information Technology. doi: 10.1007/s10676-006-0004-4.Google Scholar
  2. Archard, D. (2011). Why moral philosophers are not and should not be moral experts. Bioethics, 25(3), 119–127.CrossRefGoogle Scholar
  3. Arkoudas, K., Bringsjord, S., & Bello, P. (2005). Toward ethical robots via mechanized deontic logic. In Machine ethics: Papers from the 2005 AAAI fall symposium (pp. 17–23). Retrieved from http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Toward+ethical+robots+via+mechanized+deontic+logic#0.
  4. Bello, P., & Bringsjord, S. (2012). On how to build a moral machine. Topoi, 32(2), 251–266. doi: 10.1007/s11245-012-9129-8.CrossRefGoogle Scholar
  5. Bostrom, N. (2009). Moral uncertainty—towards a solution? http://www.overcomingbias.com/2009/01/moral-uncertainty-towards-a-solution.html.
  6. Bostrom, N. (2012). The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds and Machines, 22(2), 71–85. doi: 10.1007/s11023-012-9281-3.MathSciNetCrossRefGoogle Scholar
  7. Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford: Oxford University Press.Google Scholar
  8. Brundage, M. (2014). Limitations and risks of machine ethics. Journal of Experimental & Theoretical Artificial Intelligence, 26(3), 355–372. doi: 10.1080/0952813X.2014.895108.CrossRefGoogle Scholar
  9. Cotton-Barratt, O. (2013). Geometric reasons for normalising variance to aggregate preferences. http://users.ox.ac.uk/~ball1714/Variance%20normalisation.pdf.
  10. Cross, B. (2016). Moral philosophy, moral expertise, and the argument from disagreement. Bioethics, 30(3), 188–194.CrossRefGoogle Scholar
  11. Driver, J. (2014). Moral expertise: Judgement, practice, and analysis. Social Philosophy and Policy, 30(1–2), 280–296.Google Scholar
  12. Gloor, L. (2016). Suffering-focused AI safety: Why “fail-safe” measures might be our top intervention. In Foundational Research Institute, Report FRI-16-1. https://foundational-research.org/files/suffering-focused-ai-safety.pdf.
  13. Greene, J. D., Sommerville, R. B., Nystrom, L., Darley, J., & Cohen, J. (2001). An fMRI investigation of emotional engagement in moral judgment. Science, 293(5537), 2105–2108. doi: 10.1126/science.1062872.CrossRefGoogle Scholar
  14. Jones, K., & Schroeter, F. (2012). Moral expertise. Analyse & Kritik, 34(2), 217–230.CrossRefGoogle Scholar
  15. Lockhart, T. (2000). Moral uncertainty and its consequences. Oxford: Oxford University Press. ISBN 978-0195126105.Google Scholar
  16. MacAskill, W. (2013). The infectiousness of nihilism. Ethics, 123(3), 508–520.CrossRefGoogle Scholar
  18. MacAskill, W. (2016). Normative uncertainty as a voting problem. Mind, 125(500), 967–1004. doi: 10.1093/mind/fzv169.MathSciNetCrossRefGoogle Scholar
  19. Nissan-Rozen, I. (2015). Against moral hedging. Economics and Philosophy, 3, 1–21. doi: 10.1017/S0266267115000206.Google Scholar
  20. Oesterheld, C. (2015). Formalizing preference utilitarianism in physical world models. Synthese. doi: 10.1007/s11229-015-0883-1.MathSciNetGoogle Scholar
  21. Oesterheld, C. (2016). Backup utility functions as a fail-safe AI technique. In Foundational Research Institute, Report FRI-16-2. https://foundational-research.org/files/backup-utility-functions.pdf.
  22. PhilPapers Foundation (2009). Preliminary survey results. http://philpapers.org/surveys/results.pl.
  23. Schultz, E., Cokely, E. T., & Feltz, A. (2011). Persistent bias in expert judgments about free will and moral responsibility: A test of the expertise defense. Consciousness and Cognition, 20(4), 1722–1731.CrossRefGoogle Scholar
  24. Schwitzgebel, E., & Cushman, F. (2012). Expertise in moral reasoning? Order effects on moral judgment in professional philosophers and non-philosophers. Mind and Language, 27(2), 135–153.CrossRefGoogle Scholar
  25. Shulman, C., Tarleton, N., & Jonsson, H. (2009). Which consequentialism? Machine ethics and moral divergence. In AP-CAP 2009: The 5th Asia-Pacific computing and philosophy conference. https://intelligence.org/files/WhichConsequentialism.pdf.
  26. Williams, E. (2015). The possibility of an ongoing moral catastrophe. Ethical Theory and Moral Practice, 18(5), 971–982. doi: 10.1007/s10677-015-9567-7.CrossRefGoogle Scholar
  27. Wiltshire, T. J. (2015). A prospective framework for the design of ideal artificial moral agents: Insights from the science of heroism in humans. Minds and Machines. doi: 10.1007/s11023-015-9361-2.Google Scholar
  28. Żuradzki, T. (2016). Meta-reasoning in making moral decisions under normative uncertainty. In Argumentation and reasoned action: Proceedings of the 1st European conference on argumentation, Lisbon, 2015 (Vol. 2, pp. 1093–1104).Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.Tulane UniversityNew OrleansUSA

Personalised recommendations

Cite article

Buy options