, Volume 32, Issue 2, pp 237–250 | Cite as

Toward Modeling and Automating Ethical Decision Making: Design, Implementation, Limitations, and Responsibilities

  • Gregory S. Reed
  • Nicholaos JonesEmail author


One recent priority of the U.S. government is developing autonomous robotic systems. The U.S. Army has funded research to design a metric of evil to support military commanders with ethical decision-making and, in the future, allow robotic military systems to make autonomous ethical judgments. We use this particular project as a case study for efforts that seek to frame morality in quantitative terms. We report preliminary results from this research, describing the assumptions and limitations of a program that assesses the relative evil of two courses of action. We compare this program to other attempts to simulate ethical decision-making, assess possibilities for overcoming the trade-off between input simplification and output reliability, and discuss the responsibilities of users and designers in implementing such programs. We conclude by discussing the implications that this project highlights for the successes and challenges of developing automated mechanisms for ethical decision making.


Decision support Ethical judgment Evil Military Modeling and simulation Robotic systems 


  1. Anderson SL, Anderson M (2009) How machines can advance ethics. Philosophy Now 72:17–19Google Scholar
  2. Anderson EF, Frith KH, Caspers B (2011) Linking economics and quality: developing an evidence-based nurse staffing tool. Nurs Admin Q 35:53–60CrossRefGoogle Scholar
  3. Arkin RC (2007) Governing lethal behavior: embedding ethics in a hybrid deliberative/reactive robot architecture. U.S. Army Research Office Technical Report GIT-GVU-07-11Google Scholar
  4. Baron-Cohen S (2003) The science of evil: on empathy and the origins of cruelty. Basic Books, New YorkGoogle Scholar
  5. Beauchamp TL, Childress JF (1979) Principles of biomedical ethics. Oxford University Press, New YorkGoogle Scholar
  6. Bell MZ (1985) Why expert systems fail. J Oper Res Soc 36:613–619Google Scholar
  7. Belmont Report (1979) Belmont report—ethical principles and guidelines for the protection of human subjects of research. Available online at
  8. Center for Modeling, Simulation, and Analysis, and Center for the Management of Science and Technology [CMSA/CMOST] (2010) Developing and calibrating a quantitative metric of evil for use in course of action analysis. Final technical report. AMREDC, SSDDGoogle Scholar
  9. Dawes RM (1971) A case study of graduate admissions: applications of three principles of human decision making. Am Psychol 26:180–188CrossRefGoogle Scholar
  10. Dawes RM (1979) The robust beauty of improper linear models in decision making. Am Psychol 34:571–582CrossRefGoogle Scholar
  11. Dawes RM, Corrigan B (1974) Linear models in decision making. Psychol Bull 81:93–106CrossRefGoogle Scholar
  12. Dawes RM, Faust D, Meehl PE (1989) Clinical versus actuarial judgment. Science 243:1668–1674CrossRefGoogle Scholar
  13. DeMarco JP (2000) Principalism and moral dilemmas: a new principle. J Med Ethics 31:101–105CrossRefGoogle Scholar
  14. Dixit AK, Skeath S (2004) Games of strategy, 2nd edn. W.W. Norton & Company, Inc, New YorkGoogle Scholar
  15. Fan R (1997) Self-determination vs. family-determination: two incommensurable principles of autonomy: a report from East Asia. Bioethics 11:309–322CrossRefGoogle Scholar
  16. Finn P (2011) A future for drones: automated killing. The Washington Post. Retrieved 26 June 2012, from
  17. Gips J (1995) Towards the ethical robot. In: Ford K, Glymour C, Hayes P (eds) Android epistemology. MIT Press, Cambridge, pp 243–252Google Scholar
  18. Goodwin P, Wright G (2004) Decision analysis for management judgment, 3rd edn. Wiley, West SussexGoogle Scholar
  19. Holm S (1995) Not just autonomy—the principles of American biomedical ethics. J Med Ethics 21:332–338CrossRefGoogle Scholar
  20. McLaren BM (2003) Extensionally defining principles and cases: an AI model. Artif Intell J 150:145–181CrossRefGoogle Scholar
  21. McLaren BM (2005) Lessons in machine ethics from the perspective of two computational models of ethical reasoning. Papers from the AAAI Fall symposium, technical report FS-05-06, pp 70–77Google Scholar
  22. McLaren BM (2006) Computational models of ethical reasoning: challenges, initial steps, and future directions. IEEE Intell Syst 6:2–10Google Scholar
  23. Mental Health Advisory Team (MHAT) IV, Office of the Surgeon General (2006) Final report: operation Iraqi freedom 05-07Google Scholar
  24. Mitchell TO (1969) Observer’s hostility as a factor in judgments of behavior in hostility-provoking situations. Ph.D. DissertationGoogle Scholar
  25. Morrow L (2003) Evil: an investigation. Basic Books, New YorkGoogle Scholar
  26. Ornstein SM (1987) Computers in battle: a human overview. In: Bellin DB, Chapman G (eds) Computers in battle—will they work?. Harcourt Brace Jovanovich, Inc., Orlando, pp 1–43Google Scholar
  27. Rogerson MD, Gottlieb MC, Handelsman MH, Knapp S, Younggren J (2011) Nonrational processes in ethical decision making. Am Psychol 66:614–623CrossRefGoogle Scholar
  28. Rozoff R (2010) Decade of the drone: America’s aerial assassins. Global Res. Retrieved 26 June 2012, from
  29. Sharkey N (2008a) The ethical frontiers of robotics. Science 322:1800–1801CrossRefGoogle Scholar
  30. Sharkey N (2008b) Grounds for discrimination: autonomous robot weapons. Rusi Defense Syst 11:86–89Google Scholar
  31. Sullins JP (2010) RoboWarfare: can robots be more ethical than humans on the battlefield? Ethics Inf Technol 12:263–275CrossRefGoogle Scholar
  32. Tackett GB (2009) Framework for quantification of evil as a metric for course of action (CoA) analysis. Draft technical report. AMRDEC, RDECOMGoogle Scholar
  33. U.S. Department of Defense (2007) Unmanned systems roadmap 2007–2032Google Scholar
  34. Welner M (2007) Classifying crimes by severity: from aggravators to depravity. In: Douglass J, Ressler R, Burgess A (eds) A crime classification manual. Jossey-Bass, San Francisco, pp 55–72Google Scholar
  35. Zimbardo PG (2004) A situationist perspective on the psychology of evil: understanding how good people are transformed into perpetrators. In: Miller AG (ed) The social psychology of good and evil. Guilford Press, New York, pp 21–50Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  1. 1.Center for Modeling, Simulation, and AnalysisUniversity of Alabama in HuntsvilleHuntsvilleUSA
  2. 2.Department of PhilosophyUniversity of Alabama in HuntsvilleHuntsvilleUSA

Personalised recommendations