, Volume 32, Issue 4, pp 543–551 | Cite as

On the promotion of safe and socially beneficial artificial intelligence

  • Seth D. BaumEmail author
Open Forum


This paper discusses means for promoting artificial intelligence (AI) that is designed to be safe and beneficial for society (or simply “beneficial AI”). The promotion of beneficial AI is a social challenge because it seeks to motivate AI developers to choose beneficial AI designs. Currently, the AI field is focused mainly on building AIs that are more capable, with little regard to social impacts. Two types of measures are available for encouraging the AI field to shift more toward building beneficial AI. Extrinsic measures impose constraints or incentives on AI researchers to induce them to pursue beneficial AI even if they do not want to. Intrinsic measures encourage AI researchers to want to pursue beneficial AI. Prior research focuses on extrinsic measures, but intrinsic measures are at least as important. Indeed, intrinsic factors can determine the success of extrinsic measures. Efforts to promote beneficial AI must consider intrinsic factors by studying the social psychology of AI research communities.


Artificial intelligence Beneficial artificial intelligence Artificial intelligence safety Social psychology 



This paper has benefited from conversations with many people including Dario Amodei, Miles Brundage, Sean Legassick, Richard Mallah, and Jaan Tallinn, and from time spent at the Columbia University Center for Research on Environmental Decisions. Tony Barrett, Steven Umbrello, and Peter Howe provided helpful comments on an earlier draft. Any errors or shortcomings in the paper are the author’s alone. Work on this paper was funded in part by a grant from the Future of Life Institute. The views in this paper are the author’s and are not necessarily the views of the Future of Life Institute or the Global Catastrophic Risk Institute.


  1. Amodei D, Olah C, Steinhardt J, Christiano P, Schulman J, Mané D (2016) Concrete problems in AI safety. arXiv:1606.06565
  2. Armstrong S, Bostrom N, Shulman C (2016) Racing to the precipice: a model of artificial intelligence development. AI & Soc 31(2):201–206CrossRefGoogle Scholar
  3. Baum SD, Goertzel B, Goertzel TG (2011) How long until human-level AI? Results from an expert assessment. Technol Forecast Soc Change 78(1):185–195CrossRefGoogle Scholar
  4. Bohannon J (2015) Fears of an AI pioneer. Science 349(6245):252MathSciNetCrossRefzbMATHGoogle Scholar
  5. Borrie J (2014) Humanitarian reframing of nuclear weapons and the logic of a ban. Int Aff 90(3):625–646CrossRefGoogle Scholar
  6. Cooter RD (2000) Three effects of social norms on law: expression, deterrence, and internalization. Oregon Law Rev 79:1–22Google Scholar
  7. Deci EL (1971) Effects of externally mediated rewards on intrinsic motivation. J Pers Soc Psychol 18:105–115CrossRefGoogle Scholar
  8. Dickerson CA, Thibodeau R, Aronson E, Miller D (1992) Using cognitive dissonance to encourage water conservation. J Appl Soc Psychol 22:841–854CrossRefGoogle Scholar
  9. Eden AH, Moor JH, Soraker JH, Steinhart E (2013) Singularity hypotheses: a scientific and philosophical assessment. Springer, BerlinzbMATHGoogle Scholar
  10. Griffith E (2016) Who will build the next great car company? Fortune Magazine, 23 June. Accessed 26 Sept 2016
  11. Gurney JK (2013) Sue my car not me: products liability and accidents involving autonomous vehicles. Univ Ill J Law Technol Policy 2013(2):247–277Google Scholar
  12. Joy B (2000) Why the future doesn’t need us. Wired 8(4):238–263Google Scholar
  13. Krantz DH, Peterson N, Arora P, Milch K, Orlove B (2008) Individual values and social goals in environmental decision making. In: Smith JC, Connolly T, Son YJ (eds) Kugler T. Decision modeling and behavior in complex and uncertain environments New York, Springer, pp 165–198Google Scholar
  14. Kunda Z (1990) The case for motivated reasoning. Psychol Bull 108(3):480–498CrossRefGoogle Scholar
  15. Lapinski MK, Rimal RN (2005) An explication of social norms. Commun Theory 15(2):127–147CrossRefGoogle Scholar
  16. Lewandowsky S, Cook J, Oberauer K, Brophy S, Lloyd EA, Marriott M (2015) Recurrent fury: conspiratorial discourse in the blogosphere triggered by research on the role of conspiracist ideation in climate denial. J Soc Political Psychol 3(1):142–178CrossRefGoogle Scholar
  17. Lin P, Abney K, Bekey GA (eds) (2011) Robot ethics: the ethical and social implications of robotics. MIT Press, CambridgeGoogle Scholar
  18. Markowitz EM, Shariff AF (2012) Climate change and moral judgement. Nat Clim Change 2(4):243–247CrossRefGoogle Scholar
  19. McGinnis JO (2010) Accelerating AI. Northwest Univ Law Rev 104:366–381Google Scholar
  20. Moses LB (2007) Recurring dilemmas: the law’s race to keep up with technological change. Univ Ill J Law Technol Policy 2007(2):239–285MathSciNetGoogle Scholar
  21. Posner EA (2000) Law and social norms: the case of tax compliance. Va Law Rev 86:1781–1819CrossRefGoogle Scholar
  22. Posner RA (2004) Catastrophe: risk and response. Oxford University Press, Oxford Google Scholar
  23. Russell S, Dewey D, Tegmark M (2015) Research priorities for robust and beneficial artificial intelligence. AI Mag 36(4):105–114CrossRefGoogle Scholar
  24. Schienke EW, Tuana N, Brown DA, Davis KJ, Keller K, Shortle JS, Stickler M, Baum SD (2009) The role of the NSF Broader Impacts Criterion in enhancing research ethics pedagogy. Soc Epistemol 23(3–4):317–336CrossRefGoogle Scholar
  25. Schienke EW, Baum SD, Tuana N, Davis KJ, Keller K (2011) Intrinsic ethics regarding integrated assessment models for climate management. Sci Eng Ethics 17(3):503–523CrossRefGoogle Scholar
  26. Schultz PW, Nolan JM, Cialdini RB, Goldstein NJ, Griskevicius V (2007) The constructive, destructive, and reconstructive power of social norms. Psychol Sci 18(5):429–434CrossRefGoogle Scholar
  27. Scruggs L, Benegal S (2012) Declining public concern about climate change: can we blame the great recession? Glob Environ Change 22(2):505–515CrossRefGoogle Scholar
  28. Shome D, Marx S (2009) The psychology of climate change communication: a guide for scientists, journalists, educators, political aides, and the interested public. Columbia University Center for Research on Environmental Decisions, New YorkGoogle Scholar
  29. Shulman C (2009) Arms control and intelligence explosions. In: 7th European Conference on Computing and Philosophy (ECAP), Bellaterra, Spain, July 2–4Google Scholar
  30. Sotala K, Yampolskiy RV (2014) Responses to catastrophic AGI risk: a survey. Physica Scripta 90(1):018001. doi: 10.1088/0031-8949/90/1/018001
  31. Stone J, Fernandez NC (2008) To practice what we preach: the use of hypocrisy and cognitive dissonance to motivate behavior change. Soc Pers Psychol Compass 2(2):1024–1051CrossRefGoogle Scholar
  32. Sunstein CR (1996) On the expressive function of law. Univ Pa Law Rev 144(5):2021–2053CrossRefGoogle Scholar
  33. Vohs KD, Mead NL, Goode MR (2006) The psychological consequences of money. Science 314:1154–1156CrossRefGoogle Scholar
  34. Wilson G (2013) Minimizing global catastrophic and existential risks from emerging technologies through international law. Va Environ Law J 31:307–364Google Scholar
  35. Yampolskiy R, Fox J (2013) Safety engineering for artificial general intelligence. Topoi 32(2):217–226Google Scholar

Copyright information

© Springer-Verlag London 2016

Authors and Affiliations

  1. 1.Global Catastrophic Risk InstituteWashingtonUSA

Personalised recommendations