, Volume 31, Issue 2, pp 201–206

Racing to the precipice: a model of artificial intelligence development

Open Forum

DOI: 10.1007/s00146-015-0590-y

Cite this article as:
Armstrong, S., Bostrom, N. & Shulman, C. AI & Soc (2016) 31: 201. doi:10.1007/s00146-015-0590-y


This paper presents a simple model of an AI (artificial intelligence) arms race, where several development teams race to build the first AI. Under the assumption that the first AI will be very powerful and transformative, each team is incentivised to finish first—by skimping on safety precautions if need be. This paper presents the Nash equilibrium of this process, where each team takes the correct amount of safety precautions in the arms race. Having extra development teams and extra enmity between teams can increase the danger of an AI disaster, especially if risk-taking is more important than skill in developing the AI. Surprisingly, information also increases the risks: the more teams know about each others’ capabilities (and about their own), the more the danger increases. Should these results persist in more realistic models and analysis, it points the way to methods of increasing the chance of the safe development of AI.


AI Artificial intelligence Risk Arms race  Coordination problem Model 

Copyright information

© Springer-Verlag London 2015

Authors and Affiliations

  1. 1.Department of Philosophy, Future of Humanity InstituteOxford UniversityOxfordUK

Personalised recommendations