Minds and Machines

, Volume 23, Issue 2, pp 163–177 | Cite as

Some Implications of a Sample of Practical Turing Tests

  • Kevin WarwickEmail author
  • Huma Shah
  • James Moor


A series of imitation games involving 3-participant (simultaneous comparison of two hidden entities) and 2-participant (direct interrogation of a hidden entity) were conducted at Bletchley Park on the 100th anniversary of Alan Turing’s birth: 23 June 2012. From the ongoing analysis of over 150 games involving (expert and non-expert, males and females, adults and child) judges, machines and hidden humans (foils for the machines), we present six particular conversations that took place between human judges and a hidden entity that produced unexpected results. From this sample we focus on features of Turing’s machine intelligence test that the mathematician/code breaker did not consider in his examination for machine thinking: the subjective nature of attributing intelligence to another mind.


Chatbots Practical Turing tests Imitation game Intelligence Philosophy of mind Understanding Nature of thought 



The authors wish to express their gratitude to Bletchley Park Trust for allowing the tests to go ahead at their venue. They also wish to thank the humans who acted as judges and those who acted as hidden humans for their time input. They also wish to thank the machine designers and the machines themselves. Most of all though, our gratitude goes to the team that made the event happen technically, namely Marc Allen, Ian Bland and Chris Chapman.


Such tests would not be possible without the developers involved and their machines. Our thanks go, in no particular order, to Rollo Carpenter and Cleverbot, Fred Roberts and Elbot, Robert Medeksza and Ultra Hal, Robby Garner and JFred and finally Vladimir Veselov and Eugene.


  1. Block, N. (1981). Psychologism and behaviorism. In S. Shieber (Ed.), The Turing test: Verbal behavior as the hallmark of intelligence (pp. 229–266). UK: MIT Press.Google Scholar
  2. Chomsky, N. (2008). Turing on the imitation game. In R. Epstein, G. Roberts, & G. Beber (Eds.), Parsing the Turing test: Philosophical and methodological issues in the quest for the thinking computer (pp. 103–106). USA: Springer.Google Scholar
  3. Copeland, B. J. (2004). The essential Turing—The ideas that gave birth to the computer age. Oxford: Clarendon Press.Google Scholar
  4. Demchenko, E., & Veselov, V. (2008). Who fools whom? The great mystification, or methodological issues on making fools of human beings. In R. Epstein, G. Roberts, & G. Beber (Eds.), Parsing the Turing test: Philosophical and methodological issues in the quest for the thinking computer. Berlin: Springer.Google Scholar
  5. Epstein, R. (2008). The quest for the thinking computer. In R. Epstein, G. Roberts, & G. Beber (Eds.), Parsing the Turing test: Philosophical and methodological issues in the quest for the thinking computer (pp. 3–12). Berlin: Springer.Google Scholar
  6. Floridi, L., Taddeo, M., & Turilli, M. (2009). Turing’s imitation game: Still an impossible challenge for all machines and some judges. Minds and Machines, 19(1), 145–150.CrossRefGoogle Scholar
  7. Fowler, H., & Fowler, F. (Eds.). (1995). The concise oxford dictionary of current English (9th ed., p. 486). Oxford: Clarendon Press.Google Scholar
  8. French, R. M. (2000). The Turing test: The first 50 years. Trends in Cognitive Sciences, 4(3), 115–122.CrossRefGoogle Scholar
  9. Grice, H. P. (1989). Studies in the way of words. Cambridge, MA: Harvard University Press.Google Scholar
  10. Harnad, S. (1992). The Turing test is not a trick: Turing indistinguishability is a scientific criterion. ACM SIGART Bulletin, 3(4), 9–10.CrossRefGoogle Scholar
  11. Hayes, P., & Ford, K. (1995). Turing test considered harmful. In Proceedings of international joint conference on artificial intelligence (Vol. 1, pp. 972–977), Montreal.Google Scholar
  12. Hodges, A. (1992). Alan Turing: The enigma. New York: Vintage Press.Google Scholar
  13. Levesque, H. J. (2009). Is it enough to get the behavior right? In Proceedings of the twenty-first international joint conference on artificial intelligence (pp. 1439–1444), Pasadena, USA, July 11–17.Google Scholar
  14. Moor, J. H. (1976). An analysis of the Turing test. Philosophical Studies, 30(4), 249–257.CrossRefGoogle Scholar
  15. Moor, J. H. (2003). The status and future of the Turing test. In J. H. Moor (Ed.), The Turing test–The elusive standard of artificial intelligence (pp. 197–214). Dordrecht, The Netherlands: Kluwer.Google Scholar
  16. Penrose, R. (1994). Shadows of the mind. Oxford: Oxford University Press.Google Scholar
  17. Preston, J., & Bishop, J. M. (Eds.). (2002). Views into the Chinese room. Oxford: Clarendon Press.zbMATHGoogle Scholar
  18. Purtill, R. L. (1971). Beating the imitation game. Mind, 80(318), 290–294.CrossRefGoogle Scholar
  19. Searle, J. (1997). The mystery of consciousness. New York: New York Review of Books.Google Scholar
  20. Shah, H. (2011). Turing’s misunderstood imitation game and IBM Watson’s success. In Proceedings of the 2nd towards a comprehensive intelligence test (TCIT)—Reconsidering the Turing test for the 21st century, symposium in AISB 2011 convention (pp. 1–5), York University, UK. Available here:
  21. Shah, H., & Warwick, K. (2010a). From the buzzing in Turing’s head to machine intelligence contests. In Proceedings of symposium for 1st towards a comprehensive intelligence test. AISB Convention, De Montfort, UK, 29 March–1 April. Available here:
  22. Shah, H., & Warwick, K. (2010b). Testing Turing’s five minutes, parallel-paired imitation game. Kybernetes, 39(3), 449–465.CrossRefGoogle Scholar
  23. Shah, H., & Warwick, K. (2010c). Hidden interlocutor misidentification in practical Turing tests. Minds and Machines, 20, 441–454.CrossRefGoogle Scholar
  24. Turing, A. M. (1950). Computing, machinery and intelligence. Mind, LIX(236), 433–460.MathSciNetCrossRefGoogle Scholar
  25. Warwick, K. (2011). Artificial intelligence: The basics. London: Routledge.Google Scholar
  26. Wheeler, M. (2010). Plastic machines: Behavioural diversity and the Turing test. Kybernetes, 39(3), 466–480.MathSciNetCrossRefGoogle Scholar
  27. Whitby, B. (1996). The Turing test: AI’s biggest blind alley? In P. J. R. Millican & A. Clark (Eds.), Machine and thought: The legacy of Alan Turing (Vol. 1, pp. 53–62). Oxford: Clarendon Press.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.School of Systems EngineeringUniversity of ReadingReadingUK
  2. 2.Department of PhilosophyDartmouth CollegeHanoverUSA

Personalised recommendations