Minds and Machines

, Volume 20, Issue 3, pp 441–454 | Cite as

Hidden Interlocutor Misidentification in Practical Turing Tests

  • Huma ShahEmail author
  • Kevin Warwick


Based on insufficient evidence, and inadequate research, Floridi and his students report inaccuracies and draw false conclusions in their Minds and Machines evaluation, which this paper aims to clarify. Acting as invited judges, Floridi et al. participated in nine, of the ninety-six, Turing tests staged in the finals of the 18th Loebner Prize for Artificial Intelligence in October 2008. From the transcripts it appears that they used power over solidarity as an interrogation technique. As a result, they were fooled on several occasions into believing that a machine was a human and that a human was a machine. Worse still, they did not realise their mistake. This resulted in a combined correct identification rate of less than 56%. In their paper they assumed that they had made correct identifications when they in fact had been incorrect.


18th Loebner prize for artificial intelligence Confederate effect Elbot Eliza effect Gender-blurring effect Jury-service Parallel-paired Practical Turing tests Turing’s imitation game 


  1. Carpenter, R. (2009). Jabberwacky: Communication, companionship intelligence. Accessed October 25, 2009.
  2. Floridi, L., Taddeo, M., & Turilli, M. (2009). Turing’s imitation game—Still an impossible challenge for all machines and some judges. An Evaluation of the 2008 Loebner Contest. Minds and Machines, 19(1), 145–150.CrossRefGoogle Scholar
  3. Loebner Prize (2008). Loebner prize for artificial intelligence—Home of the first Turing test. Accessed October 29, 2009.
  4. Roberts, F. & Gulsdorff, B. (2007). IVA2007—LNAI 4722, pp. 420–421.Google Scholar
  5. Roberts, F. (2005). The AI of elbot. Unpublished.Google Scholar
  6. Roberts, F. (2008). A social psychological approach to dialogue simulation. Unpublished.Google Scholar
  7. Shah, H., & Henry, O. (2005). The confederate effect in human–machine textual interaction. In A. Zemliak & N. Mastorakis (Eds.), Proceedings of the 5th WSEAS international conference on information science, communications and applications (ISCA 2005), Cancun, Mexico, May 11–14, ISBN: 960-8457-22-X, pp. 109–114.Google Scholar
  8. Shah, H., & Pavlika, V. (2005). Text-based dialogical e-query systems: Gimmick or convenience? Proceedings of the 10th international conference on speech and computers (SPECOM), Patras, Greece, October 17–19, ISBN: 5-7452-0110-X, Vol. II, pp. 425–428.Google Scholar
  9. Shah, H., & Warwick, K. (2009). Emotion in the Turing test: A downward trend for machines in recent Loebner prizes. Chapter XVII (Section V). In J. Vallverdú & D. Casacuberta (Eds.), Handbook of research on synthetic emotions and sociable robotics: New applications in affective computing and artificial intelligence. USA: Information Science Reference, ISBN: 978-1-60566-354-8.Google Scholar
  10. Shah, H., & Warwick, K. (2010). Testing Turing’s five minutes parallel-paired imitation game. Kybernetes, 39(3), 449–465.Google Scholar
  11. Turing, A. M., Braithwaite, R., Jefferson, G., & Newman, M. (1952). Can automatic calculating machines be said to think? In J. Copeland (Eds.), The essential Turing—The ideas that gave birth to the computer age (pp. 487–506). Oxford: Clarendon Press.Google Scholar
  12. Turing, A. M. (1950). Computing, machinery and intelligence. Mind, LIX(236), 433–460.Google Scholar
  13. Turing, A. M. (1948). Intelligent machinery. In B. J. Copeland (Ed.), The essential Turing—The ideas that gave birth to the computer age. Oxford: Clarendon Press, 2004.Google Scholar
  14. Turkel, S. (1997). Life on the screen-identity in the age of the internet. London: Pheonix Paperback.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  1. 1.School of Systems EngineeringThe University of Reading, WhiteknightsReadingUK

Personalised recommendations