Hidden Interlocutor Misidentification in Practical Turing Tests
- 218 Downloads
Based on insufficient evidence, and inadequate research, Floridi and his students report inaccuracies and draw false conclusions in their Minds and Machines evaluation, which this paper aims to clarify. Acting as invited judges, Floridi et al. participated in nine, of the ninety-six, Turing tests staged in the finals of the 18th Loebner Prize for Artificial Intelligence in October 2008. From the transcripts it appears that they used power over solidarity as an interrogation technique. As a result, they were fooled on several occasions into believing that a machine was a human and that a human was a machine. Worse still, they did not realise their mistake. This resulted in a combined correct identification rate of less than 56%. In their paper they assumed that they had made correct identifications when they in fact had been incorrect.
Keywords18th Loebner prize for artificial intelligence Confederate effect Elbot Eliza effect Gender-blurring effect Jury-service Parallel-paired Practical Turing tests Turing’s imitation game
- Carpenter, R. (2009). Jabberwacky: Communication, companionship intelligence. http://www.jabberwacky.com/. Accessed October 25, 2009.
- Loebner Prize (2008). Loebner prize for artificial intelligence—Home of the first Turing test. http://www.loebner.net/Prizef/. Accessed October 29, 2009.
- Roberts, F. & Gulsdorff, B. (2007). IVA2007—LNAI 4722, pp. 420–421.Google Scholar
- Roberts, F. (2005). The AI of elbot. Unpublished.Google Scholar
- Roberts, F. (2008). A social psychological approach to dialogue simulation. Unpublished.Google Scholar
- Shah, H., & Henry, O. (2005). The confederate effect in human–machine textual interaction. In A. Zemliak & N. Mastorakis (Eds.), Proceedings of the 5th WSEAS international conference on information science, communications and applications (ISCA 2005), Cancun, Mexico, May 11–14, ISBN: 960-8457-22-X, pp. 109–114.Google Scholar
- Shah, H., & Pavlika, V. (2005). Text-based dialogical e-query systems: Gimmick or convenience? Proceedings of the 10th international conference on speech and computers (SPECOM), Patras, Greece, October 17–19, ISBN: 5-7452-0110-X, Vol. II, pp. 425–428.Google Scholar
- Shah, H., & Warwick, K. (2009). Emotion in the Turing test: A downward trend for machines in recent Loebner prizes. Chapter XVII (Section V). In J. Vallverdú & D. Casacuberta (Eds.), Handbook of research on synthetic emotions and sociable robotics: New applications in affective computing and artificial intelligence. USA: Information Science Reference, ISBN: 978-1-60566-354-8.Google Scholar
- Shah, H., & Warwick, K. (2010). Testing Turing’s five minutes parallel-paired imitation game. Kybernetes, 39(3), 449–465.Google Scholar
- Turing, A. M., Braithwaite, R., Jefferson, G., & Newman, M. (1952). Can automatic calculating machines be said to think? In J. Copeland (Eds.), The essential Turing—The ideas that gave birth to the computer age (pp. 487–506). Oxford: Clarendon Press.Google Scholar
- Turing, A. M. (1950). Computing, machinery and intelligence. Mind, LIX(236), 433–460.Google Scholar
- Turing, A. M. (1948). Intelligent machinery. In B. J. Copeland (Ed.), The essential Turing—The ideas that gave birth to the computer age. Oxford: Clarendon Press, 2004.Google Scholar
- Turkel, S. (1997). Life on the screen-identity in the age of the internet. London: Pheonix Paperback.Google Scholar