Skip to main content

The Genius of the 'Original Imitation Game' Test

Abstract

Twenty years ago in "Turing's Two Tests for Intelligence" I distinguished two distinct tests to be found in Alan Turing's 1950 paper "Computing Machinery and Intelligence": one by then very well-known, the other neglected. I also explained the significance of the neglected test. This paper revisits some of the points in that paper and explains why they are even more relevant today. It also discusses the value of tests for machine intelligence based on games humans play, giving an analysis of some twentieth century TV game shows and how they relate to the tests for machine intelligence in Turing's paper and in some other tests for machine intelligence that have been proposed since. Their value in distinguishing between 'wise' and simply ‘clever’ AI is discussed.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. CAP’99 (Computers and Philosophy 1999, August 6th, 1999, Carnegie Mellon University, Pittsburgh, PA. https://www.iacap.org/conferences/past-conferences/cap-1999-at-carnegie-mellon/). Also presented at the Pittsburgh Group on Theoretical Cognition and at Occidental College in early 2000. Later papers on the topic have appeared (Sterrett 2002, 2012, 2017).

  2. As I write this, the Wikipedia entry on the Turing Test says that I conflate the two tests. Whereas, the whole point of Sterrett (2000) is that there are two distinct tests; it even gives them proper names. Other authors similarly describe Sterrett as saying the exact opposite of what Sterrett (2000) actually said about the role of gender in Turing’s article, or describe it as making a historical claim about what Turing meant, which is, again, the exact opposite of what that article said.

  3. “It is a cliché that tests of intellectual skill differ from tests of purely mechanical skill in the novelty of the tasks set. The ability to tie a variety of knots, or to perform a variety of dives, is tested by asking the contestant to perform these tasks, and the test is not compromised if the contestant knows exactly what will be asked and practices until the task can be performed without stopping to reflect anew upon what is required. In contrast, we would think someone had missed the point of an intelligence test were the contestant given the answers to the questions beforehand, and coached to practice delivering them.” (Sterrett 2000).

  4. E.g., Shah and Warwick (2016).

  5. Biography of Arthur Samuel. IEEE Computer Society. https://www.computer.org/profiles/arthur-samuel.

  6. https://www.computer.org/profiles/arthur-samuel

  7. Ref: “Game Show ‘What’s My Line?’ Turns 70” by Marc Berman, February 2, 2020 in Forbes magazine. Downloaded May 4th, 2020 from https://www.forbes.com/sites/marcberman1/2020/02/02/game-show-whats-my-line-turns-70/#138d10536b11.

  8. January 13th, 1964 episode. You Tube channel “To Tell the Truth (CBS)” https://www.youtube.com/watch?v=4KJm7JKf5XQ.

  9. Colin Allen has pointed out that it is very likely that in using ‘man’ here, Turing meant no more than to indicate the player was human, of either gender. I find this very plausible.

References

  • Benjamin, R. (2019). Race after technology: Abolitionist tools for the new Jim code. Cambridge: Polity Books.

    Google Scholar 

  • Berman, M. (2020). Game show ‘What’s My Line?’ turns 70. February 2, 2020 in Forbes magazine.

  • Chu-Carroll, J. et al. (2012). Finding needles in the Haystack: search and candidate generation. IBM Journal of Research and Development 56(¾), Paper 6 May/July 2012.

  • Churchland, P. A. (1996). Learning and conceptual change: The view from the neurons. In A. Clark & P. J. R. Millican (Eds.), Connectionism, concepts and folk psychology: The legacy of alan turing (Vol. 2). Oxford: Clarendon Press.

    Google Scholar 

  • Damasino, N. (2020). The questioning turing test. Ph.D. Dissertation, Philosophy. The University of Edinburgh.

  • Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York: NYU Press.

    Book  Google Scholar 

  • O’Neill, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. New York: Crown Publishers.

    MATH  Google Scholar 

  • Ross, C. (2018). IBM’s Watson Supercomputer recommended ‘unsafe and incorrect’ cancer treatments, internal documents show. Retrieved July 25, 2018 from https://www.statnews.com/2018/07/25/ibm-watson-recommended-unsafe-incorrect-treatments/

  • Shah, H., Warwick, K. (2016). Imitating gender as a measure for artificial intelligence: Is it necessary? In Proceedings of the 8th International Conference on Agents and Artificial Intelligence (ICAART2016) (pp. 114–119).

  • Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., et al. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140–1144.

    MathSciNet  Article  Google Scholar 

  • Silver, D., Schrittwieser, J., Simonyan, K., et al. (2017). Mastering the game of Go without human knowledge. Nature, 550, 354–359. https://doi.org/10.1038/nature24270.

    Article  Google Scholar 

  • Sterrett, S. G. (2000). Turing’s two tests for intelligence. In Minds and Machines, Vol. 10, pp. 541–559. Reprinted in The Turing Test: The Elusive Standard of Artificial Intelligence. Edited by J. H. Moor. Kluwer Academic, 2003. (Preprint: https://philsci-archive.pitt.edu/8480/)

  • Sterrett, S. G. (2002). Too many instincts: Contrasting philosophical views on intelligence in humans and non-humans. JETAI (Journal of Experimental and Theoretical Artificial Intelligence), 14(1), 39–60. Reprinted in Thinking About Android Epistemology, Edited by K. Ford, C. Glymour & P. Hayes, MIT Press (March 2006).

  • Sterrett, S. G. (2012). Bringing up turing's child-machine. In How the World Computes, Springer Lecture Notes in Computer Science, 2012, Vol. 7318/2012, 703–713. (Preprint: https://philsci-archive.pitt.edu/9085/)

  • Sterrett, S. G. (2017). Turing and the integration of human and machine intelligence. In J. Floyd & A. Bokulich (Eds.) Philosophical explorations of the legacy of alan turing: Turing 100, Boston studies in the philosophy and history of science. Springer, Switzerland, 2017. (Preprint: https://philsci-archive.pitt.edu/10316/)

  • Strickland, E. (2019). IBM Watson, heal thyself. IEEE Spectrum, 56, 24–31.

    Article  Google Scholar 

  • Tannen, D. (1990). You just don’t understand: Women and men in conversation. New York: William Morrow and Company.

    Google Scholar 

  • Tannen, D. (1994). Gender and discourse. Oxford: Oxford University Press.

    Google Scholar 

  • Turing, A. M. (1950). Computing machinery and intelligence. Mind, LIX(236), 433–460

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to S. G. Sterrett.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sterrett, S.G. The Genius of the 'Original Imitation Game' Test. Minds & Machines 30, 469–486 (2020). https://doi.org/10.1007/s11023-020-09543-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11023-020-09543-6

Keywords

  • Intelligence
  • Machine intelligence
  • Turing Test
  • Artificial intelligence
  • Gender
  • Alan Turing
  • IBM Watson