Turing Test: 50 Years Later

  • Ayse Pinar Saygin
  • Ilyas Cicekli
  • Varol Akman
Part of the Studies in Cognitive Systems book series (COGS, volume 30)


The Turing Test is one of the most disputed topics in artificial intelligence, philosophy of mind, and cognitive science. This paper is a review of the past 50 years of the Turing Test. Philosophical debates, practical developments and repercussions in related disciplines are all covered. We discuss Turing’s ideas in detail and present the important comments that have been made on them. Within this context, behaviorism, consciousness, the ‘other minds’ problem, and similar topics in philosophy of mind are discussed. We also cover the sociological and psychological aspects of the Turing Test. Finally, we look at the current situation and analyze programs that have been developed with the aim of passing the Turing Test. We conclude that the Turing Test has been, and will continue to be, an influential and controversial topic.

Key words

chatbots Chinese Room consciousness Imitation Game intelligence Loebner Contest philosophy of mind Turing Test 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abelson, RP. (1968), Simulation of Social Behavior’, in G. Lindzey and E. Aronson, eds. Handbook of Social Psychology Reading, MA.: Addison Wesley, pp. 274–356.Google Scholar
  2. Alper, G. (1990), ‘A Psychoanalyst Takes the Turing Test’, Psychoanalytic Review 77 (1), pp. 59–68.Google Scholar
  3. Anderson, D. (1987), ‘Is the Chinese Room the Real Thing?’, Philosophy 62, pp. 389–393.CrossRefGoogle Scholar
  4. Barresi, J. (1987), ‘Prospects for the Cyberiad: Certain Limits on Human Self-Knowledge in the Cybernetic Age’, Journal for the Theory of Social Behavior 17, pp. 19–46.CrossRefGoogle Scholar
  5. Bieri, P. (1988), ‘Thinking Machines: Some Reflections on the Turing Test’, Poetics Today 9(1), pp. 163–186.CrossRefGoogle Scholar
  6. Block, N. (1978), ‘Troubles with Functionalism’, in C.W. Savage, ed, Minnesota Studies in the Philosophy of Science. Vol. 9: Perception and Cognition, Minneapolis, MN: University of Minneapolis Press.Google Scholar
  7. Block, N. (1981), ‘Psychologism and Behaviorism’, Philosophical Review 90, pp. 5–43.CrossRefGoogle Scholar
  8. Block, N. (1995), ‘The Mind as the Software of the Brain’, In D. Osherson, L. Gleitman, S. Kosslyn, E. Smith and S. Sternberg, eds., An Invitation to Cognitive Science. Cambridge, MA.: MIT Press. Boden M. (1988), ‘Escaping from the Chinese Room’, in Computer Models of the Mind, Cambridge,UK: Cambridge University Press.Google Scholar
  9. Bringsjord, S. (1992), What Robots Can and Can’t Be, Dordrecht, The Netherlands:Kluwer.CrossRefGoogle Scholar
  10. Bringsjord, S. (1994), ‘Could, How Could We Tell If, and Should — Androids Have Inner Lives?’, in K.M. Ford, C. Glymour and P. Hayes, eds. Android Epistemology, Cambridge, MA.: MIT Press, pp. 93–122.Google Scholar
  11. Bringsjord, S. (1996), ‘The Inverted Turing Test is Provably Redundant’. Psycoloquy 7(29).
  12. Chomsky, N. (1975), Reflections on Language, Pantheon.Google Scholar
  13. Clark, T. (1992), ‘The Turing Test as a Novel Form of Hermeneutics’, International Studies in Philosophy 24(1), pp. 17–31.Google Scholar
  14. Colby, K.M. (1981), ‘Modeling a Paranoid Mind’, Behavioral and Brain Sciences 4(4), pp. 515–560.CrossRefGoogle Scholar
  15. Colby, K.M. Hilf, F.D. and Weber, S. (1971), ‘Artificial Paranoia’, Artificial Intelligence 2, pp. 1–25.CrossRefGoogle Scholar
  16. Colby, K.M. Hilf, F.D., Weber, S. and Kraemer, (1972), ‘Turing-like Indistinguishability Tests for the Validation of a Computer Simulation of Paranoid Processes’, Artificial Intelligence 3, pp.199–222.CrossRefGoogle Scholar
  17. Cole, D.J. (1991), ‘Artificial Intelligence and Personal Identity’, Synthese 88, pp. 399–417.CrossRefGoogle Scholar
  18. Collins, H.M. (1990), Artificial Experts: Social Knowledge and Intelligent Machines, Cambridge, MA.: MIT Press.Google Scholar
  19. Collins, H.M. (1997), ‘The Editing Test for the Deep Problem of AI’, Psycoloquy 8(1). .
  20. Copeland, B.J. (1993), ‘The Curious Case of the Chinese Gym’, Synthese 95, pp. 173–186.CrossRefGoogle Scholar
  21. Cowley, S.J. and MacDorman, K.F. (1995), ‘Simulating Conversations: The Communion Game’, AI and Society 9, pp. 116–137.CrossRefGoogle Scholar
  22. Crockett, L. (1994), ‘The Turing Test and the Frame Problem: Al’s Mistaken Understanding of Intelligence, Norwood, NJ: Ablex.Google Scholar
  23. Davidson, D. (1990), ‘Turing’s Test’, in K.A. Said, M. Said, W.H. Newton-Smith, R. Viale and K.V. Wilkes, eds. Modelling the Mind, Oxford, UK: Claredon Press, pp. 1–11.Google Scholar
  24. Dennett, D (1992), Consciousness Explained, Boston, MA.: Little, Brown & Co.Google Scholar
  25. Dewdney, A. (1992), ‘Turing Test’, Scientific American 266(1), pp. 30–31.CrossRefGoogle Scholar
  26. Dyer, M. (1990), ‘Intentionality and Computationalism: Minds, Machines, Searle and Hamad’, Journal of Experimental and Theoretical Artificial Intelligence, 2, pp. 303–319.CrossRefGoogle Scholar
  27. Epstein, R. (1992), ‘The Quest for the Thinking Computer’, AI Magazine 13(2), pp. 81–95.Google Scholar
  28. Feigenbaum, E.A. (1996), ‘How the “What” Becomes the “How”’, Communications of the ACM 39(5), pp. 97–105.CrossRefGoogle Scholar
  29. Fetzer, J.H. (1993), ‘The TTT is not the Final Word’, Think 2(1), pp. 34–86.Google Scholar
  30. Fetzer, J.H. (1995), ‘Minds and Machines: Behaviorism, Dualism and Beyond’, Stanford Electronic Humanities Review 4(2).Google Scholar
  31. Flood, G. (1996), ‘1f Only They Could Think. Should the Turing Test be Blamed for the Ills that Beset Artificial Intelligence’, New Scientist 149(2012), pp. 32–35.Google Scholar
  32. Fodor, J.A. (1991), ‘Yin and Yang in the Chinese Room’, in D. Rosenthal, ed., The Nature of the Mind, Oxford, UK: Oxford University Press.Google Scholar
  33. Ford, K. and Hayes, P. (1996), ‘The Turing Test is Just as Bad When Inverted’, Psycoloquy 7(43). .
  34. Forsyth, R (1988), ‘The Trouble With AI’, Artificial Intelligence Review 2(1), pp. 67–77.CrossRefGoogle Scholar
  35. French, R. (1990), ‘Subcognition and the Limits of the Turing Test’, Mind 99(393), pp. 53–65.CrossRefGoogle Scholar
  36. French, R. (1995), ‘Refocusing the Debate on the Turing Test: A Response’. Behavior and Philosophy 23, pp. 59–60.Google Scholar
  37. French, R. (1995), ‘The Inverted Turing Test: A Simple (Mindless) Program that Could Pass It’, Psycoloquy 7(39). .
  38. Galatzer-Levy, R.M. (1991), ‘Computer Models and Psychoanalytic Ideas: Epistemological Applications’, Society for Psychoanalytic Psychotherapy Bulletin 6(1), pp. 23–33.Google Scholar
  39. Genova, J. (1994a), ‘Response to Anderson and Keith’, Social Epistemology 8(4), pp. 341–343.CrossRefGoogle Scholar
  40. Genova, J. (1994b), ‘Turing’s Sexual Guessing Game’, Social Epistemology 8(4), pp. 313–326.CrossRefGoogle Scholar
  41. Guccione, S. and Tamburrini, G. (1988), ‘Turing’s Test Revisited’, in Proceedings of the 1998 IEEE International Conference on Systems, Man and Cybernetics, Vol. 1. Beijing and Shenyang, China, pp. 38–41.Google Scholar
  42. Guillen, M.A. (1983), ‘The Test of Turing’, Psychology Today 17(12), pp. 80–81.Google Scholar
  43. Gunderson, K. (1964), ‘The Imitation Game’, Mind 73 pp. 234–245.CrossRefGoogle Scholar
  44. Gunderson, K. (1967), Menraliiy and Machines, New York, NY: Doubleday.Google Scholar
  45. Halpern, M. (1987), ‘Turing’s Test and the Ideology of Artificial Intelligence’, Artificial Intelligence Review 1(2), pp. 79–93.CrossRefGoogle Scholar
  46. Hamad, S. (1989), ‘Minds, Machines and Searle’, Journal of Experimental and Theoretical Artificial Intelligence 1(1), pp. 5–25.CrossRefGoogle Scholar
  47. Hamad, S. (1990), ‘The Symbol Grounding Problem’, Physica D 42, pp. 335–346.CrossRefGoogle Scholar
  48. Hamad, S. (1991), ‘Other Bodies, Other Minds: A Machine Incarnation of an Old Philosophical Problem’, Minds and Machines 1, pp. 43–54.Google Scholar
  49. Hamad, S. (1992), ‘The Turing Test is not a Trick: Turing Indistinguishability is a Scientific Criterion’, SIGART Bulletin 3(4), pp. 9–10.CrossRefGoogle Scholar
  50. Hamad, S. (1994), ‘Does Mind Piggyback on Robotic and Symbolic Capacity? in H. Morowitz and J. Singer, eds. The Mind, the Brain, and Complex Adaptive Systems, Reading, MA.: Addison Wesley.Google Scholar
  51. Hamad, S. (1998), ‘Turing Indistinguishability and the Blind Watchmaker’, in G. Mulhauser, ed. Evolving Consciousness Amsterdam: John Benjamins.Google Scholar
  52. Hauser, L. (1993), ‘Reaping the Whirlwind: Reply to Harnad’s “Other Bodies, Other Minds” ’, Minds and Machines 3, pp. 219–237.CrossRefGoogle Scholar
  53. Hauser, L. (1997), ‘Searle’s Chines Box: Debunking the Chinese Room Argument’, Minds and Machines 7, pp. 199–226.CrossRefGoogle Scholar
  54. Hayes, B. (1998), ‘Turing’s Test’, Muse 8.Google Scholar
  55. Hayes, P. and Ford, K. (1995), ‘Turing Test Considered Harmful’, in Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Vol. 1, pp. 972–977.Google Scholar
  56. Hayes, P., Harnard, S., Perlis, D. and Block, N. (1992), ‘Virtual Symposium on Virtual Mind’, Minds and Machines 3(2), pp. 217–238.CrossRefGoogle Scholar
  57. Henley, T.B. (1990), ‘Chauvinism and Science: Another Reply to Shanon’, Journal for the Theory of Social Behavior 20(1), pp. 93–95.CrossRefGoogle Scholar
  58. Hodges, A. (1983), Alan Turing: The Enigma, New York, NY: Simon & Schuster.Google Scholar
  59. Hofstadter, D.R. (1982), ‘The Turing Test: A Coffee-House Conversation’, in D. Hofstadter and D. Dennett, eds. The Mind’s I: Fantasies and Reflections on Self and Soul, London, UK: Penguin Books, pp. 69–95.Google Scholar
  60. Jacquette, D. (1993a), ‘A Turing Test Conversation’, Philosophy 68, pp. 231–233.CrossRefGoogle Scholar
  61. Jacquette, D. (1993b), ‘Who’s Afraid of the Turing Test’, Behavior and Philosophy 20, pp. 63–74.Google Scholar
  62. Johnson-Laird, P. (1988), The Computer and the Mind, Cambridge, MA.: Harvard University Press. Karelis, C. (1986), ‘Reflections on the Turing Test’, Journal for the Theory of Social Behavior 16, pp. 161–172.Google Scholar
  63. Kugel, P. (1986), ‘Thinking May Be More Than Computing’, Cognition 22, pp. 137–198.CrossRefGoogle Scholar
  64. Kugel, P. (1990), ‘Is It Time to Replace Turing’s Test?’, 1990 Workshop Artificial Intelligence: Emer- ging Science or Dying Art Form. Sponsored by SUNY Binghamton’s Program in Philosophy and Computer and Systems Sciences and AAAI.Google Scholar
  65. Lassegue, J. (1988), ‘What Kind of Turing Test did Turing Have in Mind?’, Tekhnema 3, pp. 37–58.Google Scholar
  66. Leiber, J. (1989), ‘Shanon on the Turing Test’, Journal for the Theory of Social Behavior 19(2), pp. 257–259.CrossRefGoogle Scholar
  67. Leiber, J. (1992), The Light Bulb and the Turing-Tested Machine’, Journal for the Theory of Social Behaviour 22, pp. 25–39.CrossRefGoogle Scholar
  68. Leiber, J. (1995), ‘On Turing’s Turing Test and Why the Matter Matters’, Synthese 105, pp. 59–69.CrossRefGoogle Scholar
  69. Loebner, H.G (1994), ‘In Response’, Communications of the Association for Computing Machinery 37, pp. 79–82.CrossRefGoogle Scholar
  70. Lucas, J. (1996), ‘Minds, Machines and Gödel’, Philosophy 36, pp. 112–127.CrossRefGoogle Scholar
  71. Lucas, J. (1996), ‘Minds, Machines and Gödel: A Retrospect’, in P. Millican and A. Clark, eds. Machines and Mind, Oxford UK: Oxford University Press.Google Scholar
  72. Maloney, J. (1987), ‘The Right Stuff’, Synthese 70, pp. 349–372.CrossRefGoogle Scholar
  73. Marinoff, L. (1995), ‘Has Turing Slain the Jabberwock?’, Informatica 19(4), pp. 513–526.Google Scholar
  74. Mauldin, M. (1994), ‘Chatterbots, Tinymuds and the Turing Test: Entering the Loebner Prize Com petition’, in Proceedings of the Twelfth National Conference on Artificial Intelligence, Vol. 1, Seattle, WA, pp. 16–21.Google Scholar
  75. Mays, W. (1952), ‘Can Machines Think?’, Philosophy 27, pp. 148–162.CrossRefGoogle Scholar
  76. Mcllvenny, P. (1993), ‘Constructing Societies and Social Machines: Stepping out of the Turing Test Discourse’, Journal of Intelligent Systems 2(2–4), pp. 2–4.CrossRefGoogle Scholar
  77. Michie, D. (1990), ‘The Superarticulacy Phenomenon in the Context of Software Manufacture’, in D. Partridge and Y. Wilks, eds.: The Foundations of Artificial Intelligence, Cambridge, MA.: MIT Press, pp. 411–439.CrossRefGoogle Scholar
  78. Michie, D. (1994), ‘Consciousness as an Engineering Issue, Part 1’. Journal of Consciousness Studies 1(2), pp. 52–66.Google Scholar
  79. Michie, D. (1995), ‘Consciousness as an Engineering Issue, Part 2’, Journal of Consciousness Studies 2(1), pp. 182–195.Google Scholar
  80. Michie, D. (1996), ‘Turing’s Test and Conscious Thought’ in P. Millican and A. Clark, eds. Machines and Thought: The Legacy of Alan Turing, Oxford, UK: Oxford University Press, pp. 27–51. Originally printed in Artificial Intelligence 60, pp. 27–51, 1993.Google Scholar
  81. Millar, P.H. (1973), ‘On the Point of the Imitation Game’, Mind 82, pp. 595–597.CrossRefGoogle Scholar
  82. Minsky, M. (1985), ‘Communication with Alien Intelligence’, in E. Regis, ed. Extraterrestrials: Science and Alien Intelligence, Cambridge, UK: Cambridge University Press.Google Scholar
  83. Moon, Y., Naas, C., Morkes, J., Kim, E.-Y. and Fogg, B. (1994), ‘Computers are Social Actors’, in Proceedings of the CHI Conference, Boston, MA, pp. 72–78.Google Scholar
  84. Moor, J.H. (1976), An Analysis of the Turing Test’, Philosophical Studies 30, pp. 249–257.CrossRefGoogle Scholar
  85. Moor, J.H. (1978), ‘Explaining Computer Behavior’, Philosophical Studies 34, pp. 325–327.CrossRefGoogle Scholar
  86. Narayanan, A. (1996), ‘The Intentional Stance and the Imitation Game’, in P. Millican and A. Clark,eds. Machines and Thought: The Legacy of Alan Turing, Oxford, UK: Oxford University Press, pp. 63–79.Google Scholar
  87. Parsons, H. (1990), ‘Turing on the Turing Test’, in W. Karwowski and M. Rahimi, eds. Ergonomics of Hybrid Automated Systems II, Amsterdam: Elsevier.Google Scholar
  88. Pinksy, L. (1951), ‘Do Machines Think About Thinking’, Mind 60(239), pp. 397–398.Google Scholar
  89. Platt, C. (1995), ‘What’s It Mean To Be Human, Anyway?’, Wired. Google Scholar
  90. Purtill, R.L. (1971), ‘Beating the Imitation Game’, Mind 80, 290–294.CrossRefGoogle Scholar
  91. Rankin, T. (1987), The Turing Paradigm: A Critical Assessment’, Dialogue 29, pp. 50–55.Google Scholar
  92. Reader, A. (1969), ‘Steps Toward Genuine Artificial Intelligence’, Acta Psychologica 29(3), pp. 279–289.CrossRefGoogle Scholar
  93. Rey, G. (1986), ‘What’s Really Going on in the Chinese Room?’, Philosophical Studies 50, pp. 196–285.CrossRefGoogle Scholar
  94. Richardson, R. (1982), ‘Turing Tests for Intelligence: Ned Block’s Defense of Psychologism’, Philosophical Studies 41, pp. 421–426.CrossRefGoogle Scholar
  95. Roberts, L. (1990), ‘Searle’s Extension of the Chinese Room to Connectionist Machines’, Journal of Experimental and Theoretical Artificial Intelligence 2, pp. 185–187.CrossRefGoogle Scholar
  96. Sampson, G. (1973), ‘In Defence of Turing’, Mind 82, pp. 592–594.CrossRefGoogle Scholar
  97. Saygin, A.P. (1999), ‘Turing Test and Conversation’, Master’s thesis, Bilkent University, Ankara, Turkey. Technical Report BU-CEIS-9911.Google Scholar
  98. Schweizer, P. (1998), ‘The Truly Total Turing Test’, Minds and Machines 8, pp. 263–272.CrossRefGoogle Scholar
  99. Searle, J.R. (1980), ‘Minds, Brains and Programs’, Behavioral and Brain Sciences 3, pp. 417–424.CrossRefGoogle Scholar
  100. Searle, J.R. (1990), ‘Is the Brain’s Mind a Computer Program?’, Scientific American 3(262), pp.26–31.CrossRefGoogle Scholar
  101. Shanon, B. (1989), ‘A Simple Comment Regarding the Turing Test’, Journal for the Theory of Social Behavior 19(2), pp. 249–256.CrossRefGoogle Scholar
  102. Shanon, B. (1991), ‘Chauvinism: A Misdirected Accusation’, Journal for the Theory of Social Behavior 21(3), pp. 369–371.CrossRefGoogle Scholar
  103. Sharma, R. and Conrath, D. (1993), ‘Evaluating Expert Systems: A Review of Applicable Choices’, Artificial Intelligence Review 7(2), pp. 77–91.CrossRefGoogle Scholar
  104. Shieber, S.M. (1994), ‘Lessons from a Restricted Turing Test’, Communications of the Association for Computing Machinery 37, pp. 70–78.CrossRefGoogle Scholar
  105. Stalker, D. (1978), ‘Why Machines Can’t Think: A Reply to James Moor’, Philosophical Studies 34, pp. 317–320.CrossRefGoogle Scholar
  106. Stevenson, J.G. (1976), ‘On the Imitation Game’, Philosophia 6, pp. 131–133.CrossRefGoogle Scholar
  107. Turing, A. (1950), ‘Computing Machinery and Intelligence’, Mind 59(236), pp. 433–460.CrossRefGoogle Scholar
  108. Turing, A. (1969), ‘Intelligent Machinery’, in D.M.B. Meltzer ed. Machine Intelligence 5, Edinburgh University Press, pp. 3–23. Originally, a National Physics Laboratory Report, 1948.Google Scholar
  109. Wallace, R.S. (1997), ‘The Lying Game’, Wired. Google Scholar
  110. Watt, S. (1996), ‘Naive Psychology and the Inverted Turing Test’, Psycoloquy 7(14)
  111. Weizenbaum, J. (1996), ’ELIZA—A Computer Program for the Study of Natural Language Commu-nication Between Men and Machines’, Communications of the ACM 9, pp. 36–45.Google Scholar
  112. Weizenbaum, J. (1976), Computer Power and Human Reason: From Judgement to Calculation, San Francisco, CA: W.H. Freeman.Google Scholar
  113. Whalen, T. (1995), ‘How I Lost the Contest and Re-Evaluated Humanity’,
  114. Whitby, B. (1996), ‘The Turing Test: AI’s Biggest Blind Alley?’, in P. Millican and A. Clarke, eds. Machines and Thought: The Legacy of Alan Turing, Oxford, UK: Oxford University Press, pp. 53–63.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2003

Authors and Affiliations

  • Ayse Pinar Saygin
    • 1
  • Ilyas Cicekli
    • 2
  • Varol Akman
    • 2
  1. 1.Department of Cognitive ScienceUniversity of CaliforniaUSA
  2. 2.Department of Computer EngineeringBilkent UniversityAnkaraTurkey

Personalised recommendations