Minds and Machines

, Volume 10, Issue 4, pp 463–518 | Cite as

Turing Test: 50 Years Later

  • Ayse Pinar Saygin
  • Ilyas Cicekli
  • Varol Akman


The Turing Test is one of the most disputed topics in artificial intelligence, philosophy of mind, and cognitive science. This paper is a review of the past 50 years of the Turing Test. Philosophical debates, practical developments and repercussions in related disciplines are all covered. We discuss Turing's ideas in detail and present the important comments that have been made on them. Within this context, behaviorism, consciousness, the `other minds' problem, and similar topics in philosophy of mind are discussed. We also cover the sociological and psychological aspects of the Turing Test. Finally, we look at the current situation and analyze programs that have been developed with the aim of passing the Turing Test. We conclude that the Turing Test has been, and will continue to be, an influential and controversial topic.

chatbots Chinese Room consciousness Imitation Game intelligence Loebner Contest philosophy of mind Turing Test 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abelson, RP. (1968), Simulation of Social Behavior', in G. Lindzey and E. Aronson, eds. Handbook of Social Psychology Reading, MA.: Addison Wesley, pp. 274-356.Google Scholar
  2. Alper, G. (1990), 'A Psychoanalyst Takes the Turing Test', Psychoanalytic Review 77 (1), pp. 59-68.Google Scholar
  3. Anderson, D. (1987), 'Is the Chinese Room the Real Thing?', Philosophy 62, pp. 389-393.Google Scholar
  4. Barresi, J. (1987), 'Prospects for the Cyberiad: Certain Limits on Human Self-Knowledge in the Cybernetic Age', Journal for the Theory of Social Behavior 17, pp. 19-46.Google Scholar
  5. Bieri, P. (1988), 'Thinking Machines: Some Reflections on the Turing Test', Poetics Today 9(1), pp. 163-186.Google Scholar
  6. Block, N. (1978), 'Troubles with Functionalism', in C.W. Savage, ed, Minnesota Studies in the Philosophy of Science. Vol. 9: Perception and Cognition, Minneapolis, MN: University of Minneapolis Press.Google Scholar
  7. Block, N. (1981), 'Psychologism and Behaviorism', Philosophical Review 90, pp. 5-43.Google Scholar
  8. Block, N. (1995), 'The Mind as the Software of the Brain', In D. Osherson, L. Gleitman, S. Kosslyn, E. Smith and S. Sternberg, eds., An Invitation to Cognitive Science. Cambridge, MA.: MIT Press.Google Scholar
  9. Boden M. (1988), 'Escaping from the Chinese Room', in Computer Models of the Mind, Cambridge, UK: Cambridge University Press.Google Scholar
  10. Bringsjord, S. (1992), What Robots Can and Can't Be, Dordrecht, The Netherlands: Kluwer.Google Scholar
  11. Bringsjord, S. (1994), 'Could, How Could We Tell If, and Should-Androids Have Inner Lives?', in K.M. Ford, C. Glymour and P. Hayes, eds. Android Epistemology, Cambridge, MA.: MIT Press, pp. 93-122.Google Scholar
  12. Bringsjord, S. (1996), 'The Inverted Turing Test is Provably Redundant'. Psycoloquy 7(29). Scholar
  13. Chomsky, N. (1975), Reflections on Language, Pantheon.Google Scholar
  14. Clark, T. (1992), 'The Turing Test as a Novel Form of Hermeneutics', International Studies in Philosophy 24(1), pp. 17-31.Google Scholar
  15. Colby, K.M. (1981), 'Modeling a Paranoid Mind', Behavioral and Brain Sciences 4(4), pp. 515-560.Google Scholar
  16. Colby, K.M. Hilf, F.D. and Weber, S. (1971), 'Artificial Paranoia', Artificial Intelligence 2, pp. 1-25.Google Scholar
  17. Colby, K.M. Hilf, F.D., Weber, S. and Kraemer, (1972), 'Turing-like Indistinguishability Tests for the Validation of a Computer Simulation of Paranoid Processes', Artificial Intelligence 3, pp. 199-222.Google Scholar
  18. Cole, D.J. (1991), 'Artificial Intelligence and Personal Identity', Synthese 88, pp. 399-417.Google Scholar
  19. Collins, H.M. (1990), Artificial Experts: Social Knowledge and Intelligent Machines, Cambridge, MA.: MIT Press.Google Scholar
  20. Collins, H.M. (1997), 'The Editing Test for the Deep Problem of AI', Psycoloquy 8(1). Scholar
  21. Copeland, B.J. (1993), 'The Curious Case of the Chinese Gym', Synthese 95, pp. 173-186.Google Scholar
  22. Cowley, S.J. and MacDorman, K.F. (1995), 'Simulating Conversations: The Communion Game', AI and Society 9, pp. 116-137.Google Scholar
  23. Crockett, L. (1994), 'The Turing Test and the Frame Problem: AI's Mistaken Understanding of Intelligence, Norwood, NJ: Ablex.Google Scholar
  24. Davidson, D. (1990), 'Turing's Test', in K.A. Said, M. Said, W.H. Newton-Smith, R. Viale and K.V. Wilkes, eds. Modelling the Mind, Oxford, UK: Claredon Press, pp. 1-11.Google Scholar
  25. Dennett, D (1992), Consciousness Explained, Boston, MA.: Little, Brown & Co.Google Scholar
  26. Dewdney, A. (1992), 'Turing Test', Scientific American 266(1), pp. 30-31.Google Scholar
  27. Dyer, M. (1990), 'Intentionality and Computationalism: Minds, Machines, Searle and Harnad', Journal of Experimental and Theoretical Artificial Intelligence, 2, pp. 303-319.Google Scholar
  28. Epstein, R. (1992), 'The Quest for the Thinking Computer', AI Magazine 13(2), pp. 81-95.Google Scholar
  29. Feigenbaum, E.A. (1996), 'How the "What" Becomes the "How"', Communications of the ACM 39(5), pp. 97-105.Google Scholar
  30. Fetzer, J.H. (1993), 'The TTT is not the Final Word', Think 2(1), pp. 34-86.Google Scholar
  31. Fetzer, J.H. (1995), 'Minds and Machines: Behaviorism, Dualism and Beyond', Stanford Electronic Humanities Review 4(2).Google Scholar
  32. Flood, G. (1996), 'If Only They Could Think: Should the Turing Test be Blamed for the Ills that Beset Artificial Intelligence', New Scientist 149(2012), pp. 32-35.Google Scholar
  33. Fodor, J.A. (1991), 'Yin and Yang in the Chinese Room', in D. Rosenthal, ed., The Nature of the Mind, Oxford, UK: Oxford University Press.Google Scholar
  34. Ford, K. and Hayes, P. (1996), 'The Turing Test is Just as Bad When Inverted', Psycoloquy 7(43). Scholar
  35. Forsyth, R (1988), 'The Trouble With AI', Artificial Intelligence Review 2(1), pp. 67-77.Google Scholar
  36. French, R. (1990), 'Subcognition and the Limits of the Turing Test', Mind 99(393), pp. 53-65.Google Scholar
  37. French, R. (1995), 'Refocusing the Debate on the Turing Test: A Response'. Behavior and Philosophy 23, pp. 59-60.Google Scholar
  38. French, R. (1995), 'The Inverted Turing Test: A Simple (Mindless) Program that Could Pass It', Psycoloquy 7(39). Scholar
  39. Galatzer-Levy, R.M. (1991), 'Computer Models and Psychoanalytic Ideas: Epistemological Applications', Society for Psychoanalytic Psychotherapy Bulletin 6(1), pp. 23-33.Google Scholar
  40. Genova, J. (1994a), 'Response to Anderson and Keith', Social Epistemology 8(4), pp. 341-343.Google Scholar
  41. Genova, J. (1994b), 'Turing's Sexual Guessing Game', Social Epistemology 8(4), pp. 313-326.Google Scholar
  42. Guccione, S. and Tamburrini, G. (1988), 'Turing's Test Revisited', in Proceedings of the 1998 IEEE International Conference on Systems, Man and Cybernetics, Vol. 1. Beijing and Shenyang, China, pp. 38-41.Google Scholar
  43. Guillen, M.A. (1983), 'The Test of Turing', Psychology Today 17(12), pp. 80-81.Google Scholar
  44. Gunderson, K. (1964), 'The Imitation Game', Mind 73 pp. 234-245.Google Scholar
  45. Gunderson, K. (1967), Mentality and Machines, New York, NY: Doubleday.Google Scholar
  46. Halpern, M. (1987), 'Turing's Test and the Ideology of Artificial Intelligence', Artificial Intelligence Review 1(2), pp. 79-93.Google Scholar
  47. Harnad, S. (1989), 'Minds, Machines and Searle', Journal of Experimental and Theoretical Artificial Intelligence 1(1), pp. 5-25.Google Scholar
  48. Harnad, S. (1990), 'The Symbol Grounding Problem', Physica D 42, pp. 335-346.Google Scholar
  49. Harnad, S. (1991), 'Other Bodies, Other Minds: A Machine Incarnation of an Old Philosophical Problem', Minds and Machines 1, pp. 43-54.Google Scholar
  50. Harnad, S. (1992), 'The Turing Test is not a Trick: Turing Indistinguishability is a Scientific Criterion', SIGART Bulletin 3(4), pp. 9-10.Google Scholar
  51. Harnad, S. (1994), 'Does Mind Piggyback on Robotic and Symbolic Capacity? in H. Morowitz and J. Singer, eds. The Mind, the Brain, and Complex Adaptive Systems, Reading, MA.: Addison Wesley.Google Scholar
  52. Harnad, S. (1998), 'Turing Indistinguishability and the Blind Watchmaker', in G. Mulhauser, ed. Evolving Consciousness Amsterdam: John Benjamins.Google Scholar
  53. Hauser, L. (1993), 'Reaping the Whirlwind: Reply to Harnad's “Other Bodies, Other Minds”', Minds and Machines 3, pp. 219-237.Google Scholar
  54. Hauser, L. (1997), 'Searle's Chines Box: Debunking the Chinese Room Argument', Minds and Machines 7, pp. 199-226.Google Scholar
  55. Hayes, B. (1998), 'Turing's Test', Muse 8.Google Scholar
  56. Hayes, P. and Ford, K. (1995), 'Turing Test Considered Harmful', in Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Vol. 1, pp. 972-977.Google Scholar
  57. Hayes, P., Harnard, S., Perlis, D. and Block, N. (1992), 'Virtual Symposium on Virtual Mind', Minds and Machines 3(2), pp. 217-238.Google Scholar
  58. Henley, T.B. (1990), 'Chauvinism and Science: Another Reply to Shanon', Journal for the Theory of Social Behavior 20(1), pp. 93-95.Google Scholar
  59. Hodges, A. (1983), Alan Turing: The Enigma, New York, NY: Simon & Schuster.Google Scholar
  60. Hofstadter, D.R. (1982), 'The Turing Test: A Coffee-House Conversation', in D. Hofstadter and D. Dennett, eds. The Mind's I: Fantasies and Reflections on Self and Soul, London, UK: Penguin Books, pp. 69-95.Google Scholar
  61. Jacquette, D. (1993a), 'A Turing Test Conversation', Philosophy 68, pp. 231-233.Google Scholar
  62. Jacquette, D. (1993b), 'Who's Afraid of the Turing Test', Behavior and Philosophy 20, pp. 63-74.Google Scholar
  63. Johnson-Laird, P. (1988), The Computer and the Mind, Cambridge, MA.: Harvard University Press.Google Scholar
  64. Karelis, C. (1986), 'Reflections on the Turing Test', Journal for the Theory of Social Behavior 16, pp. 161-172.Google Scholar
  65. Kugel, P. (1986), 'Thinking May Be More Than Computing', Cognition 22, pp. 137-198.Google Scholar
  66. Kugel, P. (1990), 'Is It Time to Replace Turing's Test?', 1990Workshop Artificial Intelligence: Emerging Science or Dying Art Form. Sponsored by SUNY Binghamton's Program in Philosophy and Computer and Systems Sciences and AAAI.Google Scholar
  67. Lassegue, J. (1988), 'What Kind of Turing Test did Turing Have in Mind?', Tekhnema 3, pp. 37-58.Google Scholar
  68. Leiber, J. (1989), 'Shanon on the Turing Test', Journal for the Theory of Social Behavior 19(2), pp. 257-259.Google Scholar
  69. Leiber, J. (1992), 'The Light Bulb and the Turing-Tested Machine', Journal for the Theory of Social Behaviour 22, pp. 25-39.Google Scholar
  70. Leiber, J. (1995), 'On Turing's Turing Test and Why the Matter Matters', Synthese 105, pp. 59-69.Google Scholar
  71. Loebner, H.G (1994), 'In Response', Communications of the Association for Computing Machinery 37, pp. 79-82.Google Scholar
  72. Lucas, J. (1996), 'Minds, Machines and Gödel', Philosophy 36, pp. 112-127.Google Scholar
  73. Lucas, J. (1996), 'Minds, Machines and Gödel: A Retrospect', in P. Millican and A. Clark, eds. Machines and Mind, Oxford UK: Oxford University Press.Google Scholar
  74. Maloney, J. (1987), 'The Right Stuff', Synthese 70, pp. 349-372.Google Scholar
  75. Marinoff, L. (1995), 'Has Turing Slain the Jabberwock?', Informatica 19(4), pp. 513-526.Google Scholar
  76. Mauldin, M. (1994), 'Chatterbots, Tinymuds and the Turing Test: Entering the Loebner Prize Competition', in Proceedings of the Twelfth National Conference on Artificial Intelligence, Vol. 1, Seattle, WA, pp. 16-21.Google Scholar
  77. Mays, W. (1952), 'Can Machines Think?', Philosophy 27, pp. 148-162.Google Scholar
  78. McIlvenny, P. (1993), 'Constructing Societies and Social Machines: Stepping out of the Turing Test Discourse', Journal of Intelligent Systems 2(2-4), pp. 119-156.Google Scholar
  79. Michie, D. (1990), 'The Superarticulacy Phenomenon in the Context of Software Manufacture', in D. Partridge and Y. Wilks, eds.: The Foundations of Artificial Intelligence, Cambridge, MA.: MIT Press, pp. 411-439.Google Scholar
  80. Michie, D. (1994), 'Consciousness as an Engineering Issue, Part 1'. Journal of Consciousness Studies 1(2), pp. 52-66.Google Scholar
  81. Michie, D. (1995), 'Consciousness as an Engineering Issue, Part 2', Journal of Consciousness Studies 2(1), pp. 182-195.Google Scholar
  82. Michie, D. (1996), 'Turing's Test and Conscious Thought' in P. Millican and A. Clark, eds. Machines and Thought: The Legacy of Alan Turing, Oxford, UK: Oxford University Press, pp. 27-51. Originally printed in Artificial Intelligence 60, pp. 1-22, 1993.Google Scholar
  83. Millar, P.H. (1973), 'On the Point of the Imitation Game', Mind 82, pp. 595-597.Google Scholar
  84. Minsky, M. (1985), 'Communication with Alien Intelligence', in E. Regis, ed. Extraterrestrials: Science and Alien Intelligence, Cambridge, UK: Cambridge University Press.Google Scholar
  85. Moon, Y., Naas, C., Morkes, J., Kim, E.-Y. and Fogg, B. (1994), 'Computers are Social Actors', in Proceedings of the CHI Conference, Boston, MA, pp. 72-78.Google Scholar
  86. Moor, J.H. (1976), 'An Analysis of the Turing Test', Philosophical Studies 30, pp. 249-257.Google Scholar
  87. Moor, J.H. (1978), 'Explaining Computer Behavior', Philosophical Studies 34, pp. 325-327.Google Scholar
  88. Narayanan, A. (1996), 'The Intentional Stance and the Imitation Game', in P. Millican and A. Clark, eds. Machines and Thought: The Legacy of Alan Turing, Oxford, UK: Oxford University Press, pp. 63-79.Google Scholar
  89. Parsons, H. (1990), 'Turing on the Turing Test', in W. Karwowski and M. Rahimi, eds. Ergonomics of Hybrid Automated Systems II, Amsterdam: Elsevier.Google Scholar
  90. Pinksy, L. (1951), 'Do Machines Think About Thinking', Mind 60(239), pp. 397-398.Google Scholar
  91. Platt, C. (1995), 'What's It Mean To Be Human, Anyway?', Wired.Google Scholar
  92. Purtill, R.L. (1971), 'Beating the Imitation Game', Mind 80, 290-294.Google Scholar
  93. Rankin, T. (1987), 'The Turing Paradigm: A Critical Assessment', Dialogue 29, pp. 50-55.Google Scholar
  94. Reader, A. (1969), 'Steps Toward Genuine Artificial Intelligence', Acta Psychologica 29(3), pp. 279-289.Google Scholar
  95. Rey, G. (1986), 'What's Really Going on in the Chinese Room?', Philosophical Studies 50, pp. 196-285.Google Scholar
  96. Richardson, R. (1982), 'Turing Tests for Intelligence: Ned Block's Defense of Psychologism', Philosophical Studies 41, pp. 421-426.Google Scholar
  97. Roberts, L. (1990), 'Searle's Extension of the Chinese Room to Connectionist Machines', Journal of Experimental and Theoretical Artificial Intelligence 2, pp. 185-187.Google Scholar
  98. Sampson, G. (1973), 'In Defence of Turing', Mind 82, pp. 592-594.Google Scholar
  99. Saygin, A.P. (1999), 'Turing Test and Conversation', Master's thesis, Bilkent University, Ankara, Turkey. Technical Report BU-CEIS-9911.Google Scholar
  100. Schweizer, P. (1998), 'The Truly Total Turing Test', Minds and Machines 8, pp. 263-272.Google Scholar
  101. Searle, J.R. (1980), 'Minds, Brains and Programs', Behavioral and Brain Sciences 3, pp. 417-424.Google Scholar
  102. Searle, J.R. (1990), 'Is the Brain's Mind a Computer Program?', Scientific American 3(262), pp. 26-31.Google Scholar
  103. Shanon, B. (1989), 'A Simple Comment Regarding the Turing Test', Journal for the Theory of Social Behavior 19(2), pp. 249-256.Google Scholar
  104. Shanon, B. (1991), 'Chauvinism: A Misdirected Accusation', Journal for the Theory of Social Behavior 21(3), pp. 369-371.Google Scholar
  105. Sharma, R. and Conrath, D. (1993), 'Evaluating Expert Systems: A Review of Applicable Choices', Artificial Intelligence Review 7(2), pp. 77-91.Google Scholar
  106. Shieber, S.M. (1994), 'Lessons from a Restricted Turing Test', Communications of the Association for Computing Machinery 37, pp. 70-78.Google Scholar
  107. Stalker, D. (1978), 'Why Machines Can't Think: A Reply to James Moor', Philosophical Studies 34, pp. 317-320.Google Scholar
  108. Stevenson, J.G. (1976), 'On the Imitation Game', Philosophia 6, pp. 131-133.Google Scholar
  109. Turing, A. (1950), 'Computing Machinery and Intelligence', Mind 59(236), pp. 433-460.Google Scholar
  110. Turing, A. (1969), 'Intelligent Machinery', in D.M.B. Meltzer ed. Machine Intelligence 5, Edinburgh University Press, pp. 3-23. Originally, a National Physics Laboratory Report, 1948.Google Scholar
  111. Wallace, R.S. (1997), 'The Lying Game', Wired.Google Scholar
  112. Watt, S. (1996), 'Naive Psychology and the Inverted Turing Test', Psycoloquy 7(14). 14.Google Scholar
  113. Weizenbaum, J. (1996), 'ELIZA-A Computer Program for the Study of Natural Language Communication Between Men and Machines', Communications of the ACM 9, pp. 36-45.Google Scholar
  114. Weizenbaum, J. (1976), Computer Power and Human Reason: From Judgement to Calculation, San Francisco, CA: W.H. Freeman.Google Scholar
  115. Whalen, T. (1995), 'How I Lost the Contest and Re-Evaluated Humanity', Scholar
  116. Whitby, B. (1996), 'The Turing Test: AI's Biggest Blind Alley?', in P. Millican and A. Clarke, eds. Machines and Thought: The Legacy of Alan Turing, Oxford, UK: Oxford University Press, pp. 53-63.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Ayse Pinar Saygin
  • Ilyas Cicekli
  • Varol Akman

There are no affiliations available

Personalised recommendations