Minds and Machines

, Volume 17, Issue 4, pp 391–444 | Cite as

Universal Intelligence: A Definition of Machine Intelligence

  • Shane LeggEmail author
  • Marcus Hutter


A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: we take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of intelligence for arbitrary machines. We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense. We then show how this formal definition is related to the theory of universal optimal learning agents. Finally, we survey the many other tests and definitions of intelligence that have been proposed for machines.


AIXI Complexity theory Intelligence Theoretical foundations Turing test Intelligence tests Measures Definitions 



This work was supported by the Swiss NSF grant 200020-107616.


  1. Albus, J. S. (1991). Outline for a theory of intelligence. IEEE Transactions Systems, Man and Cybernetics, 21(3), 473–509.CrossRefMathSciNetGoogle Scholar
  2. Alvarado, N., Adams, S., Burbeck, S., & Latta, C. (2002). Beyond the Turing test: Performance metrics for evaluating a computer simulation of the human mind. In Performance metrics for intelligent systems workshop. Gaithersburg, MD, North-Holland.Google Scholar
  3. Anastasi, A. (1992). What counselors should know about the use and interpretation of psychological tests. Journal of Counseling and Development, 70(5), 610–615.Google Scholar
  4. Asohan, A. (2003). Leading humanity forward. The Star, October 14.Google Scholar
  5. Bell, T. C., Cleary, J. G., & Witten, I. H. (1990). Text compression. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  6. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Belmont, MA: Athena Scientific.zbMATHGoogle Scholar
  7. Binet, A. (1911). Les idees modernes sur les enfants. Paris: Flammarion.Google Scholar
  8. Binet, A., & Simon, T. (1905). Methodes nouvelles por le diagnostic du niveai intellectuel des anormaux. L’Année Psychologique, 11, 191–244.Google Scholar
  9. Bingham, W. V. (1937). Aptitudes and aptitude testing. New York: Harper & Brothers.Google Scholar
  10. Block, N. (1981). Psychologism and behaviorism. Philosophical Review, 90, 5–43.CrossRefGoogle Scholar
  11. Boring, E. G. (1923). Intelligence as the tests test it. New Republic, 35, 35–37.Google Scholar
  12. Bringsjord, S., & Schimanski, B. (2003). What is artificial intelligence? Psychometric AI as an answer. Eighteenth International Joint Conference on Artificial Intelligence, 18, 887–893.Google Scholar
  13. Calude, C. S. (2002). Information and randomness (2nd ed.). Berlin: Springer.zbMATHGoogle Scholar
  14. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.Google Scholar
  15. Cattell, R. B. (1987). Intelligence: Its structure, growth, and action. New York: Elsevier.Google Scholar
  16. Chaitin, G. J. (1982). Gödel’s theorem and information. International Journal of Theoretical Physics, 22, 941–954.CrossRefMathSciNetGoogle Scholar
  17. Dowe, D. L., & Hajek, A. R. (1998). A non-behavioural, computational extension to the Turing test. In International conference on computational intelligence & multimedia applications (ICCIMA ’98) (pp. 101–106). Gippsland, Australia.Google Scholar
  18. Drever, J. (1952). A dictionary of psychology. Harmondsworth: Penguin Books.Google Scholar
  19. Edmonds, B. (2006). The social embedding of intelligence—towards producing a machine that could pass the turing test. In The Turing test sourcebook: Philosophical and methodological issues in the quest for the thinking computer. Dordrecht: Kluwer.Google Scholar
  20. Eisner, J. (1991). Cognitive science and the search for intelligence. Invited paper presented to the Socratic Society, University of Cape Town.Google Scholar
  21. Fiévet, C. (2005). Mesurer l’intelligence d’une machine. In Le Monde de l’intelligence (Vol. 1, pp. 42–45). Paris: Mondeo publishing.Google Scholar
  22. Fogel, D. B. (1995). Review of computational intelligence: Imitating life. Proceedings of the IEEE, 83(11).Google Scholar
  23. Ford, K. M., & Hayes, P. J. (1998). On computational wings: Rethinking the goals of artificial intelligence. Scientific American, 9, (4), 78–83.Google Scholar
  24. French, R. M. (1990). Subcognition and the limits of the Turing test. Mind, 99, 53–65.CrossRefMathSciNetGoogle Scholar
  25. Gardner, H. (1993). Frames of mind: Theory of multiple intelligences. London: Fontana Press.Google Scholar
  26. Goertzel, B. (2006). The hidden pattern. Brown Walker Press.Google Scholar
  27. Gottfredson, L. S. (1997). Mainstream science on intelligence: An editorial with 52 signatories, history, and bibliography. Intelligence, 24(1), 13–23.CrossRefGoogle Scholar
  28. Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24(1), 79–132.CrossRefGoogle Scholar
  29. Gottfredson, L. S. (2002). g: Highly general and highly practical. In R. J. Sternberg & E. L. Grigorenko (Eds.), The general factor of intelligence: How general is it? (pp. 331–380). Hillsdale, NJ: Erlbaum.Google Scholar
  30. Gould, S. J. (1981). The Mismeasure of man. New York: W. W. Norton & Company.Google Scholar
  31. Graham-Rowe, D. (2005). Spotting the bots with brains. In New scientist magazine (Vol. 2512, p. 27).Google Scholar
  32. Gregory, R. L. (1998). The Oxford companion to the mind. Oxford, UK: Oxford University Press.Google Scholar
  33. Gudwin, R. R. (2000). Evaluating intelligence: A computational semiotics perspective. In IEEE international conference on systems, man and cybernetics (pp. 2080–2085). Nashville, TN.Google Scholar
  34. Guilford, J. P. (1967). The nature of human intelligence. New York: McGraw-Hill.Google Scholar
  35. Gunderson, K. (1971). Mentality and machines. Garden City, NY: Doubleday and company.Google Scholar
  36. Harnad, S. (1989). Minds, machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence, 1, 5–25.CrossRefGoogle Scholar
  37. Haugeland, J. (1981). Mind design: Philosophy, psychology, and artificial intelligence. MIT Press: Bradford Books.Google Scholar
  38. Henmon, C. V. A. (1921). The measurement of intelligence. School and Society, 13, 151–158.Google Scholar
  39. Herman, L. M., & Pack, A. A. (1994). Animal intelligence: Historical perspectives and contemporary approaches. In R. Sternberg (Ed.), Encyclopedia of human intelligence (pp. 86–96). New York: Macmillan.Google Scholar
  40. Hernández-Orallo, J. (2000a). Beyond the Turing test. Journal of Logic, Language and Information, 9(4), 447–466.zbMATHCrossRefMathSciNetGoogle Scholar
  41. Hernández-Orallo, J. (2000b). On the computational measurement of intelligence factors. In Performance metrics for intelligent systems workshop (pp. 1–8). Gaithersburg, MD.Google Scholar
  42. Hernández-Orallo, J., & Minaya-Collado, N. (1998). A formal definition of intelligence based on an intensional variant of Kolmogorov complexity. In Proceedings of the international symposium of engineering of intelligent systems (EIS’98) (pp. 146–163). ICSC Press.Google Scholar
  43. Herrnstein, R. J., & Murray, C. (1996). The bell curve: Intelligence and class structure in American life. New York: Free Press.Google Scholar
  44. Horn, J. (1970). Organization of data on life-span development of human abilities. In R. Goulet & P. B. Baltes (Eds.), Life-span developmental psychology: Research and theory. New York: Academic Press.Google Scholar
  45. Horst, J. (2002). A native intelligence metric for artificial systems. In Performance metrics for intelligent systems workshop. Gaithersburg, MDGoogle Scholar
  46. Hsu, F. H., Campbell, M. S., & Hoane, A. J. (1995). Deep blue system overview. In Proceedings of the 1995 international conference on supercomputing (pp. 240–244).Google Scholar
  47. Hutchens, J. L. (1996). How to pass the Turing test by cheating.
  48. Hutter, M. (2001a). Towards a universal theory of artificial intelligence based on algorithmic probability and sequential decisions. In Proceedings of the 12th Eurpean conference on machine learning (ECML-2001) (pp. 226–238).Google Scholar
  49. Hutter, M. (2001b). Universal sequential decisions in unknown environments. In Proceedings of the 5th European workshop on reinforcement learning (EWRL-5), 27 (pp. 25–26).Google Scholar
  50. Hutter, M. (2005). Universal artificial intelligence: Sequential decisions based on algorithmic probability. Berlin: Springer, 300 pp.,
  51. Hutter, M. (2006a). General discounting versus average reward. In Proceedings of the 17th international conference on algorithmic learning theory (ALT-06) vol 4264 of LNAI (pp. 244–258). Barcelona. Google Scholar
  52. Hutter, M. (2006b). The human knowledge compression prize.
  53. Hutter, M. (2007a). On universal prediction and Bayesian confirmation. Theoretical Computer Science, 384(1), 33–48.zbMATHCrossRefMathSciNetGoogle Scholar
  54. Hutter, M. (2007b). Universal algorithmic intelligence: A mathematical top-down approach. In Artificial general intelligence (pp. 227–290). Berlin: Springer. Google Scholar
  55. Johnson, W. L. (1992). Needed: A new test of intelligence. SIGARTN: SIGART Newsletter (ACM Special Interest Group on Artificial Intelligence), 3(4), 7–9.Google Scholar
  56. Johnson-Laird, P. N., & Wason, P. C. (1977). A theoretical analysis of insight into a reasoning task. In P. N. Johnson-Laird & P. C. Wason (Eds.) Thinking: Readings in cognitive science (pp. 143–157). Cambridge, UK: Cambridge University Press.Google Scholar
  57. Kaufman, A. S. (2000). Tests of intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence. Cambridge, UK: Cambridge University Press.Google Scholar
  58. Kurzweil, R. (2000). The age of spiritual machines: When computers exceed human intelligence. East Rutherford, NJ: Penguin.Google Scholar
  59. Legg, S., & Hutter, M. (2004). Ergodic MDPs admit self-optimising policies. Technical Report IDSIA-21-04, IDSIA.Google Scholar
  60. Legg, S., & Hutter, M. (2004). A taxonomy for abstract environments. Technical Report IDSIA-20-04, IDSIA.Google Scholar
  61. Legg, S., & Hutter, M. (2005). A universal measure of intelligence for artificial agents. In Proceedings of the 21st international joint conference on artificial intelligence (IJCAI-2005) (pp. 1509–1510). Edinburgh.Google Scholar
  62. Legg, S., & Hutter, M. (2006a). A formal definition of intelligence for artificial systems. In Proceedings anniversary summit of artificial intelligence. Monte Verita, Switzerland.Google Scholar
  63. Legg, S., & Hutter, M. (2006b). A formal measure of machine intelligence. In Annual machine learning conference of Belgium and The Netherlands (Benelearn’06). Ghent.Google Scholar
  64. Legg, S., & Hutter, M. (2007). A collection of definitions of intelligence. In B. Goertzel & P. Wang (Eds.), Advances in artificial general intelligence: Concepts, architectures and algorithms vol 157 of frontiers in artificial intelligence and applications. (pp. 17–24). Amsterdam, NL: IOS press. Online version:
  65. Levin, L. A. (1973). Universal sequential search problems. Problems of Information Transmission, 9, 265–266.Google Scholar
  66. Li, M., & Vitányi, P. M. B. (1997). An introduction to Kolmogorov complexity and its applications (2nd ed.). Berlin: Springer.Google Scholar
  67. Loebner, H. G. (1990). The Loebner prize—the first Turing test.
  68. Looks, M., Goertzel, B., & Pennachin, C. (2004). Novamente: An integrative architecture for general intelligence. In AAAI fall symposium, achieving human-level intelligence.Google Scholar
  69. Macphail, E. M. (1985). Vertebrate intelligence: The null hypothesis. In L. Weiskrantz (Ed.), Animal intelligence (pp. 37–50). Oxford: Clarendon.Google Scholar
  70. Mahoney, M. V. (1999). Text compression as a test for artificial intelligence. In AAAI/IAAI.Google Scholar
  71. Masum, H., Christensen, S., & Oppacher, F. (2002). The Turing ratio: Metrics for open-ended tasks. In GECCO 2002: Proceedings of the genetic and evolutionary computation conference (pp. 973–980). New York: Morgan Kaufmann Publishers.Google Scholar
  72. McCarthy, J. (2004). What is artificial intelligence?
  73. Minsky, M. (1985). The society of mind. New York: Simon and Schuster.Google Scholar
  74. Müller, M. (2006). Stationary algorithmic probability. Technical report, TU Berlin, Berlin,
  75. Neisser, U., Boodoo, G., Bouchard, T. J. Jr., Boykin, A. W., Brody, N., Ceci, S. J., Halpern, D. F., Loehlin, J. C., Perloff, R., Sternberg, R. J., & Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51(2), 77–101.Google Scholar
  76. Newell, A., & Simon, H. A. (1976). Computer science as empirical enquiry: Symbols and search. Communications of the ACM 19, 3, 113–126.CrossRefMathSciNetGoogle Scholar
  77. Poole, D., Mackworth, A., & Goebel, R. (1998). Computational intelligence: A logical approach. New York: Oxford University Press.zbMATHGoogle Scholar
  78. Raven, J. (2000). The Raven’s progressive matrices: Change and stability over culture and time. Cognitive Psychology, 41, 1–48.CrossRefGoogle Scholar
  79. Reznikova, Zh. I., & Ryabko, B. Ya. (1986). Analysis of the language of ants by information-theoretic methods. Problems of Information Transmission, 22, 245–249.zbMATHGoogle Scholar
  80. Sanghi, P., & Dowe, D. L. (2003). A computer program capable of passing I.Q. tests. In Proceedings of the 4th ICCS international conference on cognitive science (ICCS’03) (pp. 570–575). Sydney, NSW, Australia.Google Scholar
  81. Saygin, A., Cicekli, I., & Akman, V. (2000). Turing test: 50 years later. Minds and Machines, 10(4), 463–518.CrossRefGoogle Scholar
  82. Schmidhuber, J. (2002). The speed prior: A new simplicity measure yielding near-optimal computable predictions. In Proceedings of the 15th annual conference on computational learning theory (COLT 2002), Lecture notes in artificial intelligence (pp. 216–228). Sydney, Australia: Springer.Google Scholar
  83. Schweizer, P. (1998). The truly total Turing test. Minds and Machines, 8, 263–272.CrossRefGoogle Scholar
  84. Searle, J. (1980). Minds, brains, and programs. Behavioral & Brain Sciences, 3, 417–458.CrossRefGoogle Scholar
  85. Shieber, S. (1994). Lessons from a restricted Turing test. CACM: Communications of the ACM, 37(6), 70–78.Google Scholar
  86. Simonton, D. K. (2003). An interview with Dr. Simonton. In J. A. Plucker (Ed.), Human intelligence: Historical influences, current controversies, teaching resources.∼intell
  87. Slatter, J. (2001). Assessment of children: Cognitive applications (4th ed.). San Diego: Jermone M. Satler Publisher Inc.Google Scholar
  88. Slotnick, B. M., & Katz, H. M. (1974). Olfactory learning-set formation in rats. Science, 185, 796–798.CrossRefGoogle Scholar
  89. Smith, W. D. (2006). Mathematical definition of “intelligence” (and consequences).∼wds/homepage/works.html
  90. Spearman, C. E. (1927). The abilities of man, their nature and measurement. New York: Macmillan.zbMATHGoogle Scholar
  91. Stern, W. L. (1912). Psychologischen Methoden der Intelligenz-Prüfung. Leipzig: Barth.Google Scholar
  92. Sternberg, R. J. (1985). Beyond IQ: A triacrchi theory of human intelligence. New York: Cambridge University Press.Google Scholar
  93. Sternberg, R. J. (Ed.) (2000). Handbook of intelligence. Cambridge University Press.Google Scholar
  94. Sternberg, R. J. (2003). An interview with Dr. Sternberg. In J. A. Plucker (Ed.), Human intelligence: Historical influences, current controversies, teaching resources.∼intell
  95. Sternberg, R. J., & Berg, C. A. (1986). Quantitative integration: Definitions of intelligence: A comparison of the 1921 and 1986 symposia. In R. J. Sternberg & D. K. Detterman (Eds.), What is intelligence? Contemporary wiewpoints on its nature and definition (pp. 155–162). Norwood, NJ: Ablex.Google Scholar
  96. Sternberg, R. J., & Grigorenko, E. L. (Eds.). (2002). Dynamic testing: The nature and measurement of learning potential. Cambridge, UK: Cambridge University Press.Google Scholar
  97. Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge MA: MIT Press.Google Scholar
  98. Terman, L. M., & Merrill, M. A. (1950). The Stanford-Binet intelligence scale. Boston: Houghton Mifflin.Google Scholar
  99. Thurstone, L. L. (1938). Primary mental abilities. Chicago: University of Chicago Press.Google Scholar
  100. Treister-Goren, A., Dunietz, J., & Hutchens, J. L. (2000). The developmental approach to evaluating artificial intelligence—a proposal. In Performance metrics for intelligence systems.Google Scholar
  101. Treister-Goren, A., & Hutchens, J. L. (2001). Creating AI: A unique interplay between the development of learning algorithms and their education. In Proceeding of the first international workshop on epigenetic robotics.Google Scholar
  102. Turing, A. M. (1950). Computing machinery and intelligence. Mind, LIX(236), 433–460.Google Scholar
  103. Voss, P. (2005). Essentials of general intelligence: The direct path to AGI. In B. Goertzel & C. Pennachin (Eds.), Artificial general intelligence. Berlin: Springer.Google Scholar
  104. Wallace, C. S. (2005). Statistical and inductive inference by minimum message length. Berlin: Springer.zbMATHGoogle Scholar
  105. Wang, P. (1995). On the working definition of intelligence. Technical Report 94, Center for Research on Concepts and Cognition, Indiana University.Google Scholar
  106. Watt, S. (1996). Naive psychology and the inverted Turing test. Psycoloquy, 7(14).Google Scholar
  107. Wechsler, D. (1958). The measurement and appraisal of adult intelligence (4 ed.). Baltimore: Williams & Wilkinds.Google Scholar
  108. Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transaction on Evolutionary Computation, 1(1), 67–82.CrossRefGoogle Scholar
  109. Zentall, T. R. (1997). Animal memory: The role of instructions. Learning and Motivation, 28, 248–267.CrossRefGoogle Scholar
  110. Zentall, T. R. (2000). Animal intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence. Cambridge, UK: Cambridge University Press.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  1. 1.IDSIAManno-LuganoSwitzerland
  2. 2.RSISE@ANU and SML@NICTACanberraAustralia

Personalised recommendations