Universal Intelligence: A Definition of Machine Intelligence

Abstract

A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: we take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of intelligence for arbitrary machines. We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense. We then show how this formal definition is related to the theory of universal optimal learning agents. Finally, we survey the many other tests and definitions of intelligence that have been proposed for machines.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Albus, J. S. (1991). Outline for a theory of intelligence. IEEE Transactions Systems, Man and Cybernetics, 21(3), 473–509.

    Article  MathSciNet  Google Scholar 

  2. Alvarado, N., Adams, S., Burbeck, S., & Latta, C. (2002). Beyond the Turing test: Performance metrics for evaluating a computer simulation of the human mind. In Performance metrics for intelligent systems workshop. Gaithersburg, MD, North-Holland.

  3. Anastasi, A. (1992). What counselors should know about the use and interpretation of psychological tests. Journal of Counseling and Development, 70(5), 610–615.

    Google Scholar 

  4. Asohan, A. (2003). Leading humanity forward. The Star, October 14.

  5. Bell, T. C., Cleary, J. G., & Witten, I. H. (1990). Text compression. Englewood Cliffs, NJ: Prentice Hall.

  6. Bertsekas, D. P., & Tsitsiklis, J. N. (1996). Neuro-dynamic programming. Belmont, MA: Athena Scientific.

    Google Scholar 

  7. Binet, A. (1911). Les idees modernes sur les enfants. Paris: Flammarion.

    Google Scholar 

  8. Binet, A., & Simon, T. (1905). Methodes nouvelles por le diagnostic du niveai intellectuel des anormaux. L’Année Psychologique, 11, 191–244.

    Google Scholar 

  9. Bingham, W. V. (1937). Aptitudes and aptitude testing. New York: Harper & Brothers.

    Google Scholar 

  10. Block, N. (1981). Psychologism and behaviorism. Philosophical Review, 90, 5–43.

    Article  Google Scholar 

  11. Boring, E. G. (1923). Intelligence as the tests test it. New Republic, 35, 35–37.

    Google Scholar 

  12. Bringsjord, S., & Schimanski, B. (2003). What is artificial intelligence? Psychometric AI as an answer. Eighteenth International Joint Conference on Artificial Intelligence, 18, 887–893.

    Google Scholar 

  13. Calude, C. S. (2002). Information and randomness (2nd ed.). Berlin: Springer.

    Google Scholar 

  14. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press.

    Google Scholar 

  15. Cattell, R. B. (1987). Intelligence: Its structure, growth, and action. New York: Elsevier.

    Google Scholar 

  16. Chaitin, G. J. (1982). Gödel’s theorem and information. International Journal of Theoretical Physics, 22, 941–954.

    Article  MathSciNet  Google Scholar 

  17. Dowe, D. L., & Hajek, A. R. (1998). A non-behavioural, computational extension to the Turing test. In International conference on computational intelligence & multimedia applications (ICCIMA ’98) (pp. 101–106). Gippsland, Australia.

  18. Drever, J. (1952). A dictionary of psychology. Harmondsworth: Penguin Books.

    Google Scholar 

  19. Edmonds, B. (2006). The social embedding of intelligence—towards producing a machine that could pass the turing test. In The Turing test sourcebook: Philosophical and methodological issues in the quest for the thinking computer. Dordrecht: Kluwer.

  20. Eisner, J. (1991). Cognitive science and the search for intelligence. Invited paper presented to the Socratic Society, University of Cape Town.

  21. Fiévet, C. (2005). Mesurer l’intelligence d’une machine. In Le Monde de l’intelligence (Vol. 1, pp. 42–45). Paris: Mondeo publishing.

  22. Fogel, D. B. (1995). Review of computational intelligence: Imitating life. Proceedings of the IEEE, 83(11).

  23. Ford, K. M., & Hayes, P. J. (1998). On computational wings: Rethinking the goals of artificial intelligence. Scientific American, 9, (4), 78–83.

  24. French, R. M. (1990). Subcognition and the limits of the Turing test. Mind, 99, 53–65.

    Article  MathSciNet  Google Scholar 

  25. Gardner, H. (1993). Frames of mind: Theory of multiple intelligences. London: Fontana Press.

  26. Goertzel, B. (2006). The hidden pattern. Brown Walker Press.

  27. Gottfredson, L. S. (1997). Mainstream science on intelligence: An editorial with 52 signatories, history, and bibliography. Intelligence, 24(1), 13–23.

    Article  Google Scholar 

  28. Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24(1), 79–132.

    Article  Google Scholar 

  29. Gottfredson, L. S. (2002). g: Highly general and highly practical. In R. J. Sternberg & E. L. Grigorenko (Eds.), The general factor of intelligence: How general is it? (pp. 331–380). Hillsdale, NJ: Erlbaum.

  30. Gould, S. J. (1981). The Mismeasure of man. New York: W. W. Norton & Company.

  31. Graham-Rowe, D. (2005). Spotting the bots with brains. In New scientist magazine (Vol. 2512, p. 27).

  32. Gregory, R. L. (1998). The Oxford companion to the mind. Oxford, UK: Oxford University Press.

    Google Scholar 

  33. Gudwin, R. R. (2000). Evaluating intelligence: A computational semiotics perspective. In IEEE international conference on systems, man and cybernetics (pp. 2080–2085). Nashville, TN.

  34. Guilford, J. P. (1967). The nature of human intelligence. New York: McGraw-Hill.

    Google Scholar 

  35. Gunderson, K. (1971). Mentality and machines. Garden City, NY: Doubleday and company.

    Google Scholar 

  36. Harnad, S. (1989). Minds, machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence, 1, 5–25.

    Article  Google Scholar 

  37. Haugeland, J. (1981). Mind design: Philosophy, psychology, and artificial intelligence. MIT Press: Bradford Books.

  38. Henmon, C. V. A. (1921). The measurement of intelligence. School and Society, 13, 151–158.

    Google Scholar 

  39. Herman, L. M., & Pack, A. A. (1994). Animal intelligence: Historical perspectives and contemporary approaches. In R. Sternberg (Ed.), Encyclopedia of human intelligence (pp. 86–96). New York: Macmillan.

    Google Scholar 

  40. Hernández-Orallo, J. (2000a). Beyond the Turing test. Journal of Logic, Language and Information, 9(4), 447–466.

    MATH  Article  MathSciNet  Google Scholar 

  41. Hernández-Orallo, J. (2000b). On the computational measurement of intelligence factors. In Performance metrics for intelligent systems workshop (pp. 1–8). Gaithersburg, MD.

  42. Hernández-Orallo, J., & Minaya-Collado, N. (1998). A formal definition of intelligence based on an intensional variant of Kolmogorov complexity. In Proceedings of the international symposium of engineering of intelligent systems (EIS’98) (pp. 146–163). ICSC Press.

  43. Herrnstein, R. J., & Murray, C. (1996). The bell curve: Intelligence and class structure in American life. New York: Free Press.

  44. Horn, J. (1970). Organization of data on life-span development of human abilities. In R. Goulet & P. B. Baltes (Eds.), Life-span developmental psychology: Research and theory. New York: Academic Press.

  45. Horst, J. (2002). A native intelligence metric for artificial systems. In Performance metrics for intelligent systems workshop. Gaithersburg, MD

  46. Hsu, F. H., Campbell, M. S., & Hoane, A. J. (1995). Deep blue system overview. In Proceedings of the 1995 international conference on supercomputing (pp. 240–244).

  47. Hutchens, J. L. (1996). How to pass the Turing test by cheating. http://www.cs.umbc.edu/471/current/papers/hutchens.pdf

  48. Hutter, M. (2001a). Towards a universal theory of artificial intelligence based on algorithmic probability and sequential decisions. In Proceedings of the 12th Eurpean conference on machine learning (ECML-2001) (pp. 226–238).

  49. Hutter, M. (2001b). Universal sequential decisions in unknown environments. In Proceedings of the 5th European workshop on reinforcement learning (EWRL-5), 27 (pp. 25–26).

  50. Hutter, M. (2005). Universal artificial intelligence: Sequential decisions based on algorithmic probability. Berlin: Springer, 300 pp., http://www.hutter1.net/ai/uaibook.htm

  51. Hutter, M. (2006a). General discounting versus average reward. In Proceedings of the 17th international conference on algorithmic learning theory (ALT-06) vol 4264 of LNAI (pp. 244–258). Barcelona.

  52. Hutter, M. (2006b). The human knowledge compression prize. http://prize.hutter1.net

  53. Hutter, M. (2007a). On universal prediction and Bayesian confirmation. Theoretical Computer Science, 384(1), 33–48.

    MATH  Article  MathSciNet  Google Scholar 

  54. Hutter, M. (2007b). Universal algorithmic intelligence: A mathematical top-down approach. In Artificial general intelligence (pp. 227–290). Berlin: Springer.

  55. Johnson, W. L. (1992). Needed: A new test of intelligence. SIGARTN: SIGART Newsletter (ACM Special Interest Group on Artificial Intelligence), 3(4), 7–9.

  56. Johnson-Laird, P. N., & Wason, P. C. (1977). A theoretical analysis of insight into a reasoning task. In P. N. Johnson-Laird & P. C. Wason (Eds.) Thinking: Readings in cognitive science (pp. 143–157). Cambridge, UK: Cambridge University Press.

  57. Kaufman, A. S. (2000). Tests of intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence. Cambridge, UK: Cambridge University Press.

  58. Kurzweil, R. (2000). The age of spiritual machines: When computers exceed human intelligence. East Rutherford, NJ: Penguin.

  59. Legg, S., & Hutter, M. (2004). Ergodic MDPs admit self-optimising policies. Technical Report IDSIA-21-04, IDSIA.

  60. Legg, S., & Hutter, M. (2004). A taxonomy for abstract environments. Technical Report IDSIA-20-04, IDSIA.

  61. Legg, S., & Hutter, M. (2005). A universal measure of intelligence for artificial agents. In Proceedings of the 21st international joint conference on artificial intelligence (IJCAI-2005) (pp. 1509–1510). Edinburgh.

  62. Legg, S., & Hutter, M. (2006a). A formal definition of intelligence for artificial systems. In Proceedings anniversary summit of artificial intelligence. Monte Verita, Switzerland.

  63. Legg, S., & Hutter, M. (2006b). A formal measure of machine intelligence. In Annual machine learning conference of Belgium and The Netherlands (Benelearn’06). Ghent.

  64. Legg, S., & Hutter, M. (2007). A collection of definitions of intelligence. In B. Goertzel & P. Wang (Eds.), Advances in artificial general intelligence: Concepts, architectures and algorithms vol 157 of frontiers in artificial intelligence and applications. (pp. 17–24). Amsterdam, NL: IOS press. Online version: http://www.vetta.org/shane/intelligence.html.

  65. Levin, L. A. (1973). Universal sequential search problems. Problems of Information Transmission, 9, 265–266.

    Google Scholar 

  66. Li, M., & Vitányi, P. M. B. (1997). An introduction to Kolmogorov complexity and its applications (2nd ed.). Berlin: Springer.

  67. Loebner, H. G. (1990). The Loebner prize—the first Turing test. http://www.loebner.net/Prizef/loebner-prize.html

  68. Looks, M., Goertzel, B., & Pennachin, C. (2004). Novamente: An integrative architecture for general intelligence. In AAAI fall symposium, achieving human-level intelligence.

  69. Macphail, E. M. (1985). Vertebrate intelligence: The null hypothesis. In L. Weiskrantz (Ed.), Animal intelligence (pp. 37–50). Oxford: Clarendon.

    Google Scholar 

  70. Mahoney, M. V. (1999). Text compression as a test for artificial intelligence. In AAAI/IAAI.

  71. Masum, H., Christensen, S., & Oppacher, F. (2002). The Turing ratio: Metrics for open-ended tasks. In GECCO 2002: Proceedings of the genetic and evolutionary computation conference (pp. 973–980). New York: Morgan Kaufmann Publishers.

  72. McCarthy, J. (2004). What is artificial intelligence? www-formal.stanford.edu/jmc/whatisai/whatisai.html

  73. Minsky, M. (1985). The society of mind. New York: Simon and Schuster.

    Google Scholar 

  74. Müller, M. (2006). Stationary algorithmic probability. Technical report, TU Berlin, Berlin, http://arXiv.org/abs/cs/0608095

  75. Neisser, U., Boodoo, G., Bouchard, T. J. Jr., Boykin, A. W., Brody, N., Ceci, S. J., Halpern, D. F., Loehlin, J. C., Perloff, R., Sternberg, R. J., & Urbina, S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51(2), 77–101.

  76. Newell, A., & Simon, H. A. (1976). Computer science as empirical enquiry: Symbols and search. Communications of the ACM 19, 3, 113–126.

    Article  MathSciNet  Google Scholar 

  77. Poole, D., Mackworth, A., & Goebel, R. (1998). Computational intelligence: A logical approach. New York: Oxford University Press.

    Google Scholar 

  78. Raven, J. (2000). The Raven’s progressive matrices: Change and stability over culture and time. Cognitive Psychology, 41, 1–48.

    Article  Google Scholar 

  79. Reznikova, Zh. I., & Ryabko, B. Ya. (1986). Analysis of the language of ants by information-theoretic methods. Problems of Information Transmission, 22, 245–249.

    MATH  Google Scholar 

  80. Sanghi, P., & Dowe, D. L. (2003). A computer program capable of passing I.Q. tests. In Proceedings of the 4th ICCS international conference on cognitive science (ICCS’03) (pp. 570–575). Sydney, NSW, Australia.

  81. Saygin, A., Cicekli, I., & Akman, V. (2000). Turing test: 50 years later. Minds and Machines, 10(4), 463–518.

    Article  Google Scholar 

  82. Schmidhuber, J. (2002). The speed prior: A new simplicity measure yielding near-optimal computable predictions. In Proceedings of the 15th annual conference on computational learning theory (COLT 2002), Lecture notes in artificial intelligence (pp. 216–228). Sydney, Australia: Springer.

  83. Schweizer, P. (1998). The truly total Turing test. Minds and Machines, 8, 263–272.

    Article  Google Scholar 

  84. Searle, J. (1980). Minds, brains, and programs. Behavioral & Brain Sciences, 3, 417–458.

    Article  Google Scholar 

  85. Shieber, S. (1994). Lessons from a restricted Turing test. CACM: Communications of the ACM, 37(6), 70–78.

  86. Simonton, D. K. (2003). An interview with Dr. Simonton. In J. A. Plucker (Ed.), Human intelligence: Historical influences, current controversies, teaching resources. http://www.indiana.edu/∼intell

  87. Slatter, J. (2001). Assessment of children: Cognitive applications (4th ed.). San Diego: Jermone M. Satler Publisher Inc.

  88. Slotnick, B. M., & Katz, H. M. (1974). Olfactory learning-set formation in rats. Science, 185, 796–798.

    Article  Google Scholar 

  89. Smith, W. D. (2006). Mathematical definition of “intelligence” (and consequences). http://math.temple.edu/∼wds/homepage/works.html

  90. Spearman, C. E. (1927). The abilities of man, their nature and measurement. New York: Macmillan.

    Google Scholar 

  91. Stern, W. L. (1912). Psychologischen Methoden der Intelligenz-Prüfung. Leipzig: Barth.

    Google Scholar 

  92. Sternberg, R. J. (1985). Beyond IQ: A triacrchi theory of human intelligence. New York: Cambridge University Press.

    Google Scholar 

  93. Sternberg, R. J. (Ed.) (2000). Handbook of intelligence. Cambridge University Press.

  94. Sternberg, R. J. (2003). An interview with Dr. Sternberg. In J. A. Plucker (Ed.), Human intelligence: Historical influences, current controversies, teaching resources. http://www.indiana.edu/z∼intell

  95. Sternberg, R. J., & Berg, C. A. (1986). Quantitative integration: Definitions of intelligence: A comparison of the 1921 and 1986 symposia. In R. J. Sternberg & D. K. Detterman (Eds.), What is intelligence? Contemporary wiewpoints on its nature and definition (pp. 155–162). Norwood, NJ: Ablex.

    Google Scholar 

  96. Sternberg, R. J., & Grigorenko, E. L. (Eds.). (2002). Dynamic testing: The nature and measurement of learning potential. Cambridge, UK: Cambridge University Press.

  97. Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge MA: MIT Press.

    Google Scholar 

  98. Terman, L. M., & Merrill, M. A. (1950). The Stanford-Binet intelligence scale. Boston: Houghton Mifflin.

    Google Scholar 

  99. Thurstone, L. L. (1938). Primary mental abilities. Chicago: University of Chicago Press.

    Google Scholar 

  100. Treister-Goren, A., Dunietz, J., & Hutchens, J. L. (2000). The developmental approach to evaluating artificial intelligence—a proposal. In Performance metrics for intelligence systems.

  101. Treister-Goren, A., & Hutchens, J. L. (2001). Creating AI: A unique interplay between the development of learning algorithms and their education. In Proceeding of the first international workshop on epigenetic robotics.

  102. Turing, A. M. (1950). Computing machinery and intelligence. Mind, LIX(236), 433–460.

  103. Voss, P. (2005). Essentials of general intelligence: The direct path to AGI. In B. Goertzel & C. Pennachin (Eds.), Artificial general intelligence. Berlin: Springer.

  104. Wallace, C. S. (2005). Statistical and inductive inference by minimum message length. Berlin: Springer.

    Google Scholar 

  105. Wang, P. (1995). On the working definition of intelligence. Technical Report 94, Center for Research on Concepts and Cognition, Indiana University.

  106. Watt, S. (1996). Naive psychology and the inverted Turing test. Psycoloquy, 7(14).

  107. Wechsler, D. (1958). The measurement and appraisal of adult intelligence (4 ed.). Baltimore: Williams & Wilkinds.

    Google Scholar 

  108. Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transaction on Evolutionary Computation, 1(1), 67–82.

    Article  Google Scholar 

  109. Zentall, T. R. (1997). Animal memory: The role of instructions. Learning and Motivation, 28, 248–267.

    Article  Google Scholar 

  110. Zentall, T. R. (2000). Animal intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence. Cambridge, UK: Cambridge University Press.

Download references

Acknowledgements

This work was supported by the Swiss NSF grant 200020-107616.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Shane Legg.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Legg, S., Hutter, M. Universal Intelligence: A Definition of Machine Intelligence. Minds & Machines 17, 391–444 (2007). https://doi.org/10.1007/s11023-007-9079-x

Download citation

Keywords

  • AIXI
  • Complexity theory
  • Intelligence
  • Theoretical foundations
  • Turing test
  • Intelligence tests
  • Measures
  • Definitions