On More Realistic Environment Distributions for Defining, Evaluating and Developing Intelligence

  • José Hernández-Orallo
  • David L. Dowe
  • Sergio España-Cubillo
  • M. Victoria Hernández-Lloreda
  • Javier Insa-Cabrera
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6830)


One insightful view of the notion of intelligence is the ability to perform well in a diverse set of tasks, problems or environments. One of the key issues is therefore the choice of this set, which can be formalised as a ‘distribution’. Formalising and properly defining this distribution is an important challenge to understand what intelligence is and to achieve artificial general intelligence (AGI). In this paper, we agree with previous criticisms that a universal distribution using a reference universal Turing machine (UTM) over tasks, environments, etc., is perhaps a much too general distribution, since, e.g., the probability of other agents appearing on the scene or having some social interaction is almost 0 for many reference UTMs. Instead, we propose the notion of Darwin-Wallace distribution for environments, which is inspired by biological evolution, artificial life and evolutionary computation. However, although enlightening about where and how intelligence should excel, this distribution has so many options and is uncomputable in so many ways that we certainly need a more practical alternative. We propose the use of intelligence tests over multi-agent systems, in such a way that agents with a certified level of intelligence at a certain degree are used to construct the tests for the next degree. This constructive methodology can then be used as a more realistic intelligence test and also as a testbed for developing and evaluating AGI systems.


Intelligence Evolutionary Computation Artificial Life Social Intelligence Intelligence Test Universal Distribution 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dowe, D.L.: Foreword re C. S. Wallace. Computer Journal 51(5), 523–560 (2008); Christopher Stewart WALLACE (1933-2004) memorial special issueCrossRefGoogle Scholar
  2. 2.
    Dowe, D.L.: Minimum Message Length and statistically consistent invariant (objective?) Bayesian probabilistic inference - from (medical) “evidence”. Social Epistemology 22(4), 433–460 (2008)CrossRefGoogle Scholar
  3. 3.
    Dowe, D.L.: MML, hybrid Bayesian network graphical models, statistical consistency, invariance and uniqueness. In: Bandyopadhyay, P.S., Forster, M.R. (eds.) Handbook of the Philosophy of Science. Philosophy of Statistics, vol. 7, pp. 901–982. Elsevier, Amsterdam (2011)Google Scholar
  4. 4.
    Dowe, D.L., Hajek, A.R.: A computational extension to the Turing Test. In: 4th Conf. of the Australasian Cognitive Science Society, Newcastle, Australia (1997)Google Scholar
  5. 5.
    Goertzel, B.: The Embodied Communication Prior: A characterization of general intelligence in the context of Embodied social interaction. In: 8th IEEE International Conference on, Cognitive Informatics, ICCI 2009, pp. 38–43. IEEE, Los Alamitos (2009)CrossRefGoogle Scholar
  6. 6.
    Goertzel, B., Bugaj, S.V.: AGI Preschool: a framework for evaluating early-stage human-like AGIs. In: Intl. Conf. on Artificial General Intelligence (AGI 2009) (2009)Google Scholar
  7. 7.
    Hernández-Orallo, J.: Beyond the Turing Test. J. Logic, Language & Information 9(4), 447–466 (2000)zbMATHCrossRefGoogle Scholar
  8. 8.
    Hernández-Orallo, J.: On the computational measurement of intelligence factors. In: Meystel, A. (ed.) Performance metrics for intelligent systems workshop, pp. 1–8. National Institute of Standards and Technology, Gaithersburg (2000)Google Scholar
  9. 9.
    Hernández-Orallo, J.: A (hopefully) non-biased universal environment class for measuring intelligence of biological and artificial systems. In: Hutter, M., et al. (eds.) Artificial General Intelligence, pp. 182–183 (2010)Google Scholar
  10. 10.
    Hernández-Orallo, J., Dowe, D.L.: Measuring universal intelligence: Towards an anytime intelligence test. Artificial Intelligence 174(18), 1508–1539 (2010)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Hernández-Orallo, J., Minaya-Collado, N.: A formal definition of intelligence based on an intensional variant of Kolmogorov complexity. In: Proc. Intl Symposium of Engineering of Intelligent Systems (EIS 1998), pp. 146–163. ICSC Press (1998)Google Scholar
  12. 12.
    Herrmann, E., Call, J., Hernández-Lloreda, M.V., Hare, B., Tomasello, M.: Humans have evolved specialized skills of social cognition: The cultural intelligence hypothesis. Science 317(5843), 1360–1366 (2007)CrossRefGoogle Scholar
  13. 13.
    Hibbard, B.: Bias and No Free Lunch in Formal Measures of Intelligence. Journal of Artificial General Intelligence 1(1), 54–61 (2009)Google Scholar
  14. 14.
    Krebs, J.R., Dawkins, R.: Animal signals: mind-reading and manipulation. Behavioural Ecology: an evolutionary approach 2, 380–402 (1984)Google Scholar
  15. 15.
    Langton, C.G.: Artificial life: An overview. The MIT Press, Cambridge (1997)Google Scholar
  16. 16.
    Legg, S., Hutter, M.: A collection of definitions of intelligence. In: Proc. of the 2007 Conf. on Artificial General Intelligence, pp. 17–24. IOS Press, Amsterdam (2007)Google Scholar
  17. 17.
    Legg, S., Hutter, M.: Universal intelligence: A definition of machine intelligence. Minds and Machines 17(4), 391–444 (2007)CrossRefGoogle Scholar
  18. 18.
    Levin, L.A.: Universal sequential search problems. Problems of Information Transmission 9(3), 265–266 (1973)Google Scholar
  19. 19.
    Sanghi, P., Dowe, D.L.: A computer program capable of passing IQ tests. In: Proc. 4th ICCS International Conference on Cognitive Science (ICCS 2003), Sydney, Australia, pp. 570–575 (2003)Google Scholar
  20. 20.
    Schmidhuber, J.: A computer scientist’s view of life, the universe, and everything. In: Foundations of Computer Science, p. 201. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  21. 21.
    Schmidhuber, J.: The Speed Prior: a new simplicity measure yielding near-optimal computable predictions. In: Kivinen, J., Sloan, R.H. (eds.) COLT 2002. LNCS (LNAI), vol. 2375, pp. 123–127. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  22. 22.
    Solomonoff, R.J.: A formal theory of inductive inference. Part I. Information and control 7(1), 1–22 (1964)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Stone, P., Veloso, M.: Towards collaborative and adversarial learning: A case study in robotic soccer. Intl. J. of Human-Computers Studies 48(1), 83–104 (1998)CrossRefGoogle Scholar
  24. 24.
    Tomasello, M., Herrmann, E.: Ape and human cognition: What’s the difference? Current Directions in Psychological Science 19(1), 3–8 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • José Hernández-Orallo
    • 1
  • David L. Dowe
    • 2
  • Sergio España-Cubillo
    • 3
  • M. Victoria Hernández-Lloreda
    • 4
  • Javier Insa-Cabrera
    • 1
  1. 1.DSICUniversitat Politècnica de ValènciaSpain
  2. 2.Clayton School of Information TechnologyMonash UniversityAustralia
  3. 3.ProS Research CenterUniversitat Politècnica de ValènciaSpain
  4. 4.Departamento de Metodología de las Ciencias del ComportamientoUniversidad Complutense de MadridSpain

Personalised recommendations