Skip to main content
SpringerLink
Log in

Mapping Intelligence: Requirements and Possibilities

  • Conference paper
  • First Online:
Philosophy and Theory of Artificial Intelligence 2017 (PT-AI 2017)

Part of the book series: Studies in Applied Philosophy, Epistemology and Rational Ethics ((SAPERE,volume 44))

Included in the following conference series:

Abstract

New types of artificial intelligence (AI), from cognitive assistants to social robots, are challenging meaningful comparison with other kinds of intelligence. How can such intelligent systems be catalogued, evaluated, and contrasted, with representations and projections that offer meaningful insights? To catalyse the research in AI and the future of cognition, we present the motivation, requirements and possibilities for an atlas of intelligence: an integrated framework and collaborative open repository for collecting and exhibiting information of all kinds of intelligence, including humans, non-human animals, AI systems, hybrids and collectives thereof. After presenting this initiative, we review related efforts and present the requirements of such a framework. We survey existing visualisations and representations, and discuss which criteria of inclusion should be used to configure an atlas of intelligence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A detailed analysis of the questionnaire can be found in (Bhatnagar et al. 2017).

  2. 2.

    Some of these initiatives are in genomics and brain imaging (Midford 2004; Boero and Bernardi 2014).

  3. 3.

    http://www.cognitiveatlas.org.

  4. 4.

    https://poldracklab.stanford.edu/.

  5. 5.

    http://observatory.brain-map.org/visualcoding/.

  6. 6.

    http://buros.org/mental-measurements-yearbook.

  7. 7.

    http://ipip.ori.org.

  8. 8.

    http://icar-project.com.

  9. 9.

    http://www.kaggle.com.

  10. 10.

    http://www.openml.org.

  11. 11.

    http://www.arcadelearningenvironment.org/.

  12. 12.

    https://gym.openai.com/.

  13. 13.

    https://www.microsoft.com/en-us/research/project/project-malmo/.

  14. 14.

    https://research.fb.com/projects/commai/.

  15. 15.

    https://deepmind.com/blog/open-sourcing-deepmind-lab/.

  16. 16.

    http://www.eff.org/ai/metrics.

  17. 17.

    http://aiindex.org.

  18. 18.

    http://www.roadmapinstitute.org.

References

  • Aarts, A., Anderson, J., Anderson, C., Attridge, P., Attwood, A., Fedor, A.: Estimating the reproducibility of psychological science. Science 349(6251), 1–8 (2015)

    Google Scholar 

  • Allen Institute for Brain Science: Allen brain observatory (2016). http://observatory.brain-map.org/visualcoding

  • Arsiwalla, X.D., Moulin-Frier, C., Herreros, I., Sanchez-Fibla, M., Verschure, P.: The morphospace of consciousness. arXiv preprint arXiv:1705.11190 (2017)

  • Balakhonov, D., Rose, J.: Crows rival monkeys in cognitive capacity. Sci. Rep. 7(1), 8809 (2017)

    Article  Google Scholar 

  • Bhatnagar, S., et al.: A First Survey on an Atlas of Intelligence (2017). http://www.dsic.upv.es/~flip/papers/Bhatnagar18_SurveyAtlas.pdf

  • Boero, F., Bernardi, G.: Phenotypic vs genotypic approaches to biodiversity, from conict to alliance. Mar. Genomics 17, 63–64 (2014)

    Article  Google Scholar 

  • Brooks, R.A.: From earwigs to humans. Robot. Auton. Syst. 20(2–4), 291–304 (1997)

    Article  Google Scholar 

  • Cadman, E.: AI not just a game for DeepMind’s Demis Hassabis. Financial Times (2014). https://www.ft.com/content/1c9d5410-8739

  • Castelvecchi, D.: Tech giants open virtual worlds to bevy of AI programs. Nature 540(7633), 323–324 (2016)

    Article  Google Scholar 

  • Cattell, R.B., Coulter, M.A.: Principles of behavioural taxonomy and the mathematical basis of the taxonome computer program. Br. J. Math. Stat. Psychol. 19(2), 237–269 (1966)

    Article  Google Scholar 

  • Dennett, D.C.: Darwin’s dangerous idea. Sciences 35(3), 34–40 (1995)

    Article  Google Scholar 

  • de Valadés, D.: Rhetorica christiana (1579)

    Google Scholar 

  • Eckersley, P., Nasser, Y., et al.: EFF AI Progress Measurement Project (2017). https://www.eff.org/es/ai/metrics. Accessed 10 Jan 2017

  • Einav, L., Levin, J.: Economics in the age of big data. Science 346(6210), 1243089 (2014)

    Article  Google Scholar 

  • Frey, C.B., Osborne, M.A.: The future of employment: how susceptible are jobs to computerisation? Technol. Forecast. Soc. Chang. 114, 254–280 (2017)

    Article  Google Scholar 

  • Gentner, D.: Psychology in cognitive science: 1978–2038. Top. Cogn. Sci. 2(3), 328–344 (2010)

    Article  Google Scholar 

  • Gewin, V.: Taxonomy: all living things, online. Nature 418(6896), 362–363 (2002)

    Article  Google Scholar 

  • Godfrey-Smith, P.: Towers and trees in cognitive evolution. In: Huebner, B. (ed.) The Philosophy of Daniel Dennett (Chap. 8.1). Oxford University Press (2015)

    Google Scholar 

  • Gray, H.M., Gray, K., Wegner, D.M.: Dimensions of mind perception. Science 315(5812), 619–619 (2007)

    Article  Google Scholar 

  • Hayles, N.K.: Narratives of artificial life. In: Future Natural: Nature, Science, Culture, pp. 146–164. Routledge, London (1996)

    Google Scholar 

  • Hernández-Orallo, J., et al.: A new AI evaluation cosmos: ready to play the game? AI Mag. 38(3), 66–69 (2017). https://www.aaai.org/ojs/index.php/aimagazine/article/view/2748

    Article  Google Scholar 

  • Hernández-Orallo, J.: The Measure of All Minds: Evaluating Natural and Artificial Intelligence. Cambridge University Press, Cambridge (2017)

    Book  Google Scholar 

  • Herrmann, E., Call, J., Hernández-Lloreda, M.V., Hare, B., Tomasello, M.: Humans have evolved specialized skills of social cognition: the cultural intelligence hypothesis. Science 317(5843), 1360–1366 (2007)

    Article  Google Scholar 

  • Hessel, M., et al.: Rainbow: combining improvements in deep reinforcement learning. arXiv preprint arXiv:1710.02298 (2017)

  • Kirsh, D.: Today the earwig, tomorrow man? Artif. Intell. 47(1–3), 161–184 (1991)

    Article  MathSciNet  Google Scholar 

  • Landhuis, E.: Neuroscience: big brain, big data. Nature 541(7638), 559–561 (2017)

    Article  Google Scholar 

  • Leslie, M.: Calling all taxonomists. Science 307(5712), 1021–1022 (2005)

    Google Scholar 

  • Lowndes, J.S.S., Best, B.D., Scarborough, C., Afflerbach, J.C., Frazier, M.R., O’Hara, C.C., Halpern, B.S.: Our path to better science in less time using open data science tools. Nat. Ecol. Evol. 1, 0160 (2017)

    Article  Google Scholar 

  • Macphail, E.M.: The comparative psychology of intelligence. Behav. Brain Sci. 10(4), 645–656 (1987)

    Article  Google Scholar 

  • Marcus, G.: Deep learning: a critical appraisal. arXiv preprint arXiv:1801.00631 (2018)

  • Marx, V.: Biology: the big challenges of big data. Nature 498(7453), 255–260 (2013)

    Article  Google Scholar 

  • McKinlay, R.: Use or lose our navigation skills: automatic wayfinding is eroding natural abilities, warns roger mckinlay. Nature 531(7596), 573–576 (2016)

    Article  Google Scholar 

  • Midford, P.E.: Ontologies for behavior. Bioinformatics 20(18), 3700–3701 (2004)

    Article  Google Scholar 

  • Miller, R.: Task taxonomy: science or technology? Ergonomics 10(2), 167–176 (1967)

    Article  Google Scholar 

  • Milner, G.: Pinpoint: How GPS is Changing Technology, Culture, and Our Minds. WW Norton & Company (2016)

    Google Scholar 

  • Moravec, H.: When will computer hardware match the human brain. J. Evol. Technol. 1(1), 10 (1998)

    Google Scholar 

  • National Research Council: Toward Precision Medicine: Building a Knowledge Network for Biomedical Research and a New Taxonomy of Disease. National Academies Press (2011)

    Google Scholar 

  • Ng, A.: What artificial intelligence can and can’t do right now. Harvard Business Review, November 2016

    Google Scholar 

  • Parr, C.S., et al.: The encyclopedia of life v2: Providing global access to knowledge about life on earth, vol. 2 (2014). http://www.eol.org/

  • Pfeifer, R.: Embodied artificial intelligence 10 years back, 10 years forward. In: Informatics, pp. 294–310 (2001)

    Google Scholar 

  • Pinker, S.: The Language Instinct: How the Mind Creates Language. William Morrow and Company (1994)

    Google Scholar 

  • Roskov, Y., et al.: Species 2000 ITIS catalogue of life. Aeon (2018). http://catalogueoflife.org/col

  • Schaie, K.W.: Intellectual development in adulthood. In: Birren, J.E., Schaie, K.W. (eds.) Handbook of the Psychology of Aging, vol. 4, pp. 266–286. Academic Press Inc. (1996)

    Google Scholar 

  • Shah, H., Warwick, K., Vallverdú, J., Wu, D.: Can machines talk? Comparison of eliza with modern dialogue systems. Comput. Hum. Behav. 58, 278–295 (2016)

    Article  Google Scholar 

  • Shanahan, M.: Conscious exotica. from algorithms to aliens, could humans ever understand minds that are radically unlike our own? Aeon (2016). https://aeon.co/essays/beyond-humans-what-other-kinds-of-minds-might-be-out-there

  • Shead, S.: Facebook’s AI boss: in terms of general intelligence, we’re not even close to a rat. Business Insider (2017). http://uk.businessinsider.com/facebooks-ai-boss-in-terms-of-general-intelligence-were-not-even-close-to-a-rat-2017-10

  • Silver, D., et al.: Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815 (2017)

  • Sloman, A.: The Structure and Space of Possible Minds. University of Sussex, School of Cognitive Sciences (1984)

    Google Scholar 

  • Solé, R.: Rise of the humanbot. arXiv preprint arXiv:1705.05935 (2017)

  • Solé, R.V., Macia, J.: Synthetic biocomputation: the possible and the actual. In: ECAL, pp. 29–36 (2011)

    Google Scholar 

  • Sparrow, B., Liu, J., Wegner, D.M.: Google effects on memory: cognitive consequences of having information at our fingertips. Science, p. 1207745 (2011)

    Google Scholar 

  • Stuart, S., Wilson, E., McNeely, J., Mittermeier, R., Rodríguez, J.: The barometer of life. Science 328(5975), 177–177 (2010)

    Article  Google Scholar 

  • Tegmark, M.: Life 3.0: Being human in the age of artificial intelligence. Knopf (2017)

    Google Scholar 

  • Thagard, P.: Why cognitive science needs philosophy and vice versa. Top. Cogn. Sci. 1(2), 237–254 (2009)

    Article  Google Scholar 

  • Vanschoren, J., et al.: Towards a data science collaboratory. Lecture Notes in Computer Science (IDA 2015), vol. 9385 (2015)

    Google Scholar 

  • Vanschoren, J., van Rijn, J.N., Bischl, B., Torgo, L.: OpenML: networked science in machine learning. SIGKDD Explor. 15(2), 49–60 (2013). https://doi.org/10.1145/2641190.2641198

    Article  Google Scholar 

  • Wegner, D.M., Gray, K.: The Mind Club: Who Thinks, What Feels, and Why It Matters. Penguin (2017)

    Google Scholar 

  • Winfield, A.F.: Estimating the energy cost of (artificial) evolution. In: Sayama, H., Rieffel, J., Risi, S., Doursat, R., Lipson, H. (eds.) Proceedings of 14th International Conference on the Synthesis and Simulation of Living Systems (ALife), pp. 872–875. MIT Press (2014)

    Google Scholar 

  • Winfield, A.F.: How intelligent is your intelligent robot? arXiv preprint arXiv:1712.08878 (2017)

  • Yampolskiy, R.V.: The universe of minds. arXiv preprint arXiv:1410.0369 (2014)

Download references

Acknowledgements

The initiative was supported by the Leverhulme Trust via the Leverhulme Centre for the Future of Intelligence. J. H-Orallo and F. M-Plumed were supported by EU (FEDER) and the Spanish MINECO under grant TIN 2015-69175-C4-1-R and by GVA under grant PROMETEOII/ 2015/013 and by the Air Force Office of Scientific Research under award number FA9550-17-1-0287. J. H-Orallo also received a Salvador de Madariaga grant (PRX17/00467) from the Spanish MECD for a research stay at the CFI, Cambridge, and a BEST grant (BEST/2017/045) from the GVA for another research stay also at the CFI. F. M-Plumed was also supported by INCIBE (Ayudas para la excelencia de los equipos de investigación avanzada en ciberseguridad). A. Weller acknowledges support from the David MacKay Newton research fellowship at Darwin College, the Alan Turing Institute under EPSRC grant EP/N510129/1 & TU/B/000074, and the Leverhulme Trust via the CFI.

Author information

Authors and Affiliations

  1. Leverhulme Centre for the Future of Intelligence, Cambridge, UK

    Sankalp Bhatnagar, Anna Alexandrova, Stephen Cave, Lucy Cheke, Matthew Crosby, Marta Halina, Seán Ó hÉigeartaigh, Huw Price, Henry Shevlin, Adrian Weller, Alan Winfield & José Hernández-Orallo

  2. The New School, New York, USA

    Sankalp Bhatnagar

  3. Centre for the study of Existential Risk, University of Cambridge, Cambridge, UK

    Shahar Avin, Seán Ó hÉigeartaigh & Huw Price

  4. Department of History and Philosophy of Science, University of Cambridge, Cambridge, UK

    Anna Alexandrova & Marta Halina

  5. Psychometrics Centre, University of Cambridge, Cambridge, UK

    Bao Sheng Loe

  6. Universitat Politècnica de València, València, Spain

    Fernando Martínez-Plumed & José Hernández-Orallo

  7. Imperial College, London, UK

    Matthew Crosby

  8. AI Roadmap Institute, Prague, Czech Republic

    Jan Feyereisl

  9. GoodAI, Prague, Czech Republic

    Jan Feyereisl

  10. Bristol Robotics Laboratory, UWE Bristol, Bristol, UK

    Alan Winfield

  11. Department of Psychology, University of Cambridge, Cambridge, UK

    Lucy Cheke

  12. Alan Turing Institute, London, UK

    Adrian Weller

Authors
  1. Sankalp Bhatnagar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Alexandrova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shahar Avin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephen Cave
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lucy Cheke
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matthew Crosby
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jan Feyereisl
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Marta Halina
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Bao Sheng Loe
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Seán Ó hÉigeartaigh
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Fernando Martínez-Plumed
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Huw Price
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Henry Shevlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Adrian Weller
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Alan Winfield
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. José Hernández-Orallo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José Hernández-Orallo .

Editor information

Editors and Affiliations

  1. Interdisciplinary Ethics Applied (IDEA) Centre, University of Leeds, Leeds, West Yorkshire, UK

    Vincent C. Müller

A Appendix: Why is an atlas needed? Similar initiatives

A Appendix: Why is an atlas needed? Similar initiatives

While identifying the need for an atlas, we look at how it fits in cognitive science as a whole and also whether there are initiatives in other fields that could be inspirational.

Regarding cognitive science, it is true that its goal is to cover all possible cognitive systems, understand their behaviour and mechanisms, and establish meaningful comparisons. However, the field has not yet been able to portray a systematic representation covering both natural and artificial systems. But if we do not find this systematic representation in cognitive science, do we find it in related subdisciplines? The answer is that some similar initiatives in other disciplines do existFootnote 2:

  • Life forms: Examples are Wikispecies (Leslie 2005), the All Species Foundation (Gewin 2002), the Catalogue of Life and the Encyclopedia of Life (Roskov et al. 2018; Hayles 1996; Parr et al. 2014; Stuart et al. 2010).

  • Neuroscience: the Cognitive AtlasFootnote 3 and related repositories for neuroscienceFootnote 4 include an ontology of human cognitive functions and related tasks, and the pathologies affected. The Allen brain observatoryFootnote 5 (Allen Institute for Brain Science 2016) is a more visually-oriented platform that maps perception and cognition to parts of the human brain (National Research Council 2011).

  • Psychometrics: There are several initiatives bringing together test batteries and repositories: the mental Measurement yearbookFootnote 6, and with a more open character, the International Personality Item PoolFootnote 7 and the International Cognitive Ability ResourceFootnote 8.

  • Machine learning and data science research: KaggleFootnote 9, OpenMLFootnote 10 (Vanschoren et al. 2013) and many other platforms (e.g., gitxiv.com) provide benchmarks for ML. OpenML also includes experimental results that can be compared, aggregated and represented with powerful analytical packages.

  • Artificial intelligence: there are many collections of benchmarks and associated results, such as ALEFootnote 11, OpenAI universe/gymFootnote 12, Microsoft MalmoFootnote 13, Facebook’s CommAI-envFootnote 14, DeepMind LabFootnote 15 (see Hernández-Orallo et al. 2017 for a summary) and meta-views, such as a recent EFF analysisFootnote 16 and the AI index reportFootnote 17. This is a sign that AI is looking in this direction (Castelvecchi 2016; Hernández-Orallo 2017). The tasks are rarely arranged into abilities and the data usually compares specialised AI systems against average humans.

A partially overlapping initiative is the AI Roadmap InstituteFootnote 18, which encourages, compares and studies various AI and general AI roadmaps. It focuses on the future and on AI primarily, with representations that are usually flowcharts and pathway comparisons. Besides identifying where the field of AI stands as a whole, it also aims to identify dead-ends and open research problems on the path to the development of general AI systems.

The data and conceptual framing of the above projects can be used to inform an atlas of intelligence. Still, no repositories or taxonomies exist focusing mostly on behaviour, encompassing natural and artificial systems, as we are undertaking. Of course, the fact that something does not exist yet is not a sufficient reason that it should. The need for an atlas has to be supported by a series of motivations and applications, which we do in Sect. 2.

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bhatnagar, S. et al. (2018). Mapping Intelligence: Requirements and Possibilities. In: Müller, V. (eds) Philosophy and Theory of Artificial Intelligence 2017. PT-AI 2017. Studies in Applied Philosophy, Epistemology and Rational Ethics, vol 44. Springer, Cham. https://doi.org/10.1007/978-3-319-96448-5_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-96448-5_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-96447-8

  • Online ISBN: 978-3-319-96448-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation