Skip to main content
SpringerLink
Log in

Artificial General Intelligence and the Human Mental Model

  • Chapter
  • First Online:
Singularity Hypotheses

Part of the book series: The Frontiers Collection ((FRONTCOLL))

Abstract

When the first artificial general intelligences are built, they may improve themselves to far-above-human levels. Speculations about such future entities are already affected by anthropomorphic bias, which leads to erroneous analogies with human minds. In this chapter, we apply a goal-oriented understanding of intelligence to show that humanity occupies only a tiny portion of the design space of possible minds. This space is much larger than what we are familiar with from the human example; and the mental architectures and goals of future superintelligences need not have most of the properties of human minds. A new approach to cognitive science and philosophy of mind, one not centered on the human example, is needed to help us understand the challenges which we will face when a power greater than us emerges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 84.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The term “artificial general intelligence” here is used in the general sense of an agent, implemented by humans, which is capable of optimizing across a wide range of goals. “Strong AI” is a common synonym. “Artificial General Intelligence”, capitalized, is also used as a term of art for a specific design paradigm which combines narrow AI techniques in an integrated engineered architecture; in contrast, for example, to one which is evolved or emulates the brain (Voss 2007). As discussed below, this more specific sense of AGI is also the primary focus of this article.

  2. 2.

    Change of goals is possible in a superintelligence where a stable metagoal is the true motivator. For example, discovery and refinement of goals is part of Coherent Extrapolated Volition, a goal system for a self-improving AGI. It is designed, to ultimately converge on the terminal value of helping humans achieve their goal system as extrapolated towards reflective equilibrium (Yudkowsky 2004; Tarleton 2010; Dewey 2011). Nonetheless, CEV does not violate the principle that a sufficiently powerful optimizer would lack human-like variability in its goals, since its meta-level values towards goal definition in themselves constitute a stable top-level goal system.

References

  • Anissimov, M. (2011). Anthropomorphism and moral realism in advanced artificial intelligence. Paper presented at the Society for Philosophy and Technology conference, Denton.

    Google Scholar 

  • Batson, C. D. (2010). Altruism in humans. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Bostrom, N. (2003). Ethical issues in advanced artificial intelligence. In I. Smit, G. Lasker, & W. Wallach (Eds.), Cognitive, emotive and ethical aspects of decision making in humans and in artificial intelligence (Vol. 2, pp. 12–17). Windsor: International Institute of Advanced Studies in Systems Research and Cybernetics.

    Google Scholar 

  • Bostrom, N. (2006). What is a singleton? Linguistic and Philosophical Investigations, 5(2), 48–54.

    Google Scholar 

  • Bostrom, N., & Sandberg, A. (2009). The wisdom of nature: An evolutionary heuristic for human enhancement. In J. Savulescu & N. Bostrom (Eds.), Human enhancement (pp. 375–416). Oxford: Oxford University Press.

    Google Scholar 

  • Brooks, R. A. (1999). Cambrian intelligence: The early history of the new AI. Cambridge: MIT Press.

    MATH  Google Scholar 

  • Brown, D. E. (1991). Human universals. New York: McGraw Hill.

    Google Scholar 

  • Brown, D. E. (2004). Human universals, human nature and human culture. Daedalus, 133(4), 47–54.

    Article  Google Scholar 

  • Chalmers, D. J. (2010). The singularity: A philosophical analysis. Journal of Consciousness Studies, 17, 7–65.

    Google Scholar 

  • Cosmides, L., & Tooby, J. (1992). Cognitive adaptations for social exchange. In J. Barkow, J. Tooby, & L. Cosmides (Eds.), The adapted mind: Evolutionary psychology and the generation of culture (pp. 163–228). Oxford: Oxford University Press.

    Google Scholar 

  • Dewey, D. (2011). Learning what to value. In J. Schmidhuber, K.R. Thórisson & M. Looks (Eds.), Artificial General Intelligence: Proceedings of 4th International Conference, AGI 2011, Mountain View, CA, USA, 3–6 August 2011, pp. 309–314. Berlin: Springer.

    Google Scholar 

  • Dijkstra, E.W. (1984). The threats to computing science. Paper presented at the ACM 1984 South Central Regional Conference, 16–18 Nov, Austin.

    Google Scholar 

  • Duffy, B. R. (2003). Anthropomorphism and the social robot. Robotics and Autonomous Systems, 42(3–4), 177–190.

    Article  MATH  Google Scholar 

  • Epley, N., Waytz, A., & Cacioppo, J. T. (2007). On seeing human: A three-factor theory of anthropomorphism. Psychological Review, 114(4), 864–888.

    Article  Google Scholar 

  • Fox, J., & Shulman, C. (2010). Superintelligence does not imply benevolence. In K. Mainzer (Ed.), Proceedings of the VIII European Conference on Computing and Philosophy (pp. 456–461). Munich: Verlag Dr. Hut.

    Google Scholar 

  • Freitas, R. A, Jr. (1979). Xenology: An introduction to the scientific study of extraterrestrial life, intelligence, and civilization (1st ed.). Sacramento: Xenology Research Institute.

    Google Scholar 

  • Gigerenzer, G., & Selten, R. (Eds.). (2001). Bounded rationality: The adaptive toolbox. Cambridge: MIT Press.

    Google Scholar 

  • Goertzel, B. (2006). The hidden pattern: A patternist philosophy of mind. Boca Raton: Brown Walker Press.

    Google Scholar 

  • Goertzel, B. (2009). The embodied communication prior: A characterization of general intelligence in the context of embodied social interaction. Paper presented at the 8th IEEE International Conference on Cognitive Informatics IEEE, Hong Kong.

    Google Scholar 

  • Goertzel, B. (2010). Toward a formal characterization of real-world general intelligence. In E. Baum, M. Hutter & E. Kitzelmann (Eds.), Proceedings of the Third Conference on Artificial General Intelligence, AGI 2010, Lugano, Switzerland, 5–8 March, 2010. Amsterdam: Atlantis.

    Google Scholar 

  • Goertzel, B., Iklé, M., Goertzel, I. F., & Heljakka, A. (2009). Probabilistic logic networks: A comprehensive framework for uncertain inference. Berlin: Springer.

    Book  Google Scholar 

  • Good, I. J. (1965). Speculations concerning the first ultraintelligent machine. Advances in Computers, 6, 31–88.

    Article  Google Scholar 

  • Graimann, B., Allison, B., & Pfurtscheller, G. (Eds.). (2010). Brain-computer interfaces: Revolutionizing human-computer interaction. Berlin: Springer.

    Google Scholar 

  • Greene, J.D. (2002). The terrible, horrible, no good, very bad truth about morality and what to do about it. Ph.D. Dissertation, Princeton University, Princeton.

    Google Scholar 

  • Greene, B. (2011). The hidden reality: Parallel universes and the deep laws of the cosmos. New York: Knopf.

    MATH  Google Scholar 

  • Griffin, D. R. (1992). Animal minds. Chicago: University of Chicago Press.

    Google Scholar 

  • Hall, J. S. (2007). Beyond AI: Creating the conscience of the machine. Amherst: Prometheus.

    Google Scholar 

  • Hanson, R. (1994). If uploads come first: The crack of a future dawn. Extropy, 6(2), 10–15.

    Google Scholar 

  • Hutter, M. (2005). Universal artificial intelligence: Sequential decisions based on algorithmic probability. Berlin: Springer.

    MATH  Google Scholar 

  • Jaynes, E. T. (2003). Probability theory: The logic of science (Vol. 1). Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Kahneman, D., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.

    Google Scholar 

  • Karlsson, F. (2010). Syntactic recursion and iteration. In H.v.d. Hulst (Ed.), Recursion and human language (pp. 43–67). Berlin: Mouton de Gruyter.

    Google Scholar 

  • Lakoff, G. (1987). Women, fire and dangerous things: What categories reveal about the mind. Chicago: University of Chicago Press.

    Book  Google Scholar 

  • Landauer, R. (1961). Irreversibility and heat generation in the computing process. IBM Journal of Research and Development, 5(3), 183–191.

    Article  MathSciNet  MATH  Google Scholar 

  • Legg, S. (2006). Is there an elegant universal theory of prediction? Technical Report No. IDSIA-12-06. Manno, Switzerland.

    Google Scholar 

  • Legg, S. (2008). Machine super intelligence. Ph.D. Thesis, University of Lugano, Lugano, Switzerland.

    Google Scholar 

  • Li, M., & Vitányi, P. (1993). An introduction to Kolmogorov complexity and its applications. Berlin: Springer.

    MATH  Google Scholar 

  • Lloyd, S. (2000). Ultimate physical limits to computation. Nature, 406, 1047–1054.

    Article  Google Scholar 

  • Lloyd, S. (2002). Computational capacity of the Universe. Physical Review Letters, 88(23), 237901.

    Article  Google Scholar 

  • Moravec, H. (1998). When will computer hardware match the human brain? Journal of Evolution and Technology, 1(1),  .

    Google Scholar 

  • Neisser, U., Boodoo, G., Bouchard, T. J, Jr, Boykin, A. W., Brody, N., Ceci, S. J., et al. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51(2), 77–101.

    Article  Google Scholar 

  • Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs: Prentice-Hall.

    Google Scholar 

  • Omohundro, S. M. (2008). The basic AI drives. In P. Wang, B. Goertzel, & S. Franklin (Eds.), The proceedings of the first AGI conference (pp. 483–492). Amsterdam: IOS Press.

    Google Scholar 

  • Orseau, L., & Ring, M. (2011). Self-modification and mortality in artificial agents. In J. Schmidhuber, K. Thórisson & M. Looks (Eds.), Artificial General Intelligence: Proceedings of 4th International Conference, AGI 2011, Mountain View, CA, USA, 3–6 August 2011 (pp. 1–10). Berlin: Springer.

    Google Scholar 

  • Reich, P. A. (1972). The finiteness of natural language. In F. Householder (Ed.), Syntactic theory 1: Structuralist (pp. 238–272). Harmondsworth: Penguin.

    Google Scholar 

  • Rosch, E. (1978). Principles of categorization. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization (pp. 27–48). Hillsdale: Lawrence Erlbaum Associates.

    Google Scholar 

  • Salamon, A. (2009). Shaping the intelligence explosion. Paper presented at the Singularity Summit. http://vimeo.com/7318055.

  • Sandberg, A., & Bostrom, N. (2008). Whole brain emulation: A roadmap, Future of humanity institute, Oxford University. Technical Report #2008-3.

    Google Scholar 

  • Sotala, K. (2010). From mostly harmless to civilization-threatening: pathways to dangerous artificial general intelligences. In K. Mainzer (Ed.), Proceedings of the VIII European Conference on Computing and Philosophy. Munich: Verlag Dr. Hut.

    Google Scholar 

  • Sotala, K. (2012, in press). Relative advantages of uploads, artificial general intelligences, and other digital minds. International Journal of Machine Consciousness, 4.

    Google Scholar 

  • Strannegård, C. (2007). Anthropomorphic artificial intelligence. Filosofiska Meddelanden, Web Series, 33. http://www.phil.gu.se/posters/festskrift2/mnemo_strannegard.pdf.

  • Tarleton, N. (2010). Coherent extrapolated volition: A meta-level approach to machine ethics, from http://singinst.org/upload/coherent-extrapolated-volition.pdf.

  • Tegmark, M. (2004). Parallel universes. In J. D. Barrow, P. C. W. Davies, & C. L. Harper (Eds.), Science and ultimate reality: Quantum theory, cosmology, and complexity (pp. 452–491). Cambridge: Cambridge University Press.

    Google Scholar 

  • Tenenbaum, J., Griffiths, T.L., & Kemp, C. (2006). Theory-based Bayesian models of inductive learning and reasoning. Trends in Cognitive Sciences (Special issue: Probabilistic models of cognition), 10(7), 309–318.

    Google Scholar 

  • Tooby, J., & Cosmides, L. (1992). The psychological foundations of culture. In J. Barkow, J. Tooby, & L. Cosmides (Eds.), The adapted mind: Evolutionary psychology and the generation of culture (pp. 19–136). Oxford: Oxford University Press.

    Google Scholar 

  • Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–460.

    Article  MathSciNet  Google Scholar 

  • Veness, J., Ng, K. S., Hutter, M., Uther, W., & Silver, D. (2011). A Monte Carlo AIXI approximation. Journal of Artificial Intelligence Research, 40, 95–142.

    MathSciNet  MATH  Google Scholar 

  • Vidal, J. J. (1973). Toward direct brain-computer communication. Annual Review of Biophysics and Bioengineering, 2, 157–180.

    Article  Google Scholar 

  • Voss, P. (2007). Essentials of general intelligence: The direct path to artificial general intelligence. In B. Goertzel & C. Pennachin (Eds.), Artificial general intelligence (pp. 131–158). Berlin: Springer.

    Chapter  Google Scholar 

  • Muehlhauser, L., & Helm, L. (2013). The Singularity and machine ethics. In A. Eden, J. Moor, J. Soraker & E. Steinhart (Eds.), The singularity hypothesis. Berlin: Springer.

    Google Scholar 

  • Muehlhauser, L., & Salamon, A. (2013). Intelligence explosion: Evidence and import. In A. Eden, J. Moor, J. Soraker & E. Steinhart (Eds.), The singularity hypothesis. Berlin: Springer.

    Google Scholar 

  • Yampolskiy, RV. (2011). AI-Complete CAPTCHAs as Zero Knowledge Proofs of Access to an Artificially Intelligent System. ISRN Artificial Intelligence, 2012-271878.

    Google Scholar 

  • Yampolskiy, RV. (2011a). Artificial Intelligence Safety Engineering: Why Machine Ethics is a Wrong Approach. Paper presented at the Philosophy and Theory of Artificial Intelligence (PT-AI2011), Thessaloniki, Greece.

    Google Scholar 

  • Yampolskiy, RV. (2011b). What to Do with the Singularity Paradox? Paper presented at the Philosophy and Theory of Artificial Intelligence (PT-AI2011), Thessaloniki, Greece.

    Google Scholar 

  • Yampolskiy, R. V. (2012a). Leakproofing singularity—artificial intelligence confinement problem. Journal of Consciousness Studies, 19(1–2), 194–214.

    Google Scholar 

  • Yampolskiy, RV. (2012b). Turing Test as a Defining Feature of AI-Completeness. Artificial Intelligence, Evolutionary Computation and Metaheuristics—In the footsteps of Alan Turing. Xin-She Yang (Ed.) (In Press): Springer.

    Google Scholar 

  • Yampolskiy, RV, and Fox, J. (2012). Safety engineering for artificial general intelligence. Topoi. Special issue on machine ethics & the Ethics of Building Intelligent Machines.

    Google Scholar 

  • Yudkowsky, E. (2003). Foundations of order. Paper presented at the Foresight Senior Associates Gathering. http://singinst.org/upload/foresight.pdf.

  • Yudkowsky, E. (2004). Coherent extrapolated volition, from http://singinst.org/upload/CEV.html.

  • Yudkowsky, E. (2006). The human importance of the intelligence explosion. Paper presented at the Singularity Summit, Stanford University.

    Google Scholar 

  • Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In N. Bostrom & M. M. Ćirković (Eds.), Global catastrophic risks (pp. 308–345). Oxford: Oxford University Press.

    Google Scholar 

  • Yudkowsky, E. (2011). Complex value systems in friendly AI. In J. Schmidhuber, K. Thórisson & M. Looks (Eds.), Proceedings of the 4th Annual Conference on Artificial General Intelligence, Mountain View, CA, USA, August 2011 (pp. 388–393). Berlin: Springer.

    Google Scholar 

Download references

Acknowledgments

Thanks to Carl Shulman, Anna Salamon, Brian Rabkin, Luke Muehlhauser, and Daniel Dewey for their valuable comments.

Author information

Authors and Affiliations

  1. Computer Engineering & Computer Science, University of Louisville, Louisville, KY, USA

    Roman V. Yampolskiy

  2. Machine Intelligence Research Institute, Aderet 1041, Emek Haela, 99850, Israel

    Joshua Fox

Authors
  1. Roman V. Yampolskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joshua Fox
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roman V. Yampolskiy .

Editor information

Editors and Affiliations

  1. School of Computer Science and, Electronic Engineering, University of Essex, Colchester, CO4 3SQ, United Kingdom

    Amnon H. Eden

  2. Dartmouth College, Thornton 6035, Hanover, 03755-3592, New Hampshire, USA

    James H. Moor

  3. Department of Philosophy, University of Twente, Enschede, 7500 AE, Netherlands

    Johnny H. Søraker

  4. Department of Philosophy, William Paterson University, Pompton Road 300, Wayne, 07470, New York, USA

    Eric Steinhart

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Yampolskiy, R.V., Fox, J. (2012). Artificial General Intelligence and the Human Mental Model. In: Eden, A., Moor, J., Søraker, J., Steinhart, E. (eds) Singularity Hypotheses. The Frontiers Collection. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32560-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32560-1_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32559-5

  • Online ISBN: 978-3-642-32560-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation