Intelligence Explosion: Evidence and Import

  • Luke MuehlhauserEmail author
  • Anna Salamon
Part of the The Frontiers Collection book series (FRONTCOLL)


In this chapter we review the evidence for and against three claims: that (1) there is a substantial chance we will create human-level AI before 2100, that (2) if human-level AI is created, there is a good chance vastly superhuman AI will follow via an “intelligence explosion,” and that (3) an uncontrolled intelligence explosion could destroy everything we value, but a controlled intelligence explosion would benefit humanity enormously if we can achieve it. We conclude with recommendations for increasing the odds of a controlled intelligence explosion relative to an uncontrolled intelligence explosion.


Artificial Intelligence Transcranial Magnetic Stimulation Optimization Power Machine Intelligence Intelligence Software 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Anderson, B. (1993). Evidence from the rat for a general factor that underlies cognitive performance and that relates to brain size: intelligence? Neuroscience Letters, 153(1), 98–102. doi: 10.1016/0304-3940(93)90086-Z.CrossRefGoogle Scholar
  2. Arbesman, S. (2011). Quantifying the ease of scientific discovery. Scientometrics, 86(2), 245–250. doi: 10.1007/s11192-010-0232-6.CrossRefGoogle Scholar
  3. Armstrong, J. S. (1985). Long-range forecasting: from crystal ball to computer (2nd ed.). New York: Wiley.Google Scholar
  4. Armstrong, S., Sandberg, A., & Bostrom N. Forthcoming. Thinking inside the box: using and controlling an Oracle AI. Minds and Machines. Google Scholar
  5. Ashby, F. G., & Helie S. (2011). A tutorial on computational cognitive neuroscience: modeling the neurodynamics of cognition. Journal of Mathematical Psychology, 55(4), 273–289. doi: 10.1016/ Scholar
  6. Bainbridge, W. S., & Roco, M. C. (Eds.). (2006). Managing nano-bio-info-cogno innovations: converging technologies in society. Dordrecht: Springer.Google Scholar
  7. Baum, S. D., Goertzel, B., & Goertzel, T. G. (2011). How long until human-level AI? Results from an expert assessment. Technological Forecasting and Social Change, 78(1), 185–195. doi: 10.1016/j.techfore.2010.09.006.CrossRefGoogle Scholar
  8. Bellman, R. E. (1957). Dynamic programming. Princeton: Princeton University Press.zbMATHGoogle Scholar
  9. Berger, J. O. (1993). Statistical decision theory and bayesian analysis (2nd edn). Springer Series in Statistics. New York: Springer.Google Scholar
  10. Bertsekas, D. P. (2007). Dynamic programming and optimal control (Vol. 2). Nashua: Athena Scientific.Google Scholar
  11. Block, N. (1981). Psychologism and behaviorism. Philosophical Review, 90(1), 5–43. doi: 10.2307/2184371.CrossRefGoogle Scholar
  12. Bostrom, N. (2002). Existential risks: Analyzing human extinction scenarios and related hazards. Journal of Evolution and Technology, 9
  13. Bostrom, N. (2003). Ethical issues in advanced artificial intelligence. In I. Smit & G. E. Lasker (Eds.), Cognitive, emotive and ethical aspects of decision making in humans and in artificial intelligence. Windsor: International Institute of Advanced Studies in Systems Research/Cybernetics. Vol. 2.Google Scholar
  14. Bostrom, N. (2006). What is a singleton? Linguistic and Philosophical Investigations, 5(2), 48–54.Google Scholar
  15. Bostrom, N. (2007). Technological revolutions: Ethics and policy in the dark. In M. Nigel, S. de Cameron, & M. E. Mitchell (Eds.), Nanoscale: Issues and perspectives for the nano century (pp. 129–152). Hoboken: Wiley. doi: 10.1002/9780470165874.ch10.Google Scholar
  16. Bostrom, N. Forthcoming(a). Superintelligence: A strategic analysis of the coming machine intelligence revolution. Manuscript, in preparation.Google Scholar
  17. Bostrom, N. (2012). The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds and Machines. Preprint at,
  18. Bostrom, N., & Ćirković, M. M. (Eds.). (2008). Global catastrophic risks. New York: Oxford University Press.Google Scholar
  19. Bostrom, N., & Sandberg, A. (2009). Cognitive enhancement: Methods, ethics, regulatory challenges. Science and Engineering Ethics, 15(3), 311–341. doi: 10.1007/s11948-009-9142-5.CrossRefGoogle Scholar
  20. Brynjolfsson, E., & McAfee, A. (2011). Race against the machine: How the digital revolution is accelerating innovation, driving productivity, and irreversibly transforming employment and the economy. Lexington: Digital Frontier Press. Kindle edition.Google Scholar
  21. Caplan, B. (2008). The totalitarian threat. In Bostrom and Ćirković 2008, 504–519.Google Scholar
  22. Cartwright, E. (2011). Behavioral economics. New York: Routledge Advanced Texts in Economics and Finance.Google Scholar
  23. Cattell, R, & Parker, A. (2012). Challenges for brain emulation: why is building a brain so difficult? Synaptic Link, Feb. 5.
  24. Chalmers, D. J. (1996). The conscious mind: In search of a fundamental theory. New York: Oxford University Press. (Philosophy of Mind Series).zbMATHGoogle Scholar
  25. Chalmers, D. J. (2010). The singularity: A philosophical analysis. Journal of Consciousness Studies 17(9–10), 7–65. Scholar
  26. Chalmers, D. J. Forthcoming. The singularity: A reply. Journal of Consciousness Studies 19.Google Scholar
  27. Crevier, D. (1993). AI: The tumultuous history of the search for artificial intelligence. New York: Basic Books.Google Scholar
  28. de Blanc, P. (2011). Ontological crises in artificial agents’ value systems. San Francisco: Singularity Institute for Artificial Intelligence, May 19.
  29. de Garis, H., Shuo, C., Goertzel, B., & Ruiting, L. (2010). A world survey of artificial brain projects, part I: Large-scale brain simulations. Neurocomputing, 74(1–3), 3–29. doi: 10.1016/j.neucom.2010.08.004.CrossRefGoogle Scholar
  30. Dennett, D. C. (1996). Kinds of minds: Toward an understanding of consciousness., Science Master New York: Basic Books.Google Scholar
  31. Dewey, D. (2011). Learning what to value. In Schmidhuber, J., Thórisson, KR., & Looks, M. 2011, 309–314.Google Scholar
  32. Dreyfus, H. L. (1972). What computers can’t do: A critique of artificial reason. New York: Harper & Row.Google Scholar
  33. Eden, A., Søraker, J., Moor, J. H., & Steinhart, E. (Eds.). (2012). The singularity hypothesis: A scientific and philosophical assessment. Berlin: Springer.Google Scholar
  34. Feldman, J. A., & Ballard, D. H. (1982). Connectionist models and their properties. Cognitive Science, 6(3), 205–254. doi:10.1207/s15516709cog0603_1.CrossRefGoogle Scholar
  35. Floreano, D., & Mattiussi, C. (2008). Bio-inspired artificial intelligence: Theories, methods, and technologies. Intelligent Robotics and Autonomous Agents. MIT Press: Cambridge.Google Scholar
  36. Fox, D. (2011). The limits of intelligence. Scientific American, July, 36–43.Google Scholar
  37. Fregni, F., Boggio, P. S., Nitsche, M., Bermpohl, F., Antal, A., Feredoes, E., et al. (2005). Anodal transcranial direct current stimulation of prefrontal cortex enhances working memory. Experimental Brain Research, 166(1), 23–30. doi: 10.1007/s00221-005-2334-6.CrossRefGoogle Scholar
  38. Friedman, M. (1953). The methodology of positive economics. In Essays in positive economics (pp. 3–43). Chicago: Chicago University Press.Google Scholar
  39. Friedman, James W., (Ed.) (1994). Problems of coordination in economic activity (Vol. 35). Recent Economic Thought. Boston: Kluwer Academic Publishers.Google Scholar
  40. Gödel, K. (1931). Über formal unentscheidbare sätze der Principia Mathematica und verwandter systeme I. Monatshefte für Mathematik, 38(1), 173–198. doi: 10.1007/BF01700692.CrossRefGoogle Scholar
  41. Goertzel, B. (2006). The hidden pattern: A patternist philosophy of mind. Boco Raton: BrownWalker Press.Google Scholar
  42. Goertzel, B. (2010). Toward a formal characterization of real-world general intelligence. In E. Baum, M. Hutter, & E. Kitzelmann (Eds.) Artificial general intelligence: Proceedings of the third conference on artificial general intelligence, AGI 2010, Lugano, Switzerland, March 58, 2010, 19–24. Vol. 10. Advances in Intelligent Systems Research. Amsterdam: Atlantis Press. doi: 10.2991/agi.2010.17.
  43. Goertzel, B. (2012). Should humanity build a global AI nanny to delay the singularity until it’s better understood? Journal of Consciousness Studies 19(1–2), 96–111. Scholar
  44. Goertzel, B., & Pennachin, C. (Eds.) (2007). Artificial general intelligence. Cognitive Technologies. Berlin: Springer. doi: 10.1007/978-3-540-68677-4.
  45. Goldreich, O. (2010). P, NP, and NP-Completeness: The basics of computational complexity. New York: Cambridge University Press.zbMATHCrossRefGoogle Scholar
  46. Good, I. J. (1959). Speculations on perceptrons and other automata. Research Lecture, RC-115. IBM, Yorktown Heights, New York, June 2. papers/58DC4EA36A143C218525785E00502E30/$File/rc115.pdf.
  47. Good, I. J. (1965). Speculations concerning the first ultraintelligent machine. In F. L. Alt & M. Rubinoff (Eds.) Advances in computers (pp. 31–88. Vol. 6). New York: Academic Press. doi: 10.1016/S0065-2458(08)60418-0.
  48. Good, I. J. (1970). Some future social repercussions of computers. International Journal of Environmental Studies, 1(1–4), 67–79. doi: 10.1080/00207237008709398.CrossRefGoogle Scholar
  49. Good, I. J. (1982). Ethical machines. In J. E. Hayes, D. Michie, & Y.-H. Pao (Eds.) Machine intelligence (pp. 555–560, Vol. 10). Intelligent Systems: Practice and Perspective. Chichester: Ellis Horwood.Google Scholar
  50. Greenfield, S. (2012). The singularity: Commentary on David Chalmers. Journal of Consciousness Studies 19(1–2), 112–118.
  51. Griffin, D., & Tversky, A. (1992). The weighing of evidence and the determinants of confidence. Cognitive Psychology, 24(3), 411–435. doi: 10.1016/0010-0285(92)90013-R.CrossRefGoogle Scholar
  52. Groß, D. (2009). Blessing or curse? Neurocognitive enhancement by “brain engineering”. Medicine Studies, 1(4), 379–391. doi: 10.1007/s12376-009-0032-6.CrossRefGoogle Scholar
  53. Gubrud, M. A. (1997). Nanotechnology and international security. Paper presented at the Fifth Foresight Conference on Molecular Nanotechnology, Palo Alto, CA, Nov. 5–8.
  54. Halevy, A., Norvig, P., & Pereira, F. (2009). The unreasonable effectiveness of data. IEEE Intelligent Systems, 24(2), 8–12. doi: 10.1109/MIS.2009.36.CrossRefGoogle Scholar
  55. Hanson, R. (2008). Economics of the singularity. IEEE Spectrum, 45(6), 45–50. doi: 10.1109/MSPEC.2008.4531461.CrossRefGoogle Scholar
  56. Hanson, R. (2012). Meet the new conflict, same as the old conflict. Journal of Consciousness Studies 19(1–2), 119–125. Scholar
  57. Hanson, R. Forthcoming. Economic growth given machine intelligence. Journal of Artificial Intelligence Research. Google Scholar
  58. Hanson, R., & Yudkowsky, E. (2008). The Hanson-Yudkowsky AI-foom debate. LessWrong Wiki. (accessed Mar. 13, 2012).
  59. Hibbard, B. (2011). Measuring agent intelligence via hierarchies of environments. In Schmidhuber, J., Thórisson, KR., & Looks, M. 2011, 303–308.Google Scholar
  60. Hibbard, B. Forthcoming. Model-based utility functions. Journal of Artificial General Intelligence. Google Scholar
  61. Hutter, M. (2005). Universal artificial intelligence: Sequential decisions based on algorithmic probability. Texts in Theoretical Computer Science. Berlin: Springer. doi: 10.1007/b138233.
  62. Hutter, M. (2009). Open problems in universal induction & intelligence. Algorithms, 2(3), 879–906. doi: 10.3390/a2030879.MathSciNetCrossRefGoogle Scholar
  63. Hutter, M. (2012a). Can intelligence explode? Journal of Consciousness Studies 19(1–2), 143–166.
  64. Hutter, M. (2012b). One decade of universal artificial intelligence. In P. Wang & B. Goertzel (eds.) Theoretical foundations of artificial general intelligence (Vol. 4). Atlantis Thinking Machines. Paris: Atlantis Press.Google Scholar
  65. Jaynes, E. T., & Bretthorst, G. L. (Eds.) (2003). Probability theory: The logic of science. New York: Cambridge University Press. doi: 10.2277/0521592712.
  66. Jones, B. F. (2009). The burden of knowledge and the “Death of the Renaissance Man”: Is innovation getting harder? Review of Economic Studies, 76(1), 283–317. doi: 10.1111/j.1467-937X.2008.00531.x.zbMATHCrossRefGoogle Scholar
  67. Kaas, S., Rayhawk,S., Salamon, A., & Salamon, P. (2010). Economic implications of software minds. San Francisco: Singularity Institute for Artificial Intelligence, Aug. 10.
  68. Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (Eds.). (2000). Principles of neural science. New York: McGraw-Hill.Google Scholar
  69. Kolmogorov, A. N. (1968). Three approaches to the quantitative definition of information. International Journal of Computer Mathematics, 2(1–4), 157–168. doi: 10.1080/00207166808803030.MathSciNetzbMATHCrossRefGoogle Scholar
  70. Koza, J. R. (2010). Human-competitive results produced by genetic programming. Genetic Programming and Evolvable Machines, 11(3–4), 251–284. doi: 10.1007/s10710-010-9112-3.CrossRefGoogle Scholar
  71. Krichmar, J. L., & Wagatsuma, H. (Eds.). (2011). Neuromorphic and brain-based robots. New York: Cambridge University Press.Google Scholar
  72. Kryder, M. H., & Kim, C. S. (2009). After hard drives—what comes next? IEEE Transactions on Magnetics, 2009(10), 3406–3413. doi: 10.1109/TMAG.2009.2024163.CrossRefGoogle Scholar
  73. Kurzweil, R. (2005). The singularity is near: When humans transcend biology. New York: Viking.Google Scholar
  74. Lampson, B. W. (1973). A note on the confinement problem. Communications of the ACM, 16(10), 613–615. doi: 10.1145/362375.362389.CrossRefGoogle Scholar
  75. Legg, S. (2008). Machine super intelligence. PhD diss., University of Lugano.
  76. Legg, S., & Hutter, M. (2007). A collection of definitions of intelligence. In B. Goertzel & P. Wang (Eds.) Advances in artificial general intelligence: Concepts, architectures and algorithmsproceedings of the AGI workshop 2006 (Vol. 157). Frontiers in Artificial Intelligence and Applications. Amsterdam: IOS Press.Google Scholar
  77. Li, M., & Vitányi, P. M. B. (2008). An introduction to Kolmogorov complexity and its applications. Texts in Computer Science. New York: Springer. doi: 10.1007/978-0-387-49820-1.
  78. Lichtenstein, S., Fischoff, B., & Phillips, L. D. (1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgement under uncertainty: Heuristics and biases (pp. 306–334). New York: Cambridge University Press.Google Scholar
  79. Loosmore, R., & Goertzel, B. (2011). Why an intelligence explosion is probable. H+ Magazine, Mar. 7.
  80. Lucas, J. R. (1961). Minds, machines and Gödel. Philosophy, 36(137), 112–127. doi: 10.1017/S0031819100057983.CrossRefGoogle Scholar
  81. Lundstrom, M. (2003). Moore’s law forever? Science, 299(5604), 210–211. doi: 10.1126/science.1079567.CrossRefGoogle Scholar
  82. Mack, C. A. (2011). Fifty years of Moore’s law. IEEE Transactions on Semiconductor Manufacturing, 24(2), 202–207. doi: 10.1109/TSM.2010.2096437.CrossRefGoogle Scholar
  83. Marcus, G. (2008). Kluge: The haphazard evolution of the human mind. Boston: Houghton Mifflin.Google Scholar
  84. McAfee, A., & Brynjolfsson, E. (2008). Investing in the IT that makes a competitive difference. Harvard Business Review, July.
  85. McCorduck, P. (2004). Machines who think: A personal inquiry into the history and prospects of artificial intelligence (2nd ed.). Natick: A. K. Peters.Google Scholar
  86. McDaniel, M. A. (2005). Big-brained people are smarter: A meta-analysis of the relationship between in vivo brain volume and intelligence. Intelligence, 33(4), 337–346. doi: 10.1016/j.intell.2004.11.005.CrossRefGoogle Scholar
  87. McDermott, D. (2012a). Response to “The Singularity” by David Chalmers. Journal of Consciousness Studies 19(1–2): 167–172. Scholar
  88. McDermott, D. (2012b). There are no “Extendible Methods” in David Chalmers’s sense unless P=NP. Unpublished manuscript. (accessed Mar. 19, 2012).
  89. Mehta, G. B. (1998). Preference and utility. In S. Barbera, P. J. Hammond, & C. Seidl (Eds.), Handbook of utility theory (Vol. I, pp. 1–47). Boston: Kluwer Academic Publishers.Google Scholar
  90. Minsky, M. (1984). Afterword to Vernor Vinge’s novel, “True Names.” Unpublished manuscript, Oct. 1. (accessed Mar. 26, 2012).
  91. Modha, D. S., Ananthanarayanan, R., Esser, S. K., Ndirango, A., Sherbondy, A. J., & Singh, R. (2011). Cognitive computing. Communications of the ACM, 54(8), 62–71. doi: 10.1145/1978542.1978559.CrossRefGoogle Scholar
  92. Modis, T. (2012). There will be no singularity. In Eden, Søraker, Moor, & Steinhart 2012.Google Scholar
  93. Moravec, H. P. (1976). The role of raw rower in intelligence. May 12. (accessed Mar. 13, 2012).
  94. Moravec, H. (1998). When will computer hardware match the human brain? Journal of Evolution and Technology 1.
  95. Moravec, H. (1999). Rise of the robots. Scientific American, Dec., 124–135.Google Scholar
  96. Muehlhauser, L. (2011). So you want to save the world. Last modified Mar. 2, 2012.
  97. Muehlhauser, L., & Helm, L. (2012). The singularity and machine ethics. In Eden, Søraker, Moor, & Steinhart 2012.Google Scholar
  98. Murphy, A. H., & Winkler, R. L. (1984). Probability forecasting in meteorology. Journal of the American Statistical Association, 79(387), 489–500.Google Scholar
  99. Nagy, B., Farmer, J. D., Trancik, J. E., & Bui, QM. (2010). Testing laws of technological progress. Santa Fe Institute, NM, Sept. 2. bn/workingpapers/NagyFarmerTrancikBui.pdf.
  100. Nagy, B., Farmer, J. D., Trancik, J. E., & Gonzales, J. P. (2011). Superexponential long-term trends in information technology. Technological Forecasting and Social Change, 78(8), 1356–1364. doi: 10.1016/j.techfore.2011.07.006.CrossRefGoogle Scholar
  101. Nielsen, M. (2011). What should a reasonable person believe about the singularity? Michael Nielsen (blog). Jan. 12. (accessed Mar. 13, 2012).
  102. Nilsson, N. J. (2009). The quest for artificial intelligence: A history of ideas and achievements. New York: Cambridge University Press.Google Scholar
  103. Nordmann, A. (2007). If and then: A critique of speculative nanoethics. NanoEthics, 1(1), 31–46. doi: 10.1007/s11569-007-0007-6.CrossRefGoogle Scholar
  104. Omohundro, S. M. (1987). Efficient algorithms with neural network behavior. Complex Systems 1(2), 273–347.
  105. Omohundro, S. M. (2007). The nature of self-improving artificial intelligence. Paper presented at the Singularity Summit 2007, San Francisco, CA, Sept. 8–9.
  106. Omohundro, S. M. (2008). The basic AI drives. In Wang, Goertzel, & Franklin 2008, 483–492.Google Scholar
  107. Omohundro, S. M. 2012. Rational artificial intelligence for the greater good. In Eden, Søraker, Moor, & Steinhart 2012.Google Scholar
  108. Orseau, L. (2011). Universal knowledge-seeking agents. In Algorithmic learning theory: 22nd international conference, ALT 2011, Espoo, Finland, October 57, 2011. Proceedings, ed. Jyrki Kivinen, Csaba Szepesvári, Esko Ukkonen, and Thomas Zeugmann. Vol. 6925. Lecture Notes in Computer Science. Berlin: Springer. doi 10.1007/978-3-642-24412-4_28.Google Scholar
  109. Orseau, L., & Ring, M. (2011). Self-modification and mortality in artificial agents. In Schmidhuber, Thórisson, and Looks 2011, 1–10.Google Scholar
  110. Pan, Z., Trikalinos, T. A., Kavvoura, F. K., Lau, J., & Ioannidis, J. P. A. (2005). Local literature bias in genetic epidemiology: An empirical evaluation of the Chinese literature. PLoS Medicine, 2(12), e334. doi: 10.1371/journal.pmed.0020334.CrossRefGoogle Scholar
  111. Parente, R., & Anderson-Parente, J. (2011). A case study of long-term Delphi accuracy. Technological Forecasting and Social Change, 78(9), 1705–1711. doi: 10.1016/j.techfore.2011.07.005.CrossRefGoogle Scholar
  112. Pennachin, C, & Goertzel, B. (2007). Contemporary approaches to artificial general intelligence. In Goertzel & Pennachin 2007, 1–30.Google Scholar
  113. Penrose, R. (1994). Shadows of the mind: A search for the missing science of consciousness. New York: Oxford University Press.Google Scholar
  114. Plebe, A., & Perconti, P. (2012). The slowdown hypothesis. In Eden, Søraker, Moor, & Steinhart 2012.Google Scholar
  115. Posner, R. A. (2004). Catastrophe: Risk and response. New York: Oxford University Press.Google Scholar
  116. Proudfoot, D., & Jack Copeland, B. (2012). Artificial intelligence. In E. Margolis, R. Samuels, & S. P. Stich (Eds.), The Oxford handbook of philosophy of cognitive science. New York: Oxford University Press.Google Scholar
  117. Rathmanner, S., & Hutter, M. (2011). A philosophical treatise of universal induction. Entropy, 13(6), 1076–1136. doi: 10.3390/e13061076.MathSciNetCrossRefGoogle Scholar
  118. Richards, M. A., & Shaw, G. A. (2004). Chips, architectures and algorithms: Reflections on the exponential growth of digital signal processing capability. Unpublished manuscript, Jan. 28. mrichard/Richards&Shaw_Algorithms01204.pdf (accessed Mar. 20, 2012).
  119. Rieffel, E., & Polak, W. (2011). Quantum computing: A gentle introduction. Scientific and Engineering Computation. Cambridge: MIT Press.Google Scholar
  120. Ring, M., & Orseau, L. (2011). Delusion, survival, and intelligent agents. In Schmidhuber, Thórisson, & Looks 2011, 11–20.Google Scholar
  121. Rowe, G., & Wright, G. (2001). Expert opinions in forecasting: The role of the Delphi technique. In J. S. Armstrong (Ed.), Principles of forecasting: A handbook for researchers and practitioners, (Vol. 30). International Series in Operations Research & Management Science. Boston: Kluwer Academic Publishers.Google Scholar
  122. Russell, S. J., & Norvig, P. (2009). Artificial intelligence: A modern approach (3rd ed.). Upper Saddle River: Prentice-Hall.Google Scholar
  123. Sandberg, A. (2010). An overview of models of technological singularity. Paper presented at the Roadmaps to AGI and the future of AGI workshop, Lugano, Switzerland, Mar. 8th.
  124. Sandberg, A. (2011). Cognition enhancement: Upgrading the brain. In J. Savulescu, R. ter Meulen, & G. Kahane (Eds.), Enhancing human capacities (pp. 71–91). Malden: Wiley-Blackwell.Google Scholar
  125. Sandberg, A., & Bostrom, N. (2008). Whole brain emulation: A roadmap. Technical Report, 2008-3. Future of Humanity Institute, University of Oxford.
  126. Sandberg, A., & Bostrom, N. (2011). Machine intelligence survey. Technical Report, 2011-1. Future of Humanity Institute, University of Oxford.
  127. Schaul, T., & Schmidhuber, J. (2010). Metalearning. Scholarpedia, 5(6), 4650. doi: 10.4249/scholarpedia.4650.CrossRefGoogle Scholar
  128. Schierwagen, A. (2011). Reverse engineering for biologically inspired cognitive architectures: A critical analysis. In C. Hernández, R. Sanz, J. Gómez-Ramirez, L. S. Smith, A. Hussain, A. Chella, & I. Aleksander (Eds.), From brains to systems: Brain-inspired cognitive systems 2010, (pp. 111–121, Vol. 718). Advances in Experimental Medicine and Biology. New York: Springer. doi:10.1007/978-1-4614-0164-3_10.Google Scholar
  129. Schmidhuber, J. (2002). The speed prior: A new simplicity measure yielding near-optimal computable predictions. In J. Kivinen & R. H. Sloan, Computational learning theory: 5th annual conference on computational learning theory, COLT 2002 Sydney, Australia, July 810, 2002 proceedings, (pp. 123–127, Vol. 2375). Lecture Notes in Computer Science. Berlin: Springer. doi:10.1007/3-540-45435-7_15.Google Scholar
  130. Schmidhuber, J. (2007). Gödel machines: Fully self-referential optimal universal self-improvers. In Goertzel & Pennachin 2007, 199–226.Google Scholar
  131. Schmidhuber, J. (2009). Ultimate cognition à la Gödel. Cognitive Computation, 1(2), 177–193. doi: 10.1007/s12559-009-9014-y.CrossRefGoogle Scholar
  132. Schmidhuber, J. (2012). Philosophers & futurists, catch up! Response to The Singularity. Journal of Consciousness Studies 19(1–2), 173–182.
  133. Schmidhuber, J., Thórisson, K. R., & Looks, M. (Eds.) (2011). Artificial General Intelligence: 4th International Conference, AGI 2011, Mountain View, CA, USA, August 36, 2011. Proceedings (Vol. 6830). Lecture Notes in Computer Science. Berlin: Springer. doi: 10.1007/978-3-642-22887-2.
  134. Schneider, S. (2010). Homo economicusor more like Homer Simpson? Current Issues. Deutsche Bank Research, Frankfurt, June 29.
  135. Schoenemann, P. T. (1997). An MRI study of the relationship between human neuroanatomy and behavioral ability. PhD diss., University of California, Berkeley. toms/papers/dissertation/Dissertation_title.htm.
  136. Schwartz, J. T. (1987). Limits of artificial intelligence. In S. C. Shapiro & D. Eckroth (Eds.), Encyclopedia of artificial intelligence (pp. 488–503, Vol. 1). New York: Wiley.Google Scholar
  137. Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(03), 417–424. doi: 10.1017/S0140525X00005756.CrossRefGoogle Scholar
  138. Shulman, C., & Bostrom, N. (2012). How hard is artificial intelligence? Evolutionary arguments and selection effects. Journal of Consciousness Studies 19.Google Scholar
  139. Shulman, C., & Sandberg, A. (2010). Implications of a software-limited singularity. Paper presented at the 8th European Conference on Computing and Philosophy (ECAP), Munich, Germany, Oct. 4–6.Google Scholar
  140. Simon, H. A. (1965). The shape of automation for men and management. New York: Harper & Row.Google Scholar
  141. Solomonoff, R. J. (1964a). A formal theory of inductive inference. Part I. Information and Control, 7(1), 1–22. doi: 10.1016/S0019-9958(64)90223-2.MathSciNetzbMATHCrossRefGoogle Scholar
  142. Solomonoff, R. J. (1964b). A formal theory of inductive inference. Part II. Information and Control, 7(2), 224–254. doi: 10.1016/S0019-9958(64)90131-7.MathSciNetzbMATHCrossRefGoogle Scholar
  143. Solomonoff, R. J. (1985). The time scale of artificial intelligence: Reflections on social effects. Human Systems Management, 5, 149–153.Google Scholar
  144. Sotala, K. (2012). Advantages of artificial intelligences, uploads, and digital minds. International Journal of Machine Consciousness 4.Google Scholar
  145. Stanovich, K. E. (2010). Rationality and the reflective mind. New York: Oxford University Press.CrossRefGoogle Scholar
  146. Tetlock, P. E. (2005). Expert political judgment: How good is it? How can we know?. Princeton: Princeton University Press.Google Scholar
  147. The Royal Society. (2011). Knowledge, networks and nations: Global scientific collaboration in the 21st century. RS Policy document, 03/11. The Royal Society, London.
  148. Trappenberg, T. P. (2009). Fundamentals of computational neuroscience (2nd ed.). New York: Oxford University Press.Google Scholar
  149. Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–460. doi: 10.1093/mind/LIX.236.433.MathSciNetCrossRefGoogle Scholar
  150. Turing, A. M. (1951). Intelligent machinery, a heretical theory. A lecture given to `51 Society’ at Manchester.Google Scholar
  151. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131. doi: 10.1126/science.185.4157.1124.CrossRefGoogle Scholar
  152. Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4), 293–315. doi: 10.1037/0033-295X.90.4.293.CrossRefGoogle Scholar
  153. The Uncertain Future. (2012). What is multi-generational in vitro embryo selection? The Uncertain Future. (accessed Mar. 25, 2012).
  154. Van der Velde, F. (2010). Where artificial intelligence and neuroscience meet: The search for grounded architectures of cognition. Advances in Artificial Intelligence, no. 5. doi: 10.1155/2010/918062.
  155. Van Gelder, T., & Port, R. F. (1995). It’s about time: An overview of the dynamical approach to cognition. In R. F. Port & T. van Gelder. Mind as motion: Explorations in the dynamics of cognition, Bradford Books. Cambridge: MIT Press.Google Scholar
  156. Veness, J., Ng, K. S., Hutter, M., Uther, W., & Silver, D. (2011). A Monte-Carlo AIXI approximation. Journal of Artificial Intelligence Research, 40, 95–142. doi: 10.1613/jair.3125.MathSciNetzbMATHGoogle Scholar
  157. Vinge, V. (1993). The coming technological singularity: How to survive in the post-human era. In Vision-21: Interdisciplinary science and engineering in the era of cyberspace, 11–22. NASA Conference Publication 10129. NASA Lewis Research Center.
  158. Von Neumann, J., & Burks, A. W. (Eds.) (1966). Theory of self-replicating automata. Urbana: University of Illinois Press.Google Scholar
  159. Walter, C. (2005). Kryder’s law. Scientific American, July 25. id = kryders-law.
  160. Wang, P., Goertzel, B., & Franklin, S. (Eds.). (2008). Artificial General Intelligence 2008: Proceedings of the First AGI Conference (Vol. 171). Frontiers in Artificial Intelligence and Applications. Amsterdam: IOS Press.Google Scholar
  161. Williams, L. V. (Ed.). (2011). Prediction markets: Theory and applications (Vol. 66). Routledge International Studies in Money and Banking. New York: Routledge.Google Scholar
  162. Wootters, W. K., & Zurek, W. H. (1982). A single quantum cannot be cloned. Nature, 299(5886), 802–803. doi: 10.1038/299802a0.CrossRefGoogle Scholar
  163. Woudenberg, F. (1991). An evaluation of Delphi. Technological Forecasting and Social Change, 40(2), 131–150. doi: 10.1016/0040-1625(91)90002-W.CrossRefGoogle Scholar
  164. Yampolskiy, R. V. (2012). Leakproofing the singularity: Artificial intelligence confinement problem. Journal of Consciousness Studies 19(1–2), 194–214. Scholar
  165. Yates, J. F., Lee, J.-W., Sieck, W. R., Choi, I., & Price, P. C. (2002). Probability judgment across cultures. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 271–291). New York: Cambridge University Press.CrossRefGoogle Scholar
  166. Yudkowsky, E. (2001). Creating Friendly AI 1.0: The analysis and design of benevolent goal architectures. The Singularity Institute, San Francisco, CA, June 15.
  167. Yudkowsky, E. (2008a). Artificial intelligence as a positive and negative factor in global risk. In Bostrom & Ćirković 2008, 308–345.Google Scholar
  168. Yudkowsky, E. (2008b). Efficient cross-domain optimization. LessWrong. Oct. 28. (accessed Mar. 19, 2012).
  169. Yudkowsky, E. (2011). Complex value systems in friendly AI. In Schmidhuber, Thórisson, & Looks 2011, 388–393.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Machine Intelligence Research InstituteBerkeleyUSA

Personalised recommendations