Singularity Hypotheses pp 61-82 | Cite as

# New Millennium AI and the Convergence of History: Update of 2012

## Abstract

Artificial Intelligence (AI) has recently become a real formal science: the new millennium brought the first mathematically sound, asymptotically optimal, *universal* problem solvers, providing a new, rigorous foundation for the previously largely heuristic field of General AI and embedded agents. There also has been rapid progress in not quite universal but still rather general and *practical* artificial recurrent neural networks for learning sequence-processing programs, now yielding state-of-the-art results in real world applications. And the computing power per Euro is still growing by a factor of 100–1,000 per decade, greatly increasing the feasibility of neural networks in general, which have started to yield human-competitive results in challenging pattern recognition competitions. Finally, a recent formal theory of fun and creativity identifies basic principles of curious and creative machines, laying foundations for artificial scientists and artists. Here I will briefly review some of the new results of my lab at IDSIA, and speculate about future developments, pointing out that the time intervals between the most notable events in over 40,000 years or \(2^9\) lifetimes of human history have sped up exponentially, apparently converging to zero within the next few decades. Or is this impression just a by-product of the way humans allocate memory space to past events?

## Keywords

Reinforcement Learn Proof Search Reinforcement Learn Method Chinese Handwritten Character Traffic Sign Recognition## References

- Balcan, M. F., Beygelzimer, A., & Langford, J. (2009). Agnostic active learning.
*Journal of Computer and System Sciences,**75*(1), 78–89.MathSciNetMATHCrossRefGoogle Scholar - Barto, A. (2013). Intrinsic motivation and reinforcement learning. In G. Baldassarre & M. Mirolli (Eds.), Intrinsically motivated learning in natural and artificial systems. Springer (in press).Google Scholar
- Behnke, S. (2003).
*Hierarchical neural networks for image interpretation, volume 2766 of lecture notes in computer science*. Springer.Google Scholar - Bishop, C. M. (2006).
*Pattern recognition and machine learning*. NY: Springer.MATHGoogle Scholar - Bringsjord, S. (2000), ‘A contrarian future for minds and machines’, chronicle of higher education (p. B5). Reprinted in The Education Di-gest, vol. 66(6), pp. 31–33.Google Scholar
- Ciresan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2010). Deep big simple neural nets for handwritten digit recogntion.
*Neural Computation,**22*(12), 3207–3220.CrossRefGoogle Scholar - Ciresan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2011a). Convolutional neural network committees for handwritten character classification. In 11th International Conference on Document Analysis and Recognition (ICDAR), pp 1250–1254.Google Scholar
- Ciresan, D. C., Meier, U., Masci, J., Gambardella, L. M. & Schmidhuber, J. (2011b). Flexible, high performance convolutional neural networks for image classification. In International Joint Conference on Artificial Intelligence IJCAI, pp 1237–1242.Google Scholar
- Ciresan, D. C., Meier, U., Masci, J., & Schmidhuber, J. (2011c). A committee of neural networks for traffic sign classification. In International Joint Conference on, Neural Networks, pp 1918–1921.Google Scholar
- Ciresan, D. C., Meier, U., Masci, J., & Schmidhuber, J. (2012a). Multi-column deep neural network for traffic sign classification.
*Neural Networks, 32*, 333–338.Google Scholar - Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012b). Multi-column deep neural networks for image classification. In IEEE Conference on Computer Vision and Pattern Recognition CVPR 2012, pp 3642–3649.Google Scholar
- Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012c). Multi-column deep neural networks for image classification. In IEEE Conference on Computer Vision and Pattern Recognition CVPR 2012. Long preprint arXiv:1202.2745v1 [cs.CV].Google Scholar
- Darwin, C. (1997).
*The descent of man, prometheus, amherst*. NY: A reprint edition.Google Scholar - Dayan, P. (2013). Exploration from generalization mediated by multiple controllers. In G. Baldassarre & M. Mirolli (Eds.), Intrinsically motivated learning in natural and artificial systems. Springer (in press).Google Scholar
- Fedorov, V. V. (1972).
*Theory of optimal experiments*. NY: Academic.Google Scholar - Fernandez, S., Graves, A., & Schmidhuber, J. (2007). Sequence labelling in structured domains with hierarchical recurrent neural networks. In
*Proceedings of the 20th International Joint Conference on Artificial Intelligence*(IJCAI).Google Scholar - Floridi, L. (2007). A look into the future impact of ICT on our lives.
*The Information Society,**23*(1), 59–64.CrossRefGoogle Scholar - Fukushima, K. (1980). Neocognitron: A self-organizing neural network for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics36(4), 193–202.Google Scholar
- Gers, F. A., & Schmidhuber, J. (2001). LSTM recurrent networks learn simple context free and context sensitive languages.
*IEEE Transactions on Neural Networks,**12*(6), 1333–1340.CrossRefGoogle Scholar - Gers, F. A., Schraudolph, N., & Schmidhuber, J. (2002). Learning precise timing with LSTM recurrent networks.
*Journal of Machine Learning Research,**3*, 115–143.MathSciNetGoogle Scholar - Gisslen, L., Luciw, M., Graziano, V., & Schmidhuber, J. (2011). Sequential constant size compressor for reinforcement learning. In
*Proceedings of Fourth Conference on Artificial General Intelligence*(AGI), Google, Mountain View, CA.Google Scholar - Glasmachers, T., Schaul, T., Sun, Y., Wierstra, D. & Schmidhuber, J. (2010). Exponential Natural Evolution Strategies. In
*Proceedings of the Genetic and Evolutionary Computation Conference (GECCO)*.Google Scholar - Gödel, K. (1931). Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I.
*Monatshefte für Mathematik und Physik,**38*, 173–198.CrossRefGoogle Scholar - Gomez, F. J., Schmidhuber, J., & Miikkulainen, R. (2008). Efficient non-linear control through neuroevolution.
*Journal of Machine Learning Research JMLR,**9*, 937–965.MathSciNetMATHGoogle Scholar - Graves, A., Fernandez, S., Gomez, F. J., & Schmidhuber, J. (2006). Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural nets. In ICML ’06:
*Proceedings of the International Conference on Machine Learning*.Google Scholar - Graves, A., Fernandez, S., Liwicki, M., Bunke, H., & Schmidhuber, J. (2008). Unconstrained on-line handwriting recognition with recurrent neural networks. In J. C. Platt, D. Koller, Y. Singer, & S. Roweis (Eds.),
*Advances in Neural Information Processing Systems 20*(pp. 577–584). Cambridge: MIT Press.Google Scholar - Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., & Schmidhuber, J. (2009). A novel connectionist system for improved unconstrained handwriting recognition.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 31(5), 855–868.Google Scholar - Graves, A., & Schmidhuber, J. (2009). Offline handwriting recognition with multidimensional recurrent neural networks. In
*Advances in Neural Information Processing Systems*(p. 21). Cambridge: MIT Press.Google Scholar - Hansen, N., & Ostermeier, A. (2001). Completely derandomized self-adaptation in evolution strategies.
*Evolutionary Computation,**9*(2), 159–195.CrossRefGoogle Scholar - Hart, S., Sen, S., & Grupen, R. (2008). Intrinsically motivated hierarchical manipulation. In
*Proceedings of the IEEE Conference on Robots and Automation (ICRA)*. California: Pasadena. Google Scholar - Hochreiter, S., Bengio, Y., Frasconi, P., & Schmidhuber, J. (2001). Gradient flow in recurrent nets: The difficulty of learning long-term dependencies. In S. C. Kremer & J. F. Kolen (Eds.),
*A Field Guide to Dynamical Recurrent Neural Networks*. NJ: IEEE Press.Google Scholar - Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory.
*Neural Computation,**9*(8), 1735–1780.Google Scholar - Holland, J. H. (1975).
*Adaptation in natural and artificial systems*. Ann Arbor: University of Michigan Press.Google Scholar - Hutter, M. (2002). The fastest and shortest algorithm for all well-defined problems.
*International Journal of Foundations of Computer Science*,*13*(3), 431–443 (On J. Schmidhuber’s SNF grant 20–61847).Google Scholar - Hutter, M. (2005).
*Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability*. Berlin: Springer (On J. Schmidhuber’s SNF grant 20–61847).Google Scholar - Jaeger, H. (2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication.
*Science,**304*, 78–80.CrossRefGoogle Scholar - Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey.
*Journal of AI research,**4*, 237–285.Google Scholar - Kolmogorov, A. N. (1965). Three approaches to the quantitative definition of information.
*Problems of Information Transmission,**1*, 1–11.Google Scholar - Koutnik, J., Gomez, F., & Schmidhuber, J. (2010). Evolving neural networks in compressed weight space. In
*Proceedings of the Conference on Genetic and, Evolutionary Computation (GECCO-10)*.Google Scholar - Krizhevsky, A. (2009).
*Learning multiple layers of features from tiny images*. Master’s thesis: Computer Science Department, University of Toronto.Google Scholar - Kuipers, B., Beeson, P., Modayil, J., & Provost, J. (2006). Bootstrap learning of foundational representations.
*Connection Science,**18*(2).Google Scholar - Kurzweil, R. (2005).
*The singularity is near*. NY: Wiley Interscience.Google Scholar - LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition.
*Proceedings of the IEEE,**86*(11), 2278–2324.Google Scholar - LeCun, Y., Huang, F.-J., & Bottou, L. (2004). Learning methods for generic object recognition with invariance to pose and lighting. In
*Proceedings of Computer Vision and Pattern Recognition Conference*.Google Scholar - Lenat, D. B. (1983). Theory formation by heuristic search.
*Machine Learning*, vol. 21.Google Scholar - Levin, L. A. (1973). Universal sequential search problems.
*Problems of Information Transmission,**9*(3), 265–266.Google Scholar - Li, M., & Vitányi, P. M. B. (1997).
*An introduction to kolmogorov complexity and its applications*(2nd ed.). NY: Springer.MATHCrossRefGoogle Scholar - Maass, W., Natschläger, T., & Markram, H. (2002).
*A fresh look at real-time computation in generic recurrent neural circuits*. Institute for Theoretical Computer Science, TU Graz : Technical report.Google Scholar - Mitchell, T. (1997).
*Machine learning*. NY: McGraw Hill.MATHGoogle Scholar - Moravec, H. (1999).
*Robot*. NY: Wiley Interscience.Google Scholar - Newell, A., & Simon, H. (1963). GPS, a program that simulates human thought. In E. Feigenbaum & J. Feldman (Eds.),
*Computers and thought*(pp. 279–293). New York: McGraw-Hill.Google Scholar - Oudeyer, P. -Y., Baranes, A., & Kaplan, F. (2013). Intrinsically motivated learning of real world sensorimotor skills with developmental constraints. In G. Baldassarre & M. Mirolli (Eds.), Intrinsically motivated learning in natural and artificial systems. Springer (in press).Google Scholar
- Rechenberg, I. (1971). Evolutions strategie–optimierung technischer systeme nach Prinzipien der biologischen Evolution. Dissertation, Published 1973 by Fromman-Holzboog.Google Scholar
- Robinson, A. J., & Fallside, F. (1987). The utility driven dynamic error propagation network. Technical Report CUED/F-INFENG/TR.1, Cambridge University Engineering Department.Google Scholar
- Rosenbloom, P. S., Laird, J. E., & Newell, A. (1993).
*The SOAR papers*. NY: MIT Press.Google Scholar - Schaul, T., Bayer, J., Wierstra, D., Sun, Y., Felder, M., Sehnke, F., et al. (2010). PyBrain.
*Journal of Machine Learning Research,**11*, 743–746.Google Scholar - Scherer, D., Müller, A., & Behnke, S. (2010). In
*International Conference on Artificial Neural Networks*.Google Scholar - Schmidhuber, J. (1990).
*Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem*. Dissertation: Institut für Informatik, Technische Universität München.Google Scholar - Schmidhuber, J. (1991a). Curious model-building control systems. In
*Proceedings of the International Joint Conference on Neural Networks*(vol. 2, pp. 1458–1463). Singapore: IEEE press.Google Scholar - Schmidhuber, J. (1991b). A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer & S. W. Wilson (Eds.)
*Proceedings of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats*, pp. 222–227. MIT Press/Bradford Books.Google Scholar - Schmidhuber, J. (1991c). Reinforcement learning in Markovian and non-Markovian environments. In D. S. Lippman, J. E. Moody, & D. S. Touretzky (Eds.),
*Advances in neural information processing systems 3*(NIPS 3) (pp. 500–506). NY: Morgan Kaufmann.Google Scholar - Schmidhuber, J. (1992a). A fixed size storage \(O(n^3)\) time complexity learning algorithm for fully recurrent continually running networks.
*Neural Computation,**4*(2), 243–248.CrossRefGoogle Scholar - Schmidhuber, J. (1992b). Learning factorial codes by predictability minimization.
*Neural Computation,**4*(6), 863–879.CrossRefGoogle Scholar - Schmidhuber, J. (1997). Discovering neural nets with low Kolmogorov complexity and high generalization capability.
*Neural Networks,**10*(5), 857–873.CrossRefGoogle Scholar - Schmidhuber, J. (1999). Artificial curiosity based on discovering novel algorithmic predictability through coevolution. In P. Angeline, Z. Michalewicz, M. Schoenauer, X. Yao,& Z. Zalzala (Eds.),
*Congress on evolutionary computation*(pp. 1612–1618). Piscataway: IEEE Press.Google Scholar - Schmidhuber, J. (2002a). Hierarchies of generalized Kolmogorov complexities and nonenumerable universal measures computable in the limit.
*International Journal of Foundations of Computer Science,**13*(4), 587–612.MathSciNetMATHCrossRefGoogle Scholar - Schmidhuber, J. (2002). The speed prior: A new simplicity measure yielding near-optimal computable predictions. In J. Kivinen& R. H. Sloan (Eds.),
*Proceedings of the 15th Annual Conference on Computational Learning Theory (COLT 2002)*(pp. 216–228). Lecture Notes in Artificial Intelligence Sydney, Australia: Springer.Google Scholar - Schmidhuber, J. (2003a). Exponential speed-up of computer history’s defining moments. http://www.idsia.ch/juergen/computerhistory.html
- Schmidhuber, J. (2003b). The new AI: General & sound & relevant for physics. Technical Report TR IDSIA-04-03, Version 1.0, arXiv:cs.AI/0302012 v1.Google Scholar
- Schmidhuber, J. (2004). Optimal ordered problem solver.
*Machine Learning,**54*, 211–254.MATHCrossRefGoogle Scholar - Schmidhuber, J. (2005). Completely self-referential optimal reinforcement learners. In W. Duch, J. Kacprzyk, E. Oja, & S. Zadrozny (Eds.), Artificial neural networks: Biological inspirations–ICANN 2005 (pp. 223–233), LNCS 3697. Springer: Berlin Heidelberg (Plenary talk).Google Scholar
- Schmidhuber, J. (2006a). Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts.
*Connection Science,**18*(2), 173–187.CrossRefGoogle Scholar - Schmidhuber, J. (2006b). Gödel machines: Fully self-referential optimal universal self-improvers. In B. Goertzel& C. Pennachin (Eds.),
*Artificial general intelligence*(pp. 199–226). Heidelberg: Springer (Variant available as arXiv:cs.LO/0309048).Google Scholar - Schmidhuber, J. (2006c). Celebrating 75 years of AI–history and outlook: The next 25 years. In M. Lungarella, F. Iida, J. Bongard,& R. Pfeifer (Eds.),
*50 years of artificial intelligence*(vol. LNAI 4850, pp. 29–41). Berlin/Heidelberg: Springer (Preprint available as arXiv:0708.4311).Google Scholar - Schmidhuber, J. (2007a). Gödel machines: Fully self-referential optimal universal self-improvers. In B. Goertzel& C. Pennachin (Eds.),
*Artificial general intelligence*(pp. 199–226). Springer Verlag (Variant available as arXiv:cs.LO/0309048).Google Scholar - Schmidhuber, J. (2007b). New millennium AI and the convergence of history. In W. Duch& J. Mandziuk (Eds.),
*Challenges to computational intelligence*(vol. 63, pp. 15–36). Studies in Computational Intelligence, Springer, 2007. Also available as arXiv:cs.AI/0606081.Google Scholar - Schmidhuber, J. (2009). Ultimate cognition à la Gödel.
*Cognitive Computation,**1*(2), 177–193.CrossRefGoogle Scholar - Schmidhuber, J. (2010). Formal theory of creativity, fun, and intrinsic motivation (1990–2010).
*IEEE Transactions on Autonomous Mental Development,**2*(3), 230–247.CrossRefGoogle Scholar - Schmidhuber, J. (2011). PowerPlay: Training an increasingly general problem solver by continually searching for the simplest still unsolvable problem. Technical Report arXiv:1112.5309v1 [cs.AI].Google Scholar
- Schmidhuber, J. (2012). Philosophers& futurists, catch up! response to the singularity.
*Journal of Consciousness Studies,**19*(1–2), 173–182.Google Scholar - Schmidhuber, J., Ciresan, D., Meier, U., Masci, J., & Graves, A. (2011). On fast deep nets for AGI vision. In
*Proceedings of Fourth Conference on Artificial General Intelligence*(AGI), Google, Mountain View, CA.Google Scholar - Schmidhuber, J., Eldracher, M., & Foltin, B. (1996). Semilinear predictability minimization produces well-known feature detectors.
*Neural Computation,**8*(4), 773–786.CrossRefGoogle Scholar - Schmidhuber, J., Wierstra, D., Gagliolo, M., & Gomez, F. J. (2007). Training recurrent networks by EVOLINO.
*Neural Computation,**19*(3), 757–779.MATHCrossRefGoogle Scholar - Schmidhuber, J., Zhao, J., & Schraudolph, N. (1997). Reinforcement learning with self-modifying policies. In S. Thrun& L. Pratt (Eds.),
*Learning to learn*(pp. 293–309). NY: Kluwer.Google Scholar - Schraudolph, N. N., Eldracher, M., & Schmidhuber, J. (1999). Processing images by semi-linear predictability minimization.
*Network: Computation in Neural Systems,*10(2), 133–169.Google Scholar - Schwefel, H. P. (1974). Numerische optimierung von computer-modellen. Dissertation, Published 1977 by Birkhäuser, Basel.Google Scholar
- Siegelmann, H. T., & Sontag, E. D. (1991). Turing computability with neural nets.
*Applied Mathematics Letters,**4*(6), 77–80.MathSciNetMATHCrossRefGoogle Scholar - Sims, K. (1994). Evolving virtual creatures. In A. Glassner (Ed.), Proceedings of SIGGRAPH ’94 (Orlando, Florida, July 1994), Computer Graphics Proceedings, Annual Conference (pp. 15–22). ACM SIGGRAPH, ACM Press. ISBN 0-89791-667-0.Google Scholar
- Singh, S., Barto, A. G., & Chentanez, N. (2005). Intrinsically motivated reinforcement learning. In
*Advances in Neural Information Processing Systems*17 (NIPS). Cambridge: MIT Press.Google Scholar - Sloman, A. (2011a, Oct 23). Challenge for vision: Seeing a Toy Crane. Retrieved June 8, 2012, from http://www.cs.bham.ac.uk/research/projects/cosy/photos/crane/
- Sloman, A. (2011b, June 8). Meta-morphogenesis and the creativity of evolution. Retrieved 6 June 2012, from http://www.cs.bham.ac.uk/research/projects/cogaff/evo-creativity.pdf
- Sloman, A. (2011c, Oct 29). Meta-Morphogenesis and Toddler Theorems: Case Studies. Retrieved 8 June 2012, from http://www.cs.bham.ac.uk/research/projects/cogaff/misc/toddler-theorems.html
- Sloman, A. (2011d, Sep 19). Simplicity and Ontologies: The trade-off between simplicity of theories and sophistication of ontologies. Retrieved June 8, 2012, from http://www.cs.bham.ac.uk/research/projects/cogaff/misc/simplicity-ontology.html
- Solomonoff, R. J. (1964). A formal theory of inductive inference.
*Part I. Information and Control,**7*, 1–22.MathSciNetMATHCrossRefGoogle Scholar - Stanley, K. O., & Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies.
*Evolutionary Computation,**10*, 99–127.CrossRefGoogle Scholar - Storck, J., Hochreiter, S., & Schmidhuber, J. (1995). Reinforcement driven information acquisition in non-deterministic environments. In
*Proceedings of the International Conference on Artificial Neural Networks*, Paris, vol. 2, pp. 159–164. EC2& Cie, 1995.Google Scholar - Strehl, A., Langford, J., & Kakade, S. (2010). Learning from logged implicit exploration data. Technical, Report arXiv:1003.0120.Google Scholar
- Sun, Y., Wierstra, D., Schaul, T., & Schmidhuber, J. (2009a).
*Efficient natural evolution strategies*. In Genetic and Evolutionary Computation Conference.Google Scholar - Sun, Y., Wierstra, D., Schaul, T., & Schmidhuber, J. (2009b). Stochastic search using the natural gradient. In
*International Conference on Machine Learning (ICML)*.Google Scholar - Sutskever, I., Martens, J., & Hinton, G. (2011). Generating text with recurrent neural networks. In L. Getoor& T. Scheffer (Eds.),
*Proceedings of the 28th International Conference on Machine Learning (ICML-11)*(pp. 1017–1024). ICML ’11 New York, NY, USA: ACM.Google Scholar - Sutton, R., & Barto, A. (1998).
*Reinforcement learning: An introduction*. Cambridge: MIT Press.Google Scholar - Turing, A. M. (1936). On computable numbers, with an application to the Entscheidungsproblem.
*Proceedings of the London Mathematical Society, Series,**2*(41), 230–267.MathSciNetGoogle Scholar - Utgoff, P. (1986). Shift of bias for inductive concept learning. In R. Michalski, J. Carbonell,& T. Mitchell (Eds.),
*Machine learning*(Vol. 2, pp. 163–190). Los Altos, CA: Morgan Kaufmann.Google Scholar - Vapnik, V. (1995).
*The nature of statistical learning theory*. New York: Springer.MATHCrossRefGoogle Scholar - Vinge, V. (1984).
*The peace war*. Inc. : Bluejay Books.Google Scholar - Vinge, V. (1993). The coming technological singularity. VISION-21 Symposium sponsored by NASA Lewis Research Center, and Whole Earth Review, Winter issue.Google Scholar
- Werbos, P. J. (1988). Generalization of backpropagation with application to a recurrent gas market model. Neural Networks, 1.Google Scholar
- Wierstra, D., Foerster, A., Peters, J., & Schmidhuber, J. (2010). Recurrent policy gradients.
*Logic Journal of IGPL*,18(2), 620–634.Google Scholar - Wierstra, D., Schaul, T., Peters, J., & Schmidhuber, J. (2008). Natural evolution strategies. In
*Congress of Evolutionary Computation (CEC 2008)*.Google Scholar - Williams R. J., & Zipser, D. (1994). Gradient-based learning algorithms for recurrent networks and their computational complexity. In
*back-propagation: Theory, architectures and applications*. Hillsdale, NJ: Erlbaum.Google Scholar - Yao, X. (1993). A review of evolutionary artificial neural networks.
*International Journal of Intelligent Systems,**4*, 203–222.Google Scholar - Yi, S., Gomez, F., & Schmidhuber, J. (2011). Planning to be surprised: Optimal Bayesian exploration in dynamic environments. In
*Proceedings of Fourth Conference on Artificial General Intelligence (AGI)*, Google, Mountain View, CA.Google Scholar