On Fast Deep Nets for AGI Vision

  • Jurgen Schmidhuber
  • Dan Cireşan
  • Ueli Meier
  • Jonathan Masci
  • Alex Graves
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6830)


Artificial General Intelligence will not be general without computer vision. Biologically inspired adaptive vision models have started to outperform traditional pre-programmed methods: our fast deep / recurrent neural networks recently collected a string of 1st ranks in many important visual pattern recognition benchmarks: IJCNN traffic sign competition, NORB, CIFAR10, MNIST, three ICDAR handwriting competitions. We greatly profit from recent advances in computing hardware, complementing recent progress in the AGI theory of mathematically optimal universal problem solvers.


AGI Fast Deep Neural Nets Computer Vision Hardware Advances vs Theoretical Progress 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Behnke, S.: Hierarchical Neural Networks for Image Interpretation. LNCS, vol. 2766. Springer, Heidelberg (2003)zbMATHCrossRefGoogle Scholar
  2. 2.
    Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Deep big simple neural nets for handwritten digit recogntion. Neural Computation 22(12), 3207–3220 (2010)CrossRefGoogle Scholar
  3. 3.
    Ciresan, D.C., Meier, U., Masci, J., Gambardella, L.M., Schmidhuber, J.: High-performance neural networks for visual object classification. arxiv 1102.0183 (2011)Google Scholar
  4. 4.
    Ciresan, D.C., Meier, U., Masci, J., Schmidhuber, J.: A committee of neural networks for traffic sign classification. In: International Joint Conference on Neural Networks to appear (2011)Google Scholar
  5. 5.
    Fukushima, K.: Neocognitron: A self-organizing neural network for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 36(4), 193–202 (1980)zbMATHCrossRefGoogle Scholar
  6. 6.
    Fukushima, K.: Neocognitron for handwritten digit recognition. Neurocomputing 51, 161–180 (2003)CrossRefGoogle Scholar
  7. 7.
    Graves, A., Fernández, S., Schmidhuber, J.: Multi-dimensional recurrent neural networks. In: Proceedings of the 17th International Conference on Artificial Neural Networks (September 2007)Google Scholar
  8. 8.
    Graves, A., Schmidhuber, J.: Offline handwriting recognition with multidimensional recurrent neural networks. In: Advances in Neural Information Processing Systems, vol. 21, MIT Press, Cambridge (2009)Google Scholar
  9. 9.
    Hochreiter, S., Schmidhuber, J.: Flat minima. Neural Computation 9(1), 1–42 (1997)zbMATHCrossRefGoogle Scholar
  10. 10.
    Hutter, M.: Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability. Springer, Berlin (2004); (On J. Schmidhuber’s SNF grant 20-61847)Google Scholar
  11. 11.
    Krizhevsky, A.: Learning multiple layers of features from tiny images. Master’s thesis, Computer Science Department, University of Toronto (2009)Google Scholar
  12. 12.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  13. 13.
    LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. In: Proc. of Computer Vision and Pattern Recognition Conference (2004)Google Scholar
  14. 14.
    Mitchell, T.: Machine Learning. McGraw Hill, New York (1997)zbMATHGoogle Scholar
  15. 15.
    Newell, A., Simon, H.: GPS, a program that simulates human thought. In: Feigenbaum, E., Feldman, J. (eds.) Computers and Thought, pp. 279–293. McGraw-Hill, New York (1963)Google Scholar
  16. 16.
    Rosenbloom, P.S., Laird, J.E., Newell, A.: The SOAR Papers. MIT Press, Cambridge (1993)Google Scholar
  17. 17.
    Scherer, D., Müller, A., Behnke, S.: Evaluation of pooling operations in convolutional architectures for object recognition. In: International Conference on Artificial Neural Networks (2010)Google Scholar
  18. 18.
    Schmidhuber, J.: Learning factorial codes by predictability minimization. Neural Computation 4(6), 863–879 (1992)CrossRefGoogle Scholar
  19. 19.
    Schmidhuber, J.: The new AI: General & sound & relevant for physics. In: Goertzel, B., Pennachin, C. (eds.) Artificial General Intelligence, pp. 175–198. Springer, Heidelberg (2006); also available as TR IDSIA-04-03, arXiv:cs.AI/0302012Google Scholar
  20. 20.
    Schmidhuber, J.: New millennium AI and the convergence of history. In: Duch, W., Mandziuk, J. (eds.) Challenges to Computational Intelligence. Studies in Computational Intelligence, vol. 63, pp. 15–36. Springer, Heidelberg (2007); arXiv:cs.AI/0606081CrossRefGoogle Scholar
  21. 21.
    Schmidhuber, J.: Ultimate cognition à la Gödel. Cognitive Computation 1(2), 177–193 (2009)CrossRefGoogle Scholar
  22. 22.
    Schmidhuber, J., Eldracher, M., Foltin, B.: Semilinear predictability minimization produces well-known feature detectors. Neural Computation 8(4), 773–786 (1996)CrossRefGoogle Scholar
  23. 23.
    Schmidhuber, J., Huber, R.: Learning to generate artificial fovea trajectories for target detection. International Journal of Neural Systems 2(1 & 2), 135–141 (1991)CrossRefGoogle Scholar
  24. 24.
    Schraudolph, N.N., Eldracher, M., Schmidhuber, J.: Processing images by semi-linear predictability minimization. Network: Computation in Neural Systems 10(2), 133–169 (1999)CrossRefGoogle Scholar
  25. 25.
    Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: Seventh International Conference on Document Analysis and Recognition, pp. 958–963 (2003)Google Scholar
  26. 26.
    Utgoff, P.: Shift of bias for inductive concept learning. In: Michalski, R., Carbonell, J., Mitchell, T. (eds.) Machine Learning, vol. 2, pp. 163–190. Morgan Kaufmann, Los Altos (1986)Google Scholar
  27. 27.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jurgen Schmidhuber
    • 1
  • Dan Cireşan
    • 1
  • Ueli Meier
    • 1
  • Jonathan Masci
    • 1
  • Alex Graves
    • 1
  1. 1.The Swiss AI Lab IDSIAUniversity of Lugano & SUPSISwitzerland

Personalised recommendations