Deep Learning of Representations: Looking Forward

  • Yoshua Bengio
Conference paper

DOI: 10.1007/978-3-642-39593-2_1

Part of the Lecture Notes in Computer Science book series (LNCS, volume 7978)
Cite this paper as:
Bengio Y. (2013) Deep Learning of Representations: Looking Forward. In: Dediu AH., Martín-Vide C., Mitkov R., Truthe B. (eds) Statistical Language and Speech Processing. SLSP 2013. Lecture Notes in Computer Science, vol 7978. Springer, Berlin, Heidelberg

Abstract

Deep learning research aims at discovering learning algorithms that discover multiple levels of distributed representations, with higher levels representing more abstract concepts. Although the study of deep learning has already led to impressive theoretical results, learning algorithms and breakthrough experiments, several challenges lie ahead. This paper proposes to examine some of these challenges, centering on the questions of scaling deep learning algorithms to much larger models and datasets, reducing optimization difficulties due to ill-conditioning or local minima, designing more efficient and powerful inference and sampling procedures, and learning to disentangle the factors of variation underlying the observed data. It also proposes a few forward-looking research directions aimed at overcoming these challenges.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Yoshua Bengio
    • 1
  1. 1.Department of Computer Science and Operations ResearchUniversité de MontréalCanada

Personalised recommendations