Advertisement

Challenges in Representation Learning: A Report on Three Machine Learning Contests

  • Ian J. Goodfellow
  • Dumitru Erhan
  • Pierre Luc Carrier
  • Aaron Courville
  • Mehdi Mirza
  • Ben Hamner
  • Will Cukierski
  • Yichuan Tang
  • David Thaler
  • Dong-Hyun Lee
  • Yingbo Zhou
  • Chetan Ramaiah
  • Fangxiang Feng
  • Ruifan Li
  • Xiaojie Wang
  • Dimitris Athanasakis
  • John Shawe-Taylor
  • Maxim Milakov
  • John Park
  • Radu Ionescu
  • Marius Popescu
  • Cristian Grozea
  • James Bergstra
  • Jingjing Xie
  • Lukasz Romaszko
  • Bing Xu
  • Zhang Chuang
  • Yoshua Bengio
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8228)

Abstract

The ICML 2013 Workshop on Challenges in Representation Learning focused on three challenges: the black box learning challenge, the facial expression recognition challenge, and the multimodal learning challenge. We describe the datasets created for these challenges and summarize the results of the competitions. We provide suggestions for organizers of future challenges and some comments on what kind of knowledge can be gained from machine learning competitions.

Keywords

representation learning competition dataset 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. Technical Report arXiv:1206.5538, U. Montreal (2012), http://arxiv.org/abs/1206.5538
  2. 2.
    Goodfellow, I.J., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., Bengio, Y.: Pylearn2: A machine learning research library. arXiv preprint arXiv:1308.421 (2013)Google Scholar
  3. 3.
    Guyon, I., Dror, G., Lemaire, V., Taylor, G., Aha, D.W.: Unsupervised and transfer learning challenge. In: Proc. Int. Joint Conf. on Neural Networks (2011)Google Scholar
  4. 4.
    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: Deep Learning and Unsupervised Feature Learning Workshop, NIPS (2011)Google Scholar
  5. 5.
    Ngiam, J., Koh, P.W.W., Chen, Z., Bhaskar, S.A., Ng, A.Y.: Sparse filtering. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 24, pp. 1125–1133 (2011)Google Scholar
  6. 6.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  7. 7.
    Cortes, C., Vapnik, V.: Support vector networks. Machine Learning 20, 273–297 (1995)zbMATHGoogle Scholar
  8. 8.
    Romaszko, L.: A deep learning approach with an ensemble-based neural network classifier for black box icml 2013 contest. In: Workshop on Challenges in Representation Learning, ICML (2013)Google Scholar
  9. 9.
    Lee, D.-H.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML (2013)Google Scholar
  10. 10.
    Grandvalet, Y., Bengio, Y.: Semi-supervised Learning by Entropy Minimization. In: NIPS 2004. MIT Press, Cambridge (2005)Google Scholar
  11. 11.
    Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.-A.: Extracting and composing robust features with denoising autoencoders. In: ICML 2008 (2008)Google Scholar
  12. 12.
    Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. In: ICML (2013)Google Scholar
  13. 13.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)Google Scholar
  14. 14.
    Susskind, J., Anderson, A., Hinton, G.E.: The Toronto face dataset. Technical Report UTML TR 2010-001, U. Toronto (2010)Google Scholar
  15. 15.
    Bergstra, J., Cox, D.D.: Hyperparameter optimization and boosting for classifying facial expressions: How good can a “null” model be? In: Workshop on Challenges in Representation Learning, ICML (2013)Google Scholar
  16. 16.
    Fukushima, K.: Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 36(4), 193–202 (1980)CrossRefzbMATHGoogle Scholar
  17. 17.
    Lowe, D.: Object recognition from local scale invariant features. In: ICCV 1999 (1999)Google Scholar
  18. 18.
    Tang, Y.: Deep learning using linear support vector machines. In: Workshop on Challenges in Representation Learning, ICML (2013)Google Scholar
  19. 19.
    Ionescu, R.T., Popescu, M., Grozea, C.: Local learning to improve bag of visual words model for facial expression recognition. In: Workshop on Challenges in Representation Learning, ICML (2013)Google Scholar
  20. 20.
    von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2004, pp. 319–326. ACM, New York (2004)CrossRefGoogle Scholar
  21. 21.
    Feng, F., Li, R., Wang, X.: Constructing hierarchical image-tags bimodal representations for word tags alternative choice. In: Workshop on Challenges in Representation Learning, ICML (2013)Google Scholar
  22. 22.
    Le, Q.V., Ranzato, M., Salakhutdinov, R., Ng, A., Tenenbaum, J.: NIPS Workshop on Challenges in Learning Hierarchical Models: Transfer Learning and Optimization (2011), https://sites.google.com/site/nips2011workshop
  23. 23.
    Goodfellow, I., Courville, A., Bengio, Y.: Large-scale feature learning with spike-and-slab sparse coding. In: ICML (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Ian J. Goodfellow
    • 1
  • Dumitru Erhan
    • 2
  • Pierre Luc Carrier
    • 1
  • Aaron Courville
    • 1
  • Mehdi Mirza
    • 1
  • Ben Hamner
    • 1
  • Will Cukierski
    • 1
  • Yichuan Tang
    • 1
  • David Thaler
    • 1
  • Dong-Hyun Lee
    • 1
  • Yingbo Zhou
    • 1
  • Chetan Ramaiah
    • 1
  • Fangxiang Feng
    • 1
  • Ruifan Li
    • 1
  • Xiaojie Wang
    • 1
  • Dimitris Athanasakis
    • 1
  • John Shawe-Taylor
    • 1
  • Maxim Milakov
    • 1
  • John Park
    • 1
  • Radu Ionescu
    • 1
  • Marius Popescu
    • 1
  • Cristian Grozea
    • 1
  • James Bergstra
    • 1
  • Jingjing Xie
    • 1
  • Lukasz Romaszko
    • 1
  • Bing Xu
    • 1
  • Zhang Chuang
    • 1
  • Yoshua Bengio
    • 1
  1. 1.Université de MontréalMontréalCanada
  2. 2.GoogleVeniceUSA

Personalised recommendations