Augmented Attribute Representations

  • Viktoriia Sharmanska
  • Novi Quadrianto
  • Christoph H. Lampert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7576)


We propose a new learning method to infer a mid-level feature representation that combines the advantage of semantic attribute representations with the higher expressive power of non-semantic features. The idea lies in augmenting an existing attribute-based representation with additional dimensions for which an autoencoder model is coupled with a large-margin principle. This construction allows a smooth transition between the zero-shot regime with no training example, the unsupervised regime with training examples but without class labels, and the supervised regime with training examples and with class labels. The resulting optimization problem can be solved efficiently, because several of the necessity steps have closed-form solutions. Through extensive experiments we show that the augmented representation achieves better results in terms of object categorization accuracy than the semantic representation alone.


Discriminative Autoencoder Hybrid Representations 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: CVPR, pp. 951–958 (2009)Google Scholar
  2. 2.
    Palatucci, M., Pomerleau, D., Hinton, G., Mitchell, T.: Zero-shot learning with semantic output codes. In: NIPS, pp. 1410–1418 (2009)Google Scholar
  3. 3.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.A.: Describing objects by their attributes. In: CVPR, pp. 1778–1785 (2009)Google Scholar
  4. 4.
    Kulkarni, G., Premraj, V., Dhar, S., Li, S., Choi, Y., Berg, A., Berg, T.: Baby talk: Understanding and generating simple image descriptions. In: CVPR, pp. 1601–1608 (2011)Google Scholar
  5. 5.
    Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. JMLR 10, 207–244 (2009)zbMATHGoogle Scholar
  7. 7.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. JMLR 11, 3371–3408 (2010)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Quadrianto, N., Lampert, C.H.: Learning multi-view neighborhood preserving projections. In: ICML, pp. 425–432 (2011)Google Scholar
  9. 9.
    Ferrari, V., Zisserman, A.: Learning visual attributes. In: NIPS, pp. 433–440 (2008)Google Scholar
  10. 10.
    Wang, Y., Mori, G.: A Discriminative Latent Model of Object Classes and Attributes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 155–168. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Mahajan, D., Sellamanickam, S., Nair, V.: A joint learning framework for attribute models and object descriptions. In: ICCV, pp. 1227–1234 (2011)Google Scholar
  12. 12.
    Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. PAMI 28, 594–611 (2006)CrossRefGoogle Scholar
  13. 13.
    Wolf, L., Hassner, T., Taigman, Y.: Descriptor based methods in the wild. In: ECCV Workshop on Faces in Real Life Images (2008)Google Scholar
  14. 14.
    Tommasi, T., Orabona, F., Caputo, B.: Safety in numbers: Learning categories from few examples with multi model knowledge transfer. In: CVPR, pp. 3081–3088 (2010)Google Scholar
  15. 15.
    Rohrbach, M., Stark, M., Szarvas, G., Gurevych, I., Schiele, B.: What helps where - and why? semantic relatedness for knowledge transfer. In: CVPR, pp. 910–917 (2010)Google Scholar
  16. 16.
    Berg, T., Berg, A., Shih, J.: Automatic Attribute Discovery and Characterization from Noisy Web Data. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 663–676. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Parikh, D., Grauman, K.: Interactively building a discriminative vocabulary of nameable attributes. In: CVPR, pp. 1681–1688 (2011)Google Scholar
  18. 18.
    Welling, M., Rosen-Zvi, M., Hinton, G.: Exponential family harmoniums with an application to information retrieval. In: NIPS (2005)Google Scholar
  19. 19.
    Ranzato, M., Huang, F., Boureau, Y., LeCun, Y.: Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: CVPR, pp. 1–8 (2007)Google Scholar
  20. 20.
    Gregor, K., LeCun, Y.: Emergence of complex-like cells in a temporal product network with local receptive fields. CoRR abs/1006.0448 (2010)Google Scholar
  21. 21.
    Hinton, G., Krizhevsky, A., Wang, S.: Transforming auto-encoders. In: ICANN, pp. 44–51 (2011)Google Scholar
  22. 22.
    Salakhutdinov, R., Hinton, G.: Learning a nonlinear embedding by preserving class neighbourhood structure. In: AISTATS (2007)Google Scholar
  23. 23.
    Tang, K., Tappen, M., Sukthankar, R., Lampert, C.: Optimizing one-shot recognition with micro-set learning. In: CVPR, pp. 3027–3034 (2010)Google Scholar
  24. 24.
    Osherson, D.N., Stern, J., Wilkie, O., Stob, M., Smith, E.E.: Default probability. Cognitive Science 15, 251–269 (1991)CrossRefGoogle Scholar
  25. 25.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). In: CVIU, pp. 346–359 (2008)Google Scholar
  26. 26.
    Ebert, S., Larlus, D., Schiele, B.: Extracting Structures in Image Collections for Object Recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 720–733. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Viktoriia Sharmanska
    • 1
  • Novi Quadrianto
    • 2
  • Christoph H. Lampert
    • 1
  1. 1.IST Austria (Institute of Science and Technology Austria)KlosterneuburgAustria
  2. 2.University of CambridgeCambridgeUK

Personalised recommendations