Attribute-Based Transfer Learning for Object Categorization with Zero/One Training Example

  • Xiaodong Yu
  • Yiannis Aloimonos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6315)


This paper studies the one-shot and zero-shot learning problems, where each object category has only one training example or has no training example at all. We approach this problem by transferring knowledge from known categories (a.k.a source categories) to new categories (a.k.a target categories) via object attributes. Object attributes are high level descriptions of object categories, such as color, texture, shape, etc. Since they represent common properties across different categories, they can be used to transfer knowledge from source categories to target categories effectively. Based on this insight, we propose an attribute-based transfer learning framework in this paper. We first build a generative attribute model to learn the probabilistic distributions of image features for each attribute, which we consider as attribute priors. These attribute priors can be used to (1) classify unseen images of target categories (zero-shot learning), or (2) facilitate learning classifiers for target categories when there is only one training examples per target category (one-shot learning). We demonstrate the effectiveness of the proposed approaches using the Animal with Attributes data set and show state-of-the-art performance in both zero-shot and one-shot learning tests.


Knowledge Transfer Visual Word Gibbs Sampling Object Attribute Attribute Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Fei-Fei, L., Fergus, R., Perona, P.: One-Shot Learning of Object Categories. PAMI 28, 594–611 (2006)Google Scholar
  2. 2.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning To Detect Unseen Object Classes by Between-Class Attribute Transfer. In: CVPR (2009)Google Scholar
  3. 3.
    Fei-Fei, L.: Knowledge Transfer in Learning to Recognize Visual Object Classes. In: International Conference on Development and Learning (2006)Google Scholar
  4. 4.
    Bart, E., Ullman, S.: Cross-Generalization: Learning Novel Classes from a Single Example by Feature Replacement. In: CVPR, pp. 672–679 (2005)Google Scholar
  5. 5.
    Torralba, A., Murphy, K.P., Freeman, W.T.: Sharing Features: Efficient Boosting Procedures for Multiclass Object Detection. In: CVPR, vol. 2, pp. 762–769 (2004)Google Scholar
  6. 6.
    Stark, M., Goesele, M., Schiele, B.: A Shape-Based Object Class Model for Knowledge Transfer. In: ICCV (2009)Google Scholar
  7. 7.
    Murphy, K., Torralba, A., Freeman, W.T.: Using the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes. In: NIPS (2003)Google Scholar
  8. 8.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing Objects by Their Attributes. In: CVPR (2009)Google Scholar
  9. 9.
    Wang, G., Forsyth, D.: Joint Learning of Visual Attributes, Object Classes and Visual Saliency. In: CVPR (2009)Google Scholar
  10. 10.
    Kumar, N., Belhumeur, P.N., Nayar, S.K.: FaceTracer: A Search Engine for Large Collections of Images with Faces. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 340–353. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Blei, D.M., Ng, A.Y., Jordan, M.I., Lafferty, J.: Latent Dirichlet Allocation. JMLR 3 (2003)Google Scholar
  12. 12.
    Griffiths, T.L., Steyvers, M.: Finding Scientific Topics. Proceedings of the National Academy of Sciences 101(suppl. 1), 5228–5235 (2004)CrossRefGoogle Scholar
  13. 13.
    Rosen-Zvi, M., Chemudugunta, C., Smyth, P., Steyvers, M.: Learning author-topic models from text corpora. ACM Transactions on Information System (2009)Google Scholar
  14. 14.
    Sivic, J., Russell, B., Efros, A.A., Zisserman, A., Freeman, B.: Discovering Objects and Their Location in Images. In: ICCV, pp. 370–377 (2005)Google Scholar
  15. 15.
    Russell, B.C., Efros, A.A., Sivic, J., Freeman, W.T., Zisserman, A.: Using Multiple Segmentations to Discover Objects and their Extent in Image Collections. In: CVPR (2006)Google Scholar
  16. 16.
    Sudderth, E.B., Torralba, A., Freeman, W.T., Willsky, A.S.: Learning Hierarchical Models of Scenes, Objects, and Parts. In: ICCV, vol. 2, pp. 1331–1338 (2005)Google Scholar
  17. 17.
    Fei-Fei, L., Perona, P.: A Bayesian Hierarchical Model for Learning Natural Scene Categories. In: CVPR, pp. 524–531 (2005)Google Scholar
  18. 18.
    Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual Categorization with Bags of Keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV (2004)Google Scholar
  19. 19.
    Sun, N., Haas, N., Connell, J.H., Pankanti, S.: A Model-Based Sampling and Sample Synthesis Method for Auto Identification in Computer Vision. In: IEEE Workshop on Automatic Identification Advanced Technologies, Washington, DC, USA, pp. 160–165 (2005)Google Scholar
  20. 20.
    Jiang, D., Hu, Y., Yan, S., Zhang, L., Zhang, H., Gao, W.: Efficient 3D reconstruction for face recognition. Pattern Recognition 38, 787–798 (2005)CrossRefGoogle Scholar
  21. 21.
    Lowe, D.G.: Distinctive Image Features from Scale-invariant Keypoints. IJCV 20, 91–110 (2004)CrossRefGoogle Scholar
  22. 22.
    van de Sande, K.E., Gevers, T., Snoek, C.G.: Evaluation of Color Descriptors for Object and Scene Recognition. In: CVPR (2008)Google Scholar
  23. 23.
    Shechtman, E., Irani, M.: Matching Local Self-Similarities across Images and Videos. In: CVPR, pp. 1–8 (2007)Google Scholar
  24. 24.
    Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study. IJCV 73, 213–238 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Xiaodong Yu
    • 1
  • Yiannis Aloimonos
    • 1
  1. 1.University of MarylandCollege ParkUSA

Personalised recommendations