Constrained Semi-Supervised Learning Using Attributes and Comparative Attributes

  • Abhinav Shrivastava
  • Saurabh Singh
  • Abhinav Gupta
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7574)


We consider the problem of semi-supervised bootstrap learning for scene categorization. Existing semi-supervised approaches are typically unreliable and face semantic drift because the learning task is under-constrained. This is primarily because they ignore the strong interactions that often exist between scene categories, such as the common attributes shared across categories as well as the attributes which make one scene different from another. The goal of this paper is to exploit these relationships and constrain the semi-supervised learning problem. For example, the knowledge that an image is an auditorium can improve labeling of amphitheaters by enforcing constraint that an amphitheater image should have more circular structures than an auditorium image. We propose constraints based on mutual exclusion, binary attributes and comparative attributes and show that they help us to constrain the learning problem and avoid semantic drift. We demonstrate the effectiveness of our approach through extensive experiments, including results on a very large dataset of one million images.


Unlabeled Data Mutual Exclusion Comparative Attribute Pairwise Constraint Scene Category 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: SIGCHI Conference on Human Factors in Computing Systems (2004)Google Scholar
  2. 2.
    Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: A database and web-based tool for image annotation. IJCV (2009)Google Scholar
  3. 3.
    Torralba, A., Fergus, R., Freeman, W.T.: 80 million tiny images: a large database for non-parametric object and scene recognition. PAMI (2008)Google Scholar
  4. 4.
    Curran, J.R., Murphy, T., Scholz, B.: Minimising semantic drift with mutual exclusion bootstrapping. In: Conference of the Pacific Association for Computational Linguistics (2007)Google Scholar
  5. 5.
    Carlson, A., Betteridge, J., Hruschka Jr., E.R., Mitchell, T.M.: Coupling semi-supervised learning of categories and relations. In: NAACL HLT Workskop on SSL for NLP (2009)Google Scholar
  6. 6.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: CVPR (2009)Google Scholar
  7. 7.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: CVPR (2009)Google Scholar
  8. 8.
    Parikh, D., Grauman, K.: Relative attributes. In: ICCV (November 2011)Google Scholar
  9. 9.
    Gupta, A., Davis, L.S.: Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 16–29. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Rastegari, M., Farhadi, A., Forsyth, D.: Attribute Discovery via Predictable Discriminative Binary Codes. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 876–889. Springer, Heidelberg (2012)Google Scholar
  11. 11.
    Parkash, A., Parikh, D.: Attributes for Classifier Feedback. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 355–369. Springer, Heidelberg (2012)Google Scholar
  12. 12.
    Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka Jr., E.R., Mitchell, T.M.: Toward an architecture for never-ending language learning. In: AAAI (2010)Google Scholar
  13. 13.
    Vijayanarasimhan, S., Grauman, K.: Large-scale live active learning: Training object detectors with crawled data and crowds. In: CVPR (2011)Google Scholar
  14. 14.
    Kapoor, A., Grauman, K., Urtasun, R., Darrell, T.: Active learning with gaussian processes for object categorization. In: ICCV (2007)Google Scholar
  15. 15.
    Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Zhang, H.J.: Two-dimensional active learning for image classification. In: CVPR (2008)Google Scholar
  16. 16.
    Joshi, A., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: CVPR (2009)Google Scholar
  17. 17.
    Siddiquie, B., Gupta, A.: Beyond active noun tagging: Modeling contextual interactions for multi-class active learning. In: CVPR (2010)Google Scholar
  18. 18.
    Russell, B., Freeman, W., Efros, A., Sivic, J., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: CVPR (2006)Google Scholar
  19. 19.
    Kang, H., Hebert, M., Kanade, T.: Discovering object instances from scenes of daily living. In: ICCV (2011)Google Scholar
  20. 20.
    Zhu, X.: Semi-supervised learning literature survey. Technical report, CS, UW-Madison (2005)Google Scholar
  21. 21.
    Riloff, E., Jones, R.: Learning dictionaries for information extraction by multi-level bootstrapping. In: AAAI (1999)Google Scholar
  22. 22.
    Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT (1998)Google Scholar
  23. 23.
    Ebert, S., Larlus, D., Schiele, B.: Extracting Structures in Image Collections for Object Recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 720–733. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  24. 24.
    Fergus, R., Weiss, Y., Torralba, A.: Semi-supervised learning in gigantic image collections. In: NIPS (2009)Google Scholar
  25. 25.
    Guillaumin, M., Verbeek, J., Schmid, C.: Multimodal semi-supervised learning for image classification. In: CVPR (2010)Google Scholar
  26. 26.
    Tighe, J., Lazebnik, S.: Understanding scenes on many levels. In: ICCV (2011)Google Scholar
  27. 27.
    Hoiem, D., Efros, A.A., Hebert, M.: Recovering surface layout from an image. IJCV (2007)Google Scholar
  28. 28.
    Hays, J., Efros, A.A.: Scene completion using millions of photographs. ACM Transactions on Graphics, SIGGRAPH (2007)Google Scholar
  29. 29.
    Oliva, A., Torralba, A.: Building the gist of a scene: the role of global image features in recognition. Progress in Brain Research (2006)Google Scholar
  30. 30.
    Xiao, J., Hays, J., Ehinger, K., Oliva, A., Torralba, A.: Sun database: Large scale scene recognition from abbey to zoo. In: CVPR (2010)Google Scholar
  31. 31.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: CVPR (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Abhinav Shrivastava
    • 1
  • Saurabh Singh
    • 1
  • Abhinav Gupta
    • 1
  1. 1.The Robotics InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations