Unsupervised Classification and Part Localization by Consistency Amplification

  • Leonid Karlinsky
  • Michael Dinerstein
  • Dan Levi
  • Shimon Ullman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5303)


We present a novel method for unsupervised classification, including the discovery of a new category and precise object and part localization. Given a set of unlabelled images, some of which contain an object of an unknown category, with unknown location and unknown size relative to the background, the method automatically identifies the images that contain the objects, localizes them and their parts, and reliably learns their appearance and geometry for subsequent classification. Current unsupervised methods construct classifiers based on a fixed set of initial features. Instead, we propose a new approach which iteratively extracts new features and re-learns the induced classifier, improving class vs. non-class separation at each iteration. We develop two main tools that allow this iterative combined search. The first is a novel star-like model capable of learning a geometric class representation in the unsupervised setting. The second is learning of ”part specific features” that are optimized for parts detection, and which optimally combine different part appearances discovered in the training examples. These novel aspects lead to precise part localization and to improvement in overall classification performance compared with previous methods. We applied our method to multiple object classes from Caltech-101, UIUC and a sub-classification problem from PASCAL. The obtained results are comparable to state-of-the-art supervised classification techniques and superior to state-of-the-art unsupervised approaches previously applied to the same image sets.


Class Image Part Localization Object Part Part Detector Detection Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-540-88688-4_24_MOESM1_ESM.avi (6.2 mb)
Supplementary material (6,389 KB)


  1. 1.
    Fritz, M., Schiele, B.: Towards unsupervised discovery of visual categories. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 232–241. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  2. 2.
    Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering objects and their localization in images. In: ICCV, pp. 370–377 (2005)Google Scholar
  3. 3.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. CVPR (2), 264–271 (2003)Google Scholar
  4. 4.
    Fergus, R., Perona, P., Zisserman, A.: A visual category filter for google images. In: ECCV (2004)Google Scholar
  5. 5.
    Fergus, R., Fei-Fei, L., Perona, P., Zisserman, A.: Learning object categories from google’s image search. In: ICCV, pp. 1816–1823 (2005)Google Scholar
  6. 6.
    Russell, B.C., Freeman, W.T., Efros, A.A., Sivic, J., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. CVPR (2006)Google Scholar
  7. 7.
    Cao, L., Fei-Fei, L.: Spatially coherent latent topic model for concurrent object segmentation and classification. In: ICCV (2007)Google Scholar
  8. 8.
    Nowozin, S., Tsuda, K., Uno, T., Kudo, T., Bakir, G.H.: Weighted substructure mining for image analysis. CVPR (2007)Google Scholar
  9. 9.
    Li, L.J., Wang, G., Fei-Fei, L.: Optimol: automatic object picture collection via incremental model learning. CVPR (2007)Google Scholar
  10. 10.
    Ahuja, N., Todorovic, S.: Discovering hierarchical taxonomy of categories and shared subcategories in images. In: ICCV (2007)Google Scholar
  11. 11.
    Liu, D., Chen, T.: Semantic-shift for unsupervised object detection. In: CVPR Workshop (2006)Google Scholar
  12. 12.
    Liu, D., Chen, T.: Unsupervised image categorization and object localization using topic models and correspondences between images. In: ICCV (2007)Google Scholar
  13. 13.
    Leibe, B., Leonardis, A., Schiele, B.: Combined object categorization and segmentation with an implicit shape model. In: ECCV (2004)Google Scholar
  14. 14.
    Quack, T., Ferrari, V., Leibe, B., Gool, L.V.: Efficient mining of frequent and distinctive feature configurations. In: ICCV (2007)Google Scholar
  15. 15.
    Loeff, N., Arora, H., Sorokin, A., Forsyth, D.: Efficient unsupervised learning for localization and detection in object categories. In: NIPS (2005)Google Scholar
  16. 16.
    Sudderth, E.B., Torralba, A., Freeman, W.T., Willsky, A.S.: Learning hierarchical models of scenes, objects, and parts. In: ICCV (2005)Google Scholar
  17. 17.
    Crandall, D.J., Huttenlocher, D.P.: Weakly supervised learning of part-based spatial models for visual object recognition. In: ECCV (1), pp. 16–29 (2006)Google Scholar
  18. 18.
    Epshtein, B., Ullman, S.: Semantic hierarchies for recognizing objects and parts. CVPR (2007)Google Scholar
  19. 19.
    Jurie, F., Triggs, B.: Creating efficient codebooks for visual recognition. In: ICCV (2005)Google Scholar
  20. 20.
    Friedman, N.: The bayesian structural em algorithm. UAI, 129–138 (1998)Google Scholar
  21. 21.
    Dorko, G., Schmid, C.: Object class recognition using discriminative local features. INRIA (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Leonid Karlinsky
    • 1
  • Michael Dinerstein
    • 1
  • Dan Levi
    • 1
  • Shimon Ullman
    • 1
  1. 1.Weizmann Institute of ScienceRehovotIsrael

Personalised recommendations