Constructing Category Hierarchies for Visual Recognition

  • Marcin Marszałek
  • Cordelia Schmid
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5305)


Class hierarchies are commonly used to reduce the complexity of the classification problem. This is crucial when dealing with a large number of categories. In this work, we evaluate class hierarchies currently constructed for visual recognition. We show that top-down as well as bottom-up approaches, which are commonly used to automatically construct hierarchies, incorporate assumptions about the separability of classes. Those assumptions do not hold for visual recognition of a large number of object categories. We therefore propose a modification which is appropriate for most top-down approaches. It allows to construct class hierarchies that postpone decisions in the presence of uncertainty and thus provide higher recognition accuracy. We also compare our method to a one-against-all approach and show how to control the speed-for-accuracy trade-off with our method. For the experimental evaluation, we use the Caltech-256 visual object classes dataset and compare to state-of-the-art methods.


Support Vector Machine Image Representation Object Category Visual Recognition Class Hierarchy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Everingham, M., van Gool, L., Williams, C., Winn, J., Zisserman, A.: Overview and results of classification challenge. In: The PASCAL VOC 2007 Challenge Workshop, in conj. with ICCV (2007)Google Scholar
  2. 2.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. IJCV (2004)Google Scholar
  3. 3.
    Lindeberg, T.: Feature detection with automatic scale selection. IJCV (1998)Google Scholar
  4. 4.
    Lowe, D.: Distinctive image features form scale-invariant keypoints. IJCV (2004)Google Scholar
  5. 5.
    Willamowski, J., Arregui, D., Csurka, G., Dance, C.R., Fan, L.: Categorizing nine visual classes using local appearance descriptors. In: IWLAVS (2004)Google Scholar
  6. 6.
    Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond (2002)Google Scholar
  7. 7.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar
  8. 8.
    Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: A comprehensive study. IJCV (2007)Google Scholar
  9. 9.
    Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. PAMI (2007)Google Scholar
  10. 10.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. Technical report (2007)Google Scholar
  11. 11.
    Chen, Y., Crawford, M., Ghosh, J.: Integrating support vector machines in a hierarchical output space decomposition framework. In: IGARSS (2004)Google Scholar
  12. 12.
    Liu, S., Yi, H., Chia, L.T., Deepu, R.: Adaptive hierarchical multi-class SVM classifier for texture-based image classification. In: ICME (2005)Google Scholar
  13. 13.
    Zhigang, L., Wenzhong, S., Qianqing, Q., Xiaowen, L., Donghui, X.: Hierarchical support vector machines. In: IGARSS (2005)Google Scholar
  14. 14.
    Yuan, X., Lai, W., Mei, T., Hua, X., Wu, X., Li, S.: Automatic video genre categorization using hierarchical SVM. In: ICIP (2006)Google Scholar
  15. 15.
    Zweig, A., Weinshall, D.: Exploiting object hierarchy: Combining models from different category levels. In: ICCV (2007)Google Scholar
  16. 16.
    He, X., Zemel, R.: Latent topic random fields: Learning using a taxonomy of labels. In: CVPR (2008)Google Scholar
  17. 17.
    Griffin, G., Perona, P.: Learning and using taxonomies for fast visual category recognition. In: CVPR (2008)Google Scholar
  18. 18.
    Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: CVPR (2006)Google Scholar
  19. 19.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: CVPR (2007)Google Scholar
  20. 20.
    Casasent, D., Wang, Y.C.: A hierarchical classifier using new support vector machines for automatic target recognition. Neural Networks (2005)Google Scholar
  21. 21.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. PAMI (2000)Google Scholar
  22. 22.
    Rahimi, A., Recht, B.: Clustering with normalized cuts is clustering with a hyperplane. In: SLCV (2004)Google Scholar
  23. 23.
    Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the Nyström method. PAMI (2004)Google Scholar
  24. 24.
    Everingham, M., Zisserman, A., Williams, C., van Gool, L.: The PASCAL visual object classes challenge 2006 (VOC 2006) results. Technical report (2006)Google Scholar
  25. 25.
    Marszałek, M., Schmid, C.: Semantic hierarchies for visual object recognition. In: CVPR (2007)Google Scholar
  26. 26.
    Perona, P., Griffin, G., Spain, M.: The Caltech 256 Workshop. In: Conj. with ICCV (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Marcin Marszałek
    • 1
  • Cordelia Schmid
    • 1
  1. 1.INRIA Grenoble, LEAR, LJK 

Personalised recommendations