Taxonomic Multi-class Prediction and Person Layout Using Efficient Structured Ranking

  • Arpit Mittal
  • Matthew B. Blaschko
  • Andrew Zisserman
  • Philip H. S. Torr
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


In computer vision efficient multi-class classification is becoming a key problem as the field develops and the number of object classes to be identified increases. Often objects might have some sort of structure such as a taxonomy in which the mis-classification score for object classes close by, using tree distance within the taxonomy, should be less than for those far apart. This is an example of multi-class classification in which the loss function has a special structure. Another example in vision is for the ubiquitous pictorial structure or parts based model. In this case we would like the mis-classification score to be proportional to the number of parts misclassified.

It transpires both of these are examples of structured output ranking problems. However, so far no efficient large scale algorithm for this problem has been demonstrated. In this work we propose an algorithm for structured output ranking that can be trained in a time linear in the number of samples under a mild assumption common to many computer vision problems: that the loss function can be discretized into a small number of values.

We show the feasibility of structured ranking on these two core computer vision problems and demonstrate a consistent and substantial improvement over competing techniques. Aside from this, we also achieve state-of-the art results for the PASCAL VOC human layout problem.


Ordinal Regression Layout Problem Structure Output Structure Ranking Computer Vision Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bengio, S., Weston, J., Grangier, D.: Label embedding trees for large multi-class tasks. In: NIPS (2010)Google Scholar
  2. 2.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes (VOC) challenge. IJCV (2010)Google Scholar
  3. 3.
    Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: Proc. ICML (2004)Google Scholar
  4. 4.
    Joachims, T., Finley, T., Yu, C.N.J.: Cutting-plane training of structural SVMs. Machine Learning (2009)Google Scholar
  5. 5.
    Li, Y., Huttenlocher, D.P.: Learning for stereo vision using the structured support vector machine. In: Proc. CVPR (2008)Google Scholar
  6. 6.
    Blaschko, M.B., Lampert, C.H.: Learning to Localize Objects with Structured Output Regression. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 2–15. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Szummer, M., Kohli, P., Hoiem, D.: Learning CRFs Using Graph Cuts. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 582–595. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Blaschko, M.B., Vedaldi, A., Zisserman, A.: Simultaneous object detection and ranking with weak supervision. In: NIPS (2010)Google Scholar
  9. 9.
    Rahtu, E., Kannala, J., Blaschko, M.B.: Learning a category independent object detection cascade. In: Proc. ICCV (2011)Google Scholar
  10. 10.
    Zhang, Z., Warrell, J., Torr, P.H.S.: Proposal generation for object detection using cascaded ranking SVMs. In: Proc. CVPR (2011)Google Scholar
  11. 11.
    Huang, J.C., Frey, B.J.: Structured ranking learning using cumulative distribution networks. In: NIPS (2008)Google Scholar
  12. 12.
    Herbrich, R., Graepel, T., Obermayer, K.: Large margin rank boundaries for ordinal regression. In: Advances in Large Margin Classifiers (2000)Google Scholar
  13. 13.
    Joachims, T.: Training linear SVMs in linear time. In: KDD (2006)Google Scholar
  14. 14.
    Deng, J., Berg, A.C., Li, K., Fei-Fei, L.: What Does Classifying More Than 10,000 Image Categories Tell Us? In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 71–84. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Binder, A., Müller, K.R., Kawanabe, M.: On taxonomies for multi-class image categorization. IJCV (2011)Google Scholar
  16. 16.
    Cai, L., Hofmann, T.: Exploiting known taxonomies in learning overlapping concepts. In: IJCAI (2007)Google Scholar
  17. 17.
  18. 18.
    Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: Proc. CVPR (2009)Google Scholar
  19. 19.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV (2004)Google Scholar
  20. 20.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. In: Proc. CVPR (2006)Google Scholar
  21. 21.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: Proc. CVPR (2010)Google Scholar
  22. 22.
    Eichner, M., Ferrari, V.: Better appearance models for pictorial structures. In: Proc. BMVC (2009)Google Scholar
  23. 23.
    Yang, Y., Ramanan, D.: Articulated pose estimation with flexible mixtures-of-parts. In: Proc. CVPR (2011)Google Scholar
  24. 24.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2011, VOC 2011 (2011),
  25. 25.
    Marin-Jimenez, M., Zisserman, A., Ferrari, V.: Heres looking at you, kid. detecting people looking at each other in videos. In: Proc. BMVC (2011)Google Scholar
  26. 26.
    Mittal, A., Zisserman, A., Torr, P.H.S.: Hand detection using multiple proposals. In: Proc. BMVC (2011)Google Scholar
  27. 27.
  28. 28.
    Li, F., Carreira, J., Sminchisescu, C.: Object recognition as ranking holistic figure-ground hypotheses. In: Proc. CVPR (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Arpit Mittal
    • 1
  • Matthew B. Blaschko
    • 2
  • Andrew Zisserman
    • 1
  • Philip H. S. Torr
    • 3
  1. 1.Department of Engineering ScienceUniversity of OxfordUK
  2. 2.Center for Visual ComputingÉcole Centrale ParisFrance
  3. 3.Department of ComputingOxford Brookes UniversityUK

Personalised recommendations