Hybrid Classifiers for Object Classification with a Rich Background

  • Margarita Osadchy
  • Daniel Keren
  • Bella Fadida-Specktor
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7576)


The majority of current methods in object classification use the one-against-rest training scheme. We argue that when applied to a large number of classes, this strategy is problematic: as the number of classes increases, the negative class becomes a very large and complicated collection of images. The resulting classification problem then becomes extremely unbalanced, and kernel SVM classifiers trained on such sets require long training time and are slow in prediction. To address these problems, we propose to consider the negative class as a background and characterize it by a prior distribution. Further, we propose to construct ”hybrid” classifiers, which are trained to separate this distribution from the samples of the positive class. A typical classifier first projects (by a function which may be non-linear) the inputs to a one-dimensional space, and then thresholds this projection. Theoretical results and empirical evaluation suggest that, after projection, the background has a relatively simple distribution, which is much easier to parameterize and work with. Our results show that hybrid classifiers offer an advantage over SVM classifiers, both in performance and complexity, especially when the negative (background) class is large.


Natural Image Kernel Matrix Probability Constraint Negative Class Random Projection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR, pp. 2169–2178 (2006)Google Scholar
  2. 2.
    Grauman, K., Darrell, T.: The Pyramid Match Kernel: Efficient Learning with Sets of Features. JMLR 8, 725–760 (2007)zbMATHGoogle Scholar
  3. 3.
    Maji, S., Berg, A.C., Malik, J.: Classification using intersection kernel support vector machines is efficient. In: CVPR, pp. 1–8 (2008)Google Scholar
  4. 4.
    Perronnin, F., Sánchez, J., Liu, Y.: Large-scale image categorization with explicit data embedding. In: CVPR, pp. 2297–2304 (2010)Google Scholar
  5. 5.
    Gehler, P., Nowozin, S.: On feature combination for multiclass object classification. In: Proc. of ICCV, pp. 221–228 (2009)Google Scholar
  6. 6.
    Song, Z., Chen, Q., Huang, Z., Hua, Y., Yan, S.: Contextualizing object detection and classification. In: CVPR, pp. 1585–1592 (2011)Google Scholar
  7. 7.
    Huang, Y., Huang, K., Yu, Y., Tan, T.: Salient coding for image classification. In: CVPR, pp. 1753–1760 (2011)Google Scholar
  8. 8.
    Rifkin, R., Klautau, A.: In defense of one-vs-all classification. JMLR 5, 101–141 (2004)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Akbani, R., Kwek, S.S., Japkowicz, N.: Applying Support Vector Machines to Imbalanced Datasets. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 39–50. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Kotsiantis, S., Kanellopoulosâ, D.: Handling imbalanced datasets: A review (2006)Google Scholar
  11. 11.
    Steinwart, I.: Sparseness of support vector machines. JMLR 4, 1071–1105 (2003)MathSciNetGoogle Scholar
  12. 12.
    Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: A comprehensive study. IJCV 73, 213–238 (2007)CrossRefGoogle Scholar
  13. 13.
    Keerthi, S.S., Chapelle, O., DeCoste, D.: Building support vector machines with reduced classifier complexity. JMLR 7, 1493–1515 (2006)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Joachims, T., Finley, T., Yu, C.N.: Cutting-plane training of structural svms. Machine Learning 77, 27–59 (2009)zbMATHCrossRefGoogle Scholar
  15. 15.
    Osadchy, M., Keren, D.: Incorporating the boltzmann prior in object detection using svm. In: CVPR, pp. 2095–2101 (2006)Google Scholar
  16. 16.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Dance, C., Willamowski, J., Fan, L., Bray, C., Csurka, G.: Visual categorization with bags of keypoints. In: ECCV (2004)Google Scholar
  18. 18.
    Weiss, Y., Freeman, W.T.: What makes a good model of natural images. In: CVPR, pp. 1–8 (2007)Google Scholar
  19. 19.
    Zhu, S.C., Mumford, D.: Prior learning and gibbs reaction-diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 19, 1236–1250 (1997)CrossRefGoogle Scholar
  20. 20.
    Roth, S., Black, M.J.: Fields of experts: A framework for learning image priors. In: CVPR, pp. 860–867 (2005)Google Scholar
  21. 21.
    Srivastava, A., Lee, A.B., Simoncelli, E.P., Zhu, S.-C.: On advances in statistical modeling of natural images. JMIV 18, 17–33 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Diaconis, P., Freedman, D.: Asymptotics of graphical projection pursuit. Ann. Statist. 12, 793–815 (1984)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. T. R. 7694, California Institute of Technology (2007)Google Scholar
  24. 24.
    Lanckriet, G., Ghaoui, L., Bhattacharyya, C., Jordan, M.: A robust minimax approach to classification. JMLR 3, 555–582 (2002)Google Scholar
  25. 25.
    Smola, A.J., Schölkopf, B.: Sparse greedy matrix approximation for machine learning. In: ICML, pp. 911–918 (2000)Google Scholar
  26. 26.
    Tsai, D., Jing, Y., Liu, Y., Rowley, H., Ioffe, S., Rehg, J.M.: Large-scale image annotation using visual synset. In: ICCV (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Margarita Osadchy
    • 1
  • Daniel Keren
    • 1
  • Bella Fadida-Specktor
    • 1
  1. 1.Computer ScienceUniversity of HaifaCarmelIsrael

Personalised recommendations