Contextual Object Detection Using Set-Based Classification

  • Ramazan Gokberk Cinbis
  • Stan Sclaroff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7577)


We propose a new model for object detection that is based on set representations of the contextual elements. In this formulation, relative spatial locations and relative scores between pairs of detections are considered as sets of unordered items. Directly training classification models on sets of unordered items, where each set can have varying cardinality can be difficult. In order to overcome this problem, we propose SetBoost, a discriminative learning algorithm for building set classifiers. The SetBoost classifiers are trained to rescore detected objects based on object-object and object-scene context. Our method is able to discover composite relationships, as well as intra-class and inter-class spatial relationships between objects. The experimental evidence shows that our set-based formulation performs comparable to or better than existing contextual methods on the SUN and the VOC 2007 benchmark datasets.


Average Precision Object Class Context Model Reference Object Contextual Relationship 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Blaschko, M.B., Lampert, C.H.: Object localization with global and local context kernels. In: BMVC (2009)Google Scholar
  2. 2.
    Desai, C., Ramanan, D., Fowlkes, C.: Discriminative models for multi-class object layout. In: ICCV (2009)Google Scholar
  3. 3.
    Choi, M.J., Lim, J.J., Torralba, A., Willsky, A.S.: Exploiting hierarchical context on a large database of object categories. In: CVPR (2010)Google Scholar
  4. 4.
    Gemert, J.C.V., Snoek, C.G., Veenman, C.J., Smeulders, A.W., Geusebroek, J.M.: Comparing compact codebooks for visual categorization. CVIU (2010)Google Scholar
  5. 5.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge (2007)Google Scholar
  6. 6.
    Li, C., Parikh, D., Chen, T.: Extracting adaptive contextual cues from unlabeled regions. In: ICCV (2011)Google Scholar
  7. 7.
    Grauman, K., Darrell, T.: Approximate correspondences in high dimensions. In: NIPS (2007)Google Scholar
  8. 8.
    Heitz, G., Koller, D.: Learning Spatial Context: Using Stuff to Find Things. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 30–43. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing: Label transfer via dense scene alignment. In: CVPR (2009)Google Scholar
  10. 10.
    Galleguillos, C., Rabinovich, A., Belongie, S.: Object categorization using co-occurrence, location and appearance. In: CVPR (2008)Google Scholar
  11. 11.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42 (2001)Google Scholar
  12. 12.
    Hoiem, D., Efros, A., Hebert, M.: Putting objects in perspective. IJCV (2008)Google Scholar
  13. 13.
    Galleguillos, C., Belongie, S.: Context based object categorization: A critical survey. CVIU 114 (2010)Google Scholar
  14. 14.
    Felzenszwalb, P., McAllester, D., Ramanan, D., Grishick: Object detection with discriminatively trained part based models. PAMI 32 (2010)Google Scholar
  15. 15.
    Kondor, R., Jebara, T.: A kernel between sets of vectors. In: ICML (2003)Google Scholar
  16. 16.
    Cuturi, M., Vert, J.: Semigroup kernels on finite sets. In: NIPS (2005)Google Scholar
  17. 17.
    Lyu, S.: Mercer kernels for object recognition with local features. In: CVPR (2005)Google Scholar
  18. 18.
    Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Stat. Learning in Comp. Vision (2004)Google Scholar
  19. 19.
    Moosmann, F., Nowak, E., Jurie, F.: Randomized clustering forests for image classification. PAMI 30 (2008)Google Scholar
  20. 20.
    Yang, L., Jin, R., Sukthankar, R., Jurie, F.: Unifying discriminative visual codebook generation with classifier training for object recognition. In: CVPR (2008)Google Scholar
  21. 21.
    Dollár, P., Babenko, B., Belongie, S., Perona, P., Tu, Z.: Multiple Component Learning for Object Detection. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 211–224. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Mason, L., Baxter, J., Bartlett, P., Frean, M.: Boosting algorithms as gradient descent in function space. In: NIPS (1999)Google Scholar
  23. 23.
    Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. on Scientific Comp. 16 (1995)Google Scholar
  24. 24.
    Freund, Y., Schapire, R.E.: A Decision-Theoretic generalization of On-Line learning and an application to boosting. J. of Comp. and Sys. Sci. 55 (1997)Google Scholar
  25. 25.
    Friedman, J.H.: Stochastic gradient boosting. Comp. Stat. and Data Analysis 38 (2002)Google Scholar
  26. 26.
  27. 27.

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ramazan Gokberk Cinbis
    • 1
  • Stan Sclaroff
    • 2
  1. 1.LEAR, INRIA GrenobleFrance
  2. 2.Department of Computer ScienceBoston UniversityUSA

Personalised recommendations