Shape-Based Object Discovery in Images

  • Sinisa Todorovic
  • Nadia Payet
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)


This paper presents an overview of our recent work on shape-based object discovery in images. The overview focuses on the following related problems: (i) discovery of all distinct 2D object categories frequently occurring in an unlabeled set of images; (ii) learning a model of the discovered categories; and (iii) recognition and localization of objects from the discovered categories in new images. The paper argues that using image contours as basic features, and thus directly grounding object discovery and recognition on shape, offers a number of advantages in solving (i)–(iii) over more commonly used point features. Since shape is directly encoded by layouts of image contours, similar contour layouts across the images are expected to belong rather to object occurrences, than the background. The contour layouts are captured by a graph over all pairs of matching contours from different images. The graph’s maximum a posteriori multicoloring assignment is taken to represent the shapes of discovered objects. Our empirical evaluation suggests that shape is more expressive and discriminative than photometric features for object discovery.


Dynamic Time Warping Spatial Layout Object Discovery Background Clutter Negative Edge 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Payet N, Todorovic S (2009) Matching hierarchies of deformable shapes. In: Proc 7th IAPR-TC-15 workshop graph-based representations in pattern recognition (GbR), pp 1–10 CrossRefGoogle Scholar
  2. 2.
    Payet N, Todorovic S (2010) From a set of shapes to object discovery. In: ECCV Google Scholar
  3. 3.
    Payet N, Todorovic S (2011) From contours to 3D object detection and pose estimation. In: ICCV (oral presentation) Google Scholar
  4. 4.
    Fergus R, Perona P, Zisserman A (2003) Object class recognition by unsupervised scale-invariant learning. In: CVPR, vol 2, pp 264–271 Google Scholar
  5. 5.
    Torralba A, Murphy K, Freeman W (2004) Sharing features: efficient boosting procedures for multiclass object detection. In: CVPR, vol 2, pp 762–769 Google Scholar
  6. 6.
    Leibe B, Leonardis A, Schiele B (2004) Combined object categorization and segmentation with an implicit shape model. In: Workshop on statistical learning in computer vision, ECCV, pp 17–32 Google Scholar
  7. 7.
    Agarwal S, Awan A, Roth D (2004) Learning to detect objects in images via a sparse, part-based representation. IEEE Trans Pattern Anal Mach Intell 26(11):1475–1490 CrossRefGoogle Scholar
  8. 8.
    Sudderth E, Torralba A, Freeman WT, Willsky AS (2005) Learning hierarchical models of scenes, objects, and parts. In: ICCV, vol 2, pp 1331–1338 Google Scholar
  9. 9.
    Winn J, Criminisi A, Minka T (2005) Object categorization by learned universal visual dictionary. In: ICCV, vol 2, pp 1800–1807 Google Scholar
  10. 10.
    Fei-Fei L, Fergus R, Perona P (2006) One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28(4):594–611 CrossRefGoogle Scholar
  11. 11.
    Opelt A, Pinz A, Zisserman A (2006) Incremental learning of object detectors using a visual shape alphabet. In: CVPR, vol 1, pp 3–10 Google Scholar
  12. 12.
    Biederman I (1988) Surface versus edge-based determinants of visual recognition. Cogn Psychol 20(1):38–64 CrossRefGoogle Scholar
  13. 13.
    Williams LR, Jacobs DW (1995) Stochastic completion fields: a neural model of illusory contour shape and salience. In: ICCV, pp 408–415 Google Scholar
  14. 14.
    Lindenbaum M (1995) Bounds on shape recognition performance. IEEE Trans Pattern Anal Mach Intell 17(7):666–680 CrossRefGoogle Scholar
  15. 15.
    Liu TL, Geiger D (1999) Approximate tree matching and shape similarity. In: Proc IEEE int conf computer vision, vol 1, pp 456–462 Google Scholar
  16. 16.
    Shokoufandeh A, Macrini D, Dickinson S, Siddiqi K, Zucker SW (2005) Indexing hierarchical structures using graph spectra. IEEE Trans Pattern Anal Mach Intell 27(7):1125–1140 CrossRefGoogle Scholar
  17. 17.
    Keselman Y, Dickinson S (2005) Generic model abstraction from examples. IEEE Trans Pattern Anal Mach Intell 27(7):1141–1156 CrossRefGoogle Scholar
  18. 18.
    Siddiqi K, Kimia BB (1996) A shock grammar for recognition. In: CVPR, p 507 Google Scholar
  19. 19.
    Sebastian TB, Klein PN, Kimia BB (2004) Recognition of shapes by editing their shock graphs. IEEE Trans Pattern Anal Mach Intell 26(5):550–571 CrossRefGoogle Scholar
  20. 20.
    Felzenszwalb PF (2005) Representation and detection of deformable shapes. IEEE Trans Pattern Anal Mach Intell 27(2):208–220 CrossRefGoogle Scholar
  21. 21.
    Zhu Q, Wang L, Wu Y, Shi J (2008) Contour context selection for object detection: a set-to-set contour matching approach. In: ECCV (2), pp 774–787 Google Scholar
  22. 22.
    Kokkinos I, Yuille AL (2009) HOP: hierarchical object parsing. In: CVPR Google Scholar
  23. 23.
    Ling H, Jacobs DW (2007) Shape classification using the inner-distance. IEEE Trans Pattern Anal Mach Intell 29(2):286–299 CrossRefGoogle Scholar
  24. 24.
    Torsello A, Robles-Kelly A, Hancock ER (2007) Discovering shape classes using tree edit-distance and pairwise clustering. Int J Comput Vis 72(3):259–285 CrossRefGoogle Scholar
  25. 25.
    Trinh NH, Kimia BB (2011) Skeleton search: category-specific object recognition and segmentation using a skeletal shape model. Int J Comput Vis 94(2):215–240 CrossRefGoogle Scholar
  26. 26.
    Bai X, Wang X, Liu W, Latecki LJ, Tu Z (2009) Active skeleton for non-rigid object detection. In: ICCV Google Scholar
  27. 27.
    Shotton J, Blake A, Cipolla R (2008) Multiscale categorical object recognition using contour fragments. IEEE Trans Pattern Anal Mach Intell 30(7):1270–1281 CrossRefGoogle Scholar
  28. 28.
    Ferrari V, Tuytelaars T, Gool LV (2006) Object detection by contour segment networks. In: ECCV, pp 14–28 Google Scholar
  29. 29.
    Perona P, Malik J (1991) Detecting and localizing edges composed of steps, peaks and roofs. In: ICCV, pp 52–57 Google Scholar
  30. 30.
    Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33:898–916 CrossRefGoogle Scholar
  31. 31.
    Felzenszwalb P, McAllester D (2006) A min-cover approach for finding salient curves. In: IEEE workshop on perceptual organization (POCV) Google Scholar
  32. 32.
    Russell BC, Freeman WT, Efros A, Sivic J, Zisserman A (2006) Using multiple segmentations to discover objects and their extent in image collections. In: CVPR Google Scholar
  33. 33.
    Todorovic S, Ahuja N (2008) Unsupervised category modeling, recognition, and segmentation in images. IEEE Trans Pattern Anal Mach Intell 30(12):1–17 CrossRefGoogle Scholar
  34. 34.
    Kim G, Faloutsos C, Hebert M (2008) Unsupervised modeling of object categories using link analysis techniques. In: CVPR Google Scholar
  35. 35.
    Lee YJ, Grauman K (2009) Shape discovery from unlabeled image collections. In: CVPR Google Scholar
  36. 36.
    Felzenszwalb P, McAllester D (2006) A min-cover approach for finding salient curves. In: CVPR POCV Google Scholar
  37. 37.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522 CrossRefGoogle Scholar
  38. 38.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110 CrossRefGoogle Scholar
  39. 39.
    Chib S, Greenberg E (1995) Understanding the metropolis-hastings algorithm. Am Stat 49(4):327–335 Google Scholar
  40. 40.
    Lin L, Zeng K, Liu X, Zhu SC (2009) Layered graph matching by composite cluster sampling with collaborative and competitive interactions. In: CVPR, June 2009 Google Scholar
  41. 41.
    Lee YJ, Grauman K (2008) Foreground focus: unsupervised learning from partially matching images. In: BMVC Google Scholar
  42. 42.
    Fei-Fei L, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: CVPR Google Scholar
  43. 43.
    Russell BC, Torralba A, Murphy KP, Freeman WT (2005) Labelme: a database and web-based tool for image annotation. Technical Report AIM-2005-025, MIT Google Scholar
  44. 44.
    Borenstein E, Ullman S (2002) Class-specific, top-down segmentation. In: ECCV, vol 2, pp 109–124 Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.School of Electrical Engineering and Computer ScienceOregon State UniversityCorvallisUSA

Personalised recommendations