What Is a Good Image Segment? A Unified Approach to Segment Extraction

  • Shai Bagon
  • Oren Boiman
  • Michal Irani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5305)


There is a huge diversity of definitions of “visually meaningful” image segments, ranging from simple uniformly colored segments, textured segments, through symmetric patterns, and up to complex semantically meaningful objects. This diversity has led to a wide range of different approaches for image segmentation. In this paper we present a single unified framework for addressing this problem – “Segmentation by Composition”. We define a good image segment as one which can be easily composed using its own pieces, but is difficult to compose using pieces from other parts of the image. This non-parametric approach captures a large diversity of segment types, yet requires no pre-definition or modelling of segment types, nor prior training. Based on this definition, we develop a segment extraction algorithm – i.e., given a single point-of-interest, provide the “best” image segment containing that point. This induces a figure-ground image segmentation, which applies to a range of different segmentation tasks: single image segmentation, simultaneous co-segmentation of several images, and class-based segmentations.


Image Segmentation Segmentation Algorithm Query Image Image Segment Segment Boundary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. PAMI (2002)Google Scholar
  2. 2.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. PAMI (2000)Google Scholar
  3. 3.
    Malik, J., Belongie, S., Shi, J., Leung, T.K.: Textons, contours and regions: Cue integration in image segmentation. In: ICCV (1999)Google Scholar
  4. 4.
    Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: ICCV (2003)Google Scholar
  5. 5.
    Kadir, T., Brady, M.: Unsupervised non-parametric region segmentation using level sets. In: ICCV (2003)Google Scholar
  6. 6.
    Li, Y., Sun, J., Tang, C.K., Shum, H.Y.: Lazy snapping. ACM TOG (2004)Google Scholar
  7. 7.
    Rother, C., Kolmogorov, V., Blake, A.: “grabcut”: Interactive foreground extraction using iterated graph cuts. In: SIGGRAPH (2004)Google Scholar
  8. 8.
    Riklin-Raviv, T., Kiryati, N., Sochen, N.: Segmentation by level sets and symmetry. In: CVPR (2006)Google Scholar
  9. 9.
    Borenstein, E., Ullman, S.: Class-specific, top-down segmentation. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351. Springer, Heidelberg (2002)Google Scholar
  10. 10.
    Leibe, B., Schiele, B.: Interleaved object categorization and segmentation. In: BMVC (2003)Google Scholar
  11. 11.
    Levin, A., Weiss, Y.: Learning to combine bottom-up and top-down segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Rother, C., Minka, T., Blake, A., Kolmogorov, V.: Cosegmentation of image pairs by histogram matching - incorporating a global constraint into mrfs. In: CVPR (2006)Google Scholar
  13. 13.
    Tu, Z., Chen, X., Yuille, A.L., Zhu, S.C.: Image parsing: Unifying segmentation, detection, and recognition. IJCV (2005)Google Scholar
  14. 14.
    Boiman, O., Irani, M.: Similarity by composition. In: NIPS (2006)Google Scholar
  15. 15.
    Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley, Chichester (1991)zbMATHGoogle Scholar
  16. 16.
    Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. PAMI (2004)Google Scholar
  17. 17.
    Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. PAMI (2001)Google Scholar
  18. 18.
    Boiman, O., Irani, M.: Detecting irregularities in images and in video. IJCV (2007)Google Scholar
  19. 19.
    Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: CVPR (2007)Google Scholar
  20. 20.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Shai Bagon
    • 1
  • Oren Boiman
    • 1
  • Michal Irani
    • 1
  1. 1.Weizmann Institute of ScienceRehovotIsrael

Personalised recommendations