Figure/Ground Assignment in Natural Images

  • Xiaofeng Ren
  • Charless C. Fowlkes
  • Jitendra Malik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3952)


Figure/ground assignment is a key step in perceptual organization which assigns contours to one of the two abutting regions, providing information about occlusion and allowing high-level processing to focus on non-accidental shapes of figural regions. In this paper, we develop a computational model for figure/ground assignment in complex natural scenes. We utilize a large dataset of images annotated with human-marked segmentations and figure/ground labels for training and quantitative evaluation.

We operationalize the concept of familiar configuration by constructing prototypical local shapes, i.e. shapemes, from image data. Shapemes automatically encode mid-level visual cues to figure/ground assignment such as convexity and parallelism. Based on the shapeme representation, we train a logistic classifier to locally predict figure/ground labels. We also consider a global model using a conditional random field (CRF) to enforce global figure/ground consistency at T-junctions. We use loopy belief propagation to perform approximate inference on this model and learn maximum likelihood parameters from ground-truth labels.

We find that the local shapeme model achieves an accuracy of 64% in predicting the correct figural assignment. This compares favorably to previous studies using classical figure/ground cues [1]. We evaluate the global model using either a set of contours extracted from a low-level edge detector or the set of contours given by human segmentations. The global CRF model significantly improves the performance over the local model, most notably when using human-marked boundaries (78%). These promising experimental results show that this is a feasible approach to bottom-up figure/ground assignment in natural images.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fowlkes, C., Martin, D., Malik, J.: On measuring the ecological validity of local figure/ground cues. In: ECVP (2003)Google Scholar
  2. 2.
    Rubin, E.: Visuell wahrgenommene figuren. In: Kobenhaven: Glydenalske boghandel (1921)Google Scholar
  3. 3.
    Palmer, S.: Vision Science: Photons to Phenomenology. MIT Press, Cambridge (1999)Google Scholar
  4. 4.
    Peterson, M.A., Gibson, B.S.: Must figure-ground organization precede object recognition? an assumption in peril. Psychological Science 5, 253–259 (1994)CrossRefGoogle Scholar
  5. 5.
    Kienker, P.K., Sejnowski, T.J., Hinton, G.E., Schumacher, L.E.: Separating figure from ground with a parallel network. Perception 15, 197–216 (1986)CrossRefGoogle Scholar
  6. 6.
    Heitger, F., von der Heydt, R.: A computational model of neural contour processing: figure-ground segregation and illusory contours. In: ICCV, Berlin, Germany, pp. 32–40 (1993)Google Scholar
  7. 7.
    Geiger, D., Kumaran, K., Parida, L.: Visual organization for figure/ground separation. In: CVPR, pp. 155–160 (1996)Google Scholar
  8. 8.
    Saund, E.: Perceptual organization of occluding contours of opaque surfaces. CVIU Special Issue on Perceptual Organization, pp. 70–82 (1999)Google Scholar
  9. 9.
    Yu, S., Lee, T.S., Kanade, T.: A hierarchical markov random field model for figure-ground segregation. In: EMM CVPR 2001, pp. 118–133 (2001)Google Scholar
  10. 10.
    Pao, H.K., Geiger, D., Rubin, N.: Measuring convexity for figure/ground separation. In: ICCV, pp. 948–955 (1999)Google Scholar
  11. 11.
    Lamme, V.A.F.: The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience 15, 1605–1615 (1995)Google Scholar
  12. 12.
    Zhou, H., Friedman, H.S., von der Heydt, R.: Coding border ownership in monkey visual cortex. Journal of Neuroscience 20, 6594–6611 (2000)Google Scholar
  13. 13.
    Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using brightness and texture. In: Advances in Neural Information Processing Systems 15 (2002)Google Scholar
  14. 14.
    Berg, A., Malik, J.: Geometric blur for template matching. In: CVPR (2001)Google Scholar
  15. 15.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. 18th International Conf. on Machine Learning (2001)Google Scholar
  16. 16.
    Ren, X., Fowlkes, C., Malik, J.: Scale-invariant contour completion using conditional random fields. In: ICCV (2005)Google Scholar
  17. 17.
    McDermott, J.: Psychophysics with junctions in real images. Perception 33, 1101–1127 (2004)CrossRefGoogle Scholar
  18. 18.
    Mori, G., Belongie, S., Malik, J.: Shape contexts enable efficient retrieval of similar shapes. In: CVPR, vol. 1, pp. 723–730 (2001)Google Scholar
  19. 19.
    Mori, G., Ren, X., Efros, A., Malik, J.: Recovering human body configurations: Combining segmentation and recognition. In: CVPR, vol. 2, pp. 326–333 (2004)Google Scholar
  20. 20.
    Kumar, S., Hebert, M.: Discriminative random fields: A discriminative framework for contextual interaction in classification. In: ICCV, pp. 1150–1159 (2003)Google Scholar
  21. 21.
    He, X., Zemel, R., Carreira-Perpinan, M.: Multiscale conditional random fields for image labelling. In: CVPR, vol. 2, pp. 695–702 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Xiaofeng Ren
    • 1
  • Charless C. Fowlkes
    • 1
  • Jitendra Malik
    • 1
  1. 1.Computer Science DivisionUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations