Figure/Ground Assignment in Natural Images

  • Xiaofeng Ren
  • Charless C. Fowlkes
  • Jitendra Malik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3952)


Figure/ground assignment is a key step in perceptual organization which assigns contours to one of the two abutting regions, providing information about occlusion and allowing high-level processing to focus on non-accidental shapes of figural regions. In this paper, we develop a computational model for figure/ground assignment in complex natural scenes. We utilize a large dataset of images annotated with human-marked segmentations and figure/ground labels for training and quantitative evaluation.

We operationalize the concept of familiar configuration by constructing prototypical local shapes, i.e. shapemes, from image data. Shapemes automatically encode mid-level visual cues to figure/ground assignment such as convexity and parallelism. Based on the shapeme representation, we train a logistic classifier to locally predict figure/ground labels. We also consider a global model using a conditional random field (CRF) to enforce global figure/ground consistency at T-junctions. We use loopy belief propagation to perform approximate inference on this model and learn maximum likelihood parameters from ground-truth labels.

We find that the local shapeme model achieves an accuracy of 64% in predicting the correct figural assignment. This compares favorably to previous studies using classical figure/ground cues [1]. We evaluate the global model using either a set of contours extracted from a low-level edge detector or the set of contours given by human segmentations. The global CRF model significantly improves the performance over the local model, most notably when using human-marked boundaries (78%). These promising experimental results show that this is a feasible approach to bottom-up figure/ground assignment in natural images.


Natural Image Conditional Random Field Ground Organization Conditional Random Field Model Junction Type 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Fowlkes, C., Martin, D., Malik, J.: On measuring the ecological validity of local figure/ground cues. In: ECVP (2003)Google Scholar
  2. 2.
    Rubin, E.: Visuell wahrgenommene figuren. In: Kobenhaven: Glydenalske boghandel (1921)Google Scholar
  3. 3.
    Palmer, S.: Vision Science: Photons to Phenomenology. MIT Press, Cambridge (1999)Google Scholar
  4. 4.
    Peterson, M.A., Gibson, B.S.: Must figure-ground organization precede object recognition? an assumption in peril. Psychological Science 5, 253–259 (1994)CrossRefGoogle Scholar
  5. 5.
    Kienker, P.K., Sejnowski, T.J., Hinton, G.E., Schumacher, L.E.: Separating figure from ground with a parallel network. Perception 15, 197–216 (1986)CrossRefGoogle Scholar
  6. 6.
    Heitger, F., von der Heydt, R.: A computational model of neural contour processing: figure-ground segregation and illusory contours. In: ICCV, Berlin, Germany, pp. 32–40 (1993)Google Scholar
  7. 7.
    Geiger, D., Kumaran, K., Parida, L.: Visual organization for figure/ground separation. In: CVPR, pp. 155–160 (1996)Google Scholar
  8. 8.
    Saund, E.: Perceptual organization of occluding contours of opaque surfaces. CVIU Special Issue on Perceptual Organization, pp. 70–82 (1999)Google Scholar
  9. 9.
    Yu, S., Lee, T.S., Kanade, T.: A hierarchical markov random field model for figure-ground segregation. In: EMM CVPR 2001, pp. 118–133 (2001)Google Scholar
  10. 10.
    Pao, H.K., Geiger, D., Rubin, N.: Measuring convexity for figure/ground separation. In: ICCV, pp. 948–955 (1999)Google Scholar
  11. 11.
    Lamme, V.A.F.: The neurophysiology of figure-ground segregation in primary visual cortex. Journal of Neuroscience 15, 1605–1615 (1995)Google Scholar
  12. 12.
    Zhou, H., Friedman, H.S., von der Heydt, R.: Coding border ownership in monkey visual cortex. Journal of Neuroscience 20, 6594–6611 (2000)Google Scholar
  13. 13.
    Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using brightness and texture. In: Advances in Neural Information Processing Systems 15 (2002)Google Scholar
  14. 14.
    Berg, A., Malik, J.: Geometric blur for template matching. In: CVPR (2001)Google Scholar
  15. 15.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. 18th International Conf. on Machine Learning (2001)Google Scholar
  16. 16.
    Ren, X., Fowlkes, C., Malik, J.: Scale-invariant contour completion using conditional random fields. In: ICCV (2005)Google Scholar
  17. 17.
    McDermott, J.: Psychophysics with junctions in real images. Perception 33, 1101–1127 (2004)CrossRefGoogle Scholar
  18. 18.
    Mori, G., Belongie, S., Malik, J.: Shape contexts enable efficient retrieval of similar shapes. In: CVPR, vol. 1, pp. 723–730 (2001)Google Scholar
  19. 19.
    Mori, G., Ren, X., Efros, A., Malik, J.: Recovering human body configurations: Combining segmentation and recognition. In: CVPR, vol. 2, pp. 326–333 (2004)Google Scholar
  20. 20.
    Kumar, S., Hebert, M.: Discriminative random fields: A discriminative framework for contextual interaction in classification. In: ICCV, pp. 1150–1159 (2003)Google Scholar
  21. 21.
    He, X., Zemel, R., Carreira-Perpinan, M.: Multiscale conditional random fields for image labelling. In: CVPR, vol. 2, pp. 695–702 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Xiaofeng Ren
    • 1
  • Charless C. Fowlkes
    • 1
  • Jitendra Malik
    • 1
  1. 1.Computer Science DivisionUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations