Skip to main content

Advertisement

SpringerLink
Log in

Learning Probabilistic Models for Contour Completion in Natural Images

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Using a large set of human segmented natural images, we study the statistics of region boundaries. We observe several power law distributions which likely arise from both multi-scale structure within individual objects and from arbitrary viewing distance. Accordingly, we develop a scale-invariant representation of images from the bottom up, using a piecewise linear approximation of contours and constrained Delaunay triangulation to complete gaps. We model curvilinear grouping on top of this graphical/geometric structure using a conditional random field to capture the statistics of continuity and different junction types. Quantitative evaluations on several large datasets show that our contour grouping algorithm consistently dominates and significantly improves on local edge detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alvarez, L., Gousseau, Y., & Morel, J. (1999). Scales in natural images and a consequence on their bounded variation norm. In Scale-space theories in computer vision.

  • Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review, 61, 183–193.

    Article  Google Scholar 

  • August, J., Siddiqi, K., & Zucker, S. W. (1999). Contour fragment grouping and shared, simple occluders. Computer Vision and Image Understanding, 76(2), 146–162.

    Article  Google Scholar 

  • Belongie, S., Malik, J., & Punicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on PAMI, 24(4), 509–522.

    Google Scholar 

  • Besag, J. E. (1974). Spatial interaction and the statistical analysis of lattice systems. Journal of Royal Statistical Society: Series B, 36(2), 192–236.

    MATH  MathSciNet  Google Scholar 

  • Borenstein, E., & Ullman, S. (2002). Class-specific, top-down segmentation. In Proceedings of 7th European conference on computer vision (Vol. 2, pp. 109–124).

  • Borgefors, G. (1988). Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on PAMI, 10(6), 849–865.

    Google Scholar 

  • Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on PAMI, 8, 679–698.

    Google Scholar 

  • Donoho, D., & Huo, X. (2002). Beamlets and multiscale image analysis. In Lecture notes in computational science and engineering : Vol. 20. Multiscale and multiresolution methods (pp. 149–196). New York: Springer.

    Google Scholar 

  • Elder, J., & Goldberg, R. (2002). Ecological statistics of gestalt laws for the perceptual organization of contours. Journal of Vision, 2(4), 324–353.

    Article  Google Scholar 

  • Elder, J. H., & Zucker, S. W. (1996). Computing contour closures. In Proceedings of 4th European conference on computer vision (Vol. I, pp. 399–412).

  • Felzenszwalb, P. (2001). Learning models for object recognition. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000). Learning low-level vision. International Journal of Computer Vision, 40(1), 25–47.

    Article  MATH  Google Scholar 

  • Geisler, W. S., Perry, J. S., Super, B. J., & Gallogly, D. P. (2001). Edge co-occurrence in natural images predicts contour grouping performance. Vision Research, 41, 711–724.

    Article  Google Scholar 

  • Geman, S., & Geman, D. (1984). Stochastic relaxation, gibbs distribution, and the bayesian retoration of images. IEEE Transactions on PAMI, 6, 721–741.

    MATH  Google Scholar 

  • He, X., Zemel, R., & Carreira-Perpinan, M. (2004). Multiscale conditional random fields for image labelling. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 695–702).

  • Heitger, F., & von der Heydt, R. (1993). A computational model of neural contour processing. In Proceedings of 4th international conference on computer vision (pp. 32–40). Berlin, Germany, May 1993.

  • Huang, J., & Mumford, D. (1999). Statistics of natural images and models. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 541–547).

  • Huttenlocher, D., & Wayner, P. (1992). Finding convex edge groupings in an image. International Journal of Computer Vision, 8(1), 7–27.

    Article  Google Scholar 

  • Huttenlocher, D. P., Klanderman, G., & Rucklidge, W. (1993). Comparing images using the Hausdorff distance. IEEE Transactions on PAMI, 15(9), 850–863.

    Google Scholar 

  • Jermyn, I., & Ishikawa, H. (2001). Globally optimal regions and boundaries as minimum ratio weight cycles. IEEE Transactions on PAMI, 23(10), 1075–1088.

    Google Scholar 

  • Kanizsa, G. (1979). Organization in vision: essays on gestalt perception. New York: Praeger.

    Google Scholar 

  • Kellman, P. J., & Shipley, T. F. (1991). A theory of visual interpolation in object perception. Cognitive Psychology, 23, 141–221.

    Article  Google Scholar 

  • Konishi, S., Yuille, A. L., Coughlan, J. M., & Zhu, S. C. (1999). Fundamental bounds on edge detection: An information theoretic evaluation of different edge cues. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 573–579).

  • Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47, 498–519.

    Article  MATH  MathSciNet  Google Scholar 

  • Kumar, S., & Hebert, M. (2006). Discriminative random fields. International Journal of Computer Vision, 68(2), 179–202.

    Article  Google Scholar 

  • Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of 18th international conference on machine learning.

  • Lee, A., Mumford, D., & Huang, J. (2001). Occlusion models for natural images: A statistical study of a scale-invariant dead leaves model. International Journal of Computer Vision, 41(1/2), 35–59.

    Article  MATH  Google Scholar 

  • Leuttgen, M., Karl, W. C., Willsky, A. S., & Tenney, R. R. (1993). Multiscale representation of markov random fields. Special Issue of the IEEE Transactions on Signal Processing on Wavelet Transforms, 41(12), 3377–96.

    Google Scholar 

  • Li, S. Z. (1995). Markov random field modeling in computer vision. New York: Springer.

    Google Scholar 

  • Martin, D., Fowlkes, C., & Malik, J. (2002). Berkeley segmentation dataset. http://www.cs.berkeley.edu/projects/vision/bsds.

  • Martin, D., Fowlkes, C., Walker, L., & Malik, J. (2003). Local boundary detection in natural images: Matching human and machine performance. In European conference on visual perception (Perception, 32 supp., p. 55).

  • Martin, D., Fowlkes, C., & Malik, J. (2004). Learning to detect natural image boundaries using local brightness, color and texture cues. IEEE Transactions on PAMI, 26(5), 530–549.

    Google Scholar 

  • Mori, G., Ren, X., Efros, A., & Malik, J. (2004). Recovering human body configurations: Combining segmentation and recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 326–333).

  • Mumford, D. (1994). Elastica and computer vision. In C. Bajaj (Ed.), Algebraic geometry and its applications (pp. 491–506). New York: Springer.

    Google Scholar 

  • Murphy, K., Weiss, Y., & Jordan, M. (1999). Loopy belief propagation for approximate inference: An empirical study. In Proceedings of of UAI (pp. 467–475).

  • Palmer, S. (1999). Vision science: Photons to phenomenology. Cambridge: MIT Press.

    Google Scholar 

  • Parent, P., & Zucker, S. W. (1989). Trace inference, curvature consistency, and curve detection. IEEE Transactions on PAMI, 11(8), 823–39.

    Google Scholar 

  • Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. Palo Alto: Morgan Kaufmann.

    Google Scholar 

  • Ren, X. (2007). Learning and matching line aspects for articulated objects. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Ren, X., & Malik, J. (2002). A probabilistic multi-scale model for contour completion based on image statistics. In Proceedings of 7th European conference on computer vision (Vol. 1, pp. 312–327).

  • Ren, X., Berg, A., & Malik, J. (2005). Recovering human body configurations using pairwise constraints between parts. In Proceedings of 10th international conference on computer vision (Vol. 1, pp. 824–831).

  • Ruderman, D. L. (1997). Origins of scaling in natural images. Vision Research, 37(23), 3385–98.

    Article  Google Scholar 

  • Ruderman, D. L., & Bialek, W. (1994). Statistics of natural images: Scaling in the woods. Physics Review Letters, 73(6), 814–817.

    Article  Google Scholar 

  • Sharon, E., Brandt, A., & Basri, R. (2000). Completion energies and scale. IEEE Transactions on PAMI, 22(10), 1117–1131.

    Google Scholar 

  • Shashua, A., & Ullman, S. (1988). Structural saliency: the detection of globally salient structures using a locally connected network. In Proceedings of 2nd international conference on computer vision (pp. 321–327).

  • Shental, N., Zomet, A., Hertz, T., & Weiss, Y. (2003). Learning and inferring image segmentations with the gbp typical cut algorithm. In Proceedings of 9th international conference on computer vision (pp. 1243–1250).

  • Shewchuk, J. (1996). Triangle: Engineering a 2d quality mesh generator and delaunay triangulator. In First workshop on applied computational geometry (pp. 124–133).

  • Sun, J., Shum, H.-Y., & Zheng, N.-N. (2002). Stereo matching using belief propagation. In Proceedings of 7th European conference on computer vision (pp. 510–524).

  • Thornber, K. K., & Williams, L. R. (2001). Orientation, scale, and discontinuity as emergent properties of illusory contour shape. Neural Computation, 13(8), 1683–1711.

    Article  MATH  Google Scholar 

  • Tu, Z. W., Chen, X. R., L Yuille, A., & Zhu, S. C. (2005). Image parsing: Unifying segmentation, detection, and recognition. International Journal of Computer Vision, 63(2), 113–140.

    Article  Google Scholar 

  • von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260–1262.

    Article  Google Scholar 

  • Weiss, Y. (1997). Interpreting images by propagating bayesian beliefs. Advances in Neural Information Processing Systems, 10, 908–915.

    Google Scholar 

  • Weiss, Y. (2000). Correctness of local probability propagation in graphical models with loops. Neural Computation, 1–41.

  • Wertheimer, M. (1938). Laws of organization in perceptual forms (partial translation). In W. B. Ellis (Ed.), A sourcebook of gestalt psychology (pp. 71–88). New York: Harcourt Brace.

    Chapter  Google Scholar 

  • Williams, L. R., & Jacobs, D. W. (1997). Stochastic completion fields: A neural model of illusory contour shape and salience. Neural Computation, 9(4), 837–858.

    Article  Google Scholar 

  • Williams, L. R., & Thornber, K. K. (1999). A comparison of measures for detecting natural shapes in cluttered backgrounds. International Journal of Computer Vision, 34(2/3), 81–96.

    Article  Google Scholar 

  • Wu, Q., & Yu, Y. (2003). Two-level image segmentation based on region and edge integration. In Proceedings of DICTA (pp. 957–966).

  • Yu, S., Gross, R., & Shi, J. (2002). Concurrent object segmentation and recognition with graph partitioning. Advances in Neural Information Processing Systems, 15.

  • Zhu, S. C. (1999). Embedding gestalt laws in markov random fields. IEEE Transactions on PAMI, 21(11), 1170–1187.

    Google Scholar 

  • Zhu, S. C., Wu, Y. N., & Mumford, D. B. (1998). Frame: Filters, random field and maximum entropy: Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2), 1–20.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Toyota Technological Institute at Chicago, 1427 E. 60th Street, Chicago, IL, 60637, USA

    Xiaofeng Ren

  2. School of Information and Computer Science, University of California, Irvine, CA, 92697-3425, USA

    Charless C. Fowlkes

  3. Computer Science Division, University of California at Berkeley, Berkeley, CA, 94720, USA

    Jitendra Malik

Authors
  1. Xiaofeng Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charless C. Fowlkes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jitendra Malik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaofeng Ren.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ren, X., Fowlkes, C.C. & Malik, J. Learning Probabilistic Models for Contour Completion in Natural Images. Int J Comput Vis 77, 47–63 (2008). https://doi.org/10.1007/s11263-007-0092-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0092-6

Keywords

Search

Navigation