Contour and Texture Analysis for Image Segmentation

  • Jitendra Malik
  • Serge Belongie
  • Thomas Leung
  • Jianbo Shi


This paper provides an algorithm for partitioning grayscale images into disjoint regions of coherent brightness and texture. Natural images contain both textured and untextured regions, so the cues of contour and texture differences are exploited simultaneously. Contours are treated in the intervening contour framework, while texture is analyzed using textons. Each of these cues has a domain of applicability, so to facilitate cue combination we introduce a gating operator based on the texturedness of the neighborhood at a pixel. Having obtained a local measure of how likely two nearby pixels are to belong to the same region, we use the spectral graph theoretic framework of normalized cuts to find partitions of the image into regions of coherent texture and brightness. Experimental results on a wide range of images are shown.

segmentation texture grouping cue integration texton normalized cut 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Belongie, S., Carson, C., Greenspan, H., and Malik, J. 1998. Colorand texture-based image segmentation using EM and its application to content-based image retrieval. In Proc. 6th Int. Conf. Computer Vision, Bombay, India, pp. 675-682.Google Scholar
  2. Belongie, S. and Malik, J. 1998. Finding boundaries in natural images: A new method using point descriptors and area completion. In Proc. 5th Euro. Conf. Computer Vision, Freiburg, Germany, pp. 751-766.Google Scholar
  3. Binford, T. 1981. Inferring surfaces from images. Artificial Intelligence, 17(1-3):205-244.Google Scholar
  4. Canny, J. 1986. A computational approach to edge detection. IEEE Trans. Pat. Anal. Mach. Intell., 8(6):679-698.Google Scholar
  5. Chung, F. 1997. Spectral Graph Theory, AMS. Providence, RI.Google Scholar
  6. DeValois, R. and DeValois, K. 1988. Spatial Vision. Oxford University Press. New York, N.Y.Google Scholar
  7. Duda, R. and Hart, P. 1973. Pattern Classification and Scene Analysis, John Wiley & Sons. New York, N.Y.Google Scholar
  8. Elder, J. and Zucker, S. 1996. Computing contour closures. In Proc. Euro. Conf. Computer Vision, Vol. I, Cambridge, England, pp. 399-412.Google Scholar
  9. Fogel, I. and Sagi, D. 1989. Gabor filters as texture discriminator. Biological Cybernetics, 61:103-113.Google Scholar
  10. Geman, S. and Geman, D. 1984. Stochastic relaxation, Gibbs distribution, and the Bayesian retoration of images. IEEE Trans. Pattern Anal. Mach. Intell., 6:721-741.Google Scholar
  11. Gersho, A. and Gray, R. 1992. Vector Quantization and Signal Compression, Kluwer Academic Publishers, Boston, MA.Google Scholar
  12. Heeger, D.J. and Bergen, J.R. 1995. Pyramid-based texture analysis/ synthesis. In Proceedings of SIGGRAPH '95, pp. 229-238.Google Scholar
  13. Jacobs, D. 1996. Robust and efficient detection of salient convex groups. IEEE Trans. Pattern Anal. Mach. Intell., 18(1):23-37.Google Scholar
  14. Jones, D. and Malik, J. 1992. Computational framework to determining stereo correspondence from a set of linear spatial filters. Image and Vision Computing, 10(10):699-708.Google Scholar
  15. Julesz, B. 1981. Textons, the elements of texture perception, and their interactions. Nature, 290(5802):91-97.Google Scholar
  16. Knutsson, H. and Granlund, G. 1983. Texture analysis using twodimensional quadrature filters. In Workshop on Computer Architecture for Pattern Analysis and Image Database Management, pp. 206-213.Google Scholar
  17. Koenderink, J. and van Doorn, A. 1987. Representation of local geometry in the visual system. Biological Cybernetics, 55(6):367-375.Google Scholar
  18. Koenderink, J. and van Doorn, A. 1988. Operational significance of receptive field assemblies. Biological Cybernetics, 58:163-171.Google Scholar
  19. Leung, T. and Malik, J. 1998. Contour continuity in region-based image segmentation. In Proc. Euro. Conf. Computer Vision,Vol. 1, H. Burkhardt and B. Neumann (Eds.). Freiburg, Germany, pp. 544-559.Google Scholar
  20. Leung, T. and Malik, J. 1999. Recognizing surfaces using threedimensional textons. In Proc. Int. Conf. Computer Vision, Corfu, Greece, pp. 1010-1017.Google Scholar
  21. Malik, J., Belongie, S., Shi, J., and Leung, T. 1999. Textons, contours and regions: Cue integration in image segmentation. In Proc. IEEE Intl. Conf. Computer Vision, Vol. 2, Corfu, Greece, pp. 918-925.Google Scholar
  22. Malik, J. and Perona, P. 1990. Preattentive texture discrimination with early vision mechanisms. J. Optical Society of America, 7(2):923-932.Google Scholar
  23. Malik, J. and Perona, P. 1992. Finding boundaries in images. In Neural Networks for Perception, Vol. 1, H. Wechsler (Ed.). Academic Press, pp. 315-344.Google Scholar
  24. Martin, D., Fowlkes, C., Tal, D., and Malik, J. 2000. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Technical Report UCB CSD-01-1133, University of California at Berkeley. overview.html.Google Scholar
  25. McLean, G. 1993. Vector quantization for texture classification. IEEE Transactions on Systems, Man, and Cybernetics, 23(3):637-649.Google Scholar
  26. Montanari, U. 1971. On the optimal detection of curves in noisy pictures. Comm. Ass. Comput., 14:335-345.Google Scholar
  27. Morrone, M. and Burr, D. 1988. Feature detection in human vision: Aphase dependent energy model. Proc. R. Soc. Lond. B, 235:221-245.Google Scholar
  28. Morrone, M. and Owens, R. 1987. Feature detection from local energy. Pattern Recognition Letters, 6:303-313.Google Scholar
  29. Mumford, D. and Shah, J. 1989. Optimal approximations by piecewise smooth functions, and associated variational problems. Comm. Pure Math., 42:577-684.Google Scholar
  30. Parent, P. and Zucker, S. 1989. Trace inference, curvature consistency, and curve detection. IEEE Trans. Pattern Anal. Mach. Intell., 11(8):823-839.Google Scholar
  31. Perona, P. and Malik, J. 1990. Detecting and localizing edges composed of steps, peaks and roofs. In Proc. 3rd Int. Conf. Computer Vision, Osaka, Japan, pp. 52-57.Google Scholar
  32. Puzicha, J., Hofmann, T., and Buhmann, J. 1997. Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 267-272.Google Scholar
  33. Raghu, P., Poongodi, R., and Yegnanarayana, B. 1997. Unsupervised texture classification using vector quantization and deterministic relaxation neural network. IEEE Transactions on Image Processing, 6(10):1376-1387.Google Scholar
  34. Sha'ashua, A. and Ullman, S. 1988. 'Structural saliency: The detection of globally salient structures using a locally connected network. In Proc. 2nd Int. Conf. Computer Vision, Tampa, FL, USA, pp. 321-327.Google Scholar
  35. Shi, J. and Malik, J. 1997. Normalized cuts and image segmentation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 731-737.Google Scholar
  36. Shi, J. and Malik, J. 2000. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 22(8):888-905.Google Scholar
  37. Weiss, Y. 1999. Segmentation using eigenvectors: A unifying view. In Proc. IEEE Intl. Conf. Computer Vision, Vol. 2, Corfu, Greece, pp. 975-982.Google Scholar
  38. Wertheimer, M. 1938. Laws of organization in perceptual forms (partial translation). In A Sourcebook of Gestalt Psychology, W. Ellis (Ed.). Harcourt Brace and Company, pp. 71-88.Google Scholar
  39. Williams, L. and Jacobs, D. 1995. Stochastic completion fields: A neural model of illusory contour shape and salience. In Proc. 5th Int. Conf. Computer Vision, Cambridge, MA, pp. 408-415.Google Scholar
  40. Young, R.A. 1985. The Gaussian derivative theory of spatial vision: Analysis of cortical cell receptive field lineweighting profiles. Technical Report GMR-4920, General Motors Research.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Jitendra Malik
    • 1
  • Serge Belongie
    • 1
  • Thomas Leung
    • 1
  • Jianbo Shi
    • 1
  1. 1.Computer Science DivisionUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations