International Journal of Computer Vision

, Volume 43, Issue 1, pp 29–44 | Cite as

Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons

  • Thomas Leung
  • Jitendra Malik
Article

Abstract

We study the recognition of surfaces made from different materials such as concrete, rug, marble, or leather on the basis of their textural appearance. Such natural textures arise from spatial variation of two surface attributes: (1) reflectance and (2) surface normal. In this paper, we provide a unified model to address both these aspects of natural texture. The main idea is to construct a vocabulary of prototype tiny surface patches with associated local geometric and photometric properties. We call these 3D textons. Examples might be ridges, grooves, spots or stripes or combinations thereof. Associated with each texton is an appearance vector, which characterizes the local irradiance distribution, represented as a set of linear Gaussian derivative filter outputs, under different lighting and viewing conditions.

Given a large collection of images of different materials, a clustering approach is used to acquire a small (on the order of 100) 3D texton vocabulary. Given a few (1 to 4) images of any material, it can be characterized using these textons. We demonstrate the application of this representation for recognition of the material viewed under novel lighting and viewing conditions. We also illustrate how the 3D texton model can be used to predict the appearance of materials under novel conditions.

3D texture texture recognition texture synthesis natural material recognition 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ball, G. and Hall, D. 1967. A clustering technique for summarizing multi-variate data. Behavioral Science, 12:153-155.Google Scholar
  2. Belhumeur, P. and Kriegman, D. 1998. What is the set of images of an object under all possible illumination conditions?. International Journal of Computer Vision, 28(3):245-260.Google Scholar
  3. Burt, P. and Adelson, E. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4):532-540.Google Scholar
  4. Chantler, M. 1994. Towards illuminant invariant texture classification. In Proc. IEE Coll. on Texture Classification: Theory and Applications.Google Scholar
  5. Chantler, M. and McGunnigle, G. 1995. Compensation of illuminant tilt variation for texture classification. In Proceedings Fifth International Conference on Image Processing and its Applications, pp. 767-771.Google Scholar
  6. Chellappa, R. and Chatterjee, S. 1985. Classification of textures using Gaussian Markov random fields. IEEE Transactions on Acoustics, Speech, Signal Processing, 33(4):959-963.Google Scholar
  7. Cross, G. and Jain, A. 1983. Markov random field texture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5(1):25-39.Google Scholar
  8. Dana, K. and Nayar, S. 1998. Histogram model for 3D textures. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 618-624.Google Scholar
  9. Dana, K. and Nayar, S. 1999a. 3D textured surface modelling. In Proceedings Workshop on the Integration of Appearance and Geometric Methods in Object Recognition, pp. 46-56.Google Scholar
  10. Dana, K. and Nayar, S. 1999b. Correlation model for 3D texture. In Proceedings IEEE 7th International Conference on Computer Vision, Vol. 2. Corfu, Greece, pp. 1061-1066.Google Scholar
  11. Dana, K., van Ginneken, B., Nayar, S., and Koenderink, J. 1999. Reflectance and texture of real-world surfaces. ACMTransactions on Graphics, 18(1):1-34.Google Scholar
  12. de Bonet, J. and Viola, P. 1998. Texture recognition using a nonparametric multi-scale statistical model. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 641-647.Google Scholar
  13. Duda, R. and Hart, P. 1973. Pattern Classification and Scene Analysis, John Wiley & Sons. New York, N.Y.Google Scholar
  14. Efros, A. and Leung, T. 1999. Texture synthesis by non-parametric sampling. In Proceedings IEEE 7th International Conference on Computer Vision, Vol. 2. Corfu, Greece, pp. 1033-1038.Google Scholar
  15. Fogel, I. and Sagi, D. 1989. Gabor filters as texture discriminator. Biological Cybernetics, 61:103-113.Google Scholar
  16. Geman, S. and Geman, D. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:721-741.Google Scholar
  17. Georghiades, A., Kriegman, D., and Belhumeur, P. 1998. Illumination cones for recognition under variable lightin: Faces. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 52-58.Google Scholar
  18. Gersho, A. and Gray, R. 1992. Vector Quantization and Signal Compression, Kluwer Academic Publishers: Boston, MA.Google Scholar
  19. Gilks, W., Richardson, S., and Spiegelhalter, D. 1996. Markov Chain Monte Carlo in Practice, Chapman and Hall.Google Scholar
  20. Haddon, J. and Forsyth, D. 1998. Shading primitives: Finding folds and shallow grooves. In Proceedings IEEE 6th International Conference on Computer Vision, Bombay, India, pp. 236-241.Google Scholar
  21. Heeger, D. and Bergen, J. 1995. Pyramid-based texture analysis/ synthesis. In Computer Graphics (SIGGRAPH '95 Proceedings), Los Angeles, CA, pp. 229-238.Google Scholar
  22. Jain, A. and Farrokhsia, F. 1991. Unsupervised texture segmentation using Gabor filters. Pattern Recognition, 24:1167-1186.Google Scholar
  23. Jones, D. and Malik, J. 1992. Computational framework to determining stereo correspondence from a set of linear spatial filters. Image and Vision Computing, 10(10):699-708.Google Scholar
  24. Julesz, B. 1981. Textons, the elements of texture perception, and their interactions. Nature, 290(5802):91-97.Google Scholar
  25. Koenderink, J. and van Doorn, A. 1980. Photometric invariants related to solid shape. Optica Acta, 27(7):981-996.Google Scholar
  26. Koenderink, J. and van Doorn, A. 1996. Illuminance texture due to surface mesostructure. Journal of the Optical Society America A, 13(3):452-463.Google Scholar
  27. Koenderink, J., van Doorn, A. Dana, K. and Nayar, S. 1999. Bidirectional reflection distribution function of thoroughly pitted surfaces. International Journal of Computer Vision, 31(2/3):129-144.Google Scholar
  28. Leung, T. and Malik, J. 1997. On perpendicular texture or: Why dowe see more flowers in the distance?. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 807-813.Google Scholar
  29. Leung, T. and Malik, J. 1999. Recognizing surfaces using three dimensional textons. In Proc. IEEE International Conference on Computer Vision, Corfu, Greece.Google Scholar
  30. MacQueen, J. 1967. Some methods for classification and analysis of multivariate observations. In Proc. Fifth Berkeley Symposium on Math. Stat. and Prob., Vol. I. pp. 281-297.Google Scholar
  31. Malik, J., Belongie, S., Shi, J., and Leung, T. 1999. Textons, contours and regions: Cue integration in image segmentation. In Proceedings IEEE 7th International Conference on Computer Vision, Corfu, Greece, pp. 918-925.Google Scholar
  32. Malik, J. and Perona, P. 1990. Preattentive texture discrimination with early vision mechanisms. Journal of the Optical Society of America A, 7(5):923-932.Google Scholar
  33. Mao, J. and Jain, A. 1992. Texture classification and segmentation using multiresolution simultaneous autoregressive models. Pattern Recognition, 25(2):173-188.Google Scholar
  34. Murase, H. and Nayar, S. 1995. Visual learning and recognition of 3-D objects from appearance. International Journal on Computer Vision, 14(1):5-24.Google Scholar
  35. Press, W., Flannery, B., Teukolsky, S., and Vetterling, W. 1988. Numerical Recipes in C, Cambridge University Press.Google Scholar
  36. Puzicha, J., Hofmann, T., and Buhmann, J. 1997. Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 267-272.Google Scholar
  37. Ripley, B. 1996. Pattern Recognition and Neural Networks, Cambridge University Press.Google Scholar
  38. Rubner, Y. and Tomasi, C. 1999. Texture-based image retrieval without segmentation. In Proceedings IEEE 7th International Conference on Computer Vision, Vol. 2. Corfu, Greece, pp. 1018-1024.Google Scholar
  39. Sebestyen, G. 1962. Pattern recognition by an adaptive process of sample set construction. IRE Trans. Info. Theory, 8:S82-S91.Google Scholar
  40. Shashua, A. 1997. On photometric issues in 3D visual recognition from a single 2D image. International Journal on Computer Vision, 21(1/2).Google Scholar
  41. Sirovitch, L. and Kirby, M. 1987. Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, 2:519-524.Google Scholar
  42. Turk, M. and Pentland, A. 1991. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71-86.Google Scholar
  43. Vaidyanathan, P. 1993. Multirate Systems and Filter Banks, Prentice-Hall: Englewood Cliffs, N.J.Google Scholar
  44. van Ginneken, B., Stavridi, M., and Koenderink, J. 1998. Diffuse and specular reflectance from rough surfaces. Applied Optics, 37(1):130-139.Google Scholar
  45. Yuan, J. and Rao, S. 1993. Spectral estimation for random fields with applications to Markov modeling and texture classification. In Markov Random Fields: Theory and Application, R. Chellappa and A. Jain (Eds.). Academic Press.Google Scholar
  46. Zhu, S., Wu, Y., and Mumford, D. 1998. Filters, random fields and maximum entropy (FRAME): Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2):107-126.Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Thomas Leung
    • 1
  • Jitendra Malik
    • 1
  1. 1.Computer Science DivisionUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations