Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
We study the recognition of surfaces made from different materials such as concrete, rug, marble, or leather on the basis of their textural appearance. Such natural textures arise from spatial variation of two surface attributes: (1) reflectance and (2) surface normal. In this paper, we provide a unified model to address both these aspects of natural texture. The main idea is to construct a vocabulary of prototype tiny surface patches with associated local geometric and photometric properties. We call these 3D textons. Examples might be ridges, grooves, spots or stripes or combinations thereof. Associated with each texton is an appearance vector, which characterizes the local irradiance distribution, represented as a set of linear Gaussian derivative filter outputs, under different lighting and viewing conditions.
Given a large collection of images of different materials, a clustering approach is used to acquire a small (on the order of 100) 3D texton vocabulary. Given a few (1 to 4) images of any material, it can be characterized using these textons. We demonstrate the application of this representation for recognition of the material viewed under novel lighting and viewing conditions. We also illustrate how the 3D texton model can be used to predict the appearance of materials under novel conditions.
- Ball, G. and Hall, D. 1967. A clustering technique for summarizing multi-variate data. Behavioral Science, 12:153-155.
- Belhumeur, P. and Kriegman, D. 1998. What is the set of images of an object under all possible illumination conditions?. International Journal of Computer Vision, 28(3):245-260.
- Burt, P. and Adelson, E. 1983. The laplacian pyramid as a compact image code. IEEE Transactions on Communications, 31(4):532-540.
- Chantler, M. 1994. Towards illuminant invariant texture classification. In Proc. IEE Coll. on Texture Classification: Theory and Applications.
- Chantler, M. and McGunnigle, G. 1995. Compensation of illuminant tilt variation for texture classification. In Proceedings Fifth International Conference on Image Processing and its Applications, pp. 767-771.
- Chellappa, R. and Chatterjee, S. 1985. Classification of textures using Gaussian Markov random fields. IEEE Transactions on Acoustics, Speech, Signal Processing, 33(4):959-963.
- Cross, G. and Jain, A. 1983. Markov random field texture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5(1):25-39.
- Dana, K. and Nayar, S. 1998. Histogram model for 3D textures. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 618-624.
- Dana, K. and Nayar, S. 1999a. 3D textured surface modelling. In Proceedings Workshop on the Integration of Appearance and Geometric Methods in Object Recognition, pp. 46-56.
- Dana, K. and Nayar, S. 1999b. Correlation model for 3D texture. In Proceedings IEEE 7th International Conference on Computer Vision, Vol. 2. Corfu, Greece, pp. 1061-1066.
- Dana, K., van Ginneken, B., Nayar, S., and Koenderink, J. 1999. Reflectance and texture of real-world surfaces. ACMTransactions on Graphics, 18(1):1-34.
- de Bonet, J. and Viola, P. 1998. Texture recognition using a nonparametric multi-scale statistical model. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 641-647.
- Duda, R. and Hart, P. 1973. Pattern Classification and Scene Analysis, John Wiley & Sons. New York, N.Y.
- Efros, A. and Leung, T. 1999. Texture synthesis by non-parametric sampling. In Proceedings IEEE 7th International Conference on Computer Vision, Vol. 2. Corfu, Greece, pp. 1033-1038.
- Fogel, I. and Sagi, D. 1989. Gabor filters as texture discriminator. Biological Cybernetics, 61:103-113.
- Geman, S. and Geman, D. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:721-741.
- Georghiades, A., Kriegman, D., and Belhumeur, P. 1998. Illumination cones for recognition under variable lightin: Faces. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, pp. 52-58.
- Gersho, A. and Gray, R. 1992. Vector Quantization and Signal Compression, Kluwer Academic Publishers: Boston, MA.
- Gilks, W., Richardson, S., and Spiegelhalter, D. 1996. Markov Chain Monte Carlo in Practice, Chapman and Hall.
- Haddon, J. and Forsyth, D. 1998. Shading primitives: Finding folds and shallow grooves. In Proceedings IEEE 6th International Conference on Computer Vision, Bombay, India, pp. 236-241.
- Heeger, D. and Bergen, J. 1995. Pyramid-based texture analysis/ synthesis. In Computer Graphics (SIGGRAPH '95 Proceedings), Los Angeles, CA, pp. 229-238.
- Jain, A. and Farrokhsia, F. 1991. Unsupervised texture segmentation using Gabor filters. Pattern Recognition, 24:1167-1186.
- Jones, D. and Malik, J. 1992. Computational framework to determining stereo correspondence from a set of linear spatial filters. Image and Vision Computing, 10(10):699-708.
- Julesz, B. 1981. Textons, the elements of texture perception, and their interactions. Nature, 290(5802):91-97.
- Koenderink, J. and van Doorn, A. 1980. Photometric invariants related to solid shape. Optica Acta, 27(7):981-996.
- Koenderink, J. and van Doorn, A. 1996. Illuminance texture due to surface mesostructure. Journal of the Optical Society America A, 13(3):452-463.
- Koenderink, J., van Doorn, A. Dana, K. and Nayar, S. 1999. Bidirectional reflection distribution function of thoroughly pitted surfaces. International Journal of Computer Vision, 31(2/3):129-144.
- Leung, T. and Malik, J. 1997. On perpendicular texture or: Why dowe see more flowers in the distance?. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 807-813.
- Leung, T. and Malik, J. 1999. Recognizing surfaces using three dimensional textons. In Proc. IEEE International Conference on Computer Vision, Corfu, Greece.
- MacQueen, J. 1967. Some methods for classification and analysis of multivariate observations. In Proc. Fifth Berkeley Symposium on Math. Stat. and Prob., Vol. I. pp. 281-297.
- Malik, J., Belongie, S., Shi, J., and Leung, T. 1999. Textons, contours and regions: Cue integration in image segmentation. In Proceedings IEEE 7th International Conference on Computer Vision, Corfu, Greece, pp. 918-925.
- Malik, J. and Perona, P. 1990. Preattentive texture discrimination with early vision mechanisms. Journal of the Optical Society of America A, 7(5):923-932.
- Mao, J. and Jain, A. 1992. Texture classification and segmentation using multiresolution simultaneous autoregressive models. Pattern Recognition, 25(2):173-188.
- Murase, H. and Nayar, S. 1995. Visual learning and recognition of 3-D objects from appearance. International Journal on Computer Vision, 14(1):5-24.
- Press, W., Flannery, B., Teukolsky, S., and Vetterling, W. 1988. Numerical Recipes in C, Cambridge University Press.
- Puzicha, J., Hofmann, T., and Buhmann, J. 1997. Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, pp. 267-272.
- Ripley, B. 1996. Pattern Recognition and Neural Networks, Cambridge University Press.
- Rubner, Y. and Tomasi, C. 1999. Texture-based image retrieval without segmentation. In Proceedings IEEE 7th International Conference on Computer Vision, Vol. 2. Corfu, Greece, pp. 1018-1024.
- Sebestyen, G. 1962. Pattern recognition by an adaptive process of sample set construction. IRE Trans. Info. Theory, 8:S82-S91.
- Shashua, A. 1997. On photometric issues in 3D visual recognition from a single 2D image. International Journal on Computer Vision, 21(1/2).
- Sirovitch, L. and Kirby, M. 1987. Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, 2:519-524.
- Turk, M. and Pentland, A. 1991. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71-86.
- Vaidyanathan, P. 1993. Multirate Systems and Filter Banks, Prentice-Hall: Englewood Cliffs, N.J.
- van Ginneken, B., Stavridi, M., and Koenderink, J. 1998. Diffuse and specular reflectance from rough surfaces. Applied Optics, 37(1):130-139.
- Yuan, J. and Rao, S. 1993. Spectral estimation for random fields with applications to Markov modeling and texture classification. In Markov Random Fields: Theory and Application, R. Chellappa and A. Jain (Eds.). Academic Press.
- Zhu, S., Wu, Y., and Mumford, D. 1998. Filters, random fields and maximum entropy (FRAME): Towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2):107-126.
- Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons
International Journal of Computer Vision
Volume 43, Issue 1 , pp 29-44
- Cover Date
- Print ISSN
- Online ISSN
- Kluwer Academic Publishers
- Additional Links
- 3D texture
- texture recognition
- texture synthesis
- natural material recognition
- Industry Sectors