International Journal of Computer Vision

, Volume 40, Issue 1, pp 49–70 | Cite as

A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients

  • Javier Portilla
  • Eero P. Simoncelli
Article

Abstract

We present a universal statistical model for texture images in the context of an overcomplete complex wavelet transform. The model is parameterized by a set of statistics computed on pairs of coefficients corresponding to basis functions at adjacent spatial locations, orientations, and scales. We develop an efficient algorithm for synthesizing random images subject to these constraints, by iteratively projecting onto the set of images satisfying each constraint, and we use this to test the perceptual validity of the model. In particular, we demonstrate the necessity of subgroups of the parameter set by showing examples of texture synthesis that fail when those parameters are removed from the set. We also demonstrate the power of our model by successfully synthesizing examples drawn from a diverse collection of artificial and natural textures.

textur modeling texture synthesis non Gaussian statstics Markov random field altering projections Julesz conjecture 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anderson, C.H. and Langer, W.D. 1997. Statistical models of image texture. Technical Report, Washington U. Medical School. Available at ftp://shifter.wustl.edu/pub/.Google Scholar
  2. Bell, A.J. and Sejnowski, T.J. 1997. The ‘independent components’ of natural scenes are edge filters. Vision Research, 37(23):3327–3338.Google Scholar
  3. Bergen, J.R. and Adelson, E.H. 1986. Visual texture segmentation based on energy measures. J. Opt. Soc. Am. A, 3:99.Google Scholar
  4. Bergen, J.R. and Landy, M.S. 1991. “Computational modeling of visual texture segregation: Computational models of visual processing. M.S. Landy and J.A. Morshon (Eds.). MIT Press, Cambridge, MA, pp. 253–271.Google Scholar
  5. Bouman, C.A. and Shapiro, M. 1994. A multiscale random field model for Bayesian image segmentation. IEEE Trans. Image Proc., 3(2).Google Scholar
  6. Bovik, A.C., Clark, M., and Geisler, W.S. 1990. Multichannel texture analysis using localized spatial filters. IEEE Pat. Anal. Mach. Intell., 12(1):55–73.Google Scholar
  7. Bovik, A.C., Clark, M., and Geisler, W.S. 1992. Localized measurements of emergent image frequencies by Gabor wavelets. IEEE Pat. Anal. Mach. Intell., 38:691–712.Google Scholar
  8. Brodatz, P. 1996. Textures: A Photographic Album for Artists and Designers. Dover: New YorkGoogle Scholar
  9. Buccigrossi, R.W. and Simoncelli, E.P. 1999. Image compression via joint statistical characterization in the wavelet domain. IEEE Trans. Image Proc., 8(12):1688–1701.Google Scholar
  10. Cadzow, J.A., Wilkes, D.M., Peters, R.A., II, and Li, X. 1993. Image texture synthesis-by-analysis usingmoving-average models. IEEE Trans on Aerospace and Electrical Systems, 29(4):1110–1122.Google Scholar
  11. Caelli,.M. and Julesz, B. 1978. Experiments in the visual perception of texture. Biol. Cybernetics, 28:167–175.Google Scholar
  12. Cano, D. and Minh, T.H. 1988. Texture synthesis using hierarchical linear transforms. Signal Processing, 15:131–148.Google Scholar
  13. Chen, P.C. and Pavlidis, T. 1983. Segmentation by texture using correlation. IEEE Pat. Anal. Mach. Intell., 5(1):64–69.Google Scholar
  14. Cross, G. and Jain, A. 1983. Markov random field texture models. IEEE Trans. PAMI, 5:25–39.Google Scholar
  15. Daubechies, I. 1988. Orthonormal bases of compactly supported wavelets. Comm. on Pure and Appl. Math., 41:909–996.Google Scholar
  16. Daugman. J.G. 1988. Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression. IEEE Trans. Acoust. Speech Signal Proc., 36(7):1169–1179.Google Scholar
  17. Daugman, J.G. 1989. Entropy reduction and decorrelation in visual coding by oriented neural receptive fields. IEEE Trans. Biomedical Engineering, 36(1):107–114.Google Scholar
  18. Daugman, J.G. and Kammen, D.M. 1986. Pure orientation filtering: A scale-invariant image-processing tool for perception research and data compression. Behavior Research Methods, Instruments, & Computers, 18(6):559–564.Google Scholar
  19. De Bonet, J. and Viola, P. 1997. A non-parametric multi-scale statistical model for natural images. In Adv. in Neural Info Processing, Vol. 9. MIT Press.Google Scholar
  20. Derin, H. and Elliott, H. 1987. Modeling and segmentation of noisy and textured images using Gibbs random fields. IEEE Pat. Anal. Mach. Intell., 9(1):39–55.Google Scholar
  21. Diaconis, P. and Freedman, D. 1981. On the statistics of vision: The Julesz conjecture. J. Math. Psychol., 24:112–118.Google Scholar
  22. Efros, A.A. and Leung, T.K. 1999. Texture synthesis by nonparameteric sampling. In Proc. Int'l Conference on Computer Vision, Corfu.Google Scholar
  23. Faugeras, O.D. and Pratt, W.K. 1980. Decorrelation methods of texture feature extraction. IEEE Pat. Anal. Mach. Intell., 2(4):323–332.Google Scholar
  24. Field, D.J. 1987. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A, 4(12):2379–2394.Google Scholar
  25. Francos, J.M., Meiri, A.Z., and Porat, B. 1993. A unified texture model based on a 2-DWold-like decomposition. IEEE Trans. Signal Proc., 41(8):2665–2678.Google Scholar
  26. Gagalowicz, A. 1981. A new method for texture fields synthesis: Some applications to the study of human vision. IEEE Pat. Anal. Mach. Intell., 3(5):520–533.Google Scholar
  27. Geman, S. and Geman, D. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Pat. Anal. Mach. Intell., 6:721–741.Google Scholar
  28. Graham, N. 1989. Visual Pattern Analyzers. Oxford University Press: New York.Google Scholar
  29. Hassner, M. and Sklansky, J. 1980. The use of Markov random fields as models of texture. Comp. Graphics Image Proc., 12:357–370.Google Scholar
  30. Heeger, D. and Bergen, J. 1995. Pyramid-based texture analysis/synthesis. In Proc. ACM SIGGRAPH.Google Scholar
  31. Hertzmann, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proc. ACM SigGraph, pp. 453–460.Google Scholar
  32. Hirani, A.N. and Totsuka, T. 1996. Combining frequency and spatial domain information for fast interactive image noise removal. In ACM SIGGRAPH, pp. 269–276.Google Scholar
  33. Jaynes, E.T. 1957. Information theory and statistical mechanics. Phys. Rev., 106:620–630.Google Scholar
  34. Jaynes, E.T. 1978. Where do we stand on maximum entropy? In The Maximal Entropy Formalism, R.D. Levine and M. Tribus (Eds.). MIT Press: Cambridge, MA.Google Scholar
  35. Julesz, B. 1962. Visual pattern discrimination. IRE Trans. Info Theory, IT-8:84–92.Google Scholar
  36. Julesz, B. 1981. Textons, the elements of texture perception and their interactions. Nature, 290:91–97.Google Scholar
  37. Julesz, B., Gilbert, E.N., Shepp, L.A., and Frisch, H.L. 1973. Inability of humans to discriminate between visual textures that agree in second-order statistics—revisited. Perception, 2:391–405.Google Scholar
  38. Julesz, B., Gilbert, E.N., and Victor, J.D. 1978. Visual discrimination of textures with identical third-order statistics. Biol. Cybernetics, 31:137–140.Google Scholar
  39. Kersten, D. 1987. Predictability and redundancy of natural images. J. Opt. Soc. Am. A, 4(12):2395–2400.Google Scholar
  40. Knutsson, H. and Granlund, G.H. 1983. Texture analysis using twodimensional quadrature filters. In Workshop on Computer Architecture for Pattern Analysis and Image Database Management, IEEE Computer Society, pp. 206–213.Google Scholar
  41. Malik, J. and Perona, J. 1990. Preattentive texture discrimination with early vision mechanisms. J. Opt. Soc. Am. A, 7:923–932.Google Scholar
  42. Mallat, S.G. 1989. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Pat. Anal. Mach. Intell., 11:674–693.Google Scholar
  43. Manduchi, R. and Portilla, J. 1999. Independent component analysis of textures. In Proc. Int'l Conference on Computer Vision, Corfu.Google Scholar
  44. Olshausen, B.A. and Field, D.J. 1996. Natural image statistics and efficient coding. Network: Computation in Neural Systems, 7:333–339.Google Scholar
  45. Perona, P. and Malik, J. 1990. Detecting and localizing edges composed of steps, peaks and roofs. In Proc. 3rd Intl. Conf. Computer Vision, Osaka, Japan.Google Scholar
  46. Popat, K. and Picard, R.W. 1993. Novel cluster-based probability model for texture synthesis, classification, and compression. In Proc. SPIE Vis Comm., Cambridge, MA.Google Scholar
  47. Popat, K. and Picard, R.W. 1997. Cluster-based probability model and its application to image and texture processing. IEEE Trans. Im. Proc., 6(2):268–284.Google Scholar
  48. Porat, M. and Zeevi, Y.Y. 1989. Localized texture processing in vision: Analysis and synthesis in Gaborian space. IEEE Trans. Biomedical Eng., 36(1):115–129.Google Scholar
  49. Portilla, J., Navarro, R., Nestares, O., and Tabernero, A. 1996. Texture synthesis-by-analysis based on a multiscale early-vision model. Optical Engineering, 35(8):2403–2417.Google Scholar
  50. Portilla, J. and Simoncelli, E. 1999. Texture modeling and synthesis using joint statistics of complex wavelet coefficients. In IEEE Workshop on Statistical and Computational Theories of Vision, Fort Collins, CO. Available at http://www.cis.ohiostate.edu/~szhu/SCTV99.html.Google Scholar
  51. Portilla, J. and Simoncelli, E. 2000. Image denoising via adjustment ofwavelet coefficient magnitude correlation. In Seventh IEEE Int'l Conf. on Image Proc., Vancouver, September 10– 13. IEEE Computer Society.Google Scholar
  52. Pratt, W.K., Faugeras, O.D., and Gagolowicz, A. 1978. Visual discrimination of stochastic texture fields. IEEE Trans. on Systems Man and Cybernetics, 8:796–804.Google Scholar
  53. Reed, T.R. and Wechsler, H. 1990. Segmentation of textured images and Gestalt organization using spatial/spatial-frequency representations. IEEE Pat. Anal. Mach. Intell., 12(1):1–12.Google Scholar
  54. Ruderman, D.L. and Bialek, W. 1994. Statistics of natural images: Scaling in the woods. Phys. Rev. Letters, 73(6):814–817.Google Scholar
  55. Simoncelli, E.P. 1997. Statistical models for images: Compression, restoration and synthesis. In 31st Asilomar Conf. on Signals, Systems and Computers, Pacific Grove, CA, November 1997. IEEE Computer Society, pp. 673–678.Google Scholar
  56. Simoncelli, E.P. and Freeman, W.T. 1995. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In Second Int'l Conf. on Image Proc., Washington, DC, October 1995. Vol. III, IEEE Sig. Proc. Society, pp. 444–447.Google Scholar
  57. Simoncelli, E.P., Freeman, W.T., Adelson, E.H., and Heeger, D.J. 1992. Shiftable multi-scale transforms. IEEE Trans Information Theory, 38(2):587–607. Special Issue on Wavelets.Google Scholar
  58. Simoncelli, E. and Portilla, J. 1998. Texture characterization via joint statistics of wavelet coefficient magnitudes. In Fifth IEEE Int'l Conf. on Image Proc., Chicago, October 4– 7, Vol. I. IEEE Computer Society.Google Scholar
  59. Tabernero, A., Portilla, J., and Navarro, R. 1999. Duality of logpolar image representations in the space and the spatial-frequency domains. IEEE Trans. on Signal Processing, 47(9):2469–2479.Google Scholar
  60. Turner, M.R. 1986. Texture discrimination by Gabor functions. Biol. Cybern., 55:71–82.Google Scholar
  61. Victor, J.D. 1994. Images, statistics and textures: Implications of triple correlation uniqueness for texture statistics and the Julesz conjecture: Comment. J. Opt. Soc. Am. A, 11(5):1680–1684.Google Scholar
  62. VisTex: An online collection of visual textures. MIT Media Laboratory, 1995. Available from http://www-white.media.mit.edu/vismod/imagery/VisionTexture/vistex.html.Google Scholar
  63. Wainwright, M.J. and Simoncelli, E.P. 2000. Scale mixtures of Gaussians and the statistics of natural images. In Adv. Neural Information Processing Systems, Vol. 12, S.A. Solla, T.K. Leen, and K.-R. Müller (Eds.). MIT Press: Cambridge, MA, pp. 855–861. Presented at Neural Information Processing Systems, Dec. 1999.Google Scholar
  64. Watson, A.B. 1987. Efficiency of a model human image code. J. Opt. Soc. Am. A, 12:2401–2417.Google Scholar
  65. Yellott, J.I. 1993. Images, statistics and textures: Implications of triple correlation uniqueness for texture statistics and the Julesz conjecture. J. Opt. Soc. Am. A, 10(5):777–793.Google Scholar
  66. Youla, D.C. 1978. Generalized image restoration by the method of alternating orthogonal projections. IEEE Trans. Circuits and Systems, 25:694–702.Google Scholar
  67. Youla, D.C. and Webb, H. 1982. Image restoration by the method of convex projections. IEEE Trans. Med. Imaging, 1:81–101.Google Scholar
  68. Zetzsche, C., Wegmann, B., and Bart, E. 1993. Nonlinear aspects of primary vision: Entropy reduction beyond decorrelation. In Int'l Symposium, Society for Information Display, Vol. XXIV, pp. 933–936.Google Scholar
  69. Zhu, S.C., Liu, X., and Wu, Y.N. 1999. Exploring the Julesz ensemble by efficient Markov chain Monte Carlo. In IEEEWorkshop on Statistical and Computational Theories of Vision, Fort Collins, CO. Available at http://www.cls.ohio-state.edu/~szhu/SCTV99.html.Google Scholar
  70. Zhu, S., Wu, Y.N., and Mumford, D. 1996. Filters, random fields and maximum entropy (FRAME)—Towards the unified theory for texture modeling. In IEEE Conf. Computer Vision and Pattern Recognition, 693–696.Google Scholar
  71. Zhu, S.C., Wu, Y.N., and Mumford, D. 1997. Minimax entropy principle and its application to texture modeling. In Neural Computation, Vol. 9, pp. 1627–1660.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Javier Portilla
    • 1
  • Eero P. Simoncelli
    • 1
  1. 1.Center for Neural Science, and Courant Institute of Mathematical SciencesNew York UniversityNew YorkUSA

Personalised recommendations