International Journal of Computer Vision

, Volume 51, Issue 2, pp 91–109 | Cite as

Dynamic Textures

  • Gianfranco Doretto
  • Alessandro Chiuso
  • Ying Nian Wu
  • Stefano Soatto


Dynamic textures are sequences of images of moving scenes that exhibit certain stationarity properties in time; these include sea-waves, smoke, foliage, whirlwind etc. We present a characterization of dynamic textures that poses the problems of modeling, learning, recognizing and synthesizing dynamic textures on a firm analytical footing. We borrow tools from system identification to capture the “essence” of dynamic textures; we do so by learning (i.e. identifying) models that are optimal in the sense of maximum likelihood or minimum prediction error variance. For the special case of second-order stationary processes, we identify the model sub-optimally in closed-form. Once learned, a model has predictive power and can be used for extrapolating synthetic sequences to infinite length with negligible computational cost. We present experimental evidence that, within our framework, even low-dimensional models can capture very complex visual phenomena.

textures dynamic scene analysis 3D textures minimum description length image compression generative model prediction error methods ARMA model subspace system identification canonical correlation learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amari, S. and Cardoso, J. 1997. Blind source separation-semiparametric statistical approach. IEEE Transactions on Signal Processing, 45:692–700.Google Scholar
  2. Arun, K.S. and Kung, S.Y. 1990. Balanced approximation of stochastic systems. SIAM Matrix Analysis and Applications, 42– 68.Google Scholar
  3. Bar-Joseph, Z., El-Yaniv, R., Lischinski, D., and Werman, M. 2001. Texture mixing and texture movie synthesis using statistical learning. IEEE Transactions on Visualization and Computer Graphics, 7(2):120–135.Google Scholar
  4. Barzel, R. 1992. Physically-Based Modeling for Computer Graphics: A Structured Approach. Academic Press, San Diego, CA.Google Scholar
  5. Bauer, D., Deistler, M., and Scherrer, W. 1999. Consistency and asymptotic normality of some subspace algorithms for systems without observed inputs. Automatica, 35(7):1243–1254.Google Scholar
  6. Bigun, J. and du Buf, J.M. 1994. N-folded symmetries by complex moments in gabor space and their application to unsupervised texture segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(1):80–87.Google Scholar
  7. Bissacco, A., Chiuso, A., Ma, Y., and Soatto, S. 2001. Recognition of human gaits. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HA.Google Scholar
  8. Blake, A. and Isard, M. 1998. Active Contours. Springer Verlag: Berlin.Google Scholar
  9. Chiuso, A. and Picci, G. To appear. Asymptotic variance of subspace techniques. In Journal of Econometrics.Google Scholar
  10. Cross, G. and Jain, A. 1983. Markov random field texture models.IEEE Transactions on Pattern Analysis and Machine Intelligence, 5:25–40.Google Scholar
  11. de Bonet, J. and Viola, P. 1998. Multiresolution sampling procedure for analysis and synthesis of texture images. In Proceedings IEEE Conf. Computer Vision and Pattern Recognition.Google Scholar
  12. Dempster, A.P., Laird, N.M., and Rubin, D.B. 1977. Maximum likelihood from incomplete data via the EM algorithm J. R. Statist. Soc. B, 39:185–197.Google Scholar
  13. Doretto, G. and Soatto, S. 2002. Editable dynamic textures. Technical Report 020001, UCLA Computer Science Department.Google Scholar
  14. Ebert, D., Carlson, W., and Parent, R. 1994. Solid spaces and inverse particle systems for controlling the animation of gases and fluids. The Visual Computer, 10(4):179–190.Google Scholar
  15. Efros, A. and Leung, T. 1999. Texture synthesis by non-parametric sampling. In Seventh International Conference on Computer Vision, Corfu, Greece.Google Scholar
  16. Fitzgibbon, A.W. 2001. Stochastic rigidity: Image registration for nowhere-static scenes. In Proc. of the Intl. Conf. on Comp. Vis., pp. 662–670.Google Scholar
  17. Fournier, A. and Reeves, W. 1986. A simple model of ocean waves. In ACM SIGGRAPH Proceedings, pp. 75–84.Google Scholar
  18. Freeman, W. and Adelson, E. 1991. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9):891–906.Google Scholar
  19. Golub, G. and Van Loan, C. 1989. Matrix Computations, 2nd edn. Johns Hopkins University Press: Baltimore, MD.Google Scholar
  20. Hassner, M. and Sklansky, J. 1981. The use of markov random fields as models of texture. In Image Modeling. Academic Press: San Diego, CA.Google Scholar
  21. Heeger, D. and Bergen, J. 1995. Pyramid-based texture analysis /synthesis. In ACM SIGGRAPH Conference Proceedings.Google Scholar
  22. Hodgins, J.K. and Wooten, W.L. 1998. Animating human athletes. In Robotics Research: The Eighth International Symposium, Y. Shirai and S. Hirose (Eds.). Springer-Verlag: Berlin, Germany, pp. 356–367.Google Scholar
  23. Hunt, R.W. 1996. The Reproduction of Colour, 5th edn. Fisher Books.Google Scholar
  24. Julesz, B. 1962. Visual pattern discrimination. IRE Trans. Info Theory, IT-8.Google Scholar
  25. Kailath, T. 1980. Linear Systems. Prentice Hall: Englewood Cliffs, NJ.Google Scholar
  26. Kirsch, A. 1996. An Introduction to the Mathematical Theory of Inverse Problems. Springer-Verlag: New York.Google Scholar
  27. Lindquist, A. and Picci, G. 1979. The stochastic realization problem. SIAM J. Control Optim, 17:365–389.Google Scholar
  28. Liu, J., Chen, R., and Logvinenko, T. 2000. A theoretical framework for sequential importance sampling and resampling. Technical Report, Stanford University, Department of Statistics.Google Scholar
  29. Ljung, L. 1987. System Identification—Theory for the User. Prentice Hall: Englewood Cliffs, NJ.Google Scholar
  30. Mallat, S. 1989. A theory of multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11:674–693.Google Scholar
  31. Mumford, D. and Gidas, B. 1998. Stochastic models for generic images. Technical Report, Division of Applied Mathematics, Brown University.Google Scholar
  32. Nelson, R.C. and Polana, R. 1992. Qualitative recognition of motion using temporal texture. Computer Vision, Graphics, and Image Processing. Image Understanding, 56(1):78–89.Google Scholar
  33. Paget, R. and Longstaff, D. 1996. A nonparametric multiscale markov random field model for synthesising natural textures. In Fourth International Symposium on Signal Processing and its Applications, 2:744–747.Google Scholar
  34. Peachey, D. 1986. Modeling waves and surf. In SIGGRAPH Conference Proceedings, pp. 65–74.Google Scholar
  35. Popat, K. and Picard, R. 1993. Novel cluster-based probability model for texture synthesis, classification, and compression. In Proceedings SPIE Visual Communications and Image Processing, Boston.Google Scholar
  36. Portilla, J. and Simoncelli, E. 1999. Texture representation and synthesis using correlation of complex wavelet coefficient magnitudes. In CSIC, Madrid.Google Scholar
  37. Reeves, W. 1983. Particle systems: A technique for modeling a class of fuzzy objects. ACM Trans. Graphics, 2:91–108.Google Scholar
  38. Rissanen, J. 1978. Modeling by shortest data description. Automatica, 14:465–471.Google Scholar
  39. Saisan, P., Doretto, G., Wu, Y.N., and Soatto, S. 2001. Dynamic texture recognition. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HA.Google Scholar
  40. Schödl, A., Szeliski, R., Salesin, D., and Essa, I. 2000.Video textures. In Proceedings of ACM SIGGRAPH Conference, New Orleans, LA.Google Scholar
  41. Simoncelli, E., Freeman, W., Adelson, E., and Heeger, D. 1992.Shiftable multi-scale transforms. IEEE Trans. Information Theory, 38(2):587–607.Google Scholar
  42. Sims, K. 1990. Particle animation and rendering using data parallel computation. Computer Graphics, 24(4):405–413.Google Scholar
  43. Stam, J. and Fiume, E. 1995. Depicting fire and other gaseous phenomena using diffusion processes. InSIGGRAPHConference Proceedings, pp. 129–136.Google Scholar
  44. Szummer, M. and Picard, R.W. 1996. Temporal texture modeling. In IEEE International Conference on Image Processing, Lausanne, Switzerland, vol. 3.Google Scholar
  45. Van Overschee, P. and De Moor, B. 1993. Subspace algorithms for the stochastic identification problem. Automatica, 29:649– 660.Google Scholar
  46. Van Overschee, P. and De Moor, B. 1994. N4sid: Subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica, 30:75–93.Google Scholar
  47. Wei, L.Y. and Levoy, M. 2000. Fast texture synthesis using tree structured vector quantization. In SIGGRAPH Conference Proceedings.Google Scholar
  48. Zhu, S., Wu, Y., and Mumford, D. 1997. Minimax entropy principle and its application to texture modeling. Neural Computation, 9:1627–1660.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Gianfranco Doretto
    • 1
  • Alessandro Chiuso
    • 2
  • Ying Nian Wu
    • 3
  • Stefano Soatto
    • 1
  1. 1.Computer Science DepartmentUniversity of CaliforniaLos Angeles
  2. 2.Dipartimento di Ingegneria dell'InformazioneUniversità di PadovaItaly
  3. 3.Statistics DepartmentUniversity of CaliforniaLos Angeles

Personalised recommendations