Adaptive Structure Tensors and their Applications

  • Thomas Brox
  • Rein van den Boomgaard
  • François Lauze
  • Joost van de Weijer
  • Joachim Weickert
  • Pavel Mrázek
  • Pierre Kornprobst
Part of the Mathematics and Visualization book series (MATHVISUAL)


The structure tensor, also known as second moment matrix or Förstner interest operator, is a very popular tool in image processing. Its purpose is the estimation of orientation and the local analysis of structure in general. It is based on the integration of data from a local neighborhood. Normally, this neighborhood is defined by a Gaussian window function and the structure tensor is computed by the weighted sum within this window. Some recently proposed methods, however, adapt the computation of the structure tensor to the image data. There are several ways how to do that. This chapter wants to give an overview of the different approaches, whereas the focus lies on the methods based on robust statistics and nonlinear diffusion. Furthermore, the data-adaptive structure tensors are evaluated in some applications. Here the main focus lies on optic flow estimation, but also texture analysis and corner detection are considered.


Gradient Vector Structure Tensor Robust Statistic Orientation Estimation Dominant Orientation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    F. Andreu, C. Ballester, V. Caselles, and J. M. Mazón. Minimizing total variation flow. Differential and Integral Equations, 14(3):321–360, March 2001.MathSciNetzbMATHGoogle Scholar
  2. 2.
    P. Bakker, L.J. van Vliet, and P.W. Verbeek. Edge preserving orientation adaptive filtering. In M. Boasson, J.A. Kaandorp, J.F.M. Tonino, and M.G. Vosselman, editors, ASCI’99, Proc. 5th Annual Conference of the Advanced School for Computing and Imaging, pp. 207–213, 1999.Google Scholar
  3. 3.
    J. L. Barron, D. J. Fleet, and S. S. Beauchemin. Performance of optical flow techniques. International Journal of Computer Vision, 12(1):43–77, February 1994.CrossRefGoogle Scholar
  4. 4.
    J. Bigün, G. H. Granlund, and J. Wiklund. Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(8):775–790, August 1991.CrossRefGoogle Scholar
  5. 5.
    M. J. Black and P. Anandan. The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104, January 1996.CrossRefGoogle Scholar
  6. 6.
    T. Brox, M. Rousson, R. Deriche, and J. Weickert. Unsupervised segmentation incorporating colour, texture, and motion. Technical Report 4760, INRIA Sophia-Antipolis, France, March 2003.Google Scholar
  7. 7.
    T. Brox, J. Weickert, B. Burgeth, and P. Mrázek. Nonlinear structure tensors. Technical report, Dept. of Mathematics, Saarland University, Saarbrüucken, Germany, July 2004.Google Scholar
  8. 8.
    S. Di Zenzo. A note on the gradient of a multi-image. Computer Vision, Graphics and Image Processing, 33:116–125, 1986.CrossRefGoogle Scholar
  9. 9.
    W. Förstner. A feature based corresponding algorithm for image matching. International Archive of Photogrammetry and Remote Sensing, 26:150–166, 1986.Google Scholar
  10. 10.
    W. Förstner and E. Gülch. A fast operator for detection and precise location of distinct points, corners and centres of circular features. In Proc. ISPRS Intercommission Conference on Fast Processing of Photogrammetric Data, pp. 281–305, Interlaken, Switzerland, June 1987.Google Scholar
  11. 11.
    B. Galvin, B. McCane, K. Novins, D. Mason, and S. Mills. Recovering motion fields: an analysis of eight optical flow algorithms. In Proc. 1998 British Machine Vision Conference, Southampton, England, September 1998.Google Scholar
  12. 12.
    C. G. Harris and M. Stephens. A combined corner and edge detector. In Proc. Fouth Alvey Vision Conference, pp. 147–152, Manchester, England, August 1988.Google Scholar
  13. 13.
    H. Haußecker and H. Spies. Motion. In B. Jähne, H. Haußecker, and P. Geißler, editors, Handbook on Computer Vision and Applications. Vol. 2: Signal Processing and Pattern Recognition, pp. 309–396. Academic Press, San Diego, 1999.Google Scholar
  14. 14.
    B. Horn and B. Schunck. Determining optical flow. Artificial Intelligence, 17:185–203, 1981.CrossRefGoogle Scholar
  15. 15.
    B. Jähne. Spatio-Temporal Image Processing, volume 751 of Lecture Notes in Computer Science. Springer, Berlin, 1993.zbMATHGoogle Scholar
  16. 16.
    M. Kass and A. Witkin. Analyzing oriented patterns. Computer Graphics and Image Processing, 37:363–385, 1987.Google Scholar
  17. 17.
    U. Köthe. Edge and junction detection with an improved structure tensor. In B. Michaelis and G. Krell, editors, Pattern Recognition. 25th DAGM Symposium, number 2781 in Lecture Notes in Computer Science, pp. 25–32, Berlin, 2003. Springer.Google Scholar
  18. 18.
    M. Kuwahara, K. Hachimura, S. Eiho, and M. Kinoshita. Processing of riangiocardiographic images. In K. Preston and M. Onoe, editors, Digital Processing of Biomedical Images, pp. 187–202, 1976.Google Scholar
  19. 19.
    F. Lauze, P. Kornprobst, C. Lenglet, R. Deriche, and M. Nielsen. About some optical flow methods from structure tensors: review and contribution. In RFIA 2004, Actes du 14e Congrès Francophone AFRIF-AFIA, volume 1, pp. 283–292, Toulouse, January 2004. LAAS-CNRS. In French.Google Scholar
  20. 20.
    T. Lindeberg and J. Garding. Shape from texture from a multi-scale perspective. In Proc. 4th International Conference on Computer Vision, pp. 683–691, Berlin, Germany, May 1993.Google Scholar
  21. 21.
    B. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In Proc. Seventh International Joint Conference on Artificial Intelligence, pp. 674–679, Vancouver, Canada, August 1981.Google Scholar
  22. 22.
    M. Middendorf and H.-H. Nagel. Estimation and interpretation of discontinuities in optical flow fields. In Proc. Eighth International Conference on Computer Vision, volume 1, pp. 178–183, Vancouver, Canada, July 2001. IEEE Computer Society Press.CrossRefGoogle Scholar
  23. 23.
    M. Middendorf and H.-H. Nagel. Empirically convergent adaptive estimation of grayvalue structure tensors. In L. van Gool, editor, Proc. 24th DAGM Symposium, volume 2449 of Lecture Notes in Computer Science, pp. 66–74, Zürich, Switzerland, September 2002. Springer.Google Scholar
  24. 24.
    P. Mrázek, J. Weickert, and A. Bruhn. On robust estimation and smoothing with spatial and tonal kernels. Technical Report 51, Series SPP-1114, Department of Mathematics, University of Bremen, Germany, June 2004.Google Scholar
  25. 25.
    M. Nagao and T. Matsuyama. Edge preserving smoothing. Computer Graphics and Image Processing, 9(4):394–407, April 1979.CrossRefGoogle Scholar
  26. 26.
    H.-H. Nagel and A. Gehrke. Spatiotemporally adaptive estimation and segmentation of OF-fields. In H. Burkhardt and B. Neumann, editors, Computer Vision — ECCV’ 98, volume 1407 of Lecture Notes in Computer Science, pp. 86–102. Springer, Berlin, 1998.Google Scholar
  27. 27.
    P. Perona and J. Malik. Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12:629–639, 1990.CrossRefGoogle Scholar
  28. 28.
    A. R. Rao and B. G. Schunck. Computing oriented texture fields. CVGIP: Graphical Models and Image Processing, 53:157–185, 1991.CrossRefGoogle Scholar
  29. 29.
    K. Rohr. Modelling and identification of characteristic intensity variations. Image and Vision Computing, 10(2):66–76, 1992.MathSciNetCrossRefGoogle Scholar
  30. 30.
    K. Rohr. Localization properties of direct corner detectors. Journal of Mathematical Imaging and Vision, 4:139–150, 1994.MathSciNetCrossRefGoogle Scholar
  31. 31.
    M. Rousson, T. Brox, and R. Deriche. Active unsupervised texture segmentation on a diffusion based feature space. In Proc. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 699–704, Madison, WI, June 2003.Google Scholar
  32. 32.
    L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60:259–268, 1992.zbMATHCrossRefGoogle Scholar
  33. 33.
    S. M. Smith and J. M. Brady. SUSAN — a new approach to low level image processing. International Journal of Computer Vision, 23(1):45–78, May 1997.CrossRefGoogle Scholar
  34. 34.
    I. Thomas. Anisotropic adaptation and structure detection. Technical Report F11, Institute for Applied Mathematics, University of Hamburg, Germany, August 1999.Google Scholar
  35. 35.
    D. Tschumperlé and R. Deriche. Diffusion tensor regularization with contraints preservation. In Proc. 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 1, pp. 948–953, Kauai, HI, December 2001. IEEE Computer Society Press.Google Scholar
  36. 36.
    R. van den Boomgaard. The Kuwahara-Nagao operator decomposed in terms of a linear smoothing and a morphological sharpening. In H. Talbot and R. Beare, editors, Mathematical Morphology, Proceedings of the 6th International Symposium on Mathematical Morphology, pp. 283–292, Sydney, Australia, April 2002. CSIRO Publishing.Google Scholar
  37. 37.
    R. van den Boomgaard and J. van de Weijer. Robust estimation of orientation for texture analysis. In Proc. Texture 2002, 2nd International Workshop on Texture Analysis and Synthesis, Copenhagen, June 2002.Google Scholar
  38. 38.
    J. Weickert. Anisotropic Diffusion in Image Processing. Teubner, Stuttgart, 1998.Google Scholar
  39. 39.
    J. Weickert. Coherence-enhancing diffusion filtering. International Journal of Computer Vision, 31(2/3):111–127, April 1999.CrossRefGoogle Scholar
  40. 40.
    J. Weickert. Coherence-enhancing diffusion of colour images. Image and Vision Computing, 17(3–4):199–210, March 1999.Google Scholar
  41. 41.
    J. Weickert and T. Brox. Diffusion and regularization of vector-and matrix-valued images. In M. Z. Nashed and O. Scherzer, editors, Inverse Problems, Image Analysis, and Medical Imaging, volume 313 of Contemporary Mathematics, pp. 251–268. AMS, Providence, 2002.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Thomas Brox
    • 1
  • Rein van den Boomgaard
    • 2
  • François Lauze
    • 3
  • Joost van de Weijer
    • 2
  • Joachim Weickert
    • 1
  • Pavel Mrázek
    • 1
  • Pierre Kornprobst
    • 4
  1. 1.Mathematical Image Analysis Group, Faculty of Mathematics and Computer ScienceSaarland UniversitySaarbrückenGermany
  2. 2.Intelligent Sensory Information Systems Group, Informatics InstituteUniversity of AmsterdamAmsterdamThe Netherlands
  3. 3.IT University of CopenhagenCopenhagenDenmark
  4. 4.Odyssée ProjectINRIA Sophia-AntipolisSophia AntipolisFrance

Personalised recommendations