International Journal of Computer Vision

, Volume 80, Issue 3, pp 375–405 | Cite as

Building Blocks for Computer Vision with Stochastic Partial Differential Equations

  • Tobias PreusserEmail author
  • Hanno Scharr
  • Kai Krajsek
  • Robert M. Kirby


We discuss the basic concepts of computer vision with stochastic partial differential equations (SPDEs). In typical approaches based on partial differential equations (PDEs), the end result in the best case is usually one value per pixel, the “expected” value. Error estimates or even full probability density functions PDFs are usually not available. This paper provides a framework allowing one to derive such PDFs, rendering computer vision approaches into measurements fulfilling scientific standards due to full error propagation. We identify the image data with random fields in order to model images and image sequences which carry uncertainty in their gray values, e.g. due to noise in the acquisition process.

The noisy behaviors of gray values is modeled as stochastic processes which are approximated with the method of generalized polynomial chaos (Wiener-Askey-Chaos). The Wiener-Askey polynomial chaos is combined with a standard spatial approximation based upon piecewise multi-linear finite elements. We present the basic building blocks needed for computer vision and image processing in this stochastic setting, i.e. we discuss the computation of stochastic moments, projections, gradient magnitudes, edge indicators, structure tensors, etc. Finally we show applications of our framework to derive stochastic analogs of well known PDEs for de-noising and optical flow extraction. These models are discretized with the stochastic Galerkin method. Our selection of SPDE models allows us to draw connections to the classical deterministic models as well as to stochastic image processing not based on PDEs. Several examples guide the reader through the presentation and show the usefulness of the framework.


Image processing Error propagation Random fields Polynomial chaos Stochastic partial differential equations Stochastic Galerkin method Stochastic finite element method 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amiaz, T., & Kiryati, N. (2006). Piecewise-smooth dense optical flow via level sets. International Journal of Computer Vision, 68(2), 111–124. CrossRefGoogle Scholar
  2. Avriel, M. (2003). Nonlinear programming: Analysis and methods. New York: Dover. zbMATHGoogle Scholar
  3. Bao, Y., & Krim, H. (2004). Smart nonlinear diffusion: A probabilistic approach. Pattern Analysis and Machine Intelligence, 26(1), 63–72. CrossRefGoogle Scholar
  4. Black, M. J., & Anandan, P. (1991). Robust dynamic motion estimation over time. In Proc. computer vision and pattern recognition, CVPR-91 (pp. 296–302), June 1991. Google Scholar
  5. Black, M. J., & Anandan, P. (1993). A framework for the robust estimation of optical flow. In Proc. ICCV93 (pp. 231–236). Google Scholar
  6. Bruhn, A., Weickert, J., & Schnörr, C. (2002). Combining the advantages of local and global optic flow methods. In Proc. DAGM (pp. 454–462). Google Scholar
  7. Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision, 61(3), 211–231. CrossRefGoogle Scholar
  8. Catté, F., Lions, P.-L., Morel, J.-M., & Coll, T. (1992). Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1), 182–193. zbMATHCrossRefMathSciNetGoogle Scholar
  9. Chorin, A. J. (1971). Hermite expansions in Monte Carlo computation. Journal of Computational Physics, 8, 471–482. CrossRefMathSciNetGoogle Scholar
  10. Chorin, A. J. (1974). Gaussian fields and random flow. Journal of Fluid Mechanics, 63, 21–32. zbMATHCrossRefMathSciNetGoogle Scholar
  11. Cohen, I. (1993). Nonlinear variational method for optical flow computation. In SCIA93 (pp. 523–530). Google Scholar
  12. de Laplace, P. S. (1812). Théorie analytique des probabilites. Paris: Courcier Imprimeur. Google Scholar
  13. Deb, M. K., Babuška, I. M., & Oden, J. T. (2001). Solutions of stochastic partial differential equations using Galerkin finite element techniques. Computer Methods in Applied Mechanics Engineering, 190, 6359–6372. zbMATHCrossRefGoogle Scholar
  14. Fermüller, C., Shulman, D., & Aloimonos, Y. (2001). The statistics of optical flow. Computer Vision and Image Understanding, 82(1), 1–32. zbMATHCrossRefGoogle Scholar
  15. Forsyth, D. A., & Ponce, J. (2003). Computer vision: a modern approach. Englewood Cliffs: Prentice Hall. Google Scholar
  16. Ghanem, R. G. (1999). Higher order sensitivity of heat conduction problems to random data using the spectral stochastic finite element method. ASME Journal of Heat Transfer, 121, 290–299. CrossRefGoogle Scholar
  17. Ghanem, R. G., & Spanos, P. (1991). Stochastic finite elements: a spectral approach. New York: Springer. zbMATHGoogle Scholar
  18. Haussecker, H., & Spies, H. (1999). Motion. In B. Jähne, H. Haußecker, & P. Geißler (Eds.), Handbook of computer vision and applications (pp. 309–396). San Diego: Academic Press. Google Scholar
  19. Haussecker, H., Spies, H., & Jähne, B. (1998). Tensor-based image sequence processing techniques for the study of dynamical processes. In Proceedings of the international symposium on real-time imaging and dynamic analysis, ISPRS, commission V, working group IC V/III, Hakodate, Japan, June 1998. Google Scholar
  20. Horn, B. K. P., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–204. CrossRefGoogle Scholar
  21. Van Huffel, S., & Vandewalle, J. (1991). Frontiers in applied mathematics: Vol. 9. The total least squares problem: Computational aspects and analysis. Philadelphia: SIAM. zbMATHGoogle Scholar
  22. Iijima, T. (1962). Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bulletin of the Electrotechnical Laboratory, 26, 368–388 (in Japanese). Google Scholar
  23. Iijima, T. (1963). Theory of pattern recognition. Electronics and Communications in Japan (pp. 123–134). Google Scholar
  24. Jähne, B. (1993). Spatio-temporal image processing: Theory and scientific applications. Lecture notes in computer science. Berlin: Springer. zbMATHGoogle Scholar
  25. Kearney, J. K., Thompson, W. B., & Boley, D. L. (1987). Optical flow estimation: An error analysis of gradient-based methods with local optimization. PAMI, 9(2), 229–244. Google Scholar
  26. Keese, A. (2004). Numerical solution of systems with stochastic uncertainties: A general purpose framework for stochastic finite elements. Ph.D. thesis, Technical University Braunschweig. Google Scholar
  27. Kichenassamy, S. (1997). The Perona-Malik paradox. SIAM Journal on Applied Mathematics, 57(5), 1328–1342. zbMATHCrossRefMathSciNetGoogle Scholar
  28. Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In DARPA image understanding workshop (pp. 121–130). Google Scholar
  29. Lucor, D., Su, C.-H., & Karniadakis, G. E. (2004). Generalized polynomial chaos and random oscillators. International Journal for Numerical Methods in Engineering, 60, 571–596. zbMATHCrossRefMathSciNetGoogle Scholar
  30. Le Maître, O. P., Reagan, M., Najm, H. N., Ghanem, R. G., & Knio, O. M. (2002). A stochastic projection method for fluid flow II: random process. Journal of Computational Physics, 181(1), 9–44. zbMATHCrossRefMathSciNetGoogle Scholar
  31. Malliavin, P. (1997). Stochastic analysis. New York: Springer. zbMATHGoogle Scholar
  32. Maltz, F. H., & Hitzl, D. L. (1979). Variance reduction in Monte Carlo computations using multi-dimensional Hermite polynomials. Journal of Computational Physics, 32, 345–376. zbMATHCrossRefMathSciNetGoogle Scholar
  33. Meecham, W. C., & Jeng, D. T. (1968). Use of Wiener-Hermite expansion for nearly normal turbulence. Journal of Fluid Mechanics, 32, 225–249. zbMATHCrossRefGoogle Scholar
  34. Mikula, K., Preusser, T., & Rumpf, M. (2004). Morphological image sequence processing. Computing and Visualization in Science, 6(4), 197–209. CrossRefMathSciNetGoogle Scholar
  35. Narayanan, V. A., & Zabaras, N. (2004). Stochastic inverse heat conduction using a spectral approach. International Journal for Numerical Methods in Engineering, 60, 1569–1593. zbMATHCrossRefMathSciNetGoogle Scholar
  36. Nestares, O., & Fleet, D. J. (2003). Error-in-variables likelihood functions for motion estimation. In IEEE international conference on image processing (ICIP) (Vol. III, pp. 77–80). Barcelona. Google Scholar
  37. Nestares, O., Fleet, D. J., & Heeger, D. (2000). Likelihood functions and confidence bounds for total-least-squares problems. In CVPR’00 (Vol. 1). Google Scholar
  38. Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158. CrossRefGoogle Scholar
  39. Perona, P., & Malik, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 629–639. CrossRefGoogle Scholar
  40. Preusser, T., & Rumpf, M. (1999). An adaptive finite element method for large scale image processing. In Proceedings scale-space ’99, scale space theories in computer vision, second international conference (pp. 223–234). Google Scholar
  41. Reagan, M. T., Najm, H. N., Debusschere, B. J., Le Maître, O. P., Knio, O. M., & Ghanem, R. G. (2004). Spectral stochastic uncertainty quantification in chemical systems. Combustion Theory and Modelling, 8, 607–632. CrossRefGoogle Scholar
  42. Reagan, M. T., Najm, H. N., Pebay, P. P., Knio, O. M., & Ghanem, R. G. (2005). Quantifying uncertainty in chemical systems modeling. International Journal of Chemical Kinetics, 37, 386–382. CrossRefGoogle Scholar
  43. Scharr, H. (2006). Diffusion-like reconstruction schemes from linear data models. In Lecture notes in computer science : Vol. 4174. Pattern recognition 2006 (pp. 51–60). Berlin: Springer. CrossRefGoogle Scholar
  44. Scharr, H., Black, M. J., & Haussecker, H. W. (2003). Image statistics and anisotropic diffusion. In Int. conf. on computer vision, ICCV 2003 (pp. 840–847), Nice, France. Google Scholar
  45. Sühling, M. (2006). Myocardial motion and deformation analysis from echocardiograms. Ph.D. thesis, Swiss Federal Institute of Technology Lausanne (EPFL), July 2006. Google Scholar
  46. Thomee, V. (1984). Galerkin—finite element methods for parabolic problems. New York: Springer. zbMATHGoogle Scholar
  47. Gauss, C. F. (1987). Theory of the combination of observations least subject to errors, part one and part two. Supplement (Classics in Applied Mathematics 11) (trans: Stewart, G. W.). Society for Industrial Mathematics, Facsimile edition. English version in 1987. Original version in Latin in 1820s. Google Scholar
  48. Weber, J., & Malik, J. (1994). Robust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision, 14(1), 5–19. Google Scholar
  49. Weickert, J. (1998). On discontinuity-preserving optic flow. In Proc. computer vision and mobile robotics workshop (pp. 115–122). Google Scholar
  50. Weickert, J., & Schnörr, C. (2001). Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3), 245–255. zbMATHCrossRefGoogle Scholar
  51. Wiener, N. (1938). The homogeneous chaos. American Journal of Mathematics, 60(4), 897–936. CrossRefMathSciNetGoogle Scholar
  52. Witkin, A. P. (1983). Scale-space filtering. In Proc. eighth int. joint conf. on artificial intelligence (IJCAI) (Vol. 2, pp. 1019–1022). Google Scholar
  53. Xiu, D. B., & Karniadakis, G. E. (2002a). Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Computational Methods in Applied Mechanics and Engineering, 191, 4927–4948. zbMATHCrossRefMathSciNetGoogle Scholar
  54. Xiu, D. B., & Karniadakis, G. E. (2002b). The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM Journal on Scientific Computing, 24, 619–644. zbMATHCrossRefMathSciNetGoogle Scholar
  55. Xiu, D. B., & Karniadakis, G. E. (2003a). Modeling uncertainty in flow simulations via generalized polynomial chaos. Journal of Computational Physics, 187, 137–167. zbMATHCrossRefMathSciNetGoogle Scholar
  56. Xiu, D. B., & Karniadakis, G. E. (2003b). A new stochastic approach to transient heat conduction modeling with uncertainty. International Journal of Heat and Mass Transfer, 46, 4681–4693. zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Tobias Preusser
    • 1
    Email author
  • Hanno Scharr
    • 2
  • Kai Krajsek
    • 2
  • Robert M. Kirby
    • 3
  1. 1.Center of Complex Systems and VisualizationBremen UniversityBremenGermany
  2. 2.Institute for Chemistry and Dynamics of the Geosphere, Institute 3: PhytosphereForschungszentrum Juelich GmbHJuelichGermany
  3. 3.School of Computing and Scientific Computing and Imaging InstituteUniversity of UtahSalt Lake CityUSA

Personalised recommendations