Building Blocks for Computer Vision with Stochastic Partial Differential Equations

Abstract

We discuss the basic concepts of computer vision with stochastic partial differential equations (SPDEs). In typical approaches based on partial differential equations (PDEs), the end result in the best case is usually one value per pixel, the “expected” value. Error estimates or even full probability density functions PDFs are usually not available. This paper provides a framework allowing one to derive such PDFs, rendering computer vision approaches into measurements fulfilling scientific standards due to full error propagation. We identify the image data with random fields in order to model images and image sequences which carry uncertainty in their gray values, e.g. due to noise in the acquisition process.

The noisy behaviors of gray values is modeled as stochastic processes which are approximated with the method of generalized polynomial chaos (Wiener-Askey-Chaos). The Wiener-Askey polynomial chaos is combined with a standard spatial approximation based upon piecewise multi-linear finite elements. We present the basic building blocks needed for computer vision and image processing in this stochastic setting, i.e. we discuss the computation of stochastic moments, projections, gradient magnitudes, edge indicators, structure tensors, etc. Finally we show applications of our framework to derive stochastic analogs of well known PDEs for de-noising and optical flow extraction. These models are discretized with the stochastic Galerkin method. Our selection of SPDE models allows us to draw connections to the classical deterministic models as well as to stochastic image processing not based on PDEs. Several examples guide the reader through the presentation and show the usefulness of the framework.

This is a preview of subscription content, log in to check access.

References

  1. Amiaz, T., & Kiryati, N. (2006). Piecewise-smooth dense optical flow via level sets. International Journal of Computer Vision, 68(2), 111–124.

    Article  Google Scholar 

  2. Avriel, M. (2003). Nonlinear programming: Analysis and methods. New York: Dover.

    Google Scholar 

  3. Bao, Y., & Krim, H. (2004). Smart nonlinear diffusion: A probabilistic approach. Pattern Analysis and Machine Intelligence, 26(1), 63–72.

    Article  Google Scholar 

  4. Black, M. J., & Anandan, P. (1991). Robust dynamic motion estimation over time. In Proc. computer vision and pattern recognition, CVPR-91 (pp. 296–302), June 1991.

  5. Black, M. J., & Anandan, P. (1993). A framework for the robust estimation of optical flow. In Proc. ICCV93 (pp. 231–236).

  6. Bruhn, A., Weickert, J., & Schnörr, C. (2002). Combining the advantages of local and global optic flow methods. In Proc. DAGM (pp. 454–462).

  7. Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision, 61(3), 211–231.

    Article  Google Scholar 

  8. Catté, F., Lions, P.-L., Morel, J.-M., & Coll, T. (1992). Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1), 182–193.

    MATH  Article  MathSciNet  Google Scholar 

  9. Chorin, A. J. (1971). Hermite expansions in Monte Carlo computation. Journal of Computational Physics, 8, 471–482.

    Article  MathSciNet  Google Scholar 

  10. Chorin, A. J. (1974). Gaussian fields and random flow. Journal of Fluid Mechanics, 63, 21–32.

    MATH  Article  MathSciNet  Google Scholar 

  11. Cohen, I. (1993). Nonlinear variational method for optical flow computation. In SCIA93 (pp. 523–530).

  12. de Laplace, P. S. (1812). Théorie analytique des probabilites. Paris: Courcier Imprimeur.

    Google Scholar 

  13. Deb, M. K., Babuška, I. M., & Oden, J. T. (2001). Solutions of stochastic partial differential equations using Galerkin finite element techniques. Computer Methods in Applied Mechanics Engineering, 190, 6359–6372.

    MATH  Article  Google Scholar 

  14. Fermüller, C., Shulman, D., & Aloimonos, Y. (2001). The statistics of optical flow. Computer Vision and Image Understanding, 82(1), 1–32.

    MATH  Article  Google Scholar 

  15. Forsyth, D. A., & Ponce, J. (2003). Computer vision: a modern approach. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  16. Ghanem, R. G. (1999). Higher order sensitivity of heat conduction problems to random data using the spectral stochastic finite element method. ASME Journal of Heat Transfer, 121, 290–299.

    Article  Google Scholar 

  17. Ghanem, R. G., & Spanos, P. (1991). Stochastic finite elements: a spectral approach. New York: Springer.

    Google Scholar 

  18. Haussecker, H., & Spies, H. (1999). Motion. In B. Jähne, H. Haußecker, & P. Geißler (Eds.), Handbook of computer vision and applications (pp. 309–396). San Diego: Academic Press.

    Google Scholar 

  19. Haussecker, H., Spies, H., & Jähne, B. (1998). Tensor-based image sequence processing techniques for the study of dynamical processes. In Proceedings of the international symposium on real-time imaging and dynamic analysis, ISPRS, commission V, working group IC V/III, Hakodate, Japan, June 1998.

  20. Horn, B. K. P., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–204.

    Article  Google Scholar 

  21. Van Huffel, S., & Vandewalle, J. (1991). Frontiers in applied mathematics: Vol. 9. The total least squares problem: Computational aspects and analysis. Philadelphia: SIAM.

    Google Scholar 

  22. Iijima, T. (1962). Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bulletin of the Electrotechnical Laboratory, 26, 368–388 (in Japanese).

    Google Scholar 

  23. Iijima, T. (1963). Theory of pattern recognition. Electronics and Communications in Japan (pp. 123–134).

  24. Jähne, B. (1993). Spatio-temporal image processing: Theory and scientific applications. Lecture notes in computer science. Berlin: Springer.

    Google Scholar 

  25. Kearney, J. K., Thompson, W. B., & Boley, D. L. (1987). Optical flow estimation: An error analysis of gradient-based methods with local optimization. PAMI, 9(2), 229–244.

    Google Scholar 

  26. Keese, A. (2004). Numerical solution of systems with stochastic uncertainties: A general purpose framework for stochastic finite elements. Ph.D. thesis, Technical University Braunschweig.

  27. Kichenassamy, S. (1997). The Perona-Malik paradox. SIAM Journal on Applied Mathematics, 57(5), 1328–1342.

    MATH  Article  MathSciNet  Google Scholar 

  28. Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In DARPA image understanding workshop (pp. 121–130).

  29. Lucor, D., Su, C.-H., & Karniadakis, G. E. (2004). Generalized polynomial chaos and random oscillators. International Journal for Numerical Methods in Engineering, 60, 571–596.

    MATH  Article  MathSciNet  Google Scholar 

  30. Le Maître, O. P., Reagan, M., Najm, H. N., Ghanem, R. G., & Knio, O. M. (2002). A stochastic projection method for fluid flow II: random process. Journal of Computational Physics, 181(1), 9–44.

    MATH  Article  MathSciNet  Google Scholar 

  31. Malliavin, P. (1997). Stochastic analysis. New York: Springer.

    Google Scholar 

  32. Maltz, F. H., & Hitzl, D. L. (1979). Variance reduction in Monte Carlo computations using multi-dimensional Hermite polynomials. Journal of Computational Physics, 32, 345–376.

    MATH  Article  MathSciNet  Google Scholar 

  33. Meecham, W. C., & Jeng, D. T. (1968). Use of Wiener-Hermite expansion for nearly normal turbulence. Journal of Fluid Mechanics, 32, 225–249.

    MATH  Article  Google Scholar 

  34. Mikula, K., Preusser, T., & Rumpf, M. (2004). Morphological image sequence processing. Computing and Visualization in Science, 6(4), 197–209.

    Article  MathSciNet  Google Scholar 

  35. Narayanan, V. A., & Zabaras, N. (2004). Stochastic inverse heat conduction using a spectral approach. International Journal for Numerical Methods in Engineering, 60, 1569–1593.

    MATH  Article  MathSciNet  Google Scholar 

  36. Nestares, O., & Fleet, D. J. (2003). Error-in-variables likelihood functions for motion estimation. In IEEE international conference on image processing (ICIP) (Vol. III, pp. 77–80). Barcelona.

  37. Nestares, O., Fleet, D. J., & Heeger, D. (2000). Likelihood functions and confidence bounds for total-least-squares problems. In CVPR’00 (Vol. 1).

  38. Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.

    Article  Google Scholar 

  39. Perona, P., & Malik, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 629–639.

    Article  Google Scholar 

  40. Preusser, T., & Rumpf, M. (1999). An adaptive finite element method for large scale image processing. In Proceedings scale-space ’99, scale space theories in computer vision, second international conference (pp. 223–234).

  41. Reagan, M. T., Najm, H. N., Debusschere, B. J., Le Maître, O. P., Knio, O. M., & Ghanem, R. G. (2004). Spectral stochastic uncertainty quantification in chemical systems. Combustion Theory and Modelling, 8, 607–632.

    Article  Google Scholar 

  42. Reagan, M. T., Najm, H. N., Pebay, P. P., Knio, O. M., & Ghanem, R. G. (2005). Quantifying uncertainty in chemical systems modeling. International Journal of Chemical Kinetics, 37, 386–382.

    Article  Google Scholar 

  43. Scharr, H. (2006). Diffusion-like reconstruction schemes from linear data models. In Lecture notes in computer science : Vol. 4174. Pattern recognition 2006 (pp. 51–60). Berlin: Springer.

    Google Scholar 

  44. Scharr, H., Black, M. J., & Haussecker, H. W. (2003). Image statistics and anisotropic diffusion. In Int. conf. on computer vision, ICCV 2003 (pp. 840–847), Nice, France.

  45. Sühling, M. (2006). Myocardial motion and deformation analysis from echocardiograms. Ph.D. thesis, Swiss Federal Institute of Technology Lausanne (EPFL), July 2006.

  46. Thomee, V. (1984). Galerkin—finite element methods for parabolic problems. New York: Springer.

    Google Scholar 

  47. Gauss, C. F. (1987). Theory of the combination of observations least subject to errors, part one and part two. Supplement (Classics in Applied Mathematics 11) (trans: Stewart, G. W.). Society for Industrial Mathematics, Facsimile edition. English version in 1987. Original version in Latin in 1820s.

  48. Weber, J., & Malik, J. (1994). Robust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision, 14(1), 5–19.

    Google Scholar 

  49. Weickert, J. (1998). On discontinuity-preserving optic flow. In Proc. computer vision and mobile robotics workshop (pp. 115–122).

  50. Weickert, J., & Schnörr, C. (2001). Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3), 245–255.

    MATH  Article  Google Scholar 

  51. Wiener, N. (1938). The homogeneous chaos. American Journal of Mathematics, 60(4), 897–936.

    Article  MathSciNet  Google Scholar 

  52. Witkin, A. P. (1983). Scale-space filtering. In Proc. eighth int. joint conf. on artificial intelligence (IJCAI) (Vol. 2, pp. 1019–1022).

  53. Xiu, D. B., & Karniadakis, G. E. (2002a). Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Computational Methods in Applied Mechanics and Engineering, 191, 4927–4948.

    MATH  Article  MathSciNet  Google Scholar 

  54. Xiu, D. B., & Karniadakis, G. E. (2002b). The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM Journal on Scientific Computing, 24, 619–644.

    MATH  Article  MathSciNet  Google Scholar 

  55. Xiu, D. B., & Karniadakis, G. E. (2003a). Modeling uncertainty in flow simulations via generalized polynomial chaos. Journal of Computational Physics, 187, 137–167.

    MATH  Article  MathSciNet  Google Scholar 

  56. Xiu, D. B., & Karniadakis, G. E. (2003b). A new stochastic approach to transient heat conduction modeling with uncertainty. International Journal of Heat and Mass Transfer, 46, 4681–4693.

    MATH  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Tobias Preusser.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Preusser, T., Scharr, H., Krajsek, K. et al. Building Blocks for Computer Vision with Stochastic Partial Differential Equations. Int J Comput Vis 80, 375–405 (2008). https://doi.org/10.1007/s11263-008-0145-5

Download citation

Keywords

  • Image processing
  • Error propagation
  • Random fields
  • Polynomial chaos
  • Stochastic partial differential equations
  • Stochastic Galerkin method
  • Stochastic finite element method