We discuss the basic concepts of computer vision with stochastic partial differential equations (SPDEs). In typical approaches based on partial differential equations (PDEs), the end result in the best case is usually one value per pixel, the “expected” value. Error estimates or even full probability density functions PDFs are usually not available. This paper provides a framework allowing one to derive such PDFs, rendering computer vision approaches into measurements fulfilling scientific standards due to full error propagation. We identify the image data with random fields in order to model images and image sequences which carry uncertainty in their gray values, e.g. due to noise in the acquisition process.
The noisy behaviors of gray values is modeled as stochastic processes which are approximated with the method of generalized polynomial chaos (Wiener-Askey-Chaos). The Wiener-Askey polynomial chaos is combined with a standard spatial approximation based upon piecewise multi-linear finite elements. We present the basic building blocks needed for computer vision and image processing in this stochastic setting, i.e. we discuss the computation of stochastic moments, projections, gradient magnitudes, edge indicators, structure tensors, etc. Finally we show applications of our framework to derive stochastic analogs of well known PDEs for de-noising and optical flow extraction. These models are discretized with the stochastic Galerkin method. Our selection of SPDE models allows us to draw connections to the classical deterministic models as well as to stochastic image processing not based on PDEs. Several examples guide the reader through the presentation and show the usefulness of the framework.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Amiaz, T., & Kiryati, N. (2006). Piecewise-smooth dense optical flow via level sets. International Journal of Computer Vision, 68(2), 111–124.
Avriel, M. (2003). Nonlinear programming: Analysis and methods. New York: Dover.
Bao, Y., & Krim, H. (2004). Smart nonlinear diffusion: A probabilistic approach. Pattern Analysis and Machine Intelligence, 26(1), 63–72.
Black, M. J., & Anandan, P. (1991). Robust dynamic motion estimation over time. In Proc. computer vision and pattern recognition, CVPR-91 (pp. 296–302), June 1991.
Black, M. J., & Anandan, P. (1993). A framework for the robust estimation of optical flow. In Proc. ICCV93 (pp. 231–236).
Bruhn, A., Weickert, J., & Schnörr, C. (2002). Combining the advantages of local and global optic flow methods. In Proc. DAGM (pp. 454–462).
Bruhn, A., Weickert, J., & Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision, 61(3), 211–231.
Catté, F., Lions, P.-L., Morel, J.-M., & Coll, T. (1992). Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1), 182–193.
Chorin, A. J. (1971). Hermite expansions in Monte Carlo computation. Journal of Computational Physics, 8, 471–482.
Chorin, A. J. (1974). Gaussian fields and random flow. Journal of Fluid Mechanics, 63, 21–32.
Cohen, I. (1993). Nonlinear variational method for optical flow computation. In SCIA93 (pp. 523–530).
de Laplace, P. S. (1812). Théorie analytique des probabilites. Paris: Courcier Imprimeur.
Deb, M. K., Babuška, I. M., & Oden, J. T. (2001). Solutions of stochastic partial differential equations using Galerkin finite element techniques. Computer Methods in Applied Mechanics Engineering, 190, 6359–6372.
Fermüller, C., Shulman, D., & Aloimonos, Y. (2001). The statistics of optical flow. Computer Vision and Image Understanding, 82(1), 1–32.
Forsyth, D. A., & Ponce, J. (2003). Computer vision: a modern approach. Englewood Cliffs: Prentice Hall.
Ghanem, R. G. (1999). Higher order sensitivity of heat conduction problems to random data using the spectral stochastic finite element method. ASME Journal of Heat Transfer, 121, 290–299.
Ghanem, R. G., & Spanos, P. (1991). Stochastic finite elements: a spectral approach. New York: Springer.
Haussecker, H., & Spies, H. (1999). Motion. In B. Jähne, H. Haußecker, & P. Geißler (Eds.), Handbook of computer vision and applications (pp. 309–396). San Diego: Academic Press.
Haussecker, H., Spies, H., & Jähne, B. (1998). Tensor-based image sequence processing techniques for the study of dynamical processes. In Proceedings of the international symposium on real-time imaging and dynamic analysis, ISPRS, commission V, working group IC V/III, Hakodate, Japan, June 1998.
Horn, B. K. P., & Schunck, B. (1981). Determining optical flow. Artificial Intelligence, 17, 185–204.
Van Huffel, S., & Vandewalle, J. (1991). Frontiers in applied mathematics: Vol. 9. The total least squares problem: Computational aspects and analysis. Philadelphia: SIAM.
Iijima, T. (1962). Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bulletin of the Electrotechnical Laboratory, 26, 368–388 (in Japanese).
Iijima, T. (1963). Theory of pattern recognition. Electronics and Communications in Japan (pp. 123–134).
Jähne, B. (1993). Spatio-temporal image processing: Theory and scientific applications. Lecture notes in computer science. Berlin: Springer.
Kearney, J. K., Thompson, W. B., & Boley, D. L. (1987). Optical flow estimation: An error analysis of gradient-based methods with local optimization. PAMI, 9(2), 229–244.
Keese, A. (2004). Numerical solution of systems with stochastic uncertainties: A general purpose framework for stochastic finite elements. Ph.D. thesis, Technical University Braunschweig.
Kichenassamy, S. (1997). The Perona-Malik paradox. SIAM Journal on Applied Mathematics, 57(5), 1328–1342.
Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In DARPA image understanding workshop (pp. 121–130).
Lucor, D., Su, C.-H., & Karniadakis, G. E. (2004). Generalized polynomial chaos and random oscillators. International Journal for Numerical Methods in Engineering, 60, 571–596.
Le Maître, O. P., Reagan, M., Najm, H. N., Ghanem, R. G., & Knio, O. M. (2002). A stochastic projection method for fluid flow II: random process. Journal of Computational Physics, 181(1), 9–44.
Malliavin, P. (1997). Stochastic analysis. New York: Springer.
Maltz, F. H., & Hitzl, D. L. (1979). Variance reduction in Monte Carlo computations using multi-dimensional Hermite polynomials. Journal of Computational Physics, 32, 345–376.
Meecham, W. C., & Jeng, D. T. (1968). Use of Wiener-Hermite expansion for nearly normal turbulence. Journal of Fluid Mechanics, 32, 225–249.
Mikula, K., Preusser, T., & Rumpf, M. (2004). Morphological image sequence processing. Computing and Visualization in Science, 6(4), 197–209.
Narayanan, V. A., & Zabaras, N. (2004). Stochastic inverse heat conduction using a spectral approach. International Journal for Numerical Methods in Engineering, 60, 1569–1593.
Nestares, O., & Fleet, D. J. (2003). Error-in-variables likelihood functions for motion estimation. In IEEE international conference on image processing (ICIP) (Vol. III, pp. 77–80). Barcelona.
Nestares, O., Fleet, D. J., & Heeger, D. (2000). Likelihood functions and confidence bounds for total-least-squares problems. In CVPR’00 (Vol. 1).
Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.
Perona, P., & Malik, J. (1990). Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 629–639.
Preusser, T., & Rumpf, M. (1999). An adaptive finite element method for large scale image processing. In Proceedings scale-space ’99, scale space theories in computer vision, second international conference (pp. 223–234).
Reagan, M. T., Najm, H. N., Debusschere, B. J., Le Maître, O. P., Knio, O. M., & Ghanem, R. G. (2004). Spectral stochastic uncertainty quantification in chemical systems. Combustion Theory and Modelling, 8, 607–632.
Reagan, M. T., Najm, H. N., Pebay, P. P., Knio, O. M., & Ghanem, R. G. (2005). Quantifying uncertainty in chemical systems modeling. International Journal of Chemical Kinetics, 37, 386–382.
Scharr, H. (2006). Diffusion-like reconstruction schemes from linear data models. In Lecture notes in computer science : Vol. 4174. Pattern recognition 2006 (pp. 51–60). Berlin: Springer.
Scharr, H., Black, M. J., & Haussecker, H. W. (2003). Image statistics and anisotropic diffusion. In Int. conf. on computer vision, ICCV 2003 (pp. 840–847), Nice, France.
Sühling, M. (2006). Myocardial motion and deformation analysis from echocardiograms. Ph.D. thesis, Swiss Federal Institute of Technology Lausanne (EPFL), July 2006.
Thomee, V. (1984). Galerkin—finite element methods for parabolic problems. New York: Springer.
Gauss, C. F. (1987). Theory of the combination of observations least subject to errors, part one and part two. Supplement (Classics in Applied Mathematics 11) (trans: Stewart, G. W.). Society for Industrial Mathematics, Facsimile edition. English version in 1987. Original version in Latin in 1820s.
Weber, J., & Malik, J. (1994). Robust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision, 14(1), 5–19.
Weickert, J. (1998). On discontinuity-preserving optic flow. In Proc. computer vision and mobile robotics workshop (pp. 115–122).
Weickert, J., & Schnörr, C. (2001). Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision, 14(3), 245–255.
Wiener, N. (1938). The homogeneous chaos. American Journal of Mathematics, 60(4), 897–936.
Witkin, A. P. (1983). Scale-space filtering. In Proc. eighth int. joint conf. on artificial intelligence (IJCAI) (Vol. 2, pp. 1019–1022).
Xiu, D. B., & Karniadakis, G. E. (2002a). Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Computational Methods in Applied Mechanics and Engineering, 191, 4927–4948.
Xiu, D. B., & Karniadakis, G. E. (2002b). The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM Journal on Scientific Computing, 24, 619–644.
Xiu, D. B., & Karniadakis, G. E. (2003a). Modeling uncertainty in flow simulations via generalized polynomial chaos. Journal of Computational Physics, 187, 137–167.
Xiu, D. B., & Karniadakis, G. E. (2003b). A new stochastic approach to transient heat conduction modeling with uncertainty. International Journal of Heat and Mass Transfer, 46, 4681–4693.
About this article
Cite this article
Preusser, T., Scharr, H., Krajsek, K. et al. Building Blocks for Computer Vision with Stochastic Partial Differential Equations. Int J Comput Vis 80, 375–405 (2008). https://doi.org/10.1007/s11263-008-0145-5
- Image processing
- Error propagation
- Random fields
- Polynomial chaos
- Stochastic partial differential equations
- Stochastic Galerkin method
- Stochastic finite element method