Skip to main content

A Theoretical Framework for Convex Regularizers in PDE-Based Computation of Image Motion

International Journal of Computer Vision volume 45, pages 245–264 (2001)Cite this article

Abstract

Many differential methods for the recovery of the optic flow field from an image sequence can be expressed in terms of a variational problem where the optic flow minimizes some energy. Typically, these energy functionals consist of two terms: a data term, which requires e.g. that a brightness constancy assumption holds, and a regularizer that encourages global or piecewise smoothness of the flow field. In this paper we present a systematic classification of rotation invariant convex regularizers by exploring their connection to diffusion filters for multichannel images. This taxonomy provides a unifying framework for data-driven and flow-driven, isotropic and anisotropic, as well as spatial and spatio-temporal regularizers. While some of these techniques are classic methods from the literature, others are derived here for the first time. We prove that all these methods are well-posed: they posses a unique solution that depends in a continuous way on the initial data. An interesting structural relation between isotropic and anisotropic flow-driven regularizers is identified, and a design criterion is proposed for constructing anisotropic flow-driven regularizers in a simple and direct way from isotropic ones. Its use is illustrated by several examples.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Alvarez, L., Esclarín, J., Lefébure, M., and Sánchez, J. 1999. A PDE model for computing the optical flow. In Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, C.E.D.Y.A. XVI, Las Palmas de Gran Canaria, Sept. 21–24, 1999, pp. 1349–1356.

  • Alvarez, L., Weickert, J., and Sánchez, J. 2000. Reliable estimation of dense optical flow fields with large displacements. Int. J. Comput. Vision, 39:41–56.

    Google Scholar 

  • Anandan, P. 1989. A computational framework and an algorithm for the measurement of visual motion. Int. J. Comput. Vision, 2:283–310.

    Google Scholar 

  • Aubert, G., Deriche, R., and Kornprobst, P. 1999. Computing optical flow via variational techniques. SIAM J. Appl. Math, 60:156–182.

    Google Scholar 

  • Barron, J.L., Fleet, D.J., and Beauchemin, S.S. 1994. Performance of optical flow techniques. Int. J. Comput. Vision, 12:43–77.

    Google Scholar 

  • Bertero, M., Poggio, T.A., and Torre, V. 1988. Ill-posed problems in early vision. Proc. IEEE, 76:869–889.

    Google Scholar 

  • Black, M.J. and Anandan, P. 1991. Robust dynamic motion estimation over time. In Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, CVPR ‘91, Maui, June 3–6, 1991, IEEE Computer Society Press: Los Alamitos, pp. 292–302.

    Google Scholar 

  • Black, M.J., and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63:75–104.

    Article  Google Scholar 

  • Blake, A. and Zisserman, A. 1987.Visual Reconstruction. MIT Press: Cambridge (Mass.).

    Google Scholar 

  • Blanc-Féraud, L., Barlaud, M., and Gaidon, T. 1993. Motion estimation involving discontinuities in a multiresolution scheme. Optical Engineering, 32(7):1475–1482.

    Google Scholar 

  • Charbonnier, P., Blanc-Féraud, L., Aubert, G., and Barlaud, M. 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. In Proc. IEEE Int. Conf. Image Processing, ICIP-94, Austin, Nov. 13–16, 1994, Vol. 2, IEEE Computer Society Press: Los Alamitos, pp. 168–172.

    Google Scholar 

  • Cohen, I. 1993. Nonlinear variational method for optical flow computation. In Proc. Eighth Scandinavian Conf. on Image Analysis, SCIA ‘93, Tromsø, May 25–28, 1993, Vol. 1, pp. 523–530.

    Google Scholar 

  • Courant, R. and Hilbert, D. 1953. Methods of Mathematical Physics, Vol. 1. Interscience: New York.

    Google Scholar 

  • Deriche, R., Kornprobst, P., and Aubert, G. 1995. Optical-flow estimation while preserving its discontinuities: A variational approach. In Proc. Second Asian Conf. Computer Vision, ACCV ‘95, Singapore, Dec. 5–8, 1995, Vol. 2, pp. 290–295.

    Google Scholar 

  • Di Zenzo, S. 1986. A note on the gradient of a multi-image. Computer Vision, Graphics, and Image Processing, 33:116–125.

    Google Scholar 

  • Elsgolc, L.E. 1961. Calculus of Variations. Pergamon: Oxford.

    Google Scholar 

  • Enkelmann, W. 1988. Investigation of multigrid algorithms for the estimation of optical flow fields in image sequences. Computer Vision, Graphics and Image Processing, 43:150–177.

    Google Scholar 

  • Finkbeiner, D.T. 1966. Introduction to Matrices and Linear Transformations. Freeman: San Francisco.

    Google Scholar 

  • Fleet, D.J. and Jepson, A.D. 1990. Computation of component image velocity from local phase information. Int. J. Comput. Vision, 5:77–104.

    Google Scholar 

  • Förstner, W. and Gülch, E. 1987. A fast operator for detection and precise location of distinct points, corners and centres of circular features. In Proc. ISPRS Intercommission Conf. on Fast Processing of Photogrammetric Data, Interlaken, June 2–4, 1987, pp. 281–305.

  • Galvin, B., McCane, B., Novins, K., Mason, D., and Mills, S. 1998. Recovering motion fields: An analysis of eight optical flow algorithms. In Proc. 1998 British Machine Vision Conference, BMVC '98, Southampton, Sept. 14–17, 1998.

  • Gerig, G., Kübler, O., Kikinis, R., and Jolesz, F.A. 1992. Nonlinear anisotropic filtering of MRI data. IEEE Trans. Medical Imaging, 11:221–232.

    Google Scholar 

  • ter Haar Romeny, B., Florack, L., Koenderink, J., and Viergever, M. (Eds.). 1997. Scale-space Theory in Computer Vision. Springer: Berlin. Lecture Notes in Computer Science, Vol. 1252.

    Google Scholar 

  • Heitz, F. and Bouthemy, P. 1993. Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Trans. Pattern Anal. Mach. Intell., 15:1217–1232.

    Google Scholar 

  • Hinterberger, W. 1999. Generierung eines Films zwischen zwei Bildern mit Hilfe des optischen Flusses. M.Sc. thesis, Industrial Mathematics Institute, University of Linz, Austria.

    Google Scholar 

  • Horn, B. and Schunck, B. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.

    Article  Google Scholar 

  • Kimmel, R., Malladi, R., and Sochen, N. 2000. Images as embedded maps and minimal surfaces: Movies, color, texture, and volumetric medical images. Int. J. Comput. Vision, 39:111–129.

    Google Scholar 

  • Koenderink, J.J. 1975. Invariant properties of the motion parallax field due to the movement of rigid bodies relative to an observer. Optica Acta, 22:773–791.

    Google Scholar 

  • Kreyszig, E. 1959. Differential Geometry. University of Toronto Press: Toronto.

    Google Scholar 

  • Kumar, A., Tannenbaum, A.R., and Balas, G.J. 1996. Optic flow: A curve evolution approach. IEEE Trans. Image Proc., 5:598–610.

    Google Scholar 

  • van Laarhoven, P.J.M. and Aarts, E.H.L. 1988. Simulated Annealing: Theory and Applications. Reidel: Dordrecht.

    Google Scholar 

  • Mémin, E. and Pérez, P. 1998. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. Image Proc, 7:703–719.

    Article  Google Scholar 

  • Mitiche, A. and Bouthemy, P. 1996. Computation and analysis of image motion: A synopsis of current problems and methods. Int. J. Comput. Vision, 19:29–55.

    Google Scholar 

  • Murray, D.W. and Buxton, B.F. 1987. Scene segmentation from visual motion using global optimization. IEEE Trans. Pattern Anal. Mach. Intell., 9:220–228.

    Google Scholar 

  • Nagel, H.H. 1983. Constraints for the estimation of displacement vector fields from image sequences. In Proc. Eighth Int. Joint Conf. on Artificial Intelligence, IJCAI ‘83, Karlsruhe, Aug. 8–12, 1983, pp. 945–951.

  • Nagel, H.-H. 1987. On the estimation of optical flow: Relations between different approaches and some new results. Artificial Intelligence, 33:299–324.

    Article  Google Scholar 

  • Nagel, H.-H. 1990. Extending the ‘oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In '90, O. Faugeras (Ed.). Lecture Notes in Computer Science, Vol. 427. Springer: Berlin, pp. 139–148.

    Google Scholar 

  • Nagel, H.-H. and Enkelmann, W. 1986. An investigation of smoothness constraints for the estimation of displacement vector fields from images sequences. IEEE Trans. Pattern Anal. Mach. Intell., 8:565–593.

    Google Scholar 

  • Nesi, P. 1993. Variational approach to optical flow estimation managing discontinuities. Image and Vision Computing, 11:419–439.

    Google Scholar 

  • Nielsen, M., Johansen, P., Olsen, O.F., and Weickert, J. (Eds.). 1999. Scale-Space Theories in Computer Vision. Springer: Berlin. Lecture Notes in Computer Science, Vol. 1682.

    Google Scholar 

  • Otte, M. and Nagel, H.-H. 1995. Estimation of optical flow based on higher-order spatiotemporal derivatives in interlaced and noninterlaced image sequences. Artificial Intelligence, 78:5–43.

    Google Scholar 

  • Proesmans, M., Van Gool, L., Pauwels, E., and Oosterlinck, A. 1994.Determination of optical flow and its discontinuities using nonlinear diffusion. In '94, J.-O. Eklundh (Ed.). Springer: Berlin, pp. 295–304. Lecture Notes in Computer Science, Vol. 801.

    Google Scholar 

  • Schnörr, C. 1991. Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. Int. J. Comput. Vision, 6:25–38.

    Google Scholar 

  • Schnörr, C. 1993. On functionals with greyvalue-controlled smoothness terms for determining optical flow. IEEE Trans. Pattern Anal. Mach. Intell., 15:1074–1079.

    Google Scholar 

  • Schnörr, C. 1994. Segmentation of visual motion by minimizing convex non-quadratic functionals. In Proc. 12th Int. Conf. Pattern Recognition, ICPR 12, Jerusalem, Oct. 9–13, 1994, Vol. A, IEEE Computer Society Press: Los Alamitos, pp. 661–663.

    Google Scholar 

  • Schnörr, C. 1996. Convex variational segmentation of multi-channel images. In '96: Images,Wavelets and PDEs, M.-O. Berger, R. Deriche, I. Herlin, J. Jaffré, and J.-M. Morel (Eds.). Springer: London, pp. 201–207. Lecture Notes in Control and Information Sciences, Vol. 219.

    Google Scholar 

  • Schnörr, C. and Sprengel, R. 1994. A nonlinear regularization approach to early vision. Biol. Cybernetics, 72:141–149.

    Google Scholar 

  • Schnörr, C., Sprengel, R., and Neumann, B. 1996. A variational approach to the design of early vision algorithms. Computing, Suppl., 11:149–165.

    Google Scholar 

  • Schnörr, C. and Weickert, J. 2000. Variational motion computation: Theoretical framework, problems and perspectives. In Mustererkennung 2000, G. Sommer, N. Krüger, and C. Perwass (Eds.). Springer: Berlin, pp. 476–487.

    Google Scholar 

  • Snyder, M.A. 1991. On the mathematical foundations of smoothness constraints for the determination of optical flow and for surface reconstruction. IEEE Trans. Pattern Anal. Mach. Intell., 13:1105–1114.

    Google Scholar 

  • Sporring, J., Nielsen, M., Florack, L., and Johansen, P. (Eds.). 1997.Gaussian Scale-Space Theory, Kluwer: Dordrecht.

    Google Scholar 

  • Stevenson, R.L., Schmitz, B.E., and Delp, E.J. 1994. Discontinuity preserving regularization of inverse visual problems. IEEE Trans. Systems, Man and Cybernetics, 24:455–469.

    Google Scholar 

  • Stiller, C. and Konrad, J. 1999. Estimating motion in image sequences. IEEE Signal Proc. Magazine, 16:70–91.

    Google Scholar 

  • Weber, J. and Malik, J. 1995. Robust computation of optical flow in a multi-scale differential framework. Int. J. Comput. Vision, 14:67–81.

    Google Scholar 

  • Weickert, J. 1994. Scale-space properties of nonlinear diffusion filtering with a diffusion tensor. Report No. 110, Laboratory of Technomathematics, University of Kaiserslautern, P.O. Box 3049, 67653 Kaiserslautern, Germany.

  • Weickert, J. 1998a. Anisotropic Diffusion in Image Processing. Teubner: Stuttgart.

    Google Scholar 

  • Weickert, J. 1998b. On discontinuity-preserving optic flow. In Proc. Computer Vision and Mobile Robotics Workshop, CVMR '98, Santorini, Sept. 17–18, 1998, S. Orphanoudakis, P. Trahanias, J. Crowley, and N. Katevas (Eds.). pp. 115–122.

  • Weickert, J. 1999. Coherence-enhancing diffusion of colour images. Image and Vision Computing, 17:199–210.

    Google Scholar 

  • Weickert, J. and Schnörr, C. 2001. Variational optic flow computation with a spatio-temporal smoothness constraint. J. Math. Imag. Vision, 14:245–255.

    Google Scholar 

  • Whitaker, R. and Gerig, G. 1994.Vector-valued diffusion. Geometry-Driven Diffusion in ComputerVision, B.M. ter Haar Romeny (Ed.). Kluwer: Dordrecht, pp. 93–134.

    Google Scholar 

  • Zeidler, E. 1990. Nonlinear Functional Analysis and its Applications, Vol. IIb. Springer: Berlin.

    Google Scholar 

  • Ziemer, W.P. 1989. Weakly Differentiable Functions. Springer: New York.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision, Graphics, and Pattern Recognition Group, Department of Mathematics and Computer Science, University of Mannheim, 68131, Mannheim, Germany

    Joachim Weickert & Christoph Schnörr

Authors
  1. Joachim Weickert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christoph Schnörr
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Weickert, J., Schnörr, C. A Theoretical Framework for Convex Regularizers in PDE-Based Computation of Image Motion. International Journal of Computer Vision 45, 245–264 (2001). https://doi.org/10.1023/A:1013614317973

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1013614317973

  • optic flow
  • differential methods
  • regularization
  • diffusion filtering
  • well-posedness