Discontinuity-Preserving Computation of Variational Optic Flow in Real-Time

  • Andrés Bruhn
  • Joachim Weickert
  • Timo Kohlberger
  • Christoph Schnörr
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3459)

Abstract

Variational methods are very popular for optic flow computation: They yield dense flow fields and perform well if they are adapted such that they respect discontinuities in the image sequence or the flow field. Unfortunately, this adaptation results in high computational complexity. In our paper we show that it is possible to achieve real-time performance for these methods if bidirectional multigrid strategies are used. To this end, we study two prototypes: i) For the anisotropic image-driven technique of Nagel and Enkelmann that results in a linear system of equations we derive a regular full multigrid scheme. ii) For an isotropic flow-driven approach with total variation (TV) regularisation that requires to solve a nonlinear system of equations we develop a full multigrid strategy based on a full approximation scheme (FAS). Experiments for sequences of size 160 × 120 demonstrate the excellent performance of the proposed numerical schemes. With frame rates of 6 and 12 dense flow fields per second, respectively, both implementations outperform corresponding modified explicit schemes by two to three orders of magnitude. Thus, for the first time ever, real-time performance can be achieved for these high quality methods.

Keywords

computer vision optical flow differential techniques variational methods multigrid methods partial differential equations 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alvarez, L., Esclarín, J., Lefébure, M., Sánchez, J.: A PDE model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Las Palmas de Gran Canaria, Spain, pp. 1349–1356 (September 1999)Google Scholar
  2. 2.
    Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision 2, 283–310 (1989)CrossRefGoogle Scholar
  3. 3.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. International Journal of Computer Vision 12(1), 43–77 (1994)CrossRefGoogle Scholar
  4. 4.
    Borzi, A., Ito, K., Kunisch, K.: Optimal control formulation for determining optical flow. SIAM Journal on Scientific Computing 24(3), 818–847 (2002)MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Brandt, A.: Multi-level adaptive solutions to boundary-value problems. Mathematics of Computation 31(138), 333–390 (1977)MATHMathSciNetCrossRefGoogle Scholar
  6. 6.
    Briggs, W.L., Henson, V.E., McCormick, S.F.: A Multigrid Tutorial, 2nd edn. SIAM, Philadelphia (2000)MATHCrossRefGoogle Scholar
  7. 7.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optic flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Variational optical flow computation in real-time. IEEE Transactions on Image Processing (2005) (to appear)Google Scholar
  9. 9.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. International Journal of Computer Vision 61(3), 1–21 (2005) (in press)CrossRefGoogle Scholar
  10. 10.
    Chan, R.H., Chan, T.F., Wan, W.L.: Multigrid for differential-convolution problems arising from image processing. In: Golub, G., Lui, S.H., Luk, F., Plemmons, R. (eds.) Proc. Workshop on Scientific Computing, Hong Kong, September 1997, pp. 58–72 (1997)Google Scholar
  11. 11.
    Cohen, I.: Nonlinear variational method for optical flow computation. In: Proc. Eighth Scandinavian Conference on Image Analysis, Tromsø, Norway, May 1993, vol. 1, pp. 523–530 (1993)Google Scholar
  12. 12.
    Deriche, R., Kornprobst, P., Aubert, G.: Optical-flow estimation while preserving its discontinuities: a variational approach. In: Li, S., Teoh, E.-K., Mital, D., Wang, H. (eds.) ACCV 1995. LNCS, vol. 1035, pp. 290–295. Springer, Heidelberg (1996)Google Scholar
  13. 13.
    El Kalmoun, M., Rüde, U.: A variational multigrid for computing the optical flow. In: Ertl, T., Girod, B., Greiner, G., Niemann, H., Seidel, H.-P., Steinbach, E., Westermann, R. (eds.) Vision, Modelling and Visualization, pp. 577–584. IOS Press, Amsterdam (2003)Google Scholar
  14. 14.
    Elsgolc, L.E.: Calculus of Variations. Pergamon, Oxford (1961)MATHGoogle Scholar
  15. 15.
    Enkelmann, W.: Investigation of multigrid algorithms for the estimation of optical flow fields in image sequences. Computer Vision, Graphics and Image Processing 43, 150–177 (1987)CrossRefGoogle Scholar
  16. 16.
    Frohn-Schnauf, C., Henn, S., Witsch, K.: Nonlinear multigrid methods for total variation denosing. Computating and Visualization in Sciene (2004) (to appear)Google Scholar
  17. 17.
    Ghosal, S., Vaněk, P.Č.: Scalable algorithm for discontinuous optical flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(2), 181–194 (1996)CrossRefGoogle Scholar
  18. 18.
    Glazer, F.: Multilevel relaxation in low-level computer vision. In: Rosenfeld, A. (ed.) Multiresolution Image Processing and Analysis, pp. 312–330. Springer, Berlin (1984)Google Scholar
  19. 19.
    Hackbusch, W.: Multigrid Methods and Applications. Springer, New York (1985)Google Scholar
  20. 20.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  21. 21.
    Huber, P.J.: Robust Statistics. Wiley, New York (1981)MATHCrossRefGoogle Scholar
  22. 22.
    Kimmel, A., Yavneh, I.: An algebraic multigrid approach for image analysis. SIAM Journal on Scientific Computing 24(4), 1218–1231 (2003)MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Kumar, A., Tannenbaum, A.R., Balas, G.J.: Optic flow: a curve evolution approach. IEEE Transactions on Image Processing 5(4), 598–610 (1996)CrossRefGoogle Scholar
  24. 24.
    Luettgen, M.R., Karl, W.C., Willsky, A.S.: Efficient multiscale regularization with applications to the computation of optical flow. IEEE Transactions on Image Processing 3(1), 41–64 (1994)CrossRefGoogle Scholar
  25. 25.
    Mémin, E., Pérez, P.: A multigrid approach for hierarchical motion estimation. In: Proc. 6th International Conference on Computer Vision, Bombay, India, January 1998, pp. 933–938 (1998)Google Scholar
  26. 26.
    Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 565–593 (1986)CrossRefGoogle Scholar
  27. 27.
    Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. In: Classics in Applied Mathematics. vol. 30, SIAM, Philadelphia (2000)Google Scholar
  28. 28.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)MATHCrossRefGoogle Scholar
  29. 29.
    Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: Proc. Twelfth International Conference on Pattern Recognition, Jerusalem, Israel, October 1994, vol. A, pp. 661–663. IEEE Computer Society Press, Los Alamitos (1994)CrossRefGoogle Scholar
  30. 30.
    Terzopoulos, D.: Image analysis using multigrid relaxation. IEEE Transactions on Pattern Analysis and Machine Intelligence 8(2), 129–139 (1986)CrossRefGoogle Scholar
  31. 31.
    Trottenberg, U., Oosterlee, C., Schüller, A.: Multigrid. Academic Press, San Diego (2001)MATHGoogle Scholar
  32. 32.
    Vogel, C.R.: A multigrid method for total variation-based image denosing. Computation and Control IV 20, 323–331 (1995)Google Scholar
  33. 33.
    Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. International Journal of Computer Vision 45(3), 245–264 (2001)MATHCrossRefGoogle Scholar
  34. 34.
    Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision 14(3), 245–255 (2001)MATHCrossRefGoogle Scholar
  35. 35.
    Wesseling, P.: An Introduction to Multigrid Methods. Wiley, Chichester (1992)MATHGoogle Scholar
  36. 36.
    Young, D.M.: Iterative Solution of Large Linear Systems. Academic Press, New York (1971)MATHGoogle Scholar
  37. 37.
    Zini, G., Sarti, A., Lamberti, C.: Application of continuum theory and multi-grid methods to motion evaluation from 3D echocardiography. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 44(2), 297–308 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Andrés Bruhn
    • 1
  • Joachim Weickert
    • 1
  • Timo Kohlberger
    • 2
  • Christoph Schnörr
    • 2
  1. 1.Mathematical Image Analysis Group, Faculty of Mathematics and Computer Science, Building 27.1Saarland UniversitySaarbrückenGermany
  2. 2.Computer Vision, Graphics, and Pattern Recognition Group, Department of Mathematics and Computer ScienceUniversity of MannheimMannheimGermany

Personalised recommendations