High Accuracy Optical Flow Estimation Based on a Theory for Warping

  • Thomas Brox
  • Andrés Bruhn
  • Nils Papenberg
  • Joachim Weickert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3024)


We study an energy functional for computing optical flow that combines three assumptions: a brightness constancy assumption, a gradient constancy assumption, and a discontinuity-preserving spatio-temporal smoothness constraint. In order to allow for large displacements, linearisations in the two data terms are strictly avoided. We present a consistent numerical scheme based on two nested fixed point iterations. By proving that this scheme implements a coarse-to-fine warping strategy, we give a theoretical foundation for warping which has been used on a mainly experimental basis so far. Our evaluation demonstrates that the novel method gives significantly smaller angular errors than previous techniques for optical flow estimation. We show that it is fairly insensitive to parameter variations, and we demonstrate its excellent robustness under noise.


IEEE Computer Society Motion Estimation Angular Error Point Iteration Smoothness Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alvarez, L., Esclarin, J., Lefebure, M., Sanchez, J.: A PDE model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Las Palmas de Gran Canaria, Spain, September 1999, pp. 1349–1356 (1999)Google Scholar
  2. 2.
    Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision 39(1), 41–56 (2000)zbMATHCrossRefGoogle Scholar
  3. 3.
    Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision 2, 283–310 (1989)CrossRefGoogle Scholar
  4. 4.
    Bab-Hadiashar, A., Suter, D.: Robust optic flow computation. International Journal of Computer Vision 29(1), 59–77 (1998)CrossRefGoogle Scholar
  5. 5.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. International Journal of Computer Vision 12(1), 43–77 (1994)CrossRefGoogle Scholar
  6. 6.
    Black, M.J., Anandan, P.: Robust dynamic motion estimation over time. In: Proc. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Maui, HI, June 1991, pp. 292–302. IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  7. 7.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
  8. 8.
    Cohen, I.: Nonlinear variational method for optical flow computation. In: Proc. Eighth Scan-dinavian Conference on Image Analysis, Tromsø, Norway, May 1993, vol. 1, pp. 523–530 (1993)Google Scholar
  9. 9.
    Deriche, R., Kornprobst, P., Aubert, G.: Optical-flow estimation while preserving its dis-continuities: a variational approach. In: Li, S., Teoh, E.-K., Mital, D., Wang, H. (eds.) ACCV 1995. LNCS, vol. 1035, pp. 290–295. Springer, Heidelberg (1995)Google Scholar
  10. 10.
    Farnebáck, G.: Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In: Proc. Eighth International Conference on Computer Vision, Vancouver, Canada, July 2001, vol. 1, pp. 171–177. IEEE Computer Society Press, Los Alamitos (2001)CrossRefGoogle Scholar
  11. 11.
    Horn, B., Schunck, B.: Determining optical flow. Artificial lntelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  12. 12.
    Ju, S., Black, M., Jepson, A.: Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency. In: Proc. 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, June 1996, pp. 307–314. IEEE Computer Society Press, Los Alamitos (1996)Google Scholar
  13. 13.
    Lai, S.-H., Vemuri, B.C.: Reliable and efficient computation of optical flow. International Journal of Computer Vision 29(2), 87–105 (1998)CrossRefGoogle Scholar
  14. 14.
    Lefeburé, M., Cohen, L.D.: Image registration, optical flow and local rigidity. Journal of Mathematical Imaging and Vision 14(2), 131–147 (2001)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. Seventh International Joint Conference on Artificial Intelligence, Vancouver, Canada, August 1981, pp. 674–679 (1981)Google Scholar
  16. 16.
    Mémin, E., Peréz, P.: A multigrid approach for hierarchical motion estimation. In: Proc. Sixth International Conference on Computer Vision, Bombay, India, January 1998, pp. 933–938. Narosa Publishing House, Bombay (1998)Google Scholar
  17. 17.
    Mémin, E., Peréz, P.: Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision 46(2), 129–155 (2002)zbMATHCrossRefGoogle Scholar
  18. 18.
    Nagel, H.-H.: Extending the ’oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In: Faugeras, O. (ed.) ECCV 1990. LNCS, vol. 427, pp. 139–148. Springer, Heidelberg (1990)CrossRefGoogle Scholar
  19. 19.
    Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 565–593 (1986)CrossRefGoogle Scholar
  20. 20.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)zbMATHCrossRefGoogle Scholar
  21. 21.
    Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: Proc. Twelfth International Conference on Pattern Recognition, Jerusalem, Israel, October 1994, vol. A, pp. 661–663. IEEE Computer Society Press, Los Alamitos (1994)Google Scholar
  22. 22.
    Tistarelli, M.: Multiple constraints for optical flow. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 800, pp. 61–70. Springer, Heidelberg (1994)Google Scholar
  23. 23.
    Uras, S., Girosi, F., Verri, A., Torre, V.: A computational approach to motion perception. Biological Cybernetics 60, 79–87 (1988)CrossRefGoogle Scholar
  24. 24.
    Weickert, J., Bruhn, A., Schnorr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. Technical Report 82, Dept. of Mathematics, Saarland University, Saarbrucken, Germany (Apr 2003)Google Scholar
  25. 25.
    Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. International Journal of Computer Vision 45(3), 245–264 (2001)zbMATHCrossRefGoogle Scholar
  26. 26.
    Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision 14(3), 245–255 (2001)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Thomas Brox
    • 1
  • Andrés Bruhn
    • 1
  • Nils Papenberg
    • 1
  • Joachim Weickert
    • 1
  1. 1.Mathematical Image Analysis Group, Faculty of Mathematics and Computer ScienceSaarland UniversitySaarbrückenGermany

Personalised recommendations