Abstract
We study an energy functional for computing optical flow that combines three assumptions: a brightness constancy assumption, a gradient constancy assumption, and a discontinuity-preserving spatio-temporal smoothness constraint. In order to allow for large displacements, linearisations in the two data terms are strictly avoided. We present a consistent numerical scheme based on two nested fixed point iterations. By proving that this scheme implements a coarse-to-fine warping strategy, we give a theoretical foundation for warping which has been used on a mainly experimental basis so far. Our evaluation demonstrates that the novel method gives significantly smaller angular errors than previous techniques for optical flow estimation. We show that it is fairly insensitive to parameter variations, and we demonstrate its excellent robustness under noise.
We gratefully acknowledge partial funding by the Deutsche Forschungsgemeinschaft (DFG).
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Alvarez, L., Esclarin, J., Lefebure, M., Sanchez, J.: A PDE model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Las Palmas de Gran Canaria, Spain, September 1999, pp. 1349–1356 (1999)
Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision 39(1), 41–56 (2000)
Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision 2, 283–310 (1989)
Bab-Hadiashar, A., Suter, D.: Robust optic flow computation. International Journal of Computer Vision 29(1), 59–77 (1998)
Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. International Journal of Computer Vision 12(1), 43–77 (1994)
Black, M.J., Anandan, P.: Robust dynamic motion estimation over time. In: Proc. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Maui, HI, June 1991, pp. 292–302. IEEE Computer Society Press, Los Alamitos (1991)
Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)
Cohen, I.: Nonlinear variational method for optical flow computation. In: Proc. Eighth Scan-dinavian Conference on Image Analysis, Tromsø, Norway, May 1993, vol. 1, pp. 523–530 (1993)
Deriche, R., Kornprobst, P., Aubert, G.: Optical-flow estimation while preserving its dis-continuities: a variational approach. In: Li, S., Teoh, E.-K., Mital, D., Wang, H. (eds.) ACCV 1995. LNCS, vol. 1035, pp. 290–295. Springer, Heidelberg (1995)
Farnebáck, G.: Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In: Proc. Eighth International Conference on Computer Vision, Vancouver, Canada, July 2001, vol. 1, pp. 171–177. IEEE Computer Society Press, Los Alamitos (2001)
Horn, B., Schunck, B.: Determining optical flow. Artificial lntelligence 17, 185–203 (1981)
Ju, S., Black, M., Jepson, A.: Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency. In: Proc. 1996 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, June 1996, pp. 307–314. IEEE Computer Society Press, Los Alamitos (1996)
Lai, S.-H., Vemuri, B.C.: Reliable and efficient computation of optical flow. International Journal of Computer Vision 29(2), 87–105 (1998)
Lefeburé, M., Cohen, L.D.: Image registration, optical flow and local rigidity. Journal of Mathematical Imaging and Vision 14(2), 131–147 (2001)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. Seventh International Joint Conference on Artificial Intelligence, Vancouver, Canada, August 1981, pp. 674–679 (1981)
Mémin, E., Peréz, P.: A multigrid approach for hierarchical motion estimation. In: Proc. Sixth International Conference on Computer Vision, Bombay, India, January 1998, pp. 933–938. Narosa Publishing House, Bombay (1998)
Mémin, E., Peréz, P.: Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision 46(2), 129–155 (2002)
Nagel, H.-H.: Extending the ’oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In: Faugeras, O. (ed.) ECCV 1990. LNCS, vol. 427, pp. 139–148. Springer, Heidelberg (1990)
Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 565–593 (1986)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: Proc. Twelfth International Conference on Pattern Recognition, Jerusalem, Israel, October 1994, vol. A, pp. 661–663. IEEE Computer Society Press, Los Alamitos (1994)
Tistarelli, M.: Multiple constraints for optical flow. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 800, pp. 61–70. Springer, Heidelberg (1994)
Uras, S., Girosi, F., Verri, A., Torre, V.: A computational approach to motion perception. Biological Cybernetics 60, 79–87 (1988)
Weickert, J., Bruhn, A., Schnorr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. Technical Report 82, Dept. of Mathematics, Saarland University, Saarbrucken, Germany (Apr 2003)
Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. International Journal of Computer Vision 45(3), 245–264 (2001)
Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. Journal of Mathematical Imaging and Vision 14(3), 245–255 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Brox, T., Bruhn, A., Papenberg, N., Weickert, J. (2004). High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J. (eds) Computer Vision - ECCV 2004. ECCV 2004. Lecture Notes in Computer Science, vol 3024. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24673-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-24673-2_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21981-1
Online ISBN: 978-3-540-24673-2
eBook Packages: Springer Book Archive