Skip to main content
SpringerLink
Log in

Optical Flow Estimation

  • Chapter
Stereo Scene Flow for 3D Motion Analysis

Abstract

In this chapter we review the estimation of the two-dimensional apparent motion field of two consecutive images in an image sequence. This apparent motion field is referred to as optical flow field, a two-dimensional vector field on the image plane. Because it is nearly impossible to cover the vast amount of approaches in the literature, in this chapter we set the focus on energy minimization approaches which estimate a dense flow field. The term dense refers to the fact that a flow vector is assigned to every (non-occluded) image pixel. Most dense approaches are based on the variational formulation of the optical flow problem, firstly suggested by Horn and Schunk. Depending on the application, density might be one important property besides accuracy and robustness. In many cases computational speed and real-time capability is a crucial issue. In this chapter we therefore discuss the latest progress in accuracy, robustness and real-time capability of dense optical flow algorithms.

figure a

Space is a still of time, while time is space in motion. Christopher R. Hallpike

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alvarez, L., Esclarín, J., Lefébure, M., Sánchez, J.: A PDE model for computing the optical flow. In: Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, Gran Canaria, Spain, pp. 1349–1356 (1999)

    Google Scholar 

  2. Aubert, G., Deriche, R., Kornprobst, P.: Computing optical flow via variational techniques. SIAM J. Appl. Math. 60(1), 156–182 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  3. Aujol, J.F., Gilboa, G., Chan, T.F., Osher, S.J.: Structure-texture image decomposition: modeling, algorithms, and parameter selection. Int. J. Comput. Vis. 67(1) (2006). doi:10.1007/s11263-006-4331-z

  4. Baker, S., Matthews, I.: Lucas-Kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004)

    Article  Google Scholar 

  5. Baker, S., Roth, S., Scharstein, D., Black, M.J., Lewis, J.P., Szeliski, R.: A database and evaluation methodology for optical flow. In: Online-Proc. International Conference on Computer Vision, Rio de Janeiro, Brazil, October 2007

    Google Scholar 

  6. Barni, M., Cappellini, V., Mecocci, A.: Fast vector median filter based on Euclidean norm approximation. IEEE Signal Process. Lett. 1(6), 92–94 (2004)

    Article  Google Scholar 

  7. Black, M.J., Anandan, P.: A framework for the robust estimation of optical flow. In: Proc. International Conference on Computer Vision, Nice, France, pp. 231–236 (1993)

    Google Scholar 

  8. Brox, T.: From pixels to regions: partial differential equations in image analysis. Ph.D. thesis, Faculty of Mathematics and Computer Science, Saarland University, Germany (2005)

    Google Scholar 

  9. Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. (2010). doi:10.1109/TPAMI.2010.143

    Google Scholar 

  10. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Proc. European Conference on Computer Vision, Prague, Czech Republic, pp. 25–36 (2004)

    Google Scholar 

  11. Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Variational optic flow computation in real-time. IEEE Trans. Image Process. 14(5), 608–615 (2005)

    Article  MathSciNet  Google Scholar 

  12. Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C.: A multigrid platform for real-time motion computation with discontinuity-preserving variational methods. Int. J. Comput. Vis. 70(3), 257–277 (2006)

    Article  Google Scholar 

  13. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1), 89–97 (2004)

    Article  MathSciNet  Google Scholar 

  14. Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Proc. International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, St. Augustine, FL, USA, pp. 136–152 (2005)

    Chapter  Google Scholar 

  15. Chambolle, A., Caselles, V., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. In: Theoretical Foundations and Numerical Methods for Sparse Recovery. De Gruyter, Berlin (2010)

    Google Scholar 

  16. Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Appl. Math. 20(10), 1964–1977 (1999)

    MathSciNet  MATH  Google Scholar 

  17. Cohen, I.: Nonlinear variational method for optical flow computation. In: Scandinavian Conf. on Image Analysis, pp. 523–523 (1993)

    Google Scholar 

  18. Coleman, T.F., Hulbert, L.A.: A direct active set algorithm for large sparse quadratic programs with simple bounds. Math. Program., Sers. A, B 45(3), 373–406 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  19. Cooke, T.: Two applications of graph-cuts to image processing. In: Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, pp. 498–504 (2008)

    Chapter  Google Scholar 

  20. Corpetti, T., Memin, E., Perez, P.: Dense estimation of fluid flows. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 365–380 (2002)

    Article  Google Scholar 

  21. Deriche, R., Kornprobst, P., Aubert, G.: Optical flow estimation while preserving its discontinuities: a variational approach. In: Proc. Asian Conference on Computer Vision, Singapore, pp. 290–295 (1995)

    Google Scholar 

  22. Devillard, N.: Fast median search: an ANSI C implementation (1998). http://ndevilla.free.fr/median/median/index.html

  23. Felsberg, M.: On the relation between anisotropic diffusion and iterated adaptive filtering. In: Pattern Recognition (Proc. DAGM), Munich, Germany, pp. 436–445 (2008)

    Chapter  Google Scholar 

  24. Goldluecke, B., Cremers, D.: Convex relaxation for multilabel problems with product label spaces. In: Proc. European Conference on Computer Vision (2010)

    Google Scholar 

  25. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)

    Article  Google Scholar 

  26. Karush, W.: Minima of functions of several variables with inequalities as side constraints. Ph.D. thesis, Dept. of Mathematics, University of Chicago (1939)

    Google Scholar 

  27. Klappstein, J.: Optical-flow based detection of moving objects in traffic scenes. Ph.D. thesis, University of Heidelberg, Heidelberg, Germany (2008)

    Google Scholar 

  28. Lempitsky, V., Roth, S., Rother, C.: Fusionflow: discrete-continuous optimization for optical flow estimation. In: Online-Proc. International Conference on Computer Vision and Pattern Recognition, Anchorage, USA, June 2008

    Google Scholar 

  29. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. of the 7th International Joint Conference on Artificial Intelligence (IJCAI), Vancouver, British Columbia, Canada, pp. 674–679 (1981)

    Google Scholar 

  30. Luenberger, D.G.: Linear and Nonlinear Programming. Addison-Wesley, Reading (1984)

    MATH  Google Scholar 

  31. Masoomzadeh-Fard, A., Venetsanopoulos, A.N.: An efficient vector ranking filter for colour image restoration. In: Proc. Canadian Conference on Electrical and Computer Engineering, Vancouver, BC, Canada, pp. 1025–1028 (1993)

    Google Scholar 

  32. Mémin, E., Pérez, P.: A multigrid approach for hierarchical motion estimation. In: Proc. International Conference on Computer Vision, Bombay, India, pp. 933–938 (1998)

    Google Scholar 

  33. Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986)

    Article  Google Scholar 

  34. Nagel, H.H.: Constraints for the estimation of displacement vector fields from image sequences. In: Proc. Eighth Int. Conf. Artif. Intell., Karlsruhe, Germany, pp. 945–951 (1983)

    Google Scholar 

  35. Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 8(5), 565–593 (1986)

    Article  Google Scholar 

  36. Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optic flow computation with theoretically justified warping. Int. J. Comput. Vis. 67(2), 141–158 (2006)

    Article  Google Scholar 

  37. Pock, T.: Fast total variation for computer vision. Ph.D. thesis, Institute for Computer Graphics and Vision, University of Graz, Graz, Austria (2008)

    Google Scholar 

  38. Rabe, C., Volmer, C., Franke, U.: Kalman filter based detection of obstacles and lane boundary. Autonome Mobile Systeme 19(1), 51–58 (2005)

    Google Scholar 

  39. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MATH  Google Scholar 

  40. Ruhnau, P., Schnoerr, C.: Variational estimation of experimental fluid flows with physics-based spatio-temporal regularization. Meas. Sci. Technol. 18, 755–763 (2007)

    Article  Google Scholar 

  41. Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: 12th Int. Conf. on Pattern Recognition, Jerusalem, Israel, pp. 661–663 (1994)

    Chapter  Google Scholar 

  42. Spies, H., Kirchges̈ner, N., Scharr, H., Jähne, B.: Dense structure estimation via regularised optical flow. In: Proc. Vision, Modeling, and Visualization, Saarbrücken, Germany, pp. 57–64 (2000)

    Google Scholar 

  43. Stein, F.: Efficient computation of optical flow using the Census transform. In: Pattern Recognition (Proc. DAGM), Tübingen, Germany, pp. 79–86 (2004)

    Chapter  Google Scholar 

  44. Steinbruecker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: IEEE International Conference on Computer Vision (ICCV), Kyoto, Japan (2009)

    Google Scholar 

  45. Stewart, E.: Intel Integrated Performance Primitives: How to Optimize Software Applications Using Intel Ipp. Intel Press, Santa Clara (2004)

    Google Scholar 

  46. Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Proc. European Conference on Computer Vision, Marseille, France, pp. 83–91 (2008)

    Google Scholar 

  47. Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: Proc. International Conference on Computer Vision and Pattern Recognition, pp. 2432–2439 (2010)

    Google Scholar 

  48. Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon University, April 1991

    Google Scholar 

  49. Trobin, W., Pock, T., Cremers, D., Bischof, H.: Continuous energy minimization via repeated binary fusion. In: European Conference on Computer Vision (ECCV), Marseille, France, October 2008

    Google Scholar 

  50. Trobin, W., Pock, T., Cremers, D., Bischof, H.: An unbiased second-order prior for high-accuracy motion estimation. In: Pattern Recognition (Proc. DAGM), Munich, Germany, pp. 396–405 (2008)

    Chapter  Google Scholar 

  51. Unger, M., Pock, T., Bischof, H.: Continuous globally optimal image segmentation with local constraints. In: Computer Vision Winter Workshop, February 2008

    Google Scholar 

  52. Weickert, J., Brox, T.: Diffusion and regularization of vector- and matrix-valued images. In: Inverse Problems, Image Analysis and Medical Imaging. Contemporary Mathematics vol. 313, pp. 251–268. AMS, Providence (2002)

    Google Scholar 

  53. Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. Int. J. Comput. Vis. 45(3), 245–264 (2001)

    Article  Google Scholar 

  54. Yang, Z., Fox, M.D.: Speckle reduction and structure enhancement by multichannel median boosted anisotropic diffusion. EURASIP J. Appl. Signal Process. 2492–2502 (2004). doi:10.1155/S1110865704402091

  55. Yin, W., Goldfarb, D., Osher, S.: Image cartoon-texture decomposition and feature selection using the total variation regularized L 1 functional. In: Variational, Geometric, and Level Set Methods in Computer Vision, Beijing, China, pp. 73–80 (2005)

    Chapter  Google Scholar 

  56. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proc. European Conference on Computer Vision, Prague, Czech Republic, pp. 151–158 (2004)

    Google Scholar 

  57. Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. In: Pattern Recognition (Proc. DAGM), Heidelberg, Germany, pp. 214–223 (2007)

    Chapter  Google Scholar 

  58. Zach, C., Gallup, D., Frahm, J.M.: Fast gain-adaptive KLT tracking on the GPU. In: Online-Proc. International Conference on Computer Vision and Pattern Recognition, Anchorage, AK, June 2008

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Group Research, Daimler AG, HPC 050–G023, Sindelfingen, 71059, Germany

    Dr. Andreas Wedel

  2. Department of Computer Science, Technical University of Munich, Boltzmannstraße 3, Garching, 85748, Germany

    Prof. Daniel Cremers

Authors
  1. Dr. Andreas Wedel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prof. Daniel Cremers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Wedel .

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Wedel, A., Cremers, D. (2011). Optical Flow Estimation. In: Stereo Scene Flow for 3D Motion Analysis. Springer, London. https://doi.org/10.1007/978-0-85729-965-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-965-9_2

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-964-2

  • Online ISBN: 978-0-85729-965-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation