Skip to main content
SpringerLink
Log in

Over-Parameterized Optical Flow Using a Stereoscopic Constraint

  • Conference paper
Book cover Scale Space and Variational Methods in Computer Vision (SSVM 2011)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 6667))

Abstract

The success of variational methods for optical flow computation lies in their ability to regularize the problem at a differential (pixel) level and combine piecewise smoothness of the flow field with the brightness constancy assumptions. However, the piecewise smoothness assumption is often motivated by heuristic or algorithmic considerations. Lately, new priors were proposed to exploit the structural properties of the flow. Yet, most of them still utilize a generic regularization term.

In this paper we consider optical flow estimation in static scenes. We show that introducing a suitable motion model for the optical flow allows us to pose the regularization term as a geometrically meaningful one. The proposed method assumes that the visible surface can be approximated by a piecewise smooth planar manifold. Accordingly, the optical flow between two consecutive frames can be locally regarded as a homography consistent with the epipolar geometry and defined by only three parameters at each pixel. These parameters are directly related to the equation of the scene local tangent plane, so that their spatial variations should be relatively small, except for creases and depth discontinuities. This leads to a regularization term that measures the total variation of the model parameters and can be extended to a Mumford-Shah segmentation of the visible surface. This new technique yields significant improvements over state of the art optical flow computation methods for static scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)

    Article  Google Scholar 

  2. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)

    Google Scholar 

  3. Bergen, J., Anandan, P., Hanna, K., Hingorani, R.: Hierarchical model-based motion estimation. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 237–252. Springer, Heidelberg (1992)

    Chapter  Google Scholar 

  4. Mémin, E., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision 46, 129–155 (2002)

    Article  MATH  Google Scholar 

  5. Roth, S., Black, M.J.: On the spatial statistics of optical flow. International Journal of Computer Vision 74, 33–50 (2007)

    Article  Google Scholar 

  6. Black, M.J., Anandan, P.: A framework for the robust estimation of optical flow. In: International Conference on Computer Vision, pp. 231–236 (1993)

    Google Scholar 

  7. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  8. Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. International Journal of Computer Vision 61, 211–231 (2005)

    Article  Google Scholar 

  9. Cohen, I.: Nonlinear variational method for optical flow computation. In: Proc. Eighth Scandinavian Conference on Image Analysis, vol. 1, pp. 523–530 (1993)

    Google Scholar 

  10. Birchfield, S., Tomasi, C.: Depth discontinuities by pixel-to-pixel stereo. International Journal of Computer Vision 35, 269–293 (1999)

    Article  Google Scholar 

  11. Slesareva, N., Bruhn, A., Weickert, J.: Optic flow goes stereo: A variational method for estimating discontinuity-preserving dense disparity maps. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 33–40. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  12. Valgaerts, L., Bruhn, A., Weickert, J.: A variational model for the joint recovery of the fundamental matrix and the optical flow. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 314–324. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  13. Wedel, A., Pock, T., Braun, J., Franke, U., Cremers, D.: Duality TV-L1 flow with fundamental matrix prior. Image Vision and Computing New Zealand, 1–6 (2008)

    Google Scholar 

  14. Ben-Ari, R., Sochen, N.: Variational stereo vision with sharp discontinuities and occlusion handling. In: International Conference on Computer Vision, pp. 1–7. IEEE Computer Society (2007)

    Google Scholar 

  15. Pons, J.P., Keriven, R., Faugeras, O., Hermosillo, G.: Variational stereovision and 3d scene flow estimation with statistical similarity measures. In: International Conference on Computer Vision, vol. 1, p. 597 (2003)

    Google Scholar 

  16. Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: Computer Vision and Pattern Recognition, pp. 1–7 (2007)

    Google Scholar 

  17. Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: A view centered variational approach. In: Computer Vision and Pattern Recognition, pp. 1506–1513 (2010)

    Google Scholar 

  18. Nir, T., Bruckstein, A.M., Kimmel, R.: Over-parameterized variational optical flow. International Journal of Computer Vision 76, 205–216 (2008)

    Article  Google Scholar 

  19. Anonymous: Over-parameterized optical flow using a stereoscopic constraint (Technical report)

    Google Scholar 

  20. Ambrosio, L., Tortorelli, V.M.: Approximation of functional depending on jumps by elliptic functional via Γ-convergence. Communications on Pure and Applied Mathematics 43, 999–1036 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  21. Shah, J.: A common framework for curve evolution, segmentation and anisotropic diffusion. In: Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition, CVPR 1996, p. 136. IEEE Computer Society, Washington, DC (1996)

    Chapter  Google Scholar 

  22. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics 42, 577–685 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  23. Amiaz, T., Kiryati, N.: Piecewise-smooth dense optical flow via level sets. International Journal of Computer Vision 68, 111–124 (2006)

    Article  Google Scholar 

  24. Brune, C., Maurer, H., Wagner, M.: Detection of intensity and motion edges within optical flow via multidimensional control 2, 1190–1210 (2009)

    Google Scholar 

  25. Ben-Ari, R., Sochen, N.A.: Stereo matching with mumford-shah regularization and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2071–2084 (2010)

    Article  Google Scholar 

  26. Marr, D.: Vision: a computational investigation into the human representation and processing of visual information. W.H. Freeman, San Francisco (1982)

    Google Scholar 

  27. Trobin, W., Pock, T., Cremers, D., Bischof, H.: An unbiased second-order prior for high-accuracy motion estimation. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 396–405. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  28. Faugeras, O., Luong, Q.T.: The Geometry of Multiple Images. The MIT Press (2001) ISBN: 0262062208

    Google Scholar 

  29. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press (2004) ISBN: 0521540518

    Google Scholar 

  30. Mainberger, M., Bruhn, A., Weickert, J.: Is dense optic flow useful to compute the fundamental matrix? In: Campilho, A., Kamel, M.S. (eds.) ICIAR 2008. LNCS, vol. 5112, pp. 630–639. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  31. Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439 (2010)

    Google Scholar 

  32. Klappstein, J.: Optical-Flow Based Detection of Moving Objects in Traffic Scenes. PhD thesis, Ruprecht-Karls-Universität, Heidelberg (2008)

    Google Scholar 

  33. Baker, S., Roth, S., Scharstein, D., Black, M., Lewis, J., Szeliski, R.: A Database and Evaluation Methodology for Optical Flow. In: International Conference on Computer Vision, pp. 1–8 (2007)

    Google Scholar 

  34. Lempitsky, V., Rother, C., Roth, S., Blake, A.: Fusion moves for markov random field optimization. IEEE Trans. Pattern Anal. Mach. Intell. 99 (2009)

    Google Scholar 

  35. Liu, Y., Cao, X., Dai, Q., Xu, W.: Continuous depth estimation for multi-view stereo. In: Computer Vision and Pattern Recognition, pp. 2121–2128 (2009)

    Google Scholar 

  36. Kim, W., Park, J., Lee, K.: Stereo matching using population-based mcmc. International Journal of Computer Vision 83, 195–209 (2009)

    Article  Google Scholar 

  37. Amiaz, T., Lubetzky, E., Kiryati, N.: Coarse to over-fine optical flow estimation. Pattern Recognition 40, 2496–2503 (2007)

    Article  MATH  Google Scholar 

Download references

Authors
  1. Guy Rosman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shachar Shem-Tov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Bitton
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tal Nir
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gilad Adiv
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ron Kimmel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Arie Feuer
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Alfred M. Bruckstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Science Department, Technion — Israel Institute of Technology, 718 Taub Building Technion, 32000, Haifa, Israel

    Alfred M. Bruckstein

  2. Department of Biomedical Engineering, Biomedical Image Analysis and Interpretation, Eindhoven University of Technology, Den Dolech 2 – WH 2.101, 5600 MB, Eindhoven, The Netherlands

    Bart M. ter Haar Romeny

  3. Faculaty of Engineering, School of Electrical Engineering, Tel Aviv University, Ramat Aviv, 69978, Israel

    Alexander M. Bronstein

  4. Institute of Computationa Science, Faculty of Informatics SI-109, Università della Svizzera Italiana, Via Giuseppe Buffi 13, 6904, Lugano, Switzerland

    Michael M. Bronstein

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rosman, G. et al. (2012). Over-Parameterized Optical Flow Using a Stereoscopic Constraint. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2011. Lecture Notes in Computer Science, vol 6667. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24785-9_64

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-24785-9_64

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-24784-2

  • Online ISBN: 978-3-642-24785-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation