Skip to main content
SpringerLink
Log in

Optical Flow with Geometric Occlusion Estimation and Fusion of Multiple Frames

  • Conference paper
Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2015)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8932))

Abstract

Optical flow research has made significant progress in recent years and it can now be computed efficiently and accurately for many images. However, complex motions, large displacements, and difficult imaging conditions are still problematic. In this paper, we present a framework for estimating optical flow which leads to improvements on these difficult cases by 1) estimating occlusions and 2) using additional temporal information. First, we divide the image into discrete triangles and show how this allows for occluded regions to be naturally estimated and directly incorporated into the optimization algorithm. We additionally propose a novel method of dealing with temporal information in image sequences by using “inertial estimates” of the flow. These estimates are combined using a classifier-based fusion scheme, which significantly improves results. These contributions are evaluated on three different optical flow datasets, and we achieve state-of-the-art results on MPI-Sintel.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Horn, B.K., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17(1), 185–203 (1981)

    Article  Google Scholar 

  2. Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)

    Google Scholar 

  3. Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR (2010)

    Google Scholar 

  4. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV 92(1), 1–31 (2011)

    Article  Google Scholar 

  5. Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  6. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: CVPR (2012)

    Google Scholar 

  7. Lempitsky, V., Roth, S., Rother, C.: Fusionflow: Discrete-continuous optimization for optical flow estimation. In: CVPR, pp. 1–8. IEEE (2008)

    Google Scholar 

  8. Xu, L., Chen, J., Jia, J.: A segmentation based variational model for accurate optical flow estimation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 671–684. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  9. Strecha, C., Fransens, R., Van Gool, L.: A probabilistic approach to large displacement optical flow and occlusion detection. In: Comaniciu, D., Mester, R., Kanatani, K., Suter, D. (eds.) SMVP 2004. LNCS, vol. 3247, pp. 71–82. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  10. Sun, D., Sudderth, E.B., Black, M.J.: Layered image motion with explicit occlusions, temporal consistency, and depth ordering. In: NIPS, pp. 2226–2234 (2010)

    Google Scholar 

  11. Glocker, B., Heibel, T.H., Navab, N., Kohli, P., Rother, C.: TriangleFlow: Optical flow with triangulation-based higher-order likelihoods. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 272–285. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  12. Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. PAMI 34(9), 1744–1757 (2012)

    Article  Google Scholar 

  13. Kim, T.H., Lee, H.S., Lee, K.M.: Optical flow via locally adaptive fusion of complementary data costs. In: ICCV (2013)

    Google Scholar 

  14. Sun, D., Liu, C., Pfister, H.: Local layering for joint motion estimation and occlusion detection. In: CVPR (2014)

    Google Scholar 

  15. Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: ICCV, pp. 1116–1123. IEEE (2011)

    Google Scholar 

  16. Sun, D., Wulff, J., Sudderth, E.B., Pfister, H., Black, M.J.: A fully-connected layered model of foreground and background flow. In: CVPR (2013)

    Google Scholar 

  17. Mac Aodha, O., Humayun, A., Pollefeys, M., Brostow, G.J.: Learning a confidence measure for optical flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(5), 1107–1120 (2013)

    Article  Google Scholar 

  18. Jung, H.Y., Lee, K.M., Lee, S.U.: Toward global minimum through combined local minima. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 298–311. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  19. Chang, H.S., Wang, Y.C.F.: Superpixel-based large displacement optical flow. In: ICIP, pp. 3835–3839 (2013)

    Google Scholar 

  20. Negahdaripour, S.: Revised definition of optical flow: Integration of radiometric and geometric cues for dynamic scene analysis. PAMI 20(9), 961–979 (1998)

    Article  Google Scholar 

  21. Donoser, M., Schmalstieg, D.: Discrete-continuous gradient orientation estimation for faster image segmentation. In: CVPR (2014)

    Google Scholar 

  22. Cowper, G.: Gaussian quadrature formulas for triangles. International Journal for Numerical Methods in Engineering 7(3), 405–408 (1973)

    Article  MATH  Google Scholar 

  23. Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 83–97. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  24. Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. PAMI 33(3), 500–513 (2011)

    Article  Google Scholar 

  25. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893. IEEE (2005)

    Google Scholar 

  26. Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: VISAPP, pp. 331–340 (2009)

    Google Scholar 

  27. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: Large displacement optical flow with deep matching. In: ICCV (2013)

    Google Scholar 

  28. Byrne, J., Shi, J.: Nested shape descriptors. In: ICCV, pp. 1201–1208. IEEE (2013)

    Google Scholar 

  29. Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: CVPR, pp. 2464–2471. IEEE (2010)

    Google Scholar 

  30. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  31. Amestoy, P.R., Davis, T.A., Duff, I.S.: Algorithm 837: Amd, an approximate minimum degree ordering algorithm. ACM Trans. Math. Softw. 30(3), 381–388 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  32. Fortun, D., Bouthemy, P., Kervrann, C.: Aggregation of local parametric candidates with exemplar-based occlusion handling for optical flow. arXiv preprint arXiv:1407.5759v1

    Google Scholar 

  33. Yamaguchi, K., McAllester, D., Urtasun, R.: Robust monocular epipolar flow estimation. In: CVPR, pp. 1862–1869. IEEE (2013)

    Google Scholar 

  34. Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Pennsylvania, USA

    Ryan Kennedy & Camillo J. Taylor

Authors
  1. Ryan Kennedy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Camillo J. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mathematics, University of Bergen, 7800, Bergen, Norway

    Xue-Cheng Tai

  2. Department of Mathematics, University of California, Los Angeles, CA, USA

    Egil Bae

  3. The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, S.A.R.

    Tony F. Chan

  4. Telemark University College, Postboks 203, 3901, Porsgrunn, Norway

    Marius Lysaker

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Kennedy, R., Taylor, C.J. (2015). Optical Flow with Geometric Occlusion Estimation and Fusion of Multiple Frames. In: Tai, XC., Bae, E., Chan, T.F., Lysaker, M. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2015. Lecture Notes in Computer Science, vol 8932. Springer, Cham. https://doi.org/10.1007/978-3-319-14612-6_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14612-6_27

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14611-9

  • Online ISBN: 978-3-319-14612-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation