Sparse Aggregation Framework for Optical Flow Estimation

  • Denis FortunEmail author
  • Patrick Bouthemy
  • Charles Kervrann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9087)


We propose a sparse aggregation framework for optical flow estimation to overcome the limitations of variational methods introduced by coarse-to-fine strategies. The idea is to compute parametric motion candidates estimated in overlapping square windows of variable size taken in the semi-local neighborhood of a given point. In the second step, a sparse representation and an optimization procedure in the continuous setting are proposed to compute a motion vector close to motion candidates for each pixel. We demonstrate the feasibility and performance of our two-step approach on image pairs and compare its performances with competitive methods on the Middlebury benchmark.


Motion estimation Optical flow Sparse representation Optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV 92(1), 1–31 (2011)CrossRefGoogle Scholar
  2. 2.
    Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The generalized patchmatch correspondence algorithm. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 29–43. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  3. 3.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. CVIU 63(1), 75–104 (1996)Google Scholar
  4. 4.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  5. 5.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  6. 6.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. PAMI 33(3), 500–513 (2011)CrossRefGoogle Scholar
  7. 7.
    Bruhn, A., Weickert, W.: A confidence measure for variational optic flow methods. In: Geometric Properties for Incomplete Data, pp. 283–298 (2006)Google Scholar
  8. 8.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  9. 9.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision 40(1), 120–145 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Chen, Z., Jin, H., Lin, Z., Cohen, S., Wu, Y.: Large displacement optical flow from nearest neighbor fields. In: CVPR, pp. 2443–2450 (2013)Google Scholar
  11. 11.
    Fortun, D., Bouthemy, P., Kervrann, C.: Aggregation of local parametric candidates with exemplar-based occlusion handling for optical flow. arXiv:1407.5759 (2014) (preprint)
  12. 12.
    Fortun, D., Bouthemy, P., Kervrann, C.: Optical flow modeling and computation: a survey. In: CVIU (2015)Google Scholar
  13. 13.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Art. Intel. 17(1–3), 185–203 (1981)CrossRefGoogle Scholar
  14. 14.
    Kondermann, C., Mester, R., Garbe, C.S.: A statistical confidence measure for optical flows. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 290–301. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  15. 15.
    Kybic, J., Nieuwenhuis, C.: Bootstrap optical flow confidence and uncertainty measure. CVIU 115(10), 1449–1462 (2011)Google Scholar
  16. 16.
    Leordeanu, M., Zanfir, A., Sminchisescu, C.: Locally affine sparse-to-dense matching for motion and occlusion estimation. In: ICCV, Sydney, Australia, pp. 1221–1728 (2013)Google Scholar
  17. 17.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. Int. Joint Conf. Art. Intel., pp. 674–679 (1981)Google Scholar
  18. 18.
    Mémin, E., Pérez, P.: Dense estimation and object-based segmentation of the optical flow with robust techniques. TIP 7(5), 703–719 (1998)Google Scholar
  19. 19.
    Mota, C., Stuke, L., Barth, E.: Analytic solutions for multiple motions. In: ICIP, Thessaloniki, Greece, pp. 917–920 (2001)Google Scholar
  20. 20.
    Mozerov, M.: Constrained optical flow estimation as a matching problem. TIP 22(5), 2044–2055 (2013)MathSciNetGoogle Scholar
  21. 21.
    Odobez, J.M., Bouthemy, P.: Robust multiresolution estimation of parametric motion models. JVCIR 6(4), 348–365 (1995)Google Scholar
  22. 22.
    Rose, K.: Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proceedings of the IEEE 86(11), 2210–2239 (1998)CrossRefGoogle Scholar
  23. 23.
    Sun, D., Sudderth, E.B., Black, M.J.: Layered image motion with explicit occlusions, temporal consistency, and depth ordering. In: NIPS, Vancouver, Canada, pp. 2226–2234 (2010)Google Scholar
  24. 24.
    Wedel, A., Cremers, D., Pock, T., Bischof, H.: Structure-and motion-adaptive regularization for high accuracy optic flow. In: ICCV, Kyoto, Japan, pp. 1663–1668 (October 2009)Google Scholar
  25. 25.
    Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C., et al.: Deepflow: large displacement optical flow with deep matching. In: ICCV, Sydney, Australia, pp. 1385–1392 (2013)Google Scholar
  26. 26.
    Werlberger, M., Pock, T., Bischof, H.: Motion estimation with non-local total variation regularization. In: CVPR, San-Fransisco, pp. 2464–2471 (2010)Google Scholar
  27. 27.
    Zimmer, H., Bruhn, A., Weickert, J.: Optic flow in harmony. IJCV 93(3), 1–21 (2011)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Denis Fortun
    • 1
    Email author
  • Patrick Bouthemy
    • 1
  • Charles Kervrann
    • 1
  1. 1.Centre de Rennes - Bretagne AtlantiqueInriaRennesFrance

Personalised recommendations