Adaptive Integration of Feature Matches into Variational Optical Flow Methods

  • Michael Stoll
  • Sebastian Volz
  • Andrés Bruhn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7726)


Despite the significant progress in terms of accuracy achieved by recent variational optical flow methods, the correct handling of large displacements still poses a severe problem for many algorithms. In particular if the motion exceeds the size of an object, standard coarse-to-fine estimation schemes fail to produce meaningful results. While the integration of point correspondences may help to overcome this limitation, such strategies often deteriorate the performance for small displacements due to false or ambiguous matches. In this paper we address the aforementioned problem by proposing an adaptive integration strategy for feature matches. The key idea of our approach is to use the matching energy of the baseline method to carefully select those locations where feature matches may potentially improve the estimation. This adaptive selection does not only reduce the runtime compared to an exhaustive search, it also improves the reliability of the estimation by identifying unnecessary and unreliable features and thus by excluding spurious matches. Results for the Middlebury benchmark and several other image sequences demonstrate that our approach succeeds in handling large displacements in such a way that the performance for small displacements is not compromised. Moreover, experiments even indicate that image sequences with small displacements can benefit from carefully selected point correspondences.


Motion Estimation Large Displacement Small Displacement Feature Match Baseline Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alvarez, L., Weickert, J., Sánchez, J.: Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision 39(1), 41–56 (2000)zbMATHCrossRefGoogle Scholar
  2. 2.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. International Journal of Computer Vision 92(1), 1–31 (2011)CrossRefGoogle Scholar
  3. 3.
    Berg, A., Malik, J.: Geometric blur for template matching. In: Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 607–614 (2001)Google Scholar
  4. 4.
    Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical Model-Based Motion Estimation. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 237–252. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  5. 5.
    Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding 63(1), 75–104 (1996)CrossRefGoogle Scholar
  6. 6.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3), 500–513 (2011)CrossRefGoogle Scholar
  8. 8.
    Bruhn, A., Weickert, J.: Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: Proc. IEEE International Conference on Computer Vision, pp. 749–755 (2005)Google Scholar
  9. 9.
    Dalal, N., Triggs, B.: Histogram of oriented gradients for human detection. In: Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2005)Google Scholar
  10. 10.
    Gwosdek, P., Zimmer, H., Grewenig, S., Bruhn, A., Weickert, J.: A highly efficient GPU implementation for variational optic flow based on the Euler-Lagrange framework. In: Proc. ECCV Workshop on Computer Vision with GPUs (2010)Google Scholar
  11. 11.
    Harris, C.G., Stephens, M.: A combined corner and edge detector. In: Proc. Alvey Vision Conference, pp. 147–152 (1988)Google Scholar
  12. 12.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  13. 13.
    Johnson, H.J., Christensen, G.E.: Consistent landmark and intensity-based registration. IEEE Transactions on Medical Imaging 21(5), 450–461 (2002)CrossRefGoogle Scholar
  14. 14.
    Li, R.: A new three-step search algorithm for block motion estimation. IEEE Transactions on Circuits and Systems for Video Technology 4(4), 438–442 (1994)CrossRefGoogle Scholar
  15. 15.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  16. 16.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proc. International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  17. 17.
    Ochs, P., Brox, T.: Object segmentation in video: A hierarchical variational approach for turning point trajectories into dense regions. In: Proc. IEEE International Conference on Computer Vision, pp. 1583–1590 (2011)Google Scholar
  18. 18.
    Rhemann, C., Hosni, A., Bleyer, M., Rother, C., Gelautz, M.: Fast cost-volume filtering for visual correspondence and beyond. In: Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3017–3024 (2011)Google Scholar
  19. 19.
    Steinbrücker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: Proc. IEEE International Conference on Computer Vision (2009)Google Scholar
  20. 20.
    Sun, D., Sudderth, E., Black, M.J.: Layered image motion with explicit occlusions, temporal consistency, and depth ordering. In: Proc. Advances in Neural Information Processing Systems 23, pp. 2226–2234 (2010)Google Scholar
  21. 21.
    Sundaram, N., Brox, T., Keutzer, K.: Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 438–451. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  22. 22.
    Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: Proc. IEEE International Conference on Computer Vision (2011)Google Scholar
  23. 23.
    Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. In: Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1293–1300 (2010)Google Scholar
  24. 24.
    Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., Seidel, H.-P.: Complementary Optic Flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 207–220. Springer, Heidelberg (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Michael Stoll
    • 1
  • Sebastian Volz
    • 1
  • Andrés Bruhn
    • 1
  1. 1.Institute for Visualisation and Interactive SystemsUniversity of StuttgartGermany

Personalised recommendations