Hierarchical Scan-Line Dynamic Programming for Optical Flow Using Semi-Global Matching

  • Simon Hermann
  • Reinhard Klette
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7729)


Dense and robust optical flow estimation is still a major challenge in low-level computer vision. In recent years, mainly variational methods contributed to the progress in this field. One reason for their success is their suitability to be embedded into hierarchical schemes, which makes them capable of handling large pixel displacements. Matching-based regularization techniques, like dynamic programming or belief propagation concepts, can also lead to accurate optical flow fields. However, results are limited to short- or mid-scale optical flow vectors, because these techniques are usually not combined with coarse-to-fine strategies. This paper introduces fSGM, a novel algorithm that is based on scan-line dynamic programming. It uses the cost integration strategy of semi-global matching, a concept well known in the area of stereo matching. The major novelty of fSGM is that it embeds the scan-line dynamic programming approach into a hierarchical scheme, which allows it to handle large pixel displacements with an accuracy comparable to variational methods. We prove the exceptional performance of fSGM by comparing it to current state-of-the-art methods on the KITTI Vision Benchmark Suite.


Optical Flow Stereo Match Image Pyramid Hierarchical Scheme Depth Discontinuity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Analysis Machine Intelligence 33, 500–513 (2011)CrossRefGoogle Scholar
  3. 3.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. of Computer Vision (IJCV) 70, 41–54 (2006)CrossRefGoogle Scholar
  4. 4.
    Gehrig, S.K., Eberli, F., Meyer, T.: A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 134–143. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI Vision Benchmark Suite. In: Proc. Computer Vision Pattern Recognition, CVPR (2012)Google Scholar
  6. 6.
    Gong, M., Yang, Y.-H.: Estimate large motion using the reliability-based motion estimation algorithm. In: Proc. Int. Joint Conf. Artificial Intelligence (IJCAI), vol. 2, pp. 1120–1126 (1985)Google Scholar
  7. 7.
    Hermann, S., Morales, S., Vaudrey, T., Klette, R.: Illumination Invariant Cost Functions in Semi-Global Matching. In: Koch, R., Huang, F. (eds.) ACCV 2010 Workshops, Part II. LNCS, vol. 6469, pp. 245–254. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Hirschmüller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: Proc. IEEE Int. Conf. Computer Vision Pattern Recognition (CVPR), vol. 2, pp. 807–814 (2005)Google Scholar
  9. 9.
    Hirschmüller, H., Scharstein, D.: Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans. Pattern Analysis Machine Intelligence 31, 1582–1599 (2009)CrossRefGoogle Scholar
  10. 10.
    Kitt, B., Lategahn, H.: Trinocular optical flow estimation for intelligent vehicle applications. In: Proc. IEEE Int. Conf. Intelligent Transportation Systems (2012) (to appear)Google Scholar
  11. 11.
    Lempitsky, V., Roth, S., Rother, C.: FusionFlow: Discrete-continuous optimization for optical flow estimation. In: Proc. IEEE Int. Conf. Computer Vision Pattern Recognition, CVPR (2008)Google Scholar
  12. 12.
    Lei, C., Yang, Y.-H.: Optical flow estimation on coarse-to-fine region-trees using discrete optimization. In: Proc. Int. Conf. Computer Vision (ICCV), pp. 1562–1569 (2009)Google Scholar
  13. 13.
    Meister, S., Jähne, B., Kondermann, D.: Outdoor stereo camera system for the generation of real-world benchmark data sets. Optical Engineering 51, paper 021107, 6 pages (2012)Google Scholar
  14. 14.
    Ohta, Y., Kanade, T.: Stereo by two-level dynamic programming. In: Proc. Int. Joint Conf. Artificial Intelligence (IJCAI), vol. 2, pp. 1120–1126 (1985)Google Scholar
  15. 15.
    Quénot, G.M.: Computation of optical flow using dynamic programming. In: Proc. IAPR Workshop Machine Vision Appl. (MVA), vol. 3, pp. 249–252 (1996)Google Scholar
  16. 16.
    Ranftl, R., Gehrig, S., Pock, T., Bischof, H.: Pushing the limits of stereo using variational stereo estimation. In: Proc. IEEE Intelligent Vehicles Symposium (IV), pp. 401–407 (2012)Google Scholar
  17. 17.
    Sun, C.: Fast optical flow using 3D shortest path techniques. Image Vision Computing 20, 981–991 (2002)CrossRefGoogle Scholar
  18. 18.
    Warren, H.S.: Hacker’s Delight, pp. 65–72. Addison-Wesley Longman, New York (2002)Google Scholar
  19. 19.
    Werlberger, M.: Convex approaches for high performance video processing, PhD thesis (2012)Google Scholar
  20. 20.
    Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 optical flow. In: Proc. British Machine Vision Conference (BMVC), pp. 1–11 (2009)Google Scholar
  21. 21.
    Zabih, R., Woodfill, J.: Non-parametric Local Transform for Computing Visual Correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994, Part II. LNCS, vol. 801, Springer, Heidelberg (1994)Google Scholar
  22. 22.
    Zach, C., Pock, T., Bischof, H.: A Duality Based Approach for Realtime TV-L 1 Optical Flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Simon Hermann
    • 1
  • Reinhard Klette
    • 1
  1. 1.The .enpeda.. Project, Department of Computer ScienceThe University of AucklandNew Zealand

Personalised recommendations