GraphFlow – 6D Large Displacement Scene Flow via Graph Matching

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9358)


We present an approach for computing dense scene flow from two large displacement RGB-D images. When dealing with large displacements the crucial step is to estimate the overall motion correctly. While state-of-the-art approaches focus on RGB information to establish guiding correspondences, we explore the power of depth edges. To achieve this, we present a new graph matching technique that brings sparse depth edges into correspondence. An additional contribution is the formulation of a continuous-label energy which is used to densify the sparse graph matching output. We present results on challenging Kinect images, for which we outperform state-of-the-art techniques.


Scene Flow Graph Matching Depth Edges Segment Description Alpha Expansion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alvarez, L., Deriche, R., Papadopoulo, T., Sánchez, J.: Symmetrical dense optical flow estimation with occlusions detection. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 721–735. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  2. 2.
    Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: PatchMatch: a randomized correspondence algorithm for structural image editing. TOG 28(3), 24 (2009)CrossRefGoogle Scholar
  3. 3.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  4. 4.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. TPAMI 24(4), 509–522 (2002)CrossRefGoogle Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. TPAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  6. 6.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. TPAMI 33(3), 500–513 (2011)CrossRefGoogle Scholar
  7. 7.
    Canny, J.: A computational approach to edge detection. TPAMI 8(6), 679–698 (1986)CrossRefGoogle Scholar
  8. 8.
    Conte, D., Foggia, P., Sansone, C., Vento, M.: Thirty years of graph matching in pattern recognition. IJPRAI 18(03), 265–298 (2004)Google Scholar
  9. 9.
    Ferstl, D., Riegler, G., Ruether, M., Bischof, H.: CP-Census: a novel model for dense variational scene flow from RGB-D data. In: Proceedings of BMVC (2014)Google Scholar
  10. 10.
    Foggia, P., Percannella, G., Vento, M.: Graph matching and learning in pattern recognition in the last 10 years. IJPRAI 28(01) (2014)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.): Time-of-Flight and Depth Imaging. LNCS, vol. 8200. Springer, Heidelberg (2013) Google Scholar
  12. 12.
    Hadfield, S., Bowden, R.: Scene particles: unregularized particle-based scene flow estimation. TPAMI 36(3), 564–576 (2014)CrossRefGoogle Scholar
  13. 13.
    Hammer, P., Hansen, P., Simeone, B.: Roof duality, complementation and persistency in quadratic 0–1 optimization. Math. Program. 28(2), 121–155 (1984)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Herbst, E., Ren, X., Fox, D.: RGB-D flow: Dense 3-d motion estimation using color and depth. In: Proceedings of ICRA, pp. 2276–2282. IEEE (2013)Google Scholar
  15. 15.
    Hornácek, M., Fitzgibbon, A., Rother, C.: Sphereflow: 6 DoF scene flow from RGB-D pairs. In: Proceedings of CVPR. IEEE (2014)Google Scholar
  16. 16.
    Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: Proceedings of ICCV. IEEE (2007)Google Scholar
  17. 17.
    Jaimez, M., Souiai, M., Gonzalez-Jimenez, J., Cremers, D.: A primal-dual framework for real-time dense RGB-D scene flow. In: Proceedings of ICRA. IEEE (2015)Google Scholar
  18. 18.
    Leordeanu, M., Zanfir, A., Sminchisescu, C.: Locally affine sparse-to-dense matching for motion and occlusion estimation. In: Proceedings of ICCV. IEEE (2013)Google Scholar
  19. 19.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–100 (2004)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Quiroga, J., Devernay, F., Crowley, J.: Local/global scene flow estimation. In: Proceedings of ICIP. IEEE (2013)Google Scholar
  21. 21.
    Rabe, C., Müller, T., Wedel, A., Franke, U.: Dense, robust, and accurate motion field estimation from stereo image sequences in real-time. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 582–595. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  22. 22.
    Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: Epicflow: edge-preserving interpolation of correspondences for optical flow (2015). arXiv:1501.02565
  23. 23.
    Sellent, A., Ruhl, K., Magnor, M.: A loop-consistency measure for dense correspondences in multi-view video. Image Vis. Comput. 30(9), 641–654 (2012)CrossRefGoogle Scholar
  24. 24.
    Smith, D.K.: Network flows: theory, algorithms, and applications. J. Oper. Res. Soc. 45(11), 1340–1340 (1994)CrossRefGoogle Scholar
  25. 25.
    Torresani, L., Kolmogorov, V., Rother, C.: A dual decomposition approach to feature correspondence. TPAMI 35(2), 259–271 (2013)CrossRefGoogle Scholar
  26. 26.
    Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. In: Proceedings of ICCV, vol. 2, pp. 722–729. IEEE (1999)Google Scholar
  27. 27.
    Verhagen, B., Timofte, R., Van Gool, L.: Scale-invariant line descriptors for wide baseline matching. In: Proceedings of WACV. IEEE (2014)Google Scholar
  28. 28.
    Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: Proceedings of ICCV. IEEE (2013)Google Scholar
  29. 29.
    Wang, Y., Zhang, J., Liu, Z., Wu, Q., Chou, P., Zhang, Z., Jia, Y.: Completed dense scene flow in RGB-D space. In: Jawahar, C.V., Shan, S. (eds.) ACCV 2014 Workshops. LNCS, vol. 9008, pp. 191–205. Springer, Heidelberg (2015) Google Scholar
  30. 30.
    Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: Proceedings of ICCV. IEEE (2013)Google Scholar
  31. 31.
    Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. TPAMI 34(9), 1744–1757 (2012)CrossRefGoogle Scholar
  32. 32.
    Yoon, K.J., Kweon, I.S.: Locally adaptive support-weight approach for visual correspondence search. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2005, vol. 2, pp. 924–931. IEEE (2005)Google Scholar
  33. 33.
    Zhang, L., Koch, R.: An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency. J. Vis. Commun. Image Represent. 24(7), 794–805 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  1. 1.TU DresdenDresdenGermany
  2. 2.Heidelberg UniversityHeidelbergGermany
  3. 3.TU DarmstadtDarmstadtGermany

Personalised recommendations