Dense Semi-rigid Scene Flow Estimation from RGBD Images

  • Julian Quiroga
  • Thomas Brox
  • Frédéric Devernay
  • James Crowley
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8695)


Scene flow is defined as the motion field in 3D space, and can be computed from a single view when using an RGBD sensor. We propose a new scene flow approach that exploits the local and piecewise rigidity of real world scenes. By modeling the motion as a field of twists, our method encourages piecewise smooth solutions of rigid body motions. We give a general formulation to solve for local and global rigid motions by jointly using intensity and depth data. In order to deal efficiently with a moving camera, we model the motion as a rigid component plus a non-rigid residual and propose an alternating solver. The evaluation demonstrates that the proposed method achieves the best results in the most commonly used scene flow benchmark. Through additional experiments we indicate the general applicability of our approach in a variety of different scenarios.


motion scene flow RGBD image 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: A view centered variational approach. In: Conference on Computer Vision and Pattern Recognition, pp. 1506–1513 (2010)Google Scholar
  2. 2.
    Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(3), 500–513 (2011)CrossRefGoogle Scholar
  3. 3.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision 40(1), 120–145 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Goldluecke, B., Strekalovskiy, E., Cremers, D.: The natural vectorial total variation which arises from geometric measure theory. SIAM Journal on Imaging Sciences 5(2), 537–563 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Hadfield, S., Bowden, R.: Scene particles: Unregularized particle-based scene flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(3), 564–576 (2014)CrossRefGoogle Scholar
  6. 6.
    Herbst, E., Ren, X., Fox, D.: RGB-D flow: Dense 3-D motion estimation using color and depth. In: International Conference on Robotics and Automation (ICRA), pp. 2276–2282 (2013)Google Scholar
  7. 7.
    Horn, B.K., Schunck, B.G.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  8. 8.
    Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: International Conference on Computer Vision, pp. 1–7 (2007)Google Scholar
  9. 9.
    Kerl, C., Sturm, J., Cremers, D.: Robust odometry estimation for RGB-D cameras. In: International Conference on Robotics and Automation (ICRA), pp. 3748–3754 (2013)Google Scholar
  10. 10.
    Letouzey, A., Petit, B., Boyer, E.: Scene flow from depth and color images. In: British Machine Vision Conference, BMVC 2011 (2011)Google Scholar
  11. 11.
    Murray, R.M., Sastry, S.S., Zexiang, L.: A Mathematical Introduction to Robotic Manipulation, 1st edn. CRC Press, Inc., Boca Raton (1994)zbMATHGoogle Scholar
  12. 12.
    Quiroga, J., Devernay, F., Crowley, J.: Scene flow by tracking in intensity and depth data. In: Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 50–57 (2012)Google Scholar
  13. 13.
    Quiroga, J., Devernay, F., Crowley, J.: Local/global scene flow estimation. In: International Conference on Image Processing (ICIP), pp. 3850–3854 (2013)Google Scholar
  14. 14.
    Rosman, G., Bronstein, A.M., Bronstein, M.M., Tai, X.-C., Kimmel, R.: Group-valued regularization for analysis of articulated motion. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part I. LNCS, vol. 7583, pp. 52–62. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 195–202 (2003)Google Scholar
  16. 16.
    Spies, H., Jahne, B., Barron, J.: Dense range flow from depth and intensity data. In: International Conference on Pattern Recognition, vol. 1, pp. 131–134 (2000)Google Scholar
  17. 17.
    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D slam systems. In: International Conference on Intelligent Robot Systems (IROS), pp. 573–580 (2012)Google Scholar
  18. 18.
    Vedula, S., Baker, S., Rander, P., Collins, R.: Three-dimensional scene flow. In: International Conference on Computer Vision, vol. 2, pp. 722–729 (1999)Google Scholar
  19. 19.
    Vogel, C., Schindler, K., Roth, S.: 3D scene flow estimation with a rigid motion prior. In: International Conference on Computer Vision, pp. 1291–1298 (2011)Google Scholar
  20. 20.
    Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Img. Sci. 1(3), 248–272 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 739–751. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-l 1 optical flow. In: Cremers, D., Rosenhahn, B., Yuille, A.L., Schmidt, F.R. (eds.) Visual Motion Analysis. LNCS, vol. 5604, pp. 23–45. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  23. 23.
    Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 Optical Flow. In: Proceedings of the British Machine Vision Conference (BMVC) (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Julian Quiroga
    • 1
    • 3
  • Thomas Brox
    • 2
  • Frédéric Devernay
    • 1
  • James Crowley
    • 1
  1. 1.PRIMA teamINRIAGrenobleFrance
  2. 2.Department of Computer ScienceUniversity of FreiburgGermany
  3. 3.Departamento de ElectrónicaPontificia Universidad JaverianaColombia

Personalised recommendations