Direct Estimation of Dense Scene Flow and Depth from a Monocular Sequence

  • Yosra Mathlouthi
  • Amar Mitiche
  • Ismail Ben Ayed
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8887)


We propose a method that uses a monocular sequence for joint direct estimation of dense scene flow and relative depth, a problem that has been generally tackled in the literature with binocular or stereo image sequences. The problem is posed as the optimization of a functional containing two terms: a data term, which relates three-dimensional (3D) velocity to depth in terms of the spatiotemporal visual pattern and an L 2 regularization term. Based on expressing the optical flow gradient constraint in terms of scene flow velocity and depth, our formulation is analogous to the classical Horn and Schunck optical flow estimation although it involves 3D motion and depth rather than 2D image motion. The discretized Euler-Lagrange equations yield a large scale sparse system of linear equations, which we order so that the corresponding matrix is symmetric positive definite. This implies that Gauss-Seidel iterations converge, point-wise or block-wise, and afford highly efficient means of solving the equations. Examples are given to verify the scheme and its implementation.


Motion Segmentation Gradient Constraint Scene Flow Stereo Sequence Stereo Image Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: IEEE International Conference on Computer Vision, ICCV (2013)Google Scholar
  2. 2.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. International Journal of Computer Vision 92(1), 1–31 (2011)CrossRefGoogle Scholar
  3. 3.
    Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: CVPR, pp. 2432–2439 (2010)Google Scholar
  4. 4.
    Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: A view centered variational approach. International Journal of Computer Vision 101(1), 6–21 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Wedel, A., Brox, T., Vaudrey, T., Rabe, C., Franke, U., Cremers, D.: Stereoscopic scene flow computation for 3d motion understanding. International Journal of Computer Vision 95(1), 29–51 (2011)CrossRefzbMATHGoogle Scholar
  6. 6.
    Rabe, C., Müller, T., Wedel, A., Franke, U.: Dense, robust, and accurate motion field estimation from stereo image sequences in real-time. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 582–595. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: IEEE International Conference on Computer Vision (ICCV), pp. 1–7 (2007)Google Scholar
  8. 8.
    Pons, J.P., Keriven, R., Faugeras, O.D.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision 72(2), 179–193 (2007)CrossRefGoogle Scholar
  9. 9.
    Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 475–480 (2005)CrossRefGoogle Scholar
  10. 10.
    Mitiche, A., Ben Ayed, I.: Variational and level set methods in image segmentation. Springer (2010)Google Scholar
  11. 11.
    Mitiche, A., Sekkati, H.: Optical flow 3D segmentation and interpretation: A variational method with active curve evolution and level sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(11), 1818–1829 (2006)CrossRefGoogle Scholar
  12. 12.
    Sekkati, H., Mitiche, A.: Concurrent 3D motion segmentation and 3D interpretation of temporal sequences of monocular images. IEEE Transactions on Image Processing 15(3), 641–653 (2006)CrossRefGoogle Scholar
  13. 13.
    Zhang, Y., Kambhamettu, C.: Integrated 3d scene flow and structure recovery from multiview image sequences. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 674–681 (2000)Google Scholar
  14. 14.
    Mitiche, A., Letang, J.M.: Stereokinematic analysis of visual data in active, convergent stereoscopy. Journal of Robotics and Autonomous Systems 705, 43–71 (1998)CrossRefGoogle Scholar
  15. 15.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Vogel, C., Schindler, K., Roth, S.: 3d scene flow estimation with a rigid motion prior. In: IEEE International Conference on Computer Vision (ICCV), pp. 1291–1298 (2011)Google Scholar
  17. 17.
    Longuet-Higgins, H., Prazdny, K.: The interpretation of a moving retinal image. Proceedings of the Royal Society of London, B 208, 385–397 (1981)CrossRefGoogle Scholar
  18. 18.
    Pons, J.P., Keriven, R., Faugeras, O., Hermosillo, G.: Variational stereovision and 3d scene flow estimation with statistical similarity measures. In: IEEE International Conference on Computer Vision (ICCV), pp. 597–602 (2003)Google Scholar
  19. 19.
    Horn, B., Schunk, B.: Determining optical flow. Artificial Intelligence 17(17), 185–203 (1981)CrossRefGoogle Scholar
  20. 20.
    Mitiche, A., Mansouri, A.: On convergence of the horn and schunck optical flow estimation method. IEEE Transactions on Image Processing 13(11), 1473–1490 (2004)CrossRefGoogle Scholar
  21. 21.
    Ciarlet, P.: Introduction a l’analyse numerique matricielle et a l’optimisation, 5th edn., Masson (1994)Google Scholar
  22. 22.
    Stoer, J., Bulirsch, R.: Introduction to Numerical Analysis, 3rd edn. Texts in Applied Mathematics, vol. 12. Springer, New York (2002)CrossRefzbMATHGoogle Scholar
  23. 23.
    Forsythe, G., Malcolm, M., Moler, C.: Computer methods for mathematical computations. Prentice-Hall (1977)Google Scholar
  24. 24.
    Sekkati, H., Mitiche, A.: A variational method for the recovery of dense 3D structure from motion. Journal of Robotics and Autonomous Systems 55, 597–607 (2007)CrossRefGoogle Scholar
  25. 25.
    Debrunner, C., Ahuja, N.: Segmentation and factorization-based motion and structure estimation for long image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 206–211 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Yosra Mathlouthi
    • 1
  • Amar Mitiche
    • 1
  • Ismail Ben Ayed
    • 2
  1. 1.Institut National de la Recherche Scientifique (INRS-EMT)MontréalCanada
  2. 2.GE HealthcareLondonCanada

Personalised recommendations