Estimating Vehicle Ego-Motion and Piecewise Planar Scene Structure from Optical Flow in a Continuous Framework

  • Andreas Neufeld
  • Johannes Berger
  • Florian Becker
  • Frank Lenzen
  • Christoph Schnörr
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9358)


We propose a variational approach for estimating egomotion and structure of a static scene from a pair of images recorded by a single moving camera. In our approach the scene structure is described by a set of 3D planar surfaces, which are linked to a SLIC superpixel decomposition of the image domain. The continuously parametrized planes are determined along with the extrinsic camera parameters by jointly minimizing a non-convex smooth objective function, that comprises a data term based on the pre-calculated optical flow between the input images and suitable priors on the scene variables. Our experiments demonstrate that our approach estimates egomotion and scene structure with a high quality, that reaches the accuracy of state-of-the-art stereo methods, but relies on a single sensor that is more cost-efficient for autonomous systems.


  1. 1.
    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)CrossRefGoogle Scholar
  2. 2.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  3. 3.
    Becker, F., Lenzen, F., Kappes, J.H., Schnörr, C.: Variational recursive joint estimation of dense scene structure and camera motion from monocular high speed traffic sequences. Int. J. Comput. Vis. 105(3), 269–297 (2013)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chojnacki, W., Szpak, Z.L., Brooks, M.J., van den Hengel, A.: Multiple homography estimation with full consistency constraints. In: DICTA (2010)Google Scholar
  5. 5.
    Engel, J., Schöps, T., Cremers, D.: LSD-SLAM: large-scale direct monocular SLAM. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 834–849. Springer, Heidelberg (2014) Google Scholar
  6. 6.
    Eriksson, A., van den Hengel, A.: Optimization on the manifold of multiple homographies. In: ICCV (2009)Google Scholar
  7. 7.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32, 1231–1237 (2013)CrossRefGoogle Scholar
  8. 8.
    Geiger, A., Ziegler, J., Stiller, C.: Stereoscan: dense 3D reconstruction in real-time. In: IEEE Intelligent Vehicles Symposium, pp. 963–968 (2011)Google Scholar
  9. 9.
    Graber, G., Pock, T., Bischof, H.: Online 3D reconstruction using convex optimization. In: ICCV (2011)Google Scholar
  10. 10.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004) CrossRefGoogle Scholar
  11. 11.
    Moré, J.J.: The Levenberg-Marquardt algorithm: implementation and theory. In: Cleaveland, W.R. (ed.) CONCUR 1992. LNCS, vol. 630, pp. 105–116. Springer, Heidelberg (1992) Google Scholar
  12. 12.
    Newcombe, R.A., Davison, A.J.: Live dense reconstruction with a single moving camera. In: CVPR (2010)Google Scholar
  13. 13.
    Newcombe, R.A., Lovegrove, S.J., Davison, A.J.: DTAM: dense tracking and mapping in real-time. In: ICCV, pp. 2320–2327 (2011)Google Scholar
  14. 14.
    Shashua, A., Avidan, S.: The Rank 4 Constraint in Multiple (\({\ge }3\)) View Geometry. In: Buxton, B.F., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1065, pp. 196–206. Springer, Heidelberg (1996) CrossRefGoogle Scholar
  15. 15.
    Stühmer, J., Gumhold, S., Cremers, D.: Parallel generalized thresholding scheme for live dense geometry from a handheld camera. In: CVGPU (2010)Google Scholar
  16. 16.
    Stühmer, J., Gumhold, S., Cremers, D.: Real-time dense geometry from a handheld camera. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) Pattern Recognition. LNCS, vol. 6376, pp. 11–20. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  17. 17.
    Szpak, Z.L., Chojnacki, W., Eriksson, A., van den Hengel, A.: Sampson distance based joint estimation of multiple homographies with uncalibrated cameras. Comput. Vis. Image Und. 125, 200–213 (2014)CrossRefGoogle Scholar
  18. 18.
    Tola, E., Strecha, C., Fua, P.: Efficient large-scale multi-view stereo for ultra high-resolution image sets. Mach. Vis. Appl. 23(5), 903–920 (2012)CrossRefGoogle Scholar
  19. 19.
    Vogel, C., Roth, S., Schindler, K.: An evaluation of data costs for optical flow. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 343–353. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  20. 20.
    Vogel, C., Roth, S., Schindler, K.: View-consistent 3D scene flow estimation over multiple frames. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 263–278. Springer, Heidelberg (2014) Google Scholar
  21. 21.
    Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: ICCV (2013)Google Scholar
  22. 22.
    Wendel, A., Maurer, M., Graber, G., Pock, T., Bischof, H.: Dense reconstruction on-the-fly. In: CVPR (2012)Google Scholar
  23. 23.
    Yamaguchi, K., Hazan, T., McAllester, D., Urtasun, R.: Continuous Markov random fields for robust stereo estimation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 45–58. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  24. 24.
    Yamaguchi, K., McAllester, D., Urtasun, R.: Efficient joint segmentation, occlusion labeling, stereo and flow estimation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part V. LNCS, vol. 8693, pp. 756–771. Springer, Heidelberg (2014) Google Scholar
  25. 25.
    Yamaguchi, K., McAllester, D.A., Urtasun, R.: Robust monocular epipolar flow estimation. In: CVPR (2013)Google Scholar
  26. 26.
    Zelnik-Manor, L., Irani, M.: Multi-view subspace constraints on homographies. In: ICCV (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Andreas Neufeld
    • 1
  • Johannes Berger
    • 1
  • Florian Becker
    • 1
  • Frank Lenzen
    • 1
  • Christoph Schnörr
    • 1
  1. 1.IPA and HCIUniversity of HeidelbergHeidelbergGermany

Personalised recommendations