International Journal of Computer Vision

, Volume 115, Issue 1, pp 1–28

3D Scene Flow Estimation with a Piecewise Rigid Scene Model

Article

DOI: 10.1007/s11263-015-0806-0

Cite this article as:
Vogel, C., Schindler, K. & Roth, S. Int J Comput Vis (2015) 115: 1. doi:10.1007/s11263-015-0806-0

Abstract

3D scene flow estimation aims to jointly recover dense geometry and 3D motion from stereoscopic image sequences, thus generalizes classical disparity and 2D optical flow estimation. To realize its conceptual benefits and overcome limitations of many existing methods, we propose to represent the dynamic scene as a collection of rigidly moving planes, into which the input images are segmented. Geometry and 3D motion are then jointly recovered alongside an over-segmentation of the scene. This piecewise rigid scene model is significantly more parsimonious than conventional pixel-based representations, yet retains the ability to represent real-world scenes with independent object motion. It, furthermore, enables us to define suitable scene priors, perform occlusion reasoning, and leverage discrete optimization schemes toward stable and accurate results. Assuming the rigid motion to persist approximately over time additionally enables us to incorporate multiple frames into the inference. To that end, each view holds its own representation, which is encouraged to be consistent across all other viewpoints and frames in a temporal window. We show that such a view-consistent multi-frame scheme significantly improves accuracy, especially in the presence of occlusions, and increases robustness against adverse imaging conditions. Our method currently achieves leading performance on the KITTI benchmark, for both flow and stereo.

Keywords

3D scene flowStereoMotion estimation Piecewise planarityPiecewise rigiditySegmentation

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Christoph Vogel
    • 1
  • Konrad Schindler
    • 1
  • Stefan Roth
    • 2
  1. 1.Photogrammetry & Remote SensingETH ZurichZurichSwitzerland
  2. 2.Department of Computer ScienceTU DarmstadtDarmstadtGermany