Advertisement

3D Szenenfluss – bildbasierte Schätzung dichter Bewegungsfelder

  • Christoph VogelEmail author
  • Stefan Roth
  • Konrad Schindler
Living reference work entry
Part of the Springer Reference Naturwissenschaften book series (SRN)

Zusammenfassung

Der 3D Szenenfluss (scene flow) ist eine dichte Beschreibung der Geometrie und des Bewegungsfeldes einer dynamischen Szene. Entsprechend ist die Bestimmung des Szenenflusses aus binokularen Videosequenzen eine Generalisierung zweier klassischer Aufgaben der bildbasierten Messtechnik, der Schätzung von Stereokorrespondenz und optischem Fluss. Im folgenden wird ein Modell vorgestellt, in dem die dynamische 3D Szene durch eine Menge von planaren Segmenten repräsentiert wird, wobei jedes Segment eine Starrkörperbewegung (Translation und Rotation) ausführt. Die (Über-)Segmentierung in starre, ebene Segmente wird gemeinsam mit deren 3D Geometrie und 3D Bewegung geschätzt. Das beschriebene Modell ist wesentlich kompakter als die konventionelle pixelweise Repräsentation, verfügt aber dennoch über genügend Flexibilität, um reale Szenen mit mehreren unabhängigen Bewegungen zu beschreiben. Darüber hinaus erlaubt es, a-priori Annahmen über die Szene einzubinden und Verdeckungen zu berücksichtigen, und ermöglicht den Einsatz robuster diskreter Optimierungsmethoden. Weiters ist das Modell, in Kombination mit einem dynamischen Modell, direkt auf mehrere aufeinanderfolgende Zeitschritte anwendbar. Dazu wird für die einzelnen Bilder jeweils eine eigene Repräsentation instanziiert. Entsprechende Bedingungen stellen sicher, dass die Schätzung über verschiedene Ansichten und verschiedene Zeitpunkte konsistent ist. Das beschriebene Modell verbessert die Genauigkeit und Zuverlässigkeit der Szenenfluss-Schätzung speziell bei ungünstigen Aufnahmebedingungen.

Literatur

  1. 1.
    Adiv, G.: Determining three-dimensional motion and structure from optical flow generated by several moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 7(4), 384–401 (1985)CrossRefGoogle Scholar
  2. 2.
    Badino, H., Kanade, T.: A head-wearable short-baseline stereo system for the simultaneous estimation of structure and motion. In: IAPR Conference on Machine Vision Application, Nara, S. 185–189 (2011)Google Scholar
  3. 3.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011). vision.middlebury.edu/flow CrossRefGoogle Scholar
  4. 4.
    Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: a view centered variational approach. In: IEEE Conference on Computer Vision and Pattern Recognition, San Francisco (2010)Google Scholar
  5. 5.
    Black, M.J., Anandan, P.: Robust dynamic motion estimation over time. In: IEEE Conference on Computer Vision and Pattern Recognition, Lahaina (1991)CrossRefGoogle Scholar
  6. 6.
    Bleyer, M., Rother, C., Kohli, P.: Surface stereo with soft segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, San Francisco (2010)Google Scholar
  7. 7.
    Bleyer, M., Rother, C., Kohli, P., Scharstein, D., Sinha, S.N.: Object stereo – joint stereo matching and object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs (2011)CrossRefGoogle Scholar
  8. 8.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 500–513 (2011)CrossRefGoogle Scholar
  9. 9.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: European Conference on Computer Vision, Prague (2004)CrossRefGoogle Scholar
  10. 10.
    Carceroni, R.L., Kutulakos, K.N.: Multi-view scene capture by surfel sampling: from video streams to non-rigid 3D motion, shape and reflectance. Int. J. Comput. Vis. 49, 175–214 (2002)CrossRefGoogle Scholar
  11. 11.
    Courchay, J., Pons, J.P., Monasse, P., Keriven, R.: Dense and accurate spatio-temporal multi-view stereovision. In: Asian Conference on Computer Vision, Xi’an (2009)Google Scholar
  12. 12.
    Devernay, F., Mateus, D., Guilbert, M.: Multi-camera scene flow by tracking 3-D points and surfels. In: IEEE Conference on Computer Vision and Pattern Recognition, New York (2006)CrossRefGoogle Scholar
  13. 13.
    Furukawa, Y., Ponce, J.: Dense 3D motion capture from synchronized video streams. In: IEEE Conference on Computer Vision and Pattern Recognition, Anchorage (2008)Google Scholar
  14. 14.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? In: IEEE Conference on Computer Vision and Pattern Recognition, Providence (2012). www.cvlibs.net/datasets/kitti/ Google Scholar
  15. 15.
    Gorelick, L., Veksler, O., Boykov, Y., Ben Ayed, I., Delong, A.: Local submodular approximations for binary pairwise energies. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus (2014)Google Scholar
  16. 16.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRefGoogle Scholar
  17. 17.
    Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: IEEE International Conference on Computer Vision, Rio de Janeiro (2007)CrossRefGoogle Scholar
  18. 18.
    Hung, C.H., Xu, L., Jia, J.: Consistent binocular depth and scene flow with chained temporal profiles. Int. J. Comput. Vis. 102(1–3), 271–292 (2013)CrossRefGoogle Scholar
  19. 19.
    Lempitsky, V., Roth, S., Rother, C.: FusionFlow: discrete-continuous optimization for optical flow estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, Anchorage (2008)Google Scholar
  20. 20.
    Lempitsky, V., Rother, C., Roth, S., Blake, A.: Fusion moves for Markov random field optimization. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1392–1405 (2010)CrossRefGoogle Scholar
  21. 21.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, Bd. 2, Vancouver (1981)Google Scholar
  22. 22.
    Meister, S., Jähne, B., Kondermann, D.: Outdoor stereo camera system for the generation of real-world benchmark data sets. Opt. Eng. 51(2), 021107 (2012)CrossRefGoogle Scholar
  23. 23.
    Müller, T., Rannacher, J., Rabe, C., Franke, U.: Feature- and depth-supported modified total variation optical flow for 3D motion field estimation in real scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs (2011)CrossRefGoogle Scholar
  24. 24.
    Murray, D.W., Buxton, B.F.: Scene segmentation from visual motion using global optimization. IEEE Trans. Pattern Anal. Mach. Intell. 9(2), 220–228 (1987)CrossRefGoogle Scholar
  25. 25.
    Nir, T., Bruckstein, A., Kimmel, R.: Over-parameterized variational optical flow. Int. J. Comput. Vis. 76(2), 205–216 (2008)CrossRefGoogle Scholar
  26. 26.
    Park, J., Oh, T.H., Jung, J., Tai, Y.W., Kweon, I.S.: A tensor voting approach for multi-view 3D scene flow estimation and refinement. In: European Conference on Computer Vision, Florence (2012)CrossRefGoogle Scholar
  27. 27.
    Rabe, C., Müller, T., Wedel, A., Franke, U.: Dense, robust, and accurate motion field estimation from stereo image sequences in real-time. In: European Conference on Computer Vision, Heraklion (2010)CrossRefGoogle Scholar
  28. 28.
    Rother, C., Kolmogorov, V., Lempitsky, V., Szummer, M.: Optimizing binary MRFs via extended roof duality. In: IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis (2007)CrossRefGoogle Scholar
  29. 29.
    Schoenemann, T., Cremers, D.: High resolution motion layer decomposition using dual-space graph cuts. In: IEEE Conference on Computer Vision and Pattern Recognition, Anchorage (2008)CrossRefGoogle Scholar
  30. 30.
    Sun, D., Sudderth, E.B., Black, M.J.: Layered image motion with explicit occlusions, temporal consistency, and depth ordering. In: Neural Information Processing Systems, Vancouver (2010)Google Scholar
  31. 31.
    Sun, D., Wulff, J., Sudderth, E., Pfister, H., Black, M.: A fully-connected layered model of foreground and background flow. In: IEEE Conference on Computer Vision and Pattern Recognition, Portland (2013)CrossRefGoogle Scholar
  32. 32.
    Tao, H., Sawhney, H.S.: Global matching criterion and color segmentation based stereo. In: IEEE Workshop on Applications in Computer Vision, Palm Springs (2000)Google Scholar
  33. 33.
    Unger, M., Werlberger, M., Pock, T., Bischof, H.: Joint motion estimation and segmentation of complex scenes with label costs and occlusion modeling. In: IEEE Conference on Computer Vision and Pattern Recognition, Providence (2012)CrossRefGoogle Scholar
  34. 34.
    Valgaerts, L., Bruhn, A., Zimmer, H., Weickert, J., Stoll, C., Theobalt, C.: Joint estimation of motion, structure and geometry from stereo sequences. In: European Conference on Computer Vision, Heraklion (2010)CrossRefGoogle Scholar
  35. 35.
    Vaudrey, T., Rabe, C., Klette, R., Milburn, J.: Differences between stereo and motion behaviour on synthetic and real-world stereo sequences. In: International Conference on Image and Vision Computing New Zealand, Christ Church (2008)CrossRefGoogle Scholar
  36. 36.
    Vedula, S., Baker, S., Collins, R., Kanade, T., Rander, P.: Three-dimensional scene flow. In: IEEE Conference on Computer Vision and Pattern Recognition, Ft. Collins (1999)CrossRefGoogle Scholar
  37. 37.
    Veksler, O., Boykov, Y., Mehrani, P.: Superpixels and supervoxels in an energy optimization framework. In: European Conference on Computer Vision, Heraklion (2010)CrossRefGoogle Scholar
  38. 38.
    Vogel, C., Schindler, K., Roth, S.: 3D scene flow estimation with a rigid motion prior. In: IEEE International Conference on Computer Vision, Barcelona (2011)CrossRefGoogle Scholar
  39. 39.
    Vogel, C., Roth, S., Schindler, K.: An evaluation of data costs for optical flow. In: Pattern Recognition (Proceedings of GCPR), Saarbrücken, S. 343–353 (2013)Google Scholar
  40. 40.
    Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: IEEE International Conference on Computer Vision, Sydney (2013)CrossRefGoogle Scholar
  41. 41.
    Vogel, C., Roth, S., Schindler, K.: 3D scene flow estimation with a piecewise rigid scene model. Int. J. Comput. Vis. 111(3), 1–28 (2015)CrossRefGoogle Scholar
  42. 42.
    Vogel, C., Roth, S., Schindler, K.: View-consistent 3D scene flow estimation over multiple frames. In: European Conference on Computer Vision, Zurich (2014)CrossRefGoogle Scholar
  43. 43.
    Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: IEEE International Conference on Computer Vision, Barcelona (2011)CrossRefGoogle Scholar
  44. 44.
    Wang, J.Y.A., Adelson, E.H.: Representing moving images with layers. IEEE Trans. Image Process. 3, 625–638 (1994)CrossRefGoogle Scholar
  45. 45.
    Wedel, A., Rabe, C., Vaudrey, T., Brox, T., Franke, U., Cremers, D.: Efficient dense scene flow from sparse or dense stereo data. In: European Conference on Computer Vision, Marseille (2008)CrossRefGoogle Scholar
  46. 46.
    Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 optical flow. In: British Machine Vision Conference, London (2009)CrossRefGoogle Scholar
  47. 47.
    Yamaguchi, K., Hazan, T., McAllester, D., Urtasun, R.: Continuous Markov random fields for robust stereo estimation. In: European Conference on Computer Vision, Florence (2012)CrossRefGoogle Scholar
  48. 48.
    Yamaguchi, K., McAllester, D., Urtasun, R.: Robust monocular epipolar flow estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, Portland (2013)CrossRefGoogle Scholar
  49. 49.
    Yamaguchi, K., McAllester, D., Urtasun, R.: Efficient joint segmentation, occlusion labeling, stereo and flow estimation. In: European Conference on Computer Vision, Zurich (2014)CrossRefGoogle Scholar
  50. 50.
    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: European Conference on Computer Vision, Stockholm (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Christoph Vogel
    • 1
    Email author
  • Stefan Roth
    • 2
  • Konrad Schindler
    • 3
  1. 1.Photogrammetry and Remote SensingETH ZürichZürichSchweiz
  2. 2.Visual Inference, TU DarmstadtDarmstadtDeutschland
  3. 3.Photogrammetry and Remote SensingETH ZürichZürichSchweiz

Personalised recommendations