International Journal of Computer Vision

, Volume 49, Issue 2, pp 175–214

Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion, Shape and Reflectance


  • Rodrigo L. Carceroni
    • Departamento de Ciência da ComputaçãoUniversidade Federal de Minas Gerais
  • Kiriakos N. Kutulakos
    • Department of Computer ScienceUniversity of Toronto

DOI: 10.1023/A:1020145606604

Cite this article as:
Carceroni, R.L. & Kutulakos, K.N. International Journal of Computer Vision (2002) 49: 175. doi:10.1023/A:1020145606604


In this paper we study the problem of recovering the 3D shape, reflectance, and non-rigid motion properties of a dynamic 3D scene. Because these properties are completely unknown and because the scene's shape and motion may be non-smooth, our approach uses multiple views to build a piecewise-continuous geometric and radiometric representation of the scene's trace in space-time. A basic primitive of this representation is the dynamic surfel, which (1) encodes the instantaneous local shape, reflectance, and motion of a small and bounded region in the scene, and (2) enables accurate prediction of the region's dynamic appearance under known illumination conditions. We show that complete surfel-based reconstructions can be created by repeatedly applying an algorithm called Surfel Sampling that combines sampling and parameter estimation to fit a single surfel to a small, bounded region of space-time. Experimental results with the Phong reflectancemodel and complex real scenes (clothing, shiny objects, skin) illustrate our method's ability to explain pixels and pixel variations in terms of their underlying causes—shape, reflectance, motion, illumination, and visibility.

stereoscopic vision 3D reconstruction multiple-view geometry multi-view stereo space carving motion analysis multi-view motion estimation direct estimation methods image warping deformation analysis 3D motion capture reflectance modeling illumination modeling Phong reflectance model

Copyright information

© Kluwer Academic Publishers 2002