General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

  • Helge Rhodin
  • Nadia Robertini
  • Dan Casas
  • Christian Richardt
  • Hans-Peter Seidel
  • Christian Theobalt
Conference paper

DOI: 10.1007/978-3-319-46454-1_31

Volume 9909 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Rhodin H., Robertini N., Casas D., Richardt C., Seidel HP., Theobalt C. (2016) General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9909. Springer, Cham

Abstract

Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model personalization a different problem and use manual or semi-automatic model initialization, which greatly reduces applicability. In this paper, we propose a fully automatic algorithm that jointly creates a rigged actor model commonly used for animation – skeleton, volumetric shape, appearance, and optionally a body surface – and estimates the actor’s motion from multi-view video input only. The approach is rigorously designed to work on footage of general outdoor scenes recorded with very few cameras and without background subtraction. Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy. For reconstruction, 3D body shape is approximated as a Gaussian density field. For pose and shape estimation, we minimize a new edge-based alignment energy inspired by volume ray casting in an absorbing medium. We further propose a new statistical human body model that represents the body surface, volumetric Gaussian density, and variability in skeleton shape. Given any multi-view sequence, our method jointly optimizes the pose and shape parameters of this model fully automatically in a spatiotemporal way.

Supplementary material

419978_1_En_31_MOESM1_ESM.pdf (3.2 mb)
Supplementary material 1 (pdf 3232 KB)

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Helge Rhodin
    • 1
  • Nadia Robertini
    • 1
    • 2
  • Dan Casas
    • 1
  • Christian Richardt
    • 1
    • 2
  • Hans-Peter Seidel
    • 1
  • Christian Theobalt
    • 1
  1. 1.MPI InformatikSaarbrückenGermany
  2. 2.Intel Visual Computing InstituteSaarbrückenGermany