Data-Driven Manifolds for Outdoor Motion Capture

  • Gerard Pons-Moll
  • Laura Leal-Taixé
  • Juergen Gall
  • Bodo Rosenhahn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7474)


Human motion capturing (HMC) from multiview image sequences is an extremely difficult problem due to depth and orientation ambiguities and the high dimensionality of the state space. In this paper, we introduce a novel hybrid HMC system that combines video input with sparse inertial sensor input. Employing an annealing particle-based optimization scheme, our idea is to use orientation cues derived from the inertial input to sample particles from the manifold of valid poses. Then, visual cues derived from the video input are used to weight these particles and to iteratively derive the final pose. As our main contribution, we propose an efficient sampling procedure where the particles are derived analytically using inverse kinematics on the orientation cues. Additionally, we introduce a novel sensor noise model to account for uncertainties based on the von Mises-Fisher distribution. Doing so, orientation constraints are naturally fulfilled and the number of needed particles can be kept very small. More generally, our method can be used to sample poses that fulfill arbitrary orientation or positional kinematic constraints. In the experiments, we show that our system can track even highly dynamic motions in an outdoor environment with changing illumination, background clutter, and shadows.


Motion Capture Inverse Kinematic Kinematic Chain Inertial Sensor Angular Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Azad, P., Asfour, T., Dillmann, R.: Robust real-time stereo-based markerless human motion capture. In: Proc. 8th IEEE-RAS Int. Conf. Humanoid Robots (2008)Google Scholar
  2. 2.
    Baak, A., Rosenhahn, B., Müller, M., Seidel, H.P.: Stabilizing motion tracking using retrieved motion priors. In: ICCV (2009)Google Scholar
  3. 3.
    Balan, A.O., Sigal, L., Black, M.J., Davis, J.E., Haussecker, H.W.: Detailed human shape and pose from images. In: CVPR (2007)Google Scholar
  4. 4.
    Bregler, C., Malik, J., Pullen, K.: Twist based acquisition and tracking of animal and human kinematics. IJCV 56(3), 179–194 (2004)CrossRefGoogle Scholar
  5. 5.
    Chen, J., Kim, M., Wang, Y., Ji, Q.: Switching gaussian process dynamic models for simultaneous composite motion tracking and recognition. In: CVPR, pp. 2655–2662. IEEE (2009)Google Scholar
  6. 6.
    Deutscher, J., Blake, A., Reid, I.: Articulated body motion capture by annealed particle filtering. In: CVPR, vol. 2, pp. 126–133 (2000)Google Scholar
  7. 7.
    Deutscher, J., Reid, I.: Articulated body motion capture by stochastic search. IJCV 61(2), 185–205 (2005)CrossRefGoogle Scholar
  8. 8.
    Fisher, R.: Dispersion on a sphere. Proceedings of the Royal Society of London. Mathematical and Physical Sciences (1953)Google Scholar
  9. 9.
    Fontmarty, M., Lerasle, F., Danes, P.: Data fusion within a modified annealed particle filter dedicated to human motion capture. In: IRS (2007)Google Scholar
  10. 10.
    Gall, J., Rosenhahn, B., Brox, T., Seidel, H.P.: Optimization and filtering for human motion capture. IJCV 87, 75–92 (2010)CrossRefGoogle Scholar
  11. 11.
    Gall, J., Yao, A., Van Gool, L.: 2D Action Recognition Serves 3D Human Pose Estimation. In: Daniilidis, K. (ed.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 425–438. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Ganapathi, V., Plagemann, C., Thrun, S., Koller, D.: Real time motion capture using a time-of-flight camera. In: CVPR (2010)Google Scholar
  13. 13.
    Gavrila, D., Davis, L.: 3D model based tracking of humans in action: a multiview approach. In: CVPR (1996)Google Scholar
  14. 14.
    Hartley, R., Zisserman, A.: Multiple view geometry, vol. 642. Cambridge University Press, Cambridge (2003)Google Scholar
  15. 15.
    Hasler, N., Rosenhahn, B., Thormählen, T., Wand, M., Gall, J., Seidel, H.P.: Markerless motion capture with unsynchronized moving cameras. In: CVPR, pp. 224–231 (2009)Google Scholar
  16. 16.
    Hauberg, S., Lapuyade, J., Engell-Nørregård, M., Erleben, K., Steenstrup Pedersen, K.: Three Dimensional Monocular Human Motion Analysis in End-Effector Space. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 235–248. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  17. 17.
    Kjellstromm, H., Kragic, D., Black, M.J.: Tracking people interacting with objects. In: CVPR, pp. 747–754 (2010)Google Scholar
  18. 18.
    Lee, C., Elgammal, A.: Coupled visual and kinematic manifold models for tracking. IJCV (2010)Google Scholar
  19. 19.
    Lee, M.W., Cohen, I.: Proposal maps driven mcmc for estimating human body pose in static images. In: CVPR, vol. 2 (2004)Google Scholar
  20. 20.
    Lehment, N., Arsic, D., Kaiser, M., Rigoll, G.: Automated pose estimation in 3D point clouds applying annealing particle filters and inverse kinematics on a gpu. In: CVPR Workshop (2010)Google Scholar
  21. 21.
    Moeslund, T., Hilton, A., Krueger, V., Sigal, L. (eds.): Visual Analysis of Humans: Looking at People. Springer (2011)Google Scholar
  22. 22.
    Murray, R., Li, Z., Sastry, S.: A Mathematical Introduction to Robotic Manipulation. CRC Press, Baton Rouge (1994)zbMATHGoogle Scholar
  23. 23.
    Paden, B.: Kinematics and control of robot manipulators. Ph.D. thesis (1985)Google Scholar
  24. 24.
    Pons-Moll, G., Baak, A., Helten, T., Müller, M., Seidel, H.P., Rosenhahn, B.: Multisensor-fusion for 3D full-body human motion capture. In: CVPR, pp. 663–670 (2010)Google Scholar
  25. 25.
    Pons-Moll, G., Rosenhahn, B.: Ball joints for marker-less human motion capture. In: WACV, pp. 1–8 (2009)Google Scholar
  26. 26.
    Pons-Moll, G., Rosenhahn, B.: Model-based pose estimation. In: Visual Analysis of Humans, pp. 139–170 (2011)Google Scholar
  27. 27.
    Pons-Moll, G., Baak, A., Gall, J., Leal-Taixe, L., Mueller, M., Seidel, H.P., Rosenhahn, B.: Outdoor human motion capture using inverse kinematics and von mises-fisher sampling. In: IEEE International Conference on Computer Vision (ICCV) (November 2011)Google Scholar
  28. 28.
    Pons-Moll, G., Leal-Taixé, L., Truong, T., Rosenhahn, B.: Efficient and Robust Shape Matching for Model Based Human Motion Capture. In: Mester, R., Felsberg, M. (eds.) DAGM 2011. LNCS, vol. 6835, pp. 416–425. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  29. 29.
    Salzmann, M., Urtasun, R.: Combining discriminative and generative methods for 3d deformable surface and articulated pose reconstruction. In: CVPR (June 2010)Google Scholar
  30. 30.
    Shakhnarovich, G., Viola, P., Darrell, T.: Fast pose estimation with parameter-sensitive hashing. In: ICCV, pp. 750–757 (2003)Google Scholar
  31. 31.
    Shoemake, K.: Animating rotation with quaternion curves. ACM SIGGRAPH 19(3), 245–254 (1985)CrossRefGoogle Scholar
  32. 32.
    Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic Tracking of 3D Human Figures Using 2D Image Motion. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 702–718. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  33. 33.
    Sigal, L., Balan, L., Black, M.: Combined discriminative and generative articulated pose and non-rigid shape estimation. In: NIPS, pp. 1337–1344 (2008)Google Scholar
  34. 34.
    Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3d human tracking. In: CVPR (2003)Google Scholar
  35. 35.
    Tao, Y., Hu, H., Zhou, H.: Integration of vision and inertial sensors for 3D arm motion tracking in home-based rehabilitation. IJRR 26(6), 607 (2007)Google Scholar
  36. 36.
    Technologies, X.M.:
  37. 37.
    Urtasun, R., Fleet, D.J., Fua, P.: 3D people tracking with gaussian process dynamical models. In: CVPR (2006)Google Scholar
  38. 38.
    Wang, P., Rehg, J.M.: A modular approach to the analysis and evaluation of particle filters for figure tracking. In: CVPR (2006)Google Scholar
  39. 39.
    Wood, A.: Simulation of the von mises-fisher distribution. Communications in Statistics - Simulation and Computation (1994)Google Scholar
  40. 40.
    Zhang, F., Hancock, E.R., Goodlett, C., Gerig, G.: Probabilistic white matter fiber tracking using particle filtering and von mises-fisher sampling. Medical Image Analysis 13(1), 5–18 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Gerard Pons-Moll
    • 1
  • Laura Leal-Taixé
    • 1
  • Juergen Gall
    • 2
    • 3
  • Bodo Rosenhahn
    • 1
  1. 1.Leibniz UniversityHannoverGermany
  2. 2.BIWI, ETH ZurichSwitzerland
  3. 3.MPI for Intelligent SystemsGermany

Personalised recommendations