Skip to main content
SpringerLink
Log in
Book cover

Outdoor and Large-Scale Real-World Scene Analysis pp 305–328Cite as

Data-Driven Manifolds for Outdoor Motion Capture

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNIP,volume 7474)

Abstract

Human motion capturing (HMC) from multiview image sequences is an extremely difficult problem due to depth and orientation ambiguities and the high dimensionality of the state space. In this paper, we introduce a novel hybrid HMC system that combines video input with sparse inertial sensor input. Employing an annealing particle-based optimization scheme, our idea is to use orientation cues derived from the inertial input to sample particles from the manifold of valid poses. Then, visual cues derived from the video input are used to weight these particles and to iteratively derive the final pose. As our main contribution, we propose an efficient sampling procedure where the particles are derived analytically using inverse kinematics on the orientation cues. Additionally, we introduce a novel sensor noise model to account for uncertainties based on the von Mises-Fisher distribution. Doing so, orientation constraints are naturally fulfilled and the number of needed particles can be kept very small. More generally, our method can be used to sample poses that fulfill arbitrary orientation or positional kinematic constraints. In the experiments, we show that our system can track even highly dynamic motions in an outdoor environment with changing illumination, background clutter, and shadows.

Keywords

  • Motion Capture
  • Inverse Kinematic
  • Kinematic Chain
  • Inertial Sensor
  • Angular Error

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Azad, P., Asfour, T., Dillmann, R.: Robust real-time stereo-based markerless human motion capture. In: Proc. 8th IEEE-RAS Int. Conf. Humanoid Robots (2008)

    Google Scholar 

  2. Baak, A., Rosenhahn, B., Müller, M., Seidel, H.P.: Stabilizing motion tracking using retrieved motion priors. In: ICCV (2009)

    Google Scholar 

  3. Balan, A.O., Sigal, L., Black, M.J., Davis, J.E., Haussecker, H.W.: Detailed human shape and pose from images. In: CVPR (2007)

    Google Scholar 

  4. Bregler, C., Malik, J., Pullen, K.: Twist based acquisition and tracking of animal and human kinematics. IJCV 56(3), 179–194 (2004)

    CrossRef  Google Scholar 

  5. Chen, J., Kim, M., Wang, Y., Ji, Q.: Switching gaussian process dynamic models for simultaneous composite motion tracking and recognition. In: CVPR, pp. 2655–2662. IEEE (2009)

    Google Scholar 

  6. Deutscher, J., Blake, A., Reid, I.: Articulated body motion capture by annealed particle filtering. In: CVPR, vol. 2, pp. 126–133 (2000)

    Google Scholar 

  7. Deutscher, J., Reid, I.: Articulated body motion capture by stochastic search. IJCV 61(2), 185–205 (2005)

    CrossRef  Google Scholar 

  8. Fisher, R.: Dispersion on a sphere. Proceedings of the Royal Society of London. Mathematical and Physical Sciences (1953)

    Google Scholar 

  9. Fontmarty, M., Lerasle, F., Danes, P.: Data fusion within a modified annealed particle filter dedicated to human motion capture. In: IRS (2007)

    Google Scholar 

  10. Gall, J., Rosenhahn, B., Brox, T., Seidel, H.P.: Optimization and filtering for human motion capture. IJCV 87, 75–92 (2010)

    CrossRef  Google Scholar 

  11. Gall, J., Yao, A., Van Gool, L.: 2D Action Recognition Serves 3D Human Pose Estimation. In: Daniilidis, K. (ed.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 425–438. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  12. Ganapathi, V., Plagemann, C., Thrun, S., Koller, D.: Real time motion capture using a time-of-flight camera. In: CVPR (2010)

    Google Scholar 

  13. Gavrila, D., Davis, L.: 3D model based tracking of humans in action: a multiview approach. In: CVPR (1996)

    Google Scholar 

  14. Hartley, R., Zisserman, A.: Multiple view geometry, vol. 642. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  15. Hasler, N., Rosenhahn, B., Thormählen, T., Wand, M., Gall, J., Seidel, H.P.: Markerless motion capture with unsynchronized moving cameras. In: CVPR, pp. 224–231 (2009)

    Google Scholar 

  16. Hauberg, S., Lapuyade, J., Engell-Nørregård, M., Erleben, K., Steenstrup Pedersen, K.: Three Dimensional Monocular Human Motion Analysis in End-Effector Space. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 235–248. Springer, Heidelberg (2009)

    CrossRef  Google Scholar 

  17. Kjellstromm, H., Kragic, D., Black, M.J.: Tracking people interacting with objects. In: CVPR, pp. 747–754 (2010)

    Google Scholar 

  18. Lee, C., Elgammal, A.: Coupled visual and kinematic manifold models for tracking. IJCV (2010)

    Google Scholar 

  19. Lee, M.W., Cohen, I.: Proposal maps driven mcmc for estimating human body pose in static images. In: CVPR, vol. 2 (2004)

    Google Scholar 

  20. Lehment, N., Arsic, D., Kaiser, M., Rigoll, G.: Automated pose estimation in 3D point clouds applying annealing particle filters and inverse kinematics on a gpu. In: CVPR Workshop (2010)

    Google Scholar 

  21. Moeslund, T., Hilton, A., Krueger, V., Sigal, L. (eds.): Visual Analysis of Humans: Looking at People. Springer (2011)

    Google Scholar 

  22. Murray, R., Li, Z., Sastry, S.: A Mathematical Introduction to Robotic Manipulation. CRC Press, Baton Rouge (1994)

    MATH  Google Scholar 

  23. Paden, B.: Kinematics and control of robot manipulators. Ph.D. thesis (1985)

    Google Scholar 

  24. Pons-Moll, G., Baak, A., Helten, T., Müller, M., Seidel, H.P., Rosenhahn, B.: Multisensor-fusion for 3D full-body human motion capture. In: CVPR, pp. 663–670 (2010)

    Google Scholar 

  25. Pons-Moll, G., Rosenhahn, B.: Ball joints for marker-less human motion capture. In: WACV, pp. 1–8 (2009)

    Google Scholar 

  26. Pons-Moll, G., Rosenhahn, B.: Model-based pose estimation. In: Visual Analysis of Humans, pp. 139–170 (2011)

    Google Scholar 

  27. Pons-Moll, G., Baak, A., Gall, J., Leal-Taixe, L., Mueller, M., Seidel, H.P., Rosenhahn, B.: Outdoor human motion capture using inverse kinematics and von mises-fisher sampling. In: IEEE International Conference on Computer Vision (ICCV) (November 2011)

    Google Scholar 

  28. Pons-Moll, G., Leal-Taixé, L., Truong, T., Rosenhahn, B.: Efficient and Robust Shape Matching for Model Based Human Motion Capture. In: Mester, R., Felsberg, M. (eds.) DAGM 2011. LNCS, vol. 6835, pp. 416–425. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

  29. Salzmann, M., Urtasun, R.: Combining discriminative and generative methods for 3d deformable surface and articulated pose reconstruction. In: CVPR (June 2010)

    Google Scholar 

  30. Shakhnarovich, G., Viola, P., Darrell, T.: Fast pose estimation with parameter-sensitive hashing. In: ICCV, pp. 750–757 (2003)

    Google Scholar 

  31. Shoemake, K.: Animating rotation with quaternion curves. ACM SIGGRAPH 19(3), 245–254 (1985)

    CrossRef  Google Scholar 

  32. Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic Tracking of 3D Human Figures Using 2D Image Motion. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 702–718. Springer, Heidelberg (2000)

    CrossRef  Google Scholar 

  33. Sigal, L., Balan, L., Black, M.: Combined discriminative and generative articulated pose and non-rigid shape estimation. In: NIPS, pp. 1337–1344 (2008)

    Google Scholar 

  34. Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3d human tracking. In: CVPR (2003)

    Google Scholar 

  35. Tao, Y., Hu, H., Zhou, H.: Integration of vision and inertial sensors for 3D arm motion tracking in home-based rehabilitation. IJRR 26(6), 607 (2007)

    Google Scholar 

  36. Technologies, X.M.: http://www.xsens.com/

  37. Urtasun, R., Fleet, D.J., Fua, P.: 3D people tracking with gaussian process dynamical models. In: CVPR (2006)

    Google Scholar 

  38. Wang, P., Rehg, J.M.: A modular approach to the analysis and evaluation of particle filters for figure tracking. In: CVPR (2006)

    Google Scholar 

  39. Wood, A.: Simulation of the von mises-fisher distribution. Communications in Statistics - Simulation and Computation (1994)

    Google Scholar 

  40. Zhang, F., Hancock, E.R., Goodlett, C., Gerig, G.: Probabilistic white matter fiber tracking using particle filtering and von mises-fisher sampling. Medical Image Analysis 13(1), 5–18 (2009)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Leibniz University, Hannover, Germany

    Gerard Pons-Moll, Laura Leal-Taixé & Bodo Rosenhahn

  2. BIWI, ETH Zurich, Switzerland

    Juergen Gall

  3. MPI for Intelligent Systems, Germany

    Juergen Gall

Authors
  1. Gerard Pons-Moll
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura Leal-Taixé
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juergen Gall
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bodo Rosenhahn
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Computing Building, Georgia Institute of Technology, 801 Atlantic Drive, 30332-0280, Atlanta, GA, USA

    Frank Dellaert

  2. Department of Computer Science, University of North Carolina at Chapel Hill, Brooks Computer Science Building, CB#3175, 27599, Chapel Hill, NC, USA

    Jan-Michael Frahm

  3. CVG - Institute of Visual Computing, Department of Computer Science, ETH Zurich, CNB G105, Universitaetstrasse 6, 8092, Zurich, Switzerland

    Marc Pollefeys

  4. Institute for Information Processing (TNT), Leibniz Universität, Appelstr. 9A, 30167, Hannover, Germany

    Laura Leal-Taixé

  5. Institute for Information Processing (TNT), Leibniz Universität, Appelstr. 9A, 30167, Hannover, Germany

    Bodo Rosenhahn

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pons-Moll, G., Leal-Taixé, L., Gall, J., Rosenhahn, B. (2012). Data-Driven Manifolds for Outdoor Motion Capture. In: Dellaert, F., Frahm, JM., Pollefeys, M., Leal-Taixé, L., Rosenhahn, B. (eds) Outdoor and Large-Scale Real-World Scene Analysis. Lecture Notes in Computer Science, vol 7474. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34091-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34091-8_14

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34090-1

  • Online ISBN: 978-3-642-34091-8

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation