Skip to main content
SpringerLink
Log in

Motion Models for People Tracking

  • Chapter
Visual Analysis of Humans

Abstract

This chapter provides an introduction to models of human pose and motion for use in 3D human pose tracking. We concentrate on probabilistic latent variable models of kinematics, most of which are learned from motion capture data, and on recent physics-based models. We briefly discuss important open problems and future research challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Global position and orientation with respect to the world coordinate frame are somewhat arbitrary, and often excluded. Global orientation with respect to gravity, height above the ground, and the change in position with respect to the body-centric coordinate frame should be included.

  2. 2.

    Because the data are joint angles, interpolation is normally accomplished using quaternion spherical interpolation [56]. Naturally, the temporal sampling rate must be sufficiently high that one can interpolate the pose signal with reasonable accuracy.

  3. 3.

    http://staffwww.dcs.shef.ac.uk/people/N.Lawrence/gpsoftware.html is a comprehensive GPLVM code base. GPLVM code is also in the Matlab toolbox for dimensionality reduction available at http://homepage.tudelft.nl/19j49/.

  4. 4.

    The GPLVM has a closed-form mapping from x to y, but there is no closed-form inverse mapping. As a consequence, optimization is required to find the optimal x for a given y.

  5. 5.

    GPDM Code: http://www.dgp.toronto.edu/~jmwang/gpdm/.

  6. 6.

    Code for the coordinated mixture of factor analyzers is included in the Matlab toolbox for dimensionality reduction available at http://homepage.tudelft.nl/19j49/.

  7. 7.

    Code: http://www.cs.nyu.edu/~gwtaylor/code/.

  8. 8.

    Several papers have used elastic solid models with depth inputs and a Kalman filter (e.g., [43, 80]); but these domains involve relatively simple dynamics with smooth, contact-free motions.

References

  1. Andriluka, M., Roth, S., Schiele, B.: Monocular 3d pose estimation and tracking by detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2010)

    Google Scholar 

  2. Brubaker, M.A., Fleet, D.J.: The kneed walker for human pose tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2008)

    Google Scholar 

  3. Brubaker, M.A., Fleet, D.J., Hertzmann, A.: Physics-based person tracking using the anthropomorphic walker. Int. J. Comput. Vis. 87(1–2), 140–155 (2010)

    Article  Google Scholar 

  4. Brubaker, M.A., Sigal, L., Fleet, D.J.: Estimating contact dynamics. In: IEEE International Conference on Computer Vision, pp. 2389–2396 (2009)

    Chapter  Google Scholar 

  5. Brubaker, M.A., Sigal, L., Fleet, D.J.: Physics-based human motion modeling for people tracking: A short tutotial. In: Notes from IEEE ICCV Tutorial (2009). (Available from http://www.cs.toronto.edu/~ls/iccv2009tutorial/)

    Google Scholar 

  6. Choo, K., Fleet, D.J.: People tracking using hybrid Monte Carlo filtering. In: IEEE International Conference on Computer Vision, vol. II, pp. 321–328 (2001)

    Google Scholar 

  7. Collins, S.H., Ruina, A.: A bipedal walking robot with efficient and human-like gait. In: International Conference on Robotics and Automation (2005)

    Google Scholar 

  8. Corazza, S., Muendermann, L., Chaudhari, A., Demattio, T., Cobelli, C., Andriacchi, T.: A markerless motion capture system to study musculoskeletal biomechanics: Visual hull and simulated annealing approach. Ann. Biomed. Eng. 34(6), 1019–1029 (2006)

    Article  Google Scholar 

  9. Darby, J., Li, B., Costens, N., Fleet, D.J., Lawrence, N.D.: Backing off: Hierarchical decomposition of activity for 3d novel pose recovery. In: British Machine Vision Conference (2009)

    Google Scholar 

  10. de La Gorce, M., Fleet, D.J., Paragios, N.: Model-based 3d hand pose estimation from monocular video. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1793–1805 (2011)

    Google Scholar 

  11. Deutscher, J., Reid, I.: Articulated body motion capture by stochastic search. Int. J. Comput. Vis. 61(2), 185–205 (2005)

    Article  Google Scholar 

  12. Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003)

    Article  MATH  Google Scholar 

  13. Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10(3), 197–208 (2000)

    Article  Google Scholar 

  14. Elgammal, A.M., Lee, C.-S.: Inferring 3d body pose from silhouettes using activity manifold learning. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 681–688 (2004)

    Google Scholar 

  15. Elgammal, A.M., Lee, C.-S.: Separating style and content on a nonlinear manifold. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 478–485 (2004)

    Google Scholar 

  16. Fox, E.B., Sudderth, E.B., Jordan, M.I., Willsky, A.S.: Nonparametric Bayesian learning of switching linear dynamical systems. In: Advances in Neural Information Processing Systems, pp. 457–464 (2008)

    Google Scholar 

  17. Full, R.J., Koditschek, D.E.: Templates and anchors: Neuromechanical hypotheses of legged locomotion on land. J. Exp. Biol. 202, 3325–3332 (1999)

    Google Scholar 

  18. Gall, J., Rosenhahn, B., Brox, T., Seidel, H.-P.: Optimization and filtering for human motion capture. Int. J. Comput. Vis. 87(1–2), 75–92 (2010)

    Article  Google Scholar 

  19. Gordon, N.J., Salmond, D.J., Smith, A.F.M.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc., F, Radar Signal Process. 140, 107–113 (1993)

    Article  Google Scholar 

  20. Grochow, K., Martin, S.L., Hertzmann, A., Popovic, Z.: Style-based inverse kinematics. ACM Trans. Graph. 23(3), 522–531 (2004)

    Article  Google Scholar 

  21. Hauberg, S., Sommer, S., Pedersen, K.S.: Gaussian-like spatial priors for articulated tracking. In: European Conference on Computer Vision, vol. 1, pp. 425–437 (2010)

    Google Scholar 

  22. Herda, L., Urtasun, R., Fua, P.: Hierarchical implicit surface joint limits for human body tracking. Comput. Vis. Image Underst. 99(2), 189–209 (2005)

    Article  Google Scholar 

  23. Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)

    Article  MATH  Google Scholar 

  24. Hinton, G.E.: A practical guide to training restricted Boltzmann machines. Technical Report UTML TR 2010-003, Department of Computer Science, University of Toronto (2010)

    Google Scholar 

  25. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hou, S., Galata, A., Caillette, F., Thacker, N.A., Bromiley, P.A.: Real-time body tracking using a Gaussian process latent variable model. In: IEEE International Conference on Computer Vision (2007)

    Google Scholar 

  27. Howe, N.R., Leventon, M.E., Freeman, W.T.: Bayesian reconstruction of 3d human motion from single-camera video. In: Advances in Neural Information Processing Systems, pp. 820–826 (1999)

    Google Scholar 

  28. Hyndman, M., Jepson, A.D., Fleet, D.J.: Higher-order autoregressive models for dynamic textures. In: British Machine Vision Conference (2007)

    Google Scholar 

  29. Isard, M., Blake, A.: CONDENSATION – Conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)

    Article  Google Scholar 

  30. Kakadiaris, L., Metaxas, D.: Model-based estimation of 3D human motion. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1453–1459 (2000)

    Article  Google Scholar 

  31. Kong, A., Liu, J.S., Wong, W.H.: Sequential imputations and Bayesian missing data problems. J. Am. Stat. Assoc. 89(425), 278–288 (1994)

    Article  MATH  Google Scholar 

  32. Kuo, A.D.: A simple model of bipedal walking predicts the preferred speed–step length relationship. J. Biomech. Eng. 123(3), 264–269 (2001)

    Article  Google Scholar 

  33. Lawrence, N.D.: Probabilistic non-linear principal component analysis with Gaussian process latent variable models. J. Mach. Learn. Res. 6, 1783–1816 (2005)

    MathSciNet  Google Scholar 

  34. Lawrence, N.D., Moore, A.J.: Hierarchical Gaussian process latent variable models. In: International Conference on Machine Learning, pp. 481–488 (2007)

    Google Scholar 

  35. Lawrence, N.D., Quiñonero-Candela, J.: Local distance preservation in the gp-lvm through back constraints. In: International Conference on Machine Learning, pp. 513–520 (2006)

    Google Scholar 

  36. Li, R.: Simultaneous learning of non-linear manifold and dynamical models for high-dimensional time series. PhD thesis, Boston University (2009)

    Google Scholar 

  37. Li, R., Tian, T.-P., Sclaroff, S., Yang, M.-H.: 3d human motion tracking with a coordinated mixture of factor analyzers. Int. J. Comput. Vis. 87(1–2), 170–190 (2010)

    Article  Google Scholar 

  38. Li, R., Yang, M.-H., Sclaroff, S., Tian, T.-P.: Monocular tracking of 3d human motion with a coordinated mixture of factor analyzers. In: European Conference on Computer Vision, vol. 2, pp. 137–150 (2006)

    Google Scholar 

  39. Liu, C.K., Hertzmann, A., Popović, Z.: Learning physics-based motion style with nonlinear inverse optimization. ACM Trans. Graph. 24(3), 1071–1081 (2005)

    Article  Google Scholar 

  40. Lu, Z., Carreira-Perpiñán, M.Á., Sminchisescu, C.: People tracking with the Laplacian eigenmaps latent variable model. In: Advances in Neural Information Processing Systems, pp. 1705–1712 (2007)

    Google Scholar 

  41. MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  42. McGeer, T.: Dynamics and control of bipedal locomotion. J. Theor. Biol. 163, 277–314 (1993)

    Article  Google Scholar 

  43. Metaxas, D., Terzopoulos, D.: Shape and nonrigid motion estimation through physics-based synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 15(6), 580–591 (1993)

    Article  Google Scholar 

  44. North, B., Blake, A., Isard, M., Rittscher, J.: Learning and classification of complex dynamics. IEEE Trans. Pattern Anal. Mach. Intell. 22(9), 1016–1034 (2000)

    Article  Google Scholar 

  45. Oh, S.M., Rehg, J.M., Balch, T.R., Dellaert, F.: Learning and inferring motion patterns using parametric segmental switching linear dynamic systems. Int. J. Comput. Vis. 77(1–3), 103–124 (2008)

    Article  Google Scholar 

  46. Pan, W., Torresani, L.: Unsupervised hierarchical modeling of locomotion styles. In: International Conference on Machine Learning, p. 99 (2009)

    Google Scholar 

  47. Pavlovic, V., Rehg, J.M., MacCormick, J.: Learning switching linear models of human motion. In: Advances in Neural Information Processing Systems, pp. 981–987 (2000)

    Google Scholar 

  48. Poon, E., Fleet, D.J.: Hybrid Monte Carlo filtering: Edge-based people tracking. In: Workshop on Motion and Video Computing, pp. 151–158 (2002)

    Chapter  Google Scholar 

  49. Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005)

    MathSciNet  Google Scholar 

  50. Raskin, L.M., Rivlin, E., Rudzsky, M.: Using Gaussian process annealing particle filter for 3d human tracking. EURASIP J. Adv. Sig. Proc. (2008)

    Google Scholar 

  51. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  52. Robertson, D.G.E., Caldwell, G.E., Hamill, J., Kamen, G., Whittlesey, S.N.: Research Methods in Biomechanics. Human Kinetics, Champaign (2004)

    Google Scholar 

  53. Rose, C., Cohen, M.F., Bodenheimer, B.: Verbs and adverbs: Multidimensional motion interpolation. IEEE Comput. Graph. Appl. 18(5), 32–40 (1998)

    Article  Google Scholar 

  54. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(550), 2323–2326 (2000)

    Article  Google Scholar 

  55. Roweis, S.T., Saul, L.K., Hinton, G.E.: Global coordination of local linear models. In: Advances in Neural Information Processing Systems, pp. 889–896 (2001)

    Google Scholar 

  56. Shoemake, K.: Animating rotation with quaternion curves. In: ACM Transactions on Graphics, pp. 245–254 (1985)

    Google Scholar 

  57. Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3d human figures using 2d image motion. In: European Conference on Computer Vision, vol. 2, pp. 702–718 (2000)

    Google Scholar 

  58. Sigal, L., Balan, A.O., Black, M.J.: Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis. 87(1–2), 4–27 (2010)

    Article  Google Scholar 

  59. Sigal, L., Fleet, D.J., Troje, N., Livne, M.: Human attributes from 3d pose tracking. In: European Conference on Computer Vision (2010)

    Google Scholar 

  60. Sminchisescu, C., Jepson, A.: Generative modeling for continuous non-linearly embedded visual inference. In: International Conference on Machine Learning, pp. 759–766 (2004)

    Google Scholar 

  61. Sminchisescu, C., Triggs, B.: Kinematic jump processes for monocular 3D human tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 69–76, Madison (2003)

    Google Scholar 

  62. Taylor, G.W., Hinton, G.E.: Factored conditional restricted Boltzmann machines for modeling motion style. In: International Conference on Machine Learning (2009)

    Google Scholar 

  63. Taylor, G.W., Hinton, G.E., Roweis, S.T.: Modeling human motion using binary latent variables. In: Advances in Neural Information Processing Systems, pp. 1345–1352 (2006)

    Google Scholar 

  64. Taylor, G.W., Sigal, L., Fleet, D.J., Hinton, G.E.: Dynamical binary latent variable models for 3d human pose tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 631–638 (2010)

    Chapter  Google Scholar 

  65. Tenenbaum, J.B., Freeman, W.T.: Separating style and content with bilinear models. Neural Comput. 12(6), 1247–1283 (2000)

    Article  Google Scholar 

  66. Troje, N.: Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. J. Vis. 2(5), 371–387 (2002)

    Article  Google Scholar 

  67. Urtasun, R., Fleet, D.J., Fua, P.: 3D people tracking with Gaussian process dynamical models. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 238–245 (2006)

    Google Scholar 

  68. Urtasun, R., Fleet, D.J., Hertzmann, A., Fua, P.: Priors for people tracking from small training sets. In: IEEE International Conference on Computer Vision, vol. 1, pp. 403–410 (2005)

    Google Scholar 

  69. Urtasun, R., Fleet, D.J., Fua, P.: Motion models for 3D people tracking. Comput. Vis. Image Underst. 104(2–3), 157–177 (2006)

    Article  Google Scholar 

  70. Urtasun, R., Fleet, D.J., Geiger, A., Popovic, J., Darrell, T., Lawrence, N.D.: Topologically-constrained latent variable models. In: International Conference on Machine Learning, pp. 1080–1087 (2008)

    Google Scholar 

  71. Van Overschee, P., De Moor, B.: N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica 30(1), 75–93 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  72. Vasilescu, M.A.O.: Human motion signatures: Analysis, synthesis, recognition. In: International Conference on Pattern Recognition, vol. III, pp. 456–460 (2002)

    Google Scholar 

  73. Verbeek, J.J.: Learning nonlinear image manifolds by global alignment of local linear models. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1236–1250 (2006)

    Article  Google Scholar 

  74. Wachter, S., Nagel, H.H.: Tracking persons in monocular image sequences. Comput. Vis. Image Underst. 74(3), 174–192 (1999)

    Article  Google Scholar 

  75. Wang, J.M.: Locomotion synthesis methods for humanoid characters. PhD thesis, University of Toronto (2010)

    Google Scholar 

  76. Wang, J.M., Fleet, D.J., Hertzmann, A.: Multifactor Gaussian process models for style-content separation. In: International Conference on Machine Learning, pp. 975–982 (2007)

    Google Scholar 

  77. Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 283–298 (2008)

    Article  Google Scholar 

  78. Wang, J.M., Fleet, D.J., Hertzmann, A.: Optimizing walking controllers. ACM Trans. Graph. 28(5) (2009)

    Google Scholar 

  79. Wang, J.M., Fleet, D.J., Hertzmann, A.: Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. 29(4) (2010)

    Google Scholar 

  80. Wren, C.R., Pentland, A.: Dynamic models of human motion. In: International Conference on Automatic Face and Gesture Recognition, pp. 22–27 (1998)

    Google Scholar 

  81. Yin, K., Loken, K., van de Panne, M.: Simbicon: Simple biped locomotion control. ACM Trans. Graph. 26(3) (2007)

    Google Scholar 

Download references

Acknowledgements

Many thanks to research colleagues with whom I have worked on modeling human motion: Michael Black, Marcus Brubaker, Aaron Hertzmann, Geoff Hinton, Hedvig Kjellström. Neil Lawrence, Roland Memisevic, Leonid Sigal, Graham Taylor, Niko Troje, Raquel Urtasun, and Jack Wang. We gratefully acknowledge generous financial support from NSERC Canada and the Canadian Institute for Advanced Research (CIfAR).

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Toronto, Toronto, Canada

    David J. Fleet

Authors
  1. David J. Fleet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David J. Fleet .

Editor information

Editors and Affiliations

  1. Department of Media Technology, Aalborg University, Niels Jernes Vej 14, Aalborg, 9220, Denmark

    Thomas B. Moeslund

  2. Centre for Vision, Speech & Signal Proc., University of Surrey, Guildford, GU2 7XH, Surrey, United Kingdom

    Adrian Hilton

  3. Copenhagen Institute of Technology, Aalborg University, Lautrupvang 2B, Ballerup, 2750, Denmark

    Volker Krüger

  4. Disney Research, Forbes Avenue 615, Pittsburgh, 15213, Pennsylvania, USA

    Leonid Sigal

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Fleet, D.J. (2011). Motion Models for People Tracking. In: Moeslund, T., Hilton, A., Krüger, V., Sigal, L. (eds) Visual Analysis of Humans. Springer, London. https://doi.org/10.1007/978-0-85729-997-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-997-0_10

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-996-3

  • Online ISBN: 978-0-85729-997-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation