Skip to main content
SpringerLink
Log in

Physics-Based Person Tracking Using the Anthropomorphic Walker

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We introduce a physics-based model for 3D person tracking. Based on a biomechanical characterization of lower-body dynamics, the model captures important physical properties of bipedal locomotion such as balance and ground contact. The model generalizes naturally to variations in style due to changes in speed, step-length, and mass, and avoids common problems (such as footskate) that arise with existing trackers. The dynamics comprise a two degree-of-freedom representation of human locomotion with inelastic ground contact. A stochastic controller generates impulsive forces during the toe-off stage of walking, and spring-like forces between the legs. A higher-dimensional kinematic body model is conditioned on the underlying dynamics. The combined model is used to track walking people in video, including examples with turning, occlusion, and varying gait. We also report quantitative monocular and binocular tracking results with the HumanEva dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Agarwal, A., & Triggs, B. (2006). Recoving 3D human pose from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 44–58.

    Article  Google Scholar 

  • Bissacco, A. (2005). Modeling and learning contact dynamics in human motion. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 421–428).

  • Blickhan, R., & Full, R. J. (1993). Similarity in multilegged locomotion: bouncing like a monopode. Journal of Comparative Physiology A: Neuroethology, Sensory, Neural, and Behavioral Physiology, 173(5), 509–517.

    Article  Google Scholar 

  • Brubaker, M. A., & Fleet, D. J. (2008). The kneed walker for human pose tracking. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Brubaker, M. A., Fleet, D. J., & Hertzmann, A. (2007). Physics-based person tracking using simplified lower-body dynamics. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Chan, M., Metaxas, D., & Dickinson, S. (1994). Physics-based tracking of 3D objects in 2D image sequences. In Proceedings of ICPR (pp. 432–436).

  • Choo, K., & Fleet, D. J. (2001). People tracking using hybrid Monte Carlo filtering. In Proceedings of IEEE international conference on computer vision (Vol. II, pp. 321–328).

  • Collins, S. H., & Ruina, A. (2005). A bipedal walking robot with efficient and human-like gait. In Proceedings of IEEE conference on robotics and automation.

  • Collins, S. H., Wisse, M., & Ruina, A. (2001). A three-dimensional passive-dynamic walking robot with two legs and knees. International Journal of Robotics Research, 20(7), 607–615.

    Article  Google Scholar 

  • Collins, S., Ruina, A., Tedrake, R., & Wisse, M. (2005). Efficient bipedal robots based on passive-dynamic walkers. Science, 307(5712), 1082–1085.

    Article  Google Scholar 

  • Delamarre, Q., & Faugeras, O. (2001). 3D articulated models and multiview tracking with physical forces. Computer Vision and Image Understanding, 81(3), 328–357.

    Article  MATH  Google Scholar 

  • Doucet, A., Godsill, S., & Andrieu, C. (2000). On sequential Monte Carlo sampling methods for Bayesian filtering. Statistics and Computing, 10(3), 197–208.

    Article  Google Scholar 

  • Elgammal, A., & Lee, C.-S. (2004). Inferring 3D body pose from silhouettes using activity manifold learning. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 681–688).

  • Fleet, D., & Weiss, Y. (2005). Optical flow estimation. In The handbook of mathematical models of computer vision (pp. 239–258). Berlin: Springer.

    Google Scholar 

  • Full, R. J., & Koditschek, D. E. (1999). Templates and anchors: neuromechanical hypotheses of legged locomotion on land. Journal of Experimental Biology, 202, 3325–3332.

    Google Scholar 

  • Goldstein, H., Poole, C. P., & Safko, J. L. (2001). Classical mechanics (3rd ed.). Reading: Addison-Wesley.

    Google Scholar 

  • Herda, L., Urtasun, R., & Fua, P. (2005). Hierarchical implicit surface joint limits for human body tracking. Computer Vision and Image Understanding, 99(2), 189–209.

    Article  Google Scholar 

  • Hodgins, J. K., Wooten, W. L., Brogan, D. C., & O’Brien, J. F. (1995). Animating human athletics. In Proceedings of SIGGRAPH (pp. 71–78).

  • Kakadiaris, L., & Metaxas, D. (2000). Model-based estimation of 3D human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1453–1459.

    Article  Google Scholar 

  • Kass, M., Witkin, A., & Terzopoulos, D. (1987). Snakes: active contour models. International Journal of Computer Vision, 1(4), 321–331.

    Article  Google Scholar 

  • Khan, Z., Balch, T., & Dellaert, F. (2004). A Rao-blackwellized particle filter for eigentracking. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 980–986).

  • Kong, A., Liu, J. S., & Wong, W. H. (1994). Sequential imputations and Bayesian missing data problems. Journal of the American Statistical Association, 89(425), 278–288.

    Article  MATH  Google Scholar 

  • Kovar, L., Schreiner, J., & Gleicher, M. (2002). Footskate cleanup for motion capture editing. In Proceedings of symposium on computer animation.

  • Kuo, A. D. (2001). A simple model of bipedal walking predicts the preferred speed-step length relationship. Journal of Biomechanical Engineering, 123(3), 264–269.

    Article  Google Scholar 

  • Kuo, A. D. (2002). Energetics of actively powered locomotion using the simplest walking model. Journal of Biomechanical Engineering, 124(1), 113–120.

    Article  Google Scholar 

  • Liu, C. K., Hertzmann, A., & Popović, Z. (2005). Learning physics-based motion style with nonlinear inverse optimization. ACM Transactions on Graphics, 24(3), 1071–1081.

    Article  Google Scholar 

  • McGeer, T. (1990a). Passive dynamic walking. International Journal of Robotics Research, 9(2), 62–82.

    Article  Google Scholar 

  • McGeer, T. (1990b). Passive walking with knees. In Proceedings of international conference on robotics and automation (Vol. 3, pp. 1640–1645).

  • McGeer, T. (1992). Principles of walking and running. In Advances in comparative and environmental physiology (Vol. 11, Chap. 4, pp. 113–139). Berlin: Springer.

    Google Scholar 

  • Metaxas, D., & Terzopoulos, D. (1993). Shape and nonrigid motion estimation through physics-based synthesis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(6), 580–591.

    Article  Google Scholar 

  • Pavlović, V., Rehg, J., Cham, T.-J., & Murphy, K. (1999). A dynamic Bayesian network approach to figure tracking using learned dynamic models. In Proceedings of IEEE international conference on computer vision (pp. 94–101).

  • Pentland, A., & Horowitz, B. (1991). Recovery of nonrigid motion and structure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 730–742.

    Article  Google Scholar 

  • Pitt, M. K., & Shepard, N. (1999). Filtering via simulation: auxiliary particle filters. Journal of the American Statistical Association, 94, 590–599.

    Article  MATH  MathSciNet  Google Scholar 

  • Popović, Z., & Witkin, A. (1999). Physically based motion transformation. In Proceedings of SIGGRAPH (pp. 11–20).

  • Pratt, G. A. (2000). Legged robots at MIT: what’s new since Raibert? IEEE Transactions on Robotics and Automation, 7(3), 15–19.

    Article  Google Scholar 

  • Rahimi, A., Recht, B., & Darrell, T. (2005). Learning appearance manifolds from video. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 868–875).

  • Robert, C. P. (1995). Simulation of truncated normal variables. Statistics and Computing, 5(2), 121–125.

    Article  MATH  Google Scholar 

  • Rosales, R., Athitsos, V., Sigal, L., & Sclaroff, S. (2001). 3D hand pose reconstruction using specialized mappings. In Proceedings of IEEE international conference on computer vision (Vol. 1, pp. 378–385).

  • Safonova, A., Hodgins, J. K., & Pollard, N. S. (2004). Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Transactions on Graphics, 23(3), 514–521.

    Article  Google Scholar 

  • Shakhnarovich, G., Viola, P., & Darrell, T. (2003). Fast pose estimation with parameter-sensitive hashing. In Proceedings of IEEE international conference on computer vision (pp. 750–757).

  • Shin, H. J., Kovar, L., & Gleicher, M. (2003). Physical touchup of human motions. In Proceedings of Pacific graphics (pp. 194–203).

  • Sidenbladh, H., Black, M. J., & Fleet, D. J. (2000). Stochastic tracking of 3D human figures using 2D image motion. In Proceedings of IEEE European conference on computer vision (Vol. 2, pp. 702–718).

  • Sigal, L., & Black, M. (2006). HumanEva: Synchronized video and motion capture dataset for evaluation of articulated human motion. Technical report CS-06-08, Computer science, Brown University.

  • Sminchisescu, C., & Jepson, A. (2004). Generative modeling for continuous non-linearly embedded visual inference. In Proceedings of international conference on machine learning (pp. 96–103).

  • Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2007). BME 3: Discriminative density propagation for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 2030–2044.

    Article  Google Scholar 

  • Terzopoulos, D., & Metaxas, D. (1990). Dynamic 3D models with local and global deformations: deformable superquadrics. In Proceedings of IEEE international conference on computer vision (pp. 606–615).

  • Urtasun, R., Fleet, D. J., Hertzmann, A., & Fua, P. (2005). Priors for people tracking from small training sets. In Proceedings of IEEE international conference on computer vision (Vol. 1, pp. 403–410).

  • Urtasun, R., Fleet, D. J., & Fua, P. (2006). 3D people tracking with Gaussian process dynamical models. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 238–245).

  • Van der Linde, R. Q., & Schwab, A. L. (2002). Lecture notes multibody dynamics B, wb1413, course 1997/1998. Lab. for Engineering Mechanics, Delft University of Technology.

  • Vondrak, M., Sigal, L., & Jenkins, O. C. (2008). Physical simulation for probabilistic motion tracking. In Proceedings of IEEE conference on computer vision and pattern recognition.

  • Wachter, S., & Nagel, H. H. (1999). Tracking persons in monocular image sequences. Computer Vision and Image Understanding, 74(3), 174–192.

    Article  Google Scholar 

  • Wisse, M., Hobbelen, D. G. E., & Schwab, A. L. (2007). Adding an upper body to passive dynamic walking robots by means of a bisecting hip mechanism. IEEE Transactions on Robotics, 23(1), 112–123.

    Article  Google Scholar 

  • Witkin, A., & Kass, M. (1988). Spacetime constraints. In Proceedings of SIGGRAPH (Vol. 22, pp. 159–168).

  • Wren, C. R., & Pentland, A. (1998). Dynamics models of human motion. In Proceedings of automatic face and gesture recognition.

  • Yin, K., Loken, K., & van de Panne, M. (2007). SIMBICON: simple biped locomotion control. ACM Transactions on Graphics, 26(3).

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Toronto, Toronto, Canada

    Marcus A. Brubaker, David J. Fleet & Aaron Hertzmann

Authors
  1. Marcus A. Brubaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David J. Fleet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aaron Hertzmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcus A. Brubaker.

Additional information

This work was financially supported in part by NSERC Canada, the Canadian Institute for Advanced Research (CIFAR), the Canada Foundation for Innovation (CFI), the Alfred P. Sloan Foundation, Microsoft Research and the Ontario Ministry of Research and Innovation. A preliminary version of this work appeared in Brubaker et al. (2007).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Brubaker, M.A., Fleet, D.J. & Hertzmann, A. Physics-Based Person Tracking Using the Anthropomorphic Walker. Int J Comput Vis 87, 140–155 (2010). https://doi.org/10.1007/s11263-009-0274-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0274-5

Keywords

Search

Navigation