International Journal of Computer Vision

, Volume 56, Issue 3, pp 179–194 | Cite as

Twist Based Acquisition and Tracking of Animal and Human Kinematics

  • Christoph Bregler
  • Jitendra Malik
  • Katherine Pullen


This paper demonstrates a new visual motion estimation technique that is able to recover high degree-of-freedom articulated human body configurations in complex video sequences. We introduce the use and integration of a mathematical technique, the product of exponential maps and twist motions, into a differential motion estimation. This results in solving simple linear systems, and enables us to recover robustly the kinematic degrees-of-freedom in noise and complex self occluded configurations. A new factorization technique lets us also recover the kinematic chain model itself. We are able to track several human walk cycles, several wallaby hop cycles, and two walk cycels of the famous movements of Eadweard Muybridge's motion studies from the last century. To the best of our knowledge, this is the first computer vision based system that is able to process such challenging footage.

human tracking motion capture kinematic chains twists exponential maps 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ayer, S. and Sawhney, H.S. 1995. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and mdl encoding. In Int. Conf. Computer Vision, Cambridge, MA, pp. 777–784.Google Scholar
  2. Basu, S., Essa, I.A., and Pentland, A.P. 1996. Motion regularization for model-based head tracking. In International Conference on Pattern Recognition.Google Scholar
  3. Bergen, J.R., Anandan, P., Hanna, K.J., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In ECCV, pp. 237– 252.Google Scholar
  4. Black, M.J. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104.Google Scholar
  5. Black, M.J. and Yacoob, Y. 1995. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In ICCV.Google Scholar
  6. Black, M.J., Yacoob, Y., Jepson, A.D., and Fleet, D.J. 1997. Learning parameterized models of image motion. In CVPR.Google Scholar
  7. Blake, A., Isard, M., and Reynard, D. 1995. Learning to track the visual motion of contours. J. Artificial Intelligence.Google Scholar
  8. Bregler, C. and Malik, J. 1998. Estimating and tracking kinematic chains. In IEEE Conf. On Computer Vision and Pattern Recognition.Google Scholar
  9. Clergue, E., Goldber, M., Madrane, N., and Merialdo, B. 1995. Automatic face and gestual recognition for video indexing. In Proc. of the Int.Workshop on Automatic Face-and Gesture-Recognition, Zurich, 1995.Google Scholar
  10. Concalves, L., Bernardo, E.D., Ursella, E., and Perona, P. 1995. Monocular tracking of the human arm in 3d. In Proc. Int. Conf. Computer Vision.Google Scholar
  11. Davis, J.W. and Bobick, A.F. 1997. The representation and recognition of human movement using temporal templates. In CVPR.Google Scholar
  12. Dempster, A.P., Laird, N.M., and Rubin, B.D. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39.Google Scholar
  13. Gavrila, D.M. and Davis, L.S. 1950. Towards 3-d model-based tracking and recognition of human movement: A multi-view approach. In Proc. Of the Int. Workshop on Automatic Face-and Gesture-Recognition, Zurich.Google Scholar
  14. Hogg, D. 1983. A program to see a walking person. Image Vision Computing, 5(20).Google Scholar
  15. Jepson, A. and Black, M.J. 1993. Mixture models for optical flow computation. In Proc. IEEE Conf. Computer Vision Plattern Recognition, New York, pp. 760–761.Google Scholar
  16. Ju, S.X., Black, M.J., and Yacoob, Y. 1996. Cardboard people: A parameterized model of articulated motion. In 2nd Int. Conf. On Automatic Face-and Gesture-Recognition, Killington, Vermon, pp. 38–44.Google Scholar
  17. Kakadiaris, I.A. and Metaxas, D. 1996. Model-based estimation of 3d human motion with occlusion based on active multiviewpoint selection. In CVPR.Google Scholar
  18. Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. 7th Int. Joinnt Conf. on Art. Intell.Google Scholar
  19. Murray, M.P., Drought, A.B., and Kory, R.C. 1964.Walking patterns of normal men. Journal of Bone and Joint Surgery, 46-A(2):335–360.Google Scholar
  20. Murray, R.M., Li, Z., and Sastry, S.S. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press.Google Scholar
  21. Muybridge, E. 1901. The Human Figure in Motion. Various Publishers, latest edition by Dover Publications.Google Scholar
  22. Pentland, A. and Horowitz, B. 1991. Recovery of nonrigid motion and structure. IEEE Transactions on PAMI, 13(7):730–742.Google Scholar
  23. Regh, J.M. and Kanade, T. 1995. Model-based tracking of selfoccluding articulated objects. In Proc. Int. Conf. Computer Vision.Google Scholar
  24. Rohr, K. 1993. Incremental recognition of pedestrians from image sequences. In Proc. IEEE Comput. Soc. Conf. Comput. Vision and Pattern Recogn. New York City, pp. 8–13.Google Scholar
  25. Shi, J. and Tomasi, C. 1994. Good features to tract. In CVPR.Google Scholar
  26. Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography:Afactorization method. Int. J. of Computer Vision, 9(2):137–154.Google Scholar
  27. Weiss, Y. and Adelson, H.E. 1995. Perceptually organized EM: A framework for motion segmentation that combines information about form and motion. Technical Report 315, M.I.T Media Lab.Google Scholar
  28. Weiss, Y. and Adelson, H.E. 1996. A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. In Proc. IEEE Conf. Computer Vision Pattern Recognition.Google Scholar
  29. Wren, C., Azarbayejani, A., Darrell, T., and Pentland, A. 1995. Pfinder: Real-time tracking of the human body. In SPIE Conference on Integration Issues in Large Commercial Media Delivery Systems, vol. 2615.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Christoph Bregler
    • 1
  • Jitendra Malik
    • 2
  • Katherine Pullen
    • 3
  1. 1.Computer Science DepartmentStanford UniversityStanfordUSA
  2. 2.Computer Science DepartmentUniversity of California at BerkeleyBerkeleyUSA
  3. 3.Physics DepartmentStanford UniversityStanfordUSA

Personalised recommendations