EigenTracking: Robust matching and tracking of articulated objects using a view-based representation

  • Michael J. Black
  • Allan D. Jepson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1064)


This paper describes a new approach for tracking rigid and articulated objects using a view-based representation. The approach builds on and extends work on eigenspace representations, robust estimation techniques, and parameterized optical flow estimation. First, we note that the least-squares image reconstruction of standard eigenspace techniques has a number of problems and we reformulate the reconstruction problem as one of robust estimation. Second we define a “subspace constancy assumption” that allows us to exploit techniques for parameterized optical flow estimation to simultaneously solve for the view of an object and the affine transformation between the eigenspace and the image. To account for large affine transformations between the eigenspace and the image we define an EigenPyramid representation and a coarse-to-fine matching strategy. Finally, we use these techniques to track objects over long image sequences in which the objects simultaneously undergo both affine image motions and changes of view. In particular we use this “EigenTracking” technique to track and recognize the gestures of a moving hand.


Input Image Optical Flow Image Region Gesture Recognition Constancy Assumption 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    A. Baumberg and D. Hogg. Learning flexible models from image sequences. In J. Eklundh, editor, ECCV-94, vol. 800 of LNCS-Series, pp. 299–308. Stockholm. 1994.Google Scholar
  2. 2.
    M. J. Black and A. D. Jepson. EigenTracking: Robust matching and tracking of articulated objects using a view-based representation. Tech. Report T95-00515, Xerox PARC, Dec. 1995.Google Scholar
  3. 3.
    M. Black and P. Anandan. The robust estimation of multiple motions: Affine and piecewise-smooth flow fields. Computer Vision and Image Understanding, in press. Also Tech. Report P93-00104, Xerox PARC, Dec. 1993.Google Scholar
  4. 4.
    M. J. Black and P. Anandan. A framework for the robust estimation of optical flow. In ICCV-93, pp. 231–236, Berlin, May 1993.Google Scholar
  5. 5.
    A. Blake, M. Isard, and D. Reynard. Learning to track curves in motion. In Proceedings of the IEEE Conf. Decision Theory and Control, pp. 3788–3793, 1994.Google Scholar
  6. 6.
    A. F. Bobick and A. D. Wilson. A state-based technique for the summarization and recognition of gesture. In ICCV-95, pp. 382–388, Boston, June 1995.Google Scholar
  7. 7.
    C. Bregler and S. M. Omohundro. Surface learning with applications to lip reading. Advances in Neural Information Processing Systems 6, pp. 43–50, San Francisco. 1994.Google Scholar
  8. 8.
    F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel. Robust Statistics: The Approach Based on Influence Functions. John Wiley and Sons, New York, NY, 1986.Google Scholar
  9. 9.
    B. Moghaddam and A. Pentland. Probabilistic visual learning for object detection. In ICCV-95, pp. 786–793, Boston., June 1995.Google Scholar
  10. 10.
    H. Murase and S. Nayar. Visual learning and recognition of 3-D objects from appearance. International Journal of Computer Vision, 14:5–24, 1995.Google Scholar
  11. 11.
    A. Pentland, B. Moghaddam, and T. Starner. View-based and modular eigenspaces for face recognition. In CVPR-94, pp. 84–91, Seattle, June 1994.Google Scholar
  12. 12.
    M. J. Tarr and S. Pinker. Mental rotation and orientation-dependence in shape recognition. Cognitive Psychology, 21:233–282, 1989.Google Scholar
  13. 13.
    M. Turk and A. Pentland. Face recognition using eigenfaces. In CVPR-91, pp. 586–591, Maui, June 1991.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • Michael J. Black
    • 1
  • Allan D. Jepson
    • 2
    • 3
  1. 1.Xerox Palo Alto Research CenterPalo Alto
  2. 2.Department of Computer ScienceUniversity of Toronto
  3. 3.Canadian Institute for Advanced ResearchCanada

Personalised recommendations