Recognizing and Tracking Human Action

  • Josephine Sullivan
  • Stefan Carlsson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2350)


Human activity can be described as a sequence of 3D body postures. The traditional approach to recognition and 3D reconstruction of human activity has been to track motion in 3D, mainly using advanced geometric and dynamic models. In this paper we reverse this process. View based activity recognition serves as an input to a human body location tracker with the ultimate goal of 3D reanimation in mind. We demonstrate that specific human actions can be detected from single frame postures in a video sequence. By recognizing the image of a person’s posture as corresponding to a particular key frame from a set of stored key frames, it is possible to map body locations from the key frames to actual frames. This is achieved using a shape matching algorithm based on qualitative similarity that computes point to point correspondence between shapes, together with information about appearance. As the mapping is from fixed key frames, our tracking does not suffer from the problem of having to reinitialise when it gets lost. It is effectively a closed loop. We present experimental results both for recognition and tracking for a sequence of a tennis player.


Human motion tracking shape correspondence 


  1. 1.
    S. Belongie and J. Malik. Matching with shape contexts. In IEEE Workshop on Content-based Access of Image and Video Libraries, June 2000.Google Scholar
  2. 2.
    A. Blake and M. Isard. Active Contours. Springer, 1998.Google Scholar
  3. 3.
    M. Brand. Shadow puppetry. In Proc. 7th Int. Conf. on Computer Vision, pages 1237–1244, 1999.Google Scholar
  4. 4.
    C. Bregler and J. Malik. Tracking people with twists and exponential maps. In Proc. Conf. Computer Vision and Pattern Recognition, pages 8–15, 1998.Google Scholar
  5. 5.
    S. Carlsson. Order structure, correspondence and shape based categories. In Shape Contour and Grouping in Computer Vision, pages 58–71. Springer LNCS 1681, 1999.CrossRefGoogle Scholar
  6. 6.
    S. Carlsson and J. Sullivan. Action recognition by shape matching to key frames. Workshop on Models versus Exemplars in Computer Vision at CVPR, 2001. Available at
  7. 7.
    D. Comaniciu, V. Ramesh, and P. Meer. Real-time tracking of non-rigid objects using mean shift. In Proc. Conf. Computer Vision and Pattern Recognition, volume 2, pages 142–149, Hilton Head Island, South Carolina, 2000.Google Scholar
  8. 8.
    J. Deutscher, A. Blake, and I. Reid. Motion capture by annealed particle filtering. Proc. Conf. Computer Vision and Pattern Recognition, 2000.Google Scholar
  9. 9.
    D.M. Gavrila. The visual analysis of human movement: A survey. Computer Vision and Image Understanding, 73(1):82–98, January 1999.Google Scholar
  10. 10.
    S.J. Godsill, A. Doucet, and M. West. Methodology for monte carlo smoothing with application to time-varying autoregressions. In Proc. International Symposium on Frontiers of Time Series Modelling, 2000.Google Scholar
  11. 11.
    D. Hogg. Model-based vision: a program to see a walking person. J. Image and Vision Computing, 1(1):5–20, 1983.CrossRefGoogle Scholar
  12. 12.
    N. R. Howe, M. E. Leventon, and W. T. Freeman. Bayesian reconstruction of 3d human motion from single-camera video. In S. A. Solla T. K. Leen and K-R. Muller, editors, Advances in Neural Information Processing Systems 12, 2000.Google Scholar
  13. 13.
    Y. Lamdan, J. Schwartz, and H. Wolfson. Object recognition by affine invariant matching. In Proc. Conf. Computer Vision and Pattern Recognition, pages 335–344, 1988.Google Scholar
  14. 14.
    D.D. Morris and J.M. Rehg. Singularity analysis for articulated object tracking. In Proc. Conf. Computer Vision and Pattern Recognition, pages 289–296, 1998.Google Scholar
  15. 15.
    N. Paragios and R. Deriche. Geodesic active regions for motion estimation and tracking. Proc. 7th Int. Conf. on Computer Vision, 1999.Google Scholar
  16. 16.
    J. Rehg and T. Kanade. Model-based tracking of self-occluding articulated objects. Proc. 5th Int. Conf. on Computer Vision, 1995.Google Scholar
  17. 17.
    K. Rohr. Towards model-based recognition of human movements in image sequences. Computer Vision, Graphics and Image Processing, 59(1):94–115, 1994.CrossRefGoogle Scholar
  18. 18.
    H. Sidenbladh, M. Black, and D.J. Fleet. Stochastic tracking of 3d human figures using 2d image motion. In Poc of European Conference on Computer Vision, pages 702–718, 2000.Google Scholar
  19. 19.
    K. Toyama and A. Blake. Probabilistic tracking in a metric space. In Proc. 8th Int. Conf. on Computer Vision, July 2001.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Josephine Sullivan
    • 1
  • Stefan Carlsson
    • 1
  1. 1.Numerical Analysis and Computing ScienceRoyal Institute of Technology, (KTH)StockholmSweden

Personalised recommendations