Recognition and Segmentation of 3-D Human Action Using HMM and Multi-class AdaBoost

  • Fengjun Lv
  • Ramakant Nevatia
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3954)


Our goal is to automatically segment and recognize basic human actions, such as stand, walk and wave hands, from a sequence of joint positions or pose angles. Such recognition is difficult due to high dimensionality of the data and large spatial and temporal variations in the same action. We decompose the high dimensional 3-D joint space into a set of feature spaces where each feature corresponds to the motion of a single joint or combination of related multiple joints. For each feature, the dynamics of each action class is learned with one HMM. Given a sequence, the observation probability is computed in each HMM and a weak classifier for that feature is formed based on those probabilities. The weak classifiers with strong discriminative power are then combined by the Multi-Class AdaBoost (AdaBoost.M2) algorithm. A dynamic programming algorithm is applied to segment and recognize actions simultaneously. Results of recognizing 22 actions on a large number of motion capture sequences as well as several annotated and automatically tracked sequences show the effectiveness of the proposed algorithms.


Hide Markov Model Action Class Action Recognition Joint Position Observation Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Campbell, L., Bobick, A.: Recognition of human body motion using phase space constraints. In: Proc. of ICCV, pp. 624–630 (1995)Google Scholar
  2. 2.
    Davis, J., Bobick, A.: The Representation and Recognition of Action Using Temporal Templates. In: Proc. of CVPR, pp. 928–934 (1997)Google Scholar
  3. 3.
    Efros, A.A., Berg, A.C., Mori, G., Malik, J.: Recognizing Action at a Distance. In: Proc. of ICCV, pp. 726–733 (2003)Google Scholar
  4. 4.
    Freund, Y., Schapire, R.E.: A decision theoretic generalization of on-line learning and application to boosting. Journal of Computer and System Science 55(1), 119–139 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Hongeng, S., Nevatia, R.: Large-Scale Event Detection Using Semi-Hidden Markov Models. In: Proc. of ICCV, pp. 1455–1462 (2003)Google Scholar
  6. 6.
    Lee, M.W., Nevatia, R.: Dynamic Human Pose Estimation using Markov chain Monte Carlo Approach. In: Proc. of the IEEE Workshop on Motion and Video Computing, WACV/MOTION 2005 (2005)Google Scholar
  7. 7.
    Lv, F., Nevatia, R.: 3D Human Action Recognition Using Spatio-Temporal Motion Templates. In: Proc. of the IEEE Workshop on Human-Computer Interaction, HCI 2005 (2005)Google Scholar
  8. 8.
    Rabiner, L.R.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proc. of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  9. 9.
    Rao, C., Yilmaz, A., Shah, M.: View-Invariant Representation and Recognition of Actions. Int’l Journal of Computer Vision 50(2), 203–226 (2002)CrossRefzbMATHGoogle Scholar
  10. 10.
    Shechtman, E., Irani, M.: Space-Time Behavior Based Correlation. In: Proc. of CVPR, vol. I, pp. 405–412 (2005)Google Scholar
  11. 11.
    Viola, P., Jones, M.: Rapid Object Detection Using a Boosted Cascade of Simple Features. In: Proc. of CVPR, pp. 511–518 (2001)Google Scholar
  12. 12.
    Yilmaz, A., Shah, M.: Actions Sketch: A Novel Action Representation. In: Proc. of CVPR, vol. I, pp. 984–989 (2005)Google Scholar
  13. 13.
    Yin, P., Essa, I., Rehg, J.M.: Asymmetrically Boosted HMM for Speech Reading. In: Proc. of CVPR, vol. II, pp. 755–761 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Fengjun Lv
    • 1
  • Ramakant Nevatia
    • 1
  1. 1.Institute for Robotics and Intelligent SystemsUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations