Fusion of Human Posture Features for Continuous Action Recognition

  • Khai Tran
  • Ioannis A. Kakadiaris
  • Shishir K. Shah
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6553)


This paper presents a real-time or online system for continuous recognition of human actions. The system recognizes actions such as walking, bending, jumping, waving, and falling and relies on spatial features computed to characterize human posture. The paper evaluates the utility of these features based on its joint or independent treatment within the context of the Hidden Markov Model (HMM) framework. A baseline approach wherein disparate spatial features are treated as an input vector to trained HMMs is used to compare three different independent feature models. In addition, an action transition constraints is introduced to stabilize the developed models and allow for continuity in recognized actions. The system is evaluated across a dataset of videos and results reported in terms of frame error rate, the frame delay in recognizing an action, action recognition rate, and the missed and false recognition rates. Experimental results shows the effectiveness of the proposed treatment of input features and the corresponding HMM formulations.


Contiuous Action Recognition HMMs Fusion of Features 


  1. 1.
    Kellokumpu, V., Pietikäinen, M., Heikkilä, J.: Human activity recognition using sequences of postures. In: Machine Vision Applications, pp. 570–573 (2005)Google Scholar
  2. 2.
    Gavrila, D.M.: The visual analysis of human movement: a survey. Comput. Vis. Image Underst. 73, 82–98 (1999)zbMATHCrossRefGoogle Scholar
  3. 3.
    Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104, 90–126 (2006)CrossRefGoogle Scholar
  4. 4.
    Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 257–267 (2001)CrossRefGoogle Scholar
  5. 5.
    Ali, A., Aggarwal, J.: Segmentation and recognition of continuous human activity. In: Proceedings of IEEE Workshop on Detection and Recognition of Events in Video, pp. 28–35 (2001)Google Scholar
  6. 6.
    Eickeler, S., Kosmala, A., Rigoll, G.: Hidden markov model based continuous online gesture recognition. In: ICPR 1998: Proceedings of the 14th International Conference on Pattern Recognition, vol. 2, p. 1206. IEEE Computer Society, Washington, DC (1998)Google Scholar
  7. 7.
    Iwai, Y., Shimizu, H., Yachida, M.: Real-time context-based gesture recognition using hmm and automaton. In: IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real -Time Systems, p. 127 (1999)Google Scholar
  8. 8.
    Laptev, I.: On space-time interest points. Int. J. Comput. Vision 64, 107–123 (2005)CrossRefGoogle Scholar
  9. 9.
    Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: VS-PETS (2005)Google Scholar
  10. 10.
    Niebles, J.C., Fei-Fei, L.: A hierarchical model of shape and appearance for human action classification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  11. 11.
    Niebles, J.C., Wang, H., Fei-Fei, L.: Unsupervised learning of human action categories using spatial-temporal words. International Journal of Computer Vision (2008)Google Scholar
  12. 12.
    Sun, X., Chen, M., Hauptmann, A.: Action recognition via local descriptors and holistic features. In: Computer Vision and Pattern Recognition Workshop, pp. 58–65 (2009)Google Scholar
  13. 13.
    Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: ICCV 2005: Proceedings of the Tenth IEEE International Conference on Computer Vision, pp. 1395–1402. IEEE Computer Society, Washington, DC (2005)Google Scholar
  14. 14.
    Shechtman, E., Irani, M.: Space-time behavior based correlation. In: CVPR 2005: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 405–412. IEEE Computer Society, Washington, DC (2005)Google Scholar
  15. 15.
    Kobayashi, T., Otsu, N.: Action and simultaneous multiple-person identification using cubic higher-order local auto-correlation. In: ICPR 2004: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 4, pp. 741–744. IEEE Computer Society, Washington, DC (2004)CrossRefGoogle Scholar
  16. 16.
    Ribeiro, P.C., Santos-victor, J.: Human activity recognition from video: modeling, feature selection and classification architecture. In: International Workshop on Human Activity Recognition and Modeling, HAREM (2005)Google Scholar
  17. 17.
    Miaou, S.G., Sung, P.H., Huang, C.Y.: A customized human fall detection system using omni-camera images and personal information. In: 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, pp. 39–42 (2006)Google Scholar
  18. 18.
    Anderson, D., Keller, J.M., Skubic, M., Chen, X., He, Z.: Recognizing falls from silhouettes. In: 28th EMBC International Conference on Engineering in Medicine and Biology Society, pp. 6388–6391 (2006)Google Scholar
  19. 19.
    Guo, F., Chellappa, R.: Video metrology using a single camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 1329–1335 (2010)CrossRefGoogle Scholar
  20. 20.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000) ISBN: 0521623049Google Scholar
  21. 21.
    DeMenthon, D., Davis, L.S.: Model-based object pose in 25 lines of code. International Journal of Computer Vision 15(1-2) (1995)Google Scholar
  22. 22.
    Rabiner, L., Juang, B.: An introduction to hidden markov models. IEEE ASSP Magazine 3, 4–16 (1986)CrossRefGoogle Scholar
  23. 23.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 257–286 (1989)Google Scholar
  24. 24.
    Yamato, J., Ohya, J., Ishii, K.: Recognizing human action in time-sequential images using hidden markov model. In: Computer Vision and Pattern Recognition, pp. 379–385 (1992)Google Scholar
  25. 25.
    Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of markov chains. Ann. Math. Statist., 164–171 (1970)Google Scholar
  26. 26.
    Kittler, J.: Combining classifiers: A theoretical framework. Pattern Analysis and Applications 1, 18–27 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Khai Tran
    • 1
  • Ioannis A. Kakadiaris
    • 1
  • Shishir K. Shah
    • 1
  1. 1.Department of Computer ScienceUniversity of HoustonHoustonUSA

Personalised recommendations