An Efficient Approach for Multi-view Human Action Recognition Based on Bag-of-Key-Poses

  • Alexandros Andre Chaaraoui
  • Pau Climent-Pérez
  • Francisco Flórez-Revuelta
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7559)


This paper presents a novel multi-view human action recognition approach based on a bag-of-key-poses. In the case of multi-view scenarios, it is especially difficult to perform accurate action recognition that still runs at an admissible recognition speed. The presented method aims to fill this gap by combining a silhouette-based pose representation with a simple, yet effective multi-view learning approach based on Model Fusion. Action classification is performed through efficient sequence matching and by the comparison of successive key poses which are evaluated on both feature similarity and match relevance. Experimentation on the MuHAVi dataset shows that the method outperforms currently available recognition rates and is exceptionally robust to actor-variance. Temporal evaluation confirms the method’s suitability for real-time recognition.


human action recognition multi-view action recognition key pose bag-of-key-poses MuHAVi dataset 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Poppe, R.: A survey on vision-based human action recognition. Image and Vision Computing 28(6), 976–990 (2010)CrossRefGoogle Scholar
  2. 2.
    Weinland, D., Ronfard, R., Boyer, E.: A survey of vision-based methods for action representation, segmentation and recognition. Comput. Vis. Image Underst. 115(2), 224–241 (2011)CrossRefGoogle Scholar
  3. 3.
    Holte, M.B., Tran, C., Trivedi, M.M., Moeslund, T.B.: Human action recognition using multiple views: a comparative perspective on recent developments. In: Proceedings of the 2011 Joint ACM Workshop on Human Gesture and Behavior Understanding, J-HGBU 2011, pp. 47–52. ACM, New York (2011)CrossRefGoogle Scholar
  4. 4.
    Bobick, A., Davis, J.: The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(3), 257–267 (2001)CrossRefGoogle Scholar
  5. 5.
    Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes. Comput. Vis. Image Underst. 104(2), 249–257 (2006)CrossRefGoogle Scholar
  6. 6.
    Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 2, pp. 1395–1402 (2005)Google Scholar
  7. 7.
    Laptev, I.: On space-time interest points. International Journal of Computer Vision 64, 107–123 (2005)CrossRefGoogle Scholar
  8. 8.
    Oikonomopoulos, A., Patras, I., Pantic, M.: Spatiotemporal salient points for visual recognition of human actions. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 36(3), 710–719 (2005)CrossRefGoogle Scholar
  9. 9.
    Scovanner, P., Ali, S., Shah, M.: A 3-dimensional SIFT descriptor and its application to action recognition. In: Proceedings of the 15th International Conference on Multimedia, MULTIMEDIA 2007, pp. 357–360. ACM, New York (2007)CrossRefGoogle Scholar
  10. 10.
    İkizler, N., Duygulu, P.: Human Action Recognition Using Distribution of Oriented Rectangular Patches. In: Elgammal, A., Rosenhahn, B., Klette, R. (eds.) Human Motion 2007. LNCS, vol. 4814, pp. 271–284. Springer, Heidelberg (2007), CrossRefGoogle Scholar
  11. 11.
    Aggarwal, J., Ryoo, M.: Human activity analysis: A review. ACM Comput. Surv. 43(3), 16:1–16:43 (2011)CrossRefGoogle Scholar
  12. 12.
    Määttä, T., Härmä, A., Aghajan, H.: On efficient use of multi-view data for activity recognition. In: Proceedings of the Fourth ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC 2010, pp. 158–165. ACM, New York (2010)CrossRefGoogle Scholar
  13. 13.
    Wu, C., Khalili, A.H., Aghajan, H.: Multiview activity recognition in smart homes with spatio-temporal features. In: Proceedings of the Fourth ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC 2010, pp. 142–149. ACM, New York (2010)CrossRefGoogle Scholar
  14. 14.
    Naiel, M.A., Abdelwahab, M.M., El-Saban, M.: Multi-view human action recognition system employing 2DPCA. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 270–275 (2011)Google Scholar
  15. 15.
    Cilla, R., Patricio, M.A., Berlanga, A., Molina, J.M.: A probabilistic, discriminative and distributed system for the recognition of human actions from multiple views. Neurocomputing 75(1), 78–87 (2012); Brazilian Symposium on Neural Networks (SBRN 2010), International Conference on Hybrid Artificial Intelligence Systems (HAIS 2010)CrossRefGoogle Scholar
  16. 16.
    Yan, S.M.P., Khan, Shah, M.: Learning 4D action feature models for arbitrary view action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–7 (2008)Google Scholar
  17. 17.
    Canton-Ferrer, C., Casas, J.R., Pardas, M.: Human model and motion based 3D action recognition in multiple view scenarios. In: Conf. on 14th European Signal Processing, Italy, pp. 1–5 (2006)Google Scholar
  18. 18.
    Suzuki, S., Abe, K.: Topological structural analysis of digitized binary images by border following. Computer Vision, Graphics, and Image Processing 30(1), 32–46 (1985)zbMATHCrossRefGoogle Scholar
  19. 19.
    Ángeles Mendoza, M., Pérez de la Blanca, N.: HMM-Based Action Recognition Using Contour Histograms. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007. LNCS, vol. 4477, pp. 394–401. Springer, Heidelberg (2007), CrossRefGoogle Scholar
  20. 20.
    Dedeoğlu, Y., Töreyin, B., Güdükbay, U., Çetin, A.: Silhouette-Based Method for Object Classification and Human Action Recognition in Video. In: Huang, T.S., Sebe, N., Lew, M., Pavlović, V., Kölsch, M., Galata, A., Kisačanin, B. (eds.) HCI/ECCV 2006. LNCS, vol. 3979, pp. 64–77. Springer, Heidelberg (2006), CrossRefGoogle Scholar
  21. 21.
    Singh, S., Velastin, S.A., Ragheb, H.: Muhavi: A multicamera human action video dataset for the evaluation of action recognition methods. In: 2010 Seventh IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 48–55 (2010)Google Scholar
  22. 22.
    Cheema, S., Eweiwi, A., Thurau, C., Bauckhage, C.: Action recognition by learning discriminative key poses. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1302–1309 (2011)Google Scholar
  23. 23.
    Martínez-Contreras, F., Orrite-Urunuela, C., Herrero-Jaraba, E., Ragheb, H., Velastin, S.A.: Recognizing human actions using silhouette-based HMM. In: Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, pp. 43–48 (2009)Google Scholar
  24. 24.
    Eweiwi, A., Cheema, S., Thurau, C., Bauckhage, C.: Temporal key poses for human action recognition. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1310–1317 (2011)Google Scholar
  25. 25.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Alexandros Andre Chaaraoui
    • 1
  • Pau Climent-Pérez
    • 1
  • Francisco Flórez-Revuelta
    • 2
  1. 1.Department of Computing TechnologyUniversity of AlicanteAlicanteSpain
  2. 2.Faculty of Science, Engineering and ComputingKingston UniversityKingston upon ThamesUnited Kingdom

Personalised recommendations