Skip to main content

Multi-view human activity recognition based on silhouette and uniform rotation invariant local binary patterns

Multimedia Systems volume 23, pages 451–467 (2017)Cite this article

Abstract

This paper addresses the problem of silhouette-based human activity recognition. Most of the previous work on silhouette based human activity recognition focus on recognition from a single view and ignores the issue of view invariance. In this paper, a system framework has been presented to recognize a view invariant human activity recognition approach that uses both contour-based pose features from silhouettes and uniform rotation local binary patterns for view invariant activity representation. The framework is composed of three consecutive modules: (1) detecting and locating people by background subtraction, (2) combined scale invariant contour-based pose features from silhouettes and uniform rotation invariant local binary patterns (LBP) are extracted, and (3) finally classifying activities of people by Multiclass Support vector machine (SVM) classifier. The rotation invariant nature of uniform LBP provides view invariant recognition of multi-view human activities. We have tested our approach successfully in the indoor and outdoor environment results on four multi-view datasets namely: our own view point dataset, VideoWeb Multi-view dataset [28], i3DPost multi-view dataset [29], and WVU multi-view human action recognition dataset [30]. The experimental results show that the proposed method of multi-view human activity recognition is robust, flexible and efficient.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2), 90–126 (2006)

    Article  Google Scholar 

  2. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 34(3), 334–352 (2004)

    Article  Google Scholar 

  3. Gavrila, D.: The visual analysis of human movement: a survey. Comput. Vis. Image Underst. 73(1), 82–98 (1999)

    Article  MATH  Google Scholar 

  4. Moeslund, T.B., Granum, E.: A survey of computer vision-based human motion capture. Comput. Vis. Image Underst. 81(3), 231–268 (2001)

    Article  MATH  Google Scholar 

  5. Haritaoglu, I., Harwood, D., Davis, L.S.: W4: real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell 22, 809–830 (2000)

    Article  Google Scholar 

  6. Olson, T., Brill, F.: Moving object detection and event recognition algorithms for smart cameras. In: Proc. DARPA Image Understanding Workshop, pp. 159–175 (1997)

  7. Lipton, A.J., Fujiyoshi, H., Patil, R.S.: Moving target classification and tracking from real-time video. In: Proc. IEEE Workshop Applications of Computer Vision, pp. 8–14 (1998)

  8. Srinivasan, K., Porkumaran, K., Sainarayanan, G.: Intelligent human body tracking, modeling, and activity analysis of video surveillance system: a survey. Int. J. Converg. Eng. Technol. Sci. 1, 1–8 (2009)

    Google Scholar 

  9. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern 34(3), 334–352 (2006)

    Article  Google Scholar 

  10. Weinland, D., Ronfard, R.: A survey of vision based methods for action representation, segmentation, and recognition. Comput. Vis. Image Underst. 115(2), 529–551 (2011)

    Article  Google Scholar 

  11. Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE Trans Pattern Anal. Mach. Intell 23(3), 257–267 (2001)

    Article  Google Scholar 

  12. Lv, F., Nevatia, R.: Single view human action recognition using key pose matching and viterbi path searching. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  13. Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes. Comput. Vis. Image Underst. 104(2–3), 249–257 (2006)

    Article  Google Scholar 

  14. Weinland, R., Ronfard, R., Boyer, E.: Automatic discovery of action taxonomies from multiple views. IEEE Conf. Comput. Vis. Pattern Recognit 2, 1639–1645 (2006)

    Google Scholar 

  15. Ahmad, M., Lee, S.: Human action recognition using shape and CLG-motion flow from multi-view image sequences. Pattern Recogn. 41(7), 2237–2252 (2008)

    Article  MATH  Google Scholar 

  16. Iosifidis, A., Tefas, A., Nikolaidis, N., Pitas, I.: Multi-view human movement recognition based on fuzzy distances and linear discriminant analysis. Comput. Vis. Image Underst. 116, 347–360 (2011)

    Article  Google Scholar 

  17. Iosifidis, A., Tefas, A., Pitas, I.: View-invariant action recognition based on artificial neural networks. IEEE Trans. Neural Netw. Learn. Syst 23(3), 412–424 (2012)

    Article  Google Scholar 

  18. Iosifidis, A., Tefas, A., Pitas, I.: Multi-view action recognition based on action volumes, fuzzy distances and cluster discriminant analysis. Sig. Process. 93, 1445–1457 (2013)

    Article  Google Scholar 

  19. Chaaraoui, A.A., Climent-P´erez, P., Fl´orez-Revuelta, F.: Silhouette-based human action recognition using sequences of key poses. Pattern Recogn. Lett. 34(15), 1799–1807 (2013)

    Article  Google Scholar 

  20. Iosifidis, A., Tefas, A., Pitas, I.: Learning sparse representations for view-independent human action recognition based on fuzzy distances. Neurocomputing 121, 334–353 (2013)

    Article  Google Scholar 

  21. Sharma, C.M., Kushwaha, A.K.S., Nigam, S., Khare, A.: Automatic human activity recognition in video using background modeling spatio-temporal template matching based technique. In: Proceedings of ACM International Conference on Advances in Computing and Artificial Intelligence, pp. 97–101 (2011)

  22. Le, Q.V., Zou, W.Y., Yeung, S.Y., Ng, A.Y.: Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3361–3368 (2011)

  23. Liu, W., Zhang, Y., Tang, S., Tang, J., Hong, R., Li, J.: Accurate estimation of human body orientation from RGB-D sensors. IEEE Trans. Cybern. 43(5), 1442–1452 (2013)

    Article  Google Scholar 

  24. Snidaro, L., Foresti, G.L.: Real-time thresholding with Euler numbers. Pattern Recognit. Lett. 24(9/10), 1533–1544 (2003)

    Article  MATH  Google Scholar 

  25. Dedeoglu, Y., Toereyin, B.U., Gueduekbay, U, Cetin, A.E.: Silhouette-based method for object classification and human action recognition. In: Proceedings of the ECCV workshop on HCI (2006)

  26. Pietikäinen, M.: Computer Vision Using Local Binary Patterns, vol. 40. Springer, Berlin (2011)

    Google Scholar 

  27. Westons, J., Wtkins, C.: Support vector machines for multiclass pattern recognition. In: Proceedings of the 7th European Symposium on Artificial Neural Networks, pp. 219–224 (1999)

  28. Denina, G., Bhanu, B., Nguyen, H., Ding, C., Kamal A., Ravishankar, C., Roy-Chowdhury, A., Ivers, A., Varda, B.: VideoWeb Dataset for multi-camera activities and non-verbal communication. In: Distributed Video Sensor Networks, Springer (2010)

  29. University of Surrey and CERTH-ITI, i3dpost multi-view human action datasets, January 2012. http://kahlan.eps.surrey.ac.uk/i3dpostaction/

  30. Kulathumani, V.: WVU Multi-view action recognition dataset available on: http://csee.wvu.edu/~vkkulathumani/wvu-action.html#download2

  31. Yuan, J., Liu Z, Wu, Y.: Discriminative subvolume search for efficient action detection. In: IEEE Conf. on Computer Vision and Pattern Recognition (2009). http://www.ece.northwestern.edu/~jyu410/index_files/actiondetection.html

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology (BHU), Varanasi, 221005, UP, India

    Alok Kumar Singh Kushwaha, Subodh Srivastava & Rajeev Srivastava

Authors
  1. Alok Kumar Singh Kushwaha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Subodh Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajeev Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alok Kumar Singh Kushwaha.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by Y. Zhang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kushwaha, A.K.S., Srivastava, S. & Srivastava, R. Multi-view human activity recognition based on silhouette and uniform rotation invariant local binary patterns. Multimedia Systems 23, 451–467 (2017). https://doi.org/10.1007/s00530-016-0505-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-016-0505-x

Keywords

  • Human activity recognition
  • Features extraction
  • Local binary patterns
  • Multiclass support vector machine (SVM)