Skip to main content
SpringerLink
Log in

Human action recognition based on 3D body mask and depth spatial-temporal maps

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, a method based on depth spatial-temporal maps(DSTMs) is presented for human action recognition from depth video sequences, which provides compact global spatial and temporal information of human motion for action recognition. In our approach, the initial frame of depth sequences is dilated to generate 3D body mask. The new depth sequences of major part of the human body are then computed after using 3D body mask on each depth frame. We project each frame of the new depth sequences onto three orthogonal axes to get three binary lists. Under each projection axis, binary lists are stitching in order through an entire depth sequence forming a DSTM. We evaluate our method on two standard databases. Experimental results show that this method could effectively capture the spatial and temporal information of human motion and improve the accuracy of human action recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Bobick AF, Davis JW (2001) The recognition of human movement using temporal templates. Pattern Anal Mach Intell IEEE Trans 23(3):257–267

    Article  Google Scholar 

  2. Chen C, Kehtarnavaz N, Jafari R (2014) A medication adherence monitoring system for pill bottles based on a wearable inertial sensor. In: Engineering in medicine and biology society. IEEE, pp 4983–4986

  3. Chen C, Liu K, Jafari R et al (2014) Home-based senior fitness test measurement system using collaborative inertial and depth sensors. In: Engineering in medicine and biology society. IEEE, pp 4135–4138

  4. Chen C, Kehtarnavaz N, Jafari R (2014) A medication adherence monitoring system for pill bottles based on a wearable inertial sensor. Conf Proc IEEE Eng Med Biol Soc 2014:4983–4986

    Google Scholar 

  5. Chen C, Jafari R, Kehtarnavaz N (2015) Action recognition from depth sequences using depth motion maps-based local binary patterns. IEEE Xplore, pp 1092–1099

  6. Davis JW (2001) Hierarchical motion history images for recognizing human motion. In: IEEE workshop on detection and recognition of events in video, 2001. Proceedings. IEEE, pp 39–46

  7. Fan X, Tjahjadi T (2017) A dynamic framework based on local Zernike moment and motion history image for facial expression recognition. Pattern Recog, pp 399–406

  8. Laptev I, Marszalek M, Schmid C et al (2008) Learning realistic human actions from movies. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–8

  9. Li W, Zhang Z, Liu Z (2010) Action recognition based on a bag of 3D points. In: Computer vision and pattern recognition workshops. IEEE, pp 9–14

  10. Oreifej O, Liu Z (2013) HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences. In: Computer vision and pattern recognition. IEEE, pp 716–723

  11. Sun J, Wu X, Yan S et al (2009) Hierarchical spatial-temporal context modeling for action recognition. Cvpr, pp 2004–2011

  12. Tian YL, Cao L, Liu Z et al (2012) Hierarchical filtered motion for action recognition in crowded videos. IEEE Trans Syst Man Cybern Part C 42 (3):313–323

    Article  Google Scholar 

  13. Vemulapalli R, Arrate F, Chellappa R (2014) Human action recognition by representing 3D skeletons as points in a lie group. In: IEEE conference on computer vision and pattern recognition. IEEE computer society, pp 588–595

  14. Xia L, Aggarwal JK (2013) Spatial-temporal depth cuboid similarity feature for activity recognition using depth camera. In: Computer vision and pattern recognition. IEEE, pp 2834–2841

  15. Yang AY, Jafari R, Sastry SS et al (2009) Distributed recognition of human actions using wearable motion sensor networks. J Ambient Intell Smart Environ 1(2):103–115

    Article  Google Scholar 

  16. Yang X, Zhang C, Tian YL (2012) Recognizing actions using depth motion maps-based histograms of oriented gradients. In: ACM international conference on multimedia. ACM, pp 1057–1060

  17. Yang X, Zhang C, Tian YL (2012) Recognizing actions using depth motion maps-based histograms of oriented gradients. In: ACM international conference on multimedia. ACM, pp 1057–1060

  18. Zhang L, Zhang L, Tao D et al (2015) A sparse and discriminative tensor to vector projection for human gait feature representation. Sig Process 106 (C):245–252

    Article  Google Scholar 

  19. Zhao N, Zhang L, Du B et al (2016) Sparse tensor discriminative locality alignment for gait recognition. In: International joint conference on neural networks. IEEE, pp 4489–4495

  20. Zhang B, Yang Y, Chen C et al (2017) Action recognition using 3D histograms of texture and A multi-class boosting classifier. IEEE Trans Image Process Publ IEEE Sig Process Soc 26(10):4648–4660

    Article  MathSciNet  Google Scholar 

  21. Yang X, Tian YL (2014) Super normal vector for activity recognition using depth sequences. In: Computer vision and pattern recognition. IEEE, pp 804–811

Download references

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Changzhou University, Changzhou, China

    Xing Li, Zhenjie Hou & Jiuzhen Liang

  2. Jiangsu Province Networking and Mobile Internet Technology Engineering Key Laboratory, Huaian, China

    Zhenjie Hou

  3. Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, USA

    Chen Chen

Authors
  1. Xing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenjie Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiuzhen Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenjie Hou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Hou, Z., Liang, J. et al. Human action recognition based on 3D body mask and depth spatial-temporal maps. Multimed Tools Appl 79, 35761–35778 (2020). https://doi.org/10.1007/s11042-020-09593-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09593-z

Keywords

Search

Navigation