Real-Time Human Action Recognition Using DMMs-Based LBP and EOH Features

  • Mohammad Farhad Bulbul
  • Yunsheng Jiang
  • Jinwen MaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9225)


This paper proposes a new feature extraction scheme for the real-time human action recognition from depth video sequences. First, three Depth Motion Maps (DMMs) are formed from the depth video. Then, on top of these DMMs, the Local Binary Patterns (LBPs) are calculated within overlapping blocks to capture the local texture information, and the Edge Oriented Histograms (EOHs) are computed within non-overlapping blocks to extract dense shape features. Finally, to increase the discriminatory power, the DMMs-based LBP and EOH features are fused in a systematic way to get the so-called DLE features. The proposed DLE features are then fed into the l 2 -regularized Collaborative Representation Classifier (l 2 -CRC) to learn the model of human action. Experimental results on the publicly available Microsoft Research Action3D dataset demonstrate that the proposed approach achieves the state-of-the-art recognition performance without compromising the processing speed for all the key steps, and thus shows the suitability for real-time implementation.


Human action recognition Depth Motion Maps Local Binary patterns Edge Oriented Histograms 



This work was supported by the Natural Science Foundation of China for Grant 61171138


  1. 1.
    Chen, C., Kehtarnavaz, N., Jafari, R.: A medication adherence monitoring system for pill bottles based on a wearable inertial sensor. In: Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. pp. 4983–4986 (2014)Google Scholar
  2. 2.
    Chen, C., Jafari, R., Kehtarnavaz, N.: Improving human action recognition using fusion of depth camera and inertial sensors. IEEE Trans. Hum.-Mach. Syst. 45(1), 51–61 (2015)CrossRefGoogle Scholar
  3. 3.
    Laptev, I.: On space-time interest points. Int. J. Comput. Vision 64(2/3), 107–123 (2005)CrossRefGoogle Scholar
  4. 4.
    Niebles, J., Wang, H., Fei-Fei, L.: Unsupervised learning of human action categories using spatial-temporal words. Int. J. Comput. Vision 79(3), 299–318 (2008)CrossRefGoogle Scholar
  5. 5.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kip- man, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1297–1304 (2011)Google Scholar
  6. 6.
    Microsoft: kinect for windows.
  7. 7.
    Chen, C., Liu, K., Kehtarnavaz, N.: Real-time human action recognition based on depth motion maps. J. Real-Time Image Process. (2013)Google Scholar
  8. 8.
    Li, W., Zhang, Z., Liu, Z.: Action recognition based on a bag of 3d Points. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 9–14 (2010)Google Scholar
  9. 9.
    Vieira, A., Nascimento, E., Oliveira, G., Liu, Z., Campos, M.: Space-time occupancy patterns for 3d action recognition from depth map sequences. In: Proceedings of Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, pp. 252–259 (2012)Google Scholar
  10. 10.
    Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y.: Robust 3d action recognition with random occupancy patterns. In: Proceedings of European Conference on Computer Vision, pp. 872–885 (2012)Google Scholar
  11. 11.
    Yang, X., Zhang, C., Tian, Y.: Recognizing actions using depth motion maps-based histograms of oriented gradients. In: Proceedings of ACM International Conference on Multimedia, pp. 1057–1060 (2012)Google Scholar
  12. 12.
    Oreifej, O., Liu, Z.: Hon4d: histogram of oriented 4d normals for activity recognition from depth sequences. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 716–723 (2013)Google Scholar
  13. 13.
    Luo, J., Wang, W., Qi, H.: Spatio-temporal feature extraction and representation for RGB-D human action recognition. Pattern Recogn. Lett. 50, 139–148 (2014)CrossRefGoogle Scholar
  14. 14.
    Lu, C., Jia, J., Tang, C.K.: Range-sample depth feature for action recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 772–779 (2014)Google Scholar
  15. 15.
    Chen, C., Jafari, R., Kehtarnavaz, N.: Action recognition from depth sequences using depth motion maps-based local binary patterns. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, pp. 1092–1099 (2015)Google Scholar
  16. 16.
    Farhad, M., Jiang, Y., Ma, J.: Human action recognition based on DMMs, HOGs and Contourlet transform. In: Proceedings of IEEE International Conference on Multimedia Big Data, pp. 389–394 (2015)Google Scholar
  17. 17.
    Yang, X., Tian, Y.: Eigen joints-based action recognition using Naive-Bayes-Nearest-Neighbor. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 14–19 (2012)Google Scholar
  18. 18.
    Xia, L., Chen, C., Aggarwal, J.: View invariant human action recognition using histograms of 3d joints. In: Proceedings of Workshop on Human Activity Under- standing from 3D Data, pp. 20–27 (2012)Google Scholar
  19. 19.
    Wang, J., Liu, Z., Wu, Y., Yuan, J.: Mining actionlet ensemble for action recognition with depth cameras. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1290–1297 (2012)Google Scholar
  20. 20.
    Luo, J., Wang, W., Qi, H.: Group sparsity and geometry constrained dictionary learning for action recognition from depth maps. In: Proceedings of the 14th IEEE International Conference on Computer Vision, pp. 1809–1816 (2013)Google Scholar
  21. 21.
    Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3d human skeletons as points in a lie group. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 588–595 (2014)Google Scholar
  22. 22.
    Chaaraoui, A.A., Padilla-Lopez, J.R., Climent-Perez, P., Florez-Revuelta, F.: Evolutionary joint selection to improve human action recognition with RGB-D devices. Expert Syst. Appl. 41(3), 786–794 (2014)CrossRefGoogle Scholar
  23. 23.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture Classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefGoogle Scholar
  24. 24.
  25. 25.
    Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., Yan, S.: Sparse representation for computer vision and pattern recognition. Proc. IEEE 98(6), 1031–1044 (2010)CrossRefzbMATHGoogle Scholar
  26. 26.
    Tikhonov, A., Arsenin, V.: Solutions of ill-posed problems. Math. Comput. 32(144), 1320–1322 (1978)CrossRefGoogle Scholar
  27. 27.
    Chen, C., Tramel, E.W., Fowler, J.E.: Compressed-sensing recovery of images and video using multi-hypothesis predictions. In: Proceedings of the 45th Asilomar Conference on signals, Systems, and Computers, pp. 1193–1198 (2011)Google Scholar
  28. 28.
    Chen, C., Li, W., Tramel, E.W., Fowler, J.E.: Reconstruction of hyperspectral imagery from random projections using multi-hypothesis prediction. IEEE Trans. Geosci. Remote Sens. 52(1), 365–374 (2014)CrossRefGoogle Scholar
  29. 29.
    Chen, C., Fowler, J.E.: Single-image Super-resolution Using Multi-hypothesis Prediction. In: Proceedings of the 46th Asilomar Conference on Signals, Systems, and Computers, pp. 608–612 (2012)Google Scholar
  30. 30.
    Golub, G., Hansen, P.C., O’Leary, D.: Tikhonov-regularization and total least squares. SIAM J. Matrix Anal. Appl. 21(1), 185–194 (1999)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Mohammad Farhad Bulbul
    • 1
  • Yunsheng Jiang
    • 1
  • Jinwen Ma
    • 1
    Email author
  1. 1.Department of Information Science, School of Mathematical Sciences and LMAMPeking UniversityBeijingChina

Personalised recommendations