Skip to main content
Book cover

Contactless Human Activity Analysis pp 43–81Cite as

Skeleton-Based Activity Recognition: Preprocessing and Approaches

Part of the Intelligent Systems Reference Library book series (ISRL,volume 200)

Abstract

Research in Activity Recognition is one of the thriving areas in the field of computer vision. This development comes into existence by introducing the skeleton-based architectures for action recognition and related research areas. By advancing the research into real-time scenarios, practitioners find it fascinating and challenging to work on human action recognition because of the following core aspects—numerous types of distinct actions, variations in the multimodal datasets, feature extraction, and view adaptiveness. Moreover, hand-crafted features and depth sequence models cannot perform efficiently on the multimodal representations. Consequently, recognizing many action classes by extracting some smart and discriminative features is a daunting task. As a result, deep learning models are adapted to work in the field of skeleton-based action recognition. This chapter entails all the fundamental aspects of skeleton-based action recognition, such as—skeleton tracking, representation, preprocessing techniques, feature extraction, and recognition methods. This chapter can be a beginning point for a researcher who wishes to work in action analysis or recognition based on skeleton joint-points.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-68590-4_2
  • Chapter length: 39 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-68590-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.99
Price excludes VAT (USA)
Hardcover Book
USD   219.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 2.1
Fig. 2.2
Fig. 2.3
Fig. 2.4
Fig. 2.5
Fig. 2.6
Fig. 2.7
Fig. 2.8
Fig. 2.9

References

  1. Baek, S., Kwang, I.K., Kim, T.-K.: Augmented skeleton space transfer for depth-based hand pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8330–8339 (2018)

    Google Scholar 

  2. Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Weipeng, X., Casas, D., Theobalt, C.: Vnect: Real-time 3d human pose estimation with a single rgb camera. ACM Trans. Graph. (TOG) 36(4), 1–14 (2017)

    CrossRef  Google Scholar 

  3. Ling, J., Tian, L., Li, C.: 3d human activity recognition using skeletal data from rgbd sensors. In: International Symposium on Visual Computing, pp. 133–142. Springer (2016)

    Google Scholar 

  4. Balakrishnan, S., Rice, J.M., Walker, S.H., Carroll, A.S., Dow-Hygelund, C.C., Goodwin, A.K., Mullin, J.M., Rattenbury, T.L., Rooke-Ley, J.M., Schmitt, J.M., et al.: Action detection and activity classification, May 31 2016. US Patent 9,352,207

    Google Scholar 

  5. Wang, J., Liu, Z., Ying, W., Yuan, J.: Learning actionlet ensemble for 3d human action recognition. IEEE Trans. Pattern Analy. Mach. Intell. 36(5), 914–927 (2013)

    CrossRef  Google Scholar 

  6. Wang, L., Gu, T., Tao, X., Lu, J.: Sensor-based human activity recognition in a multi-user scenario. In: European Conference on Ambient Intelligence, pp. 78–87. Springer (2009)

    Google Scholar 

  7. Batabyal, T., Chattopadhyay, T., Mukherjee, D.P.: Action recognition using joint coordinates of 3d skeleton data. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4107–4111. IEEE (2015)

    Google Scholar 

  8. Kong, Y., Fu, Y.: Bilinear heterogeneous information machine for rgb-d action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1054–1062 (2015)

    Google Scholar 

  9. Seidenari, L., Varano, C., Berretti, S., Bimbo, A., Pala, P.: Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 479–485 (2013)

    Google Scholar 

  10. Pham, H.-H., Khoudour, L., Crouzil, A., Zegers, P., Velastin, S.A.: Exploiting deep residual networks for human action recognition from skeletal data. Comput. Vis. Image Underst. 170, 51–66 (2018)

    CrossRef  Google Scholar 

  11. Presti, L.L., Cascia, M.L.: 3d skeleton-based human action classification: a survey. Pattern Recogn. 53, 130–147 (2016)

    CrossRef  Google Scholar 

  12. Chen, Y., Tian, Y., He, M.: Monocular human pose estimation: a survey of deep learning-based methods. Comput. Vis. Image Underst. 192, 102897, 03 (2020)

    Google Scholar 

  13. Zhang, A., Ma, X., Song, R., Rong, X., Tian, X., Tian, G., Li, Y.: Deep learning based human action recognition: a survey. In: 2017 Chinese Automation Congress (CAC), pp. 3780–3785. IEEE (2017)

    Google Scholar 

  14. Asadi-Aghbolaghi, M., Clapés, A., Bellantonio, M., Escalante, H.J., Ponce-López, V., Baró, X., Guyon, I., Kasaei, S., Escalera, S.: Deep learning for action and gesture recognition in image sequences: a survey. In: Gesture Recognition, pp. 539–578. Springer (2017)

    Google Scholar 

  15. Wang, L., Huynh, D.Q., Koniusz, D.Q.: A comparative review of recent kinect-based action recognition algorithms. IEEE Trans. Image Process. 29, 15–28 (2019)

    MathSciNet  CrossRef  Google Scholar 

  16. Jegham, I., Khalifa, A.B., Alouani, I., Mahjoub, M.A.: Vision-based human action recognition: an overview and real world challenges. Forensic Sci. Int.: Digital Investig. 32, 200901 (2020)

    Google Scholar 

  17. Cao, Z., Simon, T., Wei, S.-E., Sheikh, S.-E.: Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)

    Google Scholar 

  18. Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: CVPR (2017)

    Google Scholar 

  19. Wei, S.-E., Ramakrishna, S.-E., Kanade, T., Sheikh. Y.: Convolutional pose machines. In: CVPR (2016)

    Google Scholar 

  20. Rahmani, H., Mian, A.: 3d action recognition from novel viewpoints. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1506–1515 (2016)

    Google Scholar 

  21. Vieira, A.W., Nascimento, E.R., Oliveira, G.L., Liu, Z., Campos, M.F.M.: Stop: space-time occupancy patterns for 3d action recognition from depth map sequences. In: Iberoamerican Congress on Pattern Recognition, pp. 252–259. Springer (2012)

    Google Scholar 

  22. Cavazza, J., Zunino, A., San Biagio, M., Murino, V.: Kernelized covariance for action recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 408–413. IEEE (2016)

    Google Scholar 

  23. Materzynska, J., Xiao, J., Herzig, R., Xu, H., Wang, X., Darrell, T.: Something-else: compositional action recognition with spatial-temporal interaction networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1049–1059 (2020)

    Google Scholar 

  24. Yang, J., Liu, Wu, Yuan, J.: Mei, T: Hierarchical soft quantization for skeleton-based human action recognition. IEEE Trans, Multimedia (2020)

    Google Scholar 

  25. Huang, J., Xiang, X., Gong, X., Zhang, B., et al.: Long-short graph memory network for skeleton-based action recognition. In: The IEEE Winter Conference on Applications of Computer Vision, pp. 645–652 (2020)

    Google Scholar 

  26. Si, C., Chen, W., Wang, W., Wang, L., Tan, T.: An attention enhanced graph convolutional lstm network for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1227–1236 (2019)

    Google Scholar 

  27. Yan, S., Li, Z., Xiong, Y., Yan, H., Lin, D.: Convolutional sequence generation for skeleton-based action synthesis. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4394–4402 (2019)

    Google Scholar 

  28. Zhao, R., Wang, K., Su, K., Ji, Q.: Bayesian graph convolution lstm for skeleton based action recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6882–6892 (2019)

    Google Scholar 

  29. Shi, L., Zhang, Y., Cheng, J., Lu, H.: Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12026–12035 (2019)

    Google Scholar 

  30. Yang, H., Yan, D., Zhang, L., Li, D., Sun, Y.D., You, S.D., Maybank, S.J.: Feedback graph convolutional network for skeleton-based action recognition. arXiv preprint arXiv:2003.07564 (2020)

  31. Zhu, G., Zhang, L., Li, H., Shen, P., Afaq Ali Shah, S., Bennamoun, M.: Topology-learnable graph convolution for skeleton-based action recognition. Pattern Recogn. Lett. (2020)

    Google Scholar 

  32. Chen, Y., Ma, G., Yuan, C., Li, B., Zhang, H., Wang, F., Hu, W.: Graph convolutional network with structure pooling and joint-wise channel attention for action recognition. Pattern Recogn., p. 107321 (2020)

    Google Scholar 

  33. Huang, L., Huang, Y., Ouyang, W., Wang, L. et al.: Part-level graph convolutional network for skeleton-based action recognition (2020)

    Google Scholar 

  34. Tang, Y., Tian, Y., Lu, J., Li, P., Zhou, J.: Deep progressive reinforcement learning for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5323–5332 (2018)

    Google Scholar 

  35. Caetano, C., Brémond, F., Schwartz, W.R.: Skeleton image representation for 3d action recognition based on tree structure and reference joints. In: 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 16–23. IEEE (2019)

    Google Scholar 

  36. Ke, Q., Bennamoun, M., An, A., Sohel, F., Boussaid, F.: A new representation of skeleton sequences for 3d action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3288–3297 (2017)

    Google Scholar 

  37. Liliana [Lo Presti], Marco [La Cascia]: 3d skeleton-based human action classification: a survey. Pattern Recogn. 53, 130–147 (2016)

    Google Scholar 

  38. Chaudhry, R., Ofli, F., Kurillo, G., Bajcsy, R., Vidal, R.: Bio-inspired dynamic 3d discriminative skeletal features for human action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 471–478 (2013)

    Google Scholar 

  39. Slama, R., Wannous, H., Daoudi, M., Srivastava, A.: Accurate 3d action recognition using learning on the grassmann manifold. Pattern Recogn. 48(2), 556–567 (2015)

    CrossRef  Google Scholar 

  40. Li, X., Zhang, Y., Zhang, J.: Improved key poses model for skeleton-based action recognition. In: Pacific Rim Conference on Multimedia, pp. 358–367. Springer (2017)

    Google Scholar 

  41. Cai, L., Liu, C., Yuan, R., Ding, H.: Human action recognition using lie group features and convolutional neural networks. Nonlinear Dyn., pp. 1–11 (2020)

    Google Scholar 

  42. Ghorbel, E., Demisse, G., Aouada, D., Ottersten, B.: Fast adaptive reparametrization (far) with application to human action recognition. IEEE Signal Process. Lett. 27, 580–584 (2020)

    CrossRef  Google Scholar 

  43. Huang, Z., Wan, C., Probst, T., Van Gool, L.: Deep learning on lie groups for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6099–6108 (2017)

    Google Scholar 

  44. de Boissiere, A.M., Noumeir, R.: Infrared and 3d skeleton feature fusion for rgb-d action recognition. arXiv preprint arXiv:2002.12886 (2020)

  45. Lee, I., Kim, D., Kang, S., Lee, S.: Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1012–1020 (2017)

    Google Scholar 

  46. Rahmani, H., Bennamoun, M.: Learning action recognition model from depth and skeleton videos. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5832–5841 (2017)

    Google Scholar 

  47. Zhang, P., Lan, C., Xing, J., Zeng, W., Xue, J., Zheng, N.: View adaptive recurrent neural networks for high performance human action recognition from skeleton data. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2117–2126 (2017)

    Google Scholar 

  48. Li, R., Fu, H., Lo, W., Chi, Z., Song, Z., Wen, D.: Skeleton-based action recognition with key-segment descriptor and temporal step matrix model. IEEE Access 7, 169782–169795 (2019)

    CrossRef  Google Scholar 

  49. Rahmani, H., Bennamoun, M.: Learning action recognition model from depth and skeleton videos. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 5833–5842 (2017)

    Google Scholar 

  50. Nie, Q., Wang, J., Wang, X., Liu, Y.: View-invariant human action recognition based on a 3d bio-constrained skeleton model. IEEE Trans. Image Process. 28(8), 3959–3972 (2019)

    MathSciNet  CrossRef  Google Scholar 

  51. Li, S., Jiang, T., Tian, Y., Huang, T.: 3d human skeleton data compression for action recognition. In: 2019 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2019)

    Google Scholar 

  52. Nie, W., Wang, W., Huang, X.: Srnet: Structured relevance feature learning network from skeleton data for human action recognition. IEEE Access 7, 132161–132172 (2019)

    CrossRef  Google Scholar 

  53. Shi, L., Zhang, Y., Cheng, J., Lu, H.: Skeleton-based action recognition with directed graph neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7912–7921 (2019)

    Google Scholar 

  54. Li, S., Jiang, T., Huang, T., Tian, Y.: Global co-occurrence feature learning and active coordinate system conversion for skeleton-based action recognition. In: The IEEE Winter Conference on Applications of Computer Vision, pp. 586–594 (2020)

    Google Scholar 

  55. Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1110–1118 (2015)

    Google Scholar 

  56. Su, K., Liu, X., Shlizerman, E., Predict & cluster: Unsupervised skeleton based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9631–9640 (2020)

    Google Scholar 

  57. Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., Zheng, N.: Semantics-guided neural networks for efficient skeleton-based human action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1112–1121 (2020)

    Google Scholar 

  58. Raj, B.N., Subramanian, A., Ravichandran, K., Venkateswaran, N.: Exploring techniques to improve activity recognition using human pose skeletons. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision Workshops, pp. 165–172 (2020)

    Google Scholar 

  59. Huang, J., Huang, Z., Xiang, X., Gong, X., Zhang, B.: Long-short graph memory network for skeleton-based action recognition. In: The IEEE Winter Conference on Applications of Computer Vision (WACV), March 2020

    Google Scholar 

  60. Huynh, D.Q.: Metrics for 3d rotations: Comparison and analysis. J. Math. Imaging Vis. 35(2), 155–164 (2009)

    MathSciNet  CrossRef  Google Scholar 

  61. Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L., Xie, X.: Co-occurrence feature learning for skeleton based action recognition using regularized deep lstm networks. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)

    Google Scholar 

  62. Morais, R., Le, V., Tran, T., Saha, B., Mansour, M., Venkatesh, S.: Learning regularity in skeleton trajectories for anomaly detection in videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11996–12004 (2019)

    Google Scholar 

  63. Gaglio, S., Re, G.L., Morana, M.: Human activity recognition process using 3-d posture data. IEEE Trans. Human-Mach. Syst. 45(5), 586–597 (2014)

    CrossRef  Google Scholar 

  64. Naveenkumar, M., Domnic, S.: Skeleton joint difference maps for 3d action recognition with convolutional neural networks. In: International Conference on Recent Trends in Image Processing and Pattern Recognition, pp. 144–150. Springer (2018)

    Google Scholar 

  65. Wang, P., Li, W., Gao, Z., Zhang, J., Tang, C., Ogunbona, P.O.: Action recognition from depth maps using deep convolutional neural networks. IEEE Trans. Human-Mach. Syst. 46(4), 498–509 (2015)

    CrossRef  Google Scholar 

  66. Yang, X., Zhang, C., Tian, Y.L.: Recognizing actions using depth motion maps-based histograms of oriented gradients. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 1057–1060 (2012)

    Google Scholar 

  67. Li, B., Dai, Y., Cheng, X., Chen, H., Lin, Y., He, M.: Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep cnn. In: 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 601–604. IEEE (2017)

    Google Scholar 

  68. Huynh-The, T., Hua, C.-H., Tu, N.A., Kim, J.-W., Kim, S.-H., Kim, D.-S.: 3d action recognition exploiting hierarchical deep feature fusion model. In: 2020 14th International Conference on Ubiquitous Information Management and Communication (IMCOM), pp. 1–3. IEEE (2020)

    Google Scholar 

  69. Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., Tian, Q.: Actional-structural graph convolutional networks for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3595–3603 (2019)

    Google Scholar 

  70. Liu, J., Liu, Y., Wang, Y., Prinet, V., Xiang, S., Pan, C.: Decoupled representation learning for skeleton-based gesture recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5751–5760 (2020)

    Google Scholar 

  71. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth annual Workshop on Computational Learning Theory, pp. 144–152 (1992)

    Google Scholar 

  72. Cortes, Corinna, Vapnik, Vladimir: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  73. Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3d skeletons as points in a lie group. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 588–595 (2014)

    Google Scholar 

  74. Boulahia, S.Y., Anquetil, E., Kulpa, R., Multon, F.: Hif3d: Handwriting-inspired features for 3d skeleton-based action recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 985–990. IEEE (2016)

    Google Scholar 

  75. Li, X., Zhang, Y., Liao, D.: Mining key skeleton poses with latent svm for action recognition. Appl. Comput. Intell. Soft Comput. (2017)

    Google Scholar 

  76. Xu, D., Xiao, X., Wang, X., Wang, J.: Human action recognition based on kinect and pso-svm by representing 3d skeletons as points in lie group. In: 2016 International Conference on Audio, Language and Image Processing (ICALIP), pp. 568–573. IEEE (2016)

    Google Scholar 

  77. Liu, M., He, Q., Liu, H.: Fusing shape and motion matrices for view invariant action recognition using 3d skeletons. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3670–3674. IEEE (2017)

    Google Scholar 

  78. Weng, J., Weng, C., Yuan, J.: Spatio-temporal naive-bayes nearest-neighbor (st-nbnn) for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4171–4180 (2017)

    Google Scholar 

  79. Tang, N.C., Lin, Y.-Y., Hua, J.-H., Weng, M.-F., Mark Liao, H.-Y.: Human action recognition using associated depth and skeleton information. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4608–4612. IEEE (2014)

    Google Scholar 

  80. Ubalde, S., Gómez-Fernández, F., Goussies, N.A., Mejail, M.: Skeleton-based action recognition using citation-knn on bags of time-stamped pose descriptors. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3051–3055. IEEE (2016)

    Google Scholar 

  81. Li, Y., Guo, T., Xia, R., Liu, X.: A novel skeleton spatial pyramid model for skeleton-based action recognition. In: 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP), pp. 16–20. IEEE (2019)

    Google Scholar 

  82. Liu, Z., Zhang, C., Tian, Y.: 3d-based deep convolutional neural network for action recognition with depth sequences. Image Vis. Comput. 55, 93–100 (2016)

    CrossRef  Google Scholar 

  83. Wang, H., Wang, L.: Beyond joints: Learning representations from primitive geometries for skeleton-based action recognition and detection. IEEE Trans. Image Process. 27(9), 4382–4394 (2018)

    MathSciNet  CrossRef  Google Scholar 

  84. Zhang, P., Lan, C., Xing, J., Zeng, W., Xue, J., Zheng, N.: View adaptive neural networks for high performance skeleton-based human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1963–1978 (2019)

    CrossRef  Google Scholar 

  85. Si, C., Jing, Y., Wang, W., Wang, L., Tan, T.: Skeleton-based action recognition with hierarchical spatial reasoning and temporal stack learning network. Pattern Recogn., p. 107511 (2020)

    Google Scholar 

  86. Yang, D., Li, M.M., Fu, H., Fan, J., Leung, H.: Centrality graph convolutional networks for skeleton-based action recognition. arXiv preprint arXiv:2003.03007 (2020)

  87. Ke, Q., Bennamoun, M., An, S., Sohel, F., Boussaid, F.: Learning clip representations for skeleton-based 3d action recognition. IEEE Trans. Image Process. 27(6), 2842–2855 (2018)

    MathSciNet  CrossRef  Google Scholar 

  88. Tian, D., Lu, Z.-M., Chen, X., Ma, L.-H.: An attentional spatial temporal graph convolutional network with co-occurrence feature learning for action recognition. Multimedia Tools Appl., 1–19 (2020)

    Google Scholar 

  89. Liu, A.-A., Yu-Ting, S., Jia, P.-P., Gao, Z., Hao, T., Yang, Z.-X.: Multiple/single-view human action recognition via part-induced multitask structural learning. IEEE Trans. Cybern. 45(6), 1194–1208 (2014)

    CrossRef  Google Scholar 

  90. Yang, Y., Deng, C., Tao, D., Zhang, S., Liu, W., Gao, X.: Latent max-margin multitask learning with skelets for 3-d action recognition. IEEE Trans. Cybern. 47(2), 439–448 (2016)

    Google Scholar 

  91. Nguyen, X.S., Brun, L., Lézoray, O., Bougleux, S.: A neural network based on spd manifold learning for skeleton-based hand gesture recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12036–12045 (2019)

    Google Scholar 

  92. Zhang, T., Zheng, W., Cui, Z., Zong, Y., Li, C., Zhou, X., Yang, J.: Deep manifold-to-manifold transforming network for skeleton-based action recognition. IEEE Trans, Multimedia (2020)

    CrossRef  Google Scholar 

  93. Devanne, M., Wannous, H., Berretti, S., Pala, P., Daoudi, M., Bimbo, D.: Alberto: 3-d human action recognition by shape analysis of motion trajectories on riemannian manifold. IEEE Trans. Cybern. 45(7), 1340–1352 (2014)

    CrossRef  Google Scholar 

  94. Li, J., Xie, X., Pan, Q., Cao, Y., Zhao, Z., Shi, G.: Sgm-net: Skeleton-guided multimodal network for action recognition. Pattern Recogn., 107356 (2020)

    Google Scholar 

  95. Liu, J., Wang, G., Duan, L.-Y., Abdiyeva, K., Kot, A.C.: Skeleton-based human action recognition with global context-aware attention lstm networks. IEEE Trans. Image Process. 27(4), 1586–1599 (2017)

    MathSciNet  CrossRef  Google Scholar 

  96. Zheng, W., Li, L., Zhang, Z., Huang, Y., Wang, L.: Relational network for skeleton-based action recognition. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), pp. 826–831. IEEE (2019)

    Google Scholar 

  97. Mahasseni, B., Todorovic, S.: Regularizing long short term memory with 3d human-skeleton sequences for action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3054–3062 (2016)

    Google Scholar 

  98. Han, Y., Chung, S.-L., Ambikapathi, A., Chan, J.-S., Lin, W.-Y., Su, S.-F.: Robust human action recognition using global spatial-temporal attention for human skeleton data. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2018)

    Google Scholar 

  99. Song, S., Lan, C., Xing, J., Zeng, W., Jiaying, L.: Spatio-temporal attention-based lstm networks for 3d action recognition and detection. IEEE Trans. Image Process. 27(7), 3459–3471 (2018)

    MathSciNet  CrossRef  Google Scholar 

  100. Cheng, K., Zhang, Y., He, X., Chen, W., Cheng, J., Lu, H.: Skeleton-based action recognition with shift graph convolutional network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 183–192 (2020)

    Google Scholar 

  101. Li, M., Chen, S., Zhao, Y., Zhang, Y., Wang, Y., Tian, Q.: Dynamic multiscale graph neural networks for 3d skeleton based human motion prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 214–223 (2020)

    Google Scholar 

  102. Liu, Z., Zhang, H., Chen, Z., Wang, Z., Ouyang, W.: Disentangling and unifying graph convolutions for skeleton-based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 143–152 (2020)

    Google Scholar 

  103. Gao, X., Li, K., Zhang, Y., Miao, Q., Sheng, L., Xie, J., Xu, J.: 3d skeleton-based video action recognition by graph convolution network. In: 2019 IEEE International Conference on Smart Internet of Things (SmartIoT), pp. 500–501. IEEE (2019)

    Google Scholar 

  104. Li, C., Cui, Z., Zheng, W., Chunyan, X., Ji, R., Yang, J.: Action-attending graphic neural network. IEEE Trans. Image Process. 27(7), 3657–3670 (2018)

    MathSciNet  CrossRef  Google Scholar 

  105. Song, Y.-F., Zhang, Z., Wang, L.: Richly activated graph convolutional network for action recognition with incomplete skeletons. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1–5. IEEE (2019)

    Google Scholar 

  106. Ye, F., Tang, H., Wang, X., Liang, X.: Joints relation inference network for skeleton-based action recognition. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 16–20. IEEE (2019)

    Google Scholar 

  107. Zhang, X., Xu, C., Tao, D.: Context aware graph convolution for skeleton-based action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14333–14342 (2020)

    Google Scholar 

  108. Zhang, G., Zhang, X.: Multi-heads attention graph convolutional networks for skeleton-based action recognition. In: 2019 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4. IEEE (2019)

    Google Scholar 

  109. Si, C., Jing, Y., Wang, W., Wang, L., Tan, T.: Skeleton-based action recognition with spatial reasoning and temporal stack learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 103–118 (2018)

    Google Scholar 

  110. Zare, A., Moghaddam, H.A., Sharifi, A.: Video spatiotemporal mapping for human action recognition by convolutional neural network. Pattern Anal. Appl. 23(1), 265–279 (2020)

    CrossRef  Google Scholar 

  111. Cho, S., Maqbool, M., Liu, F., Foroosh, H.: Self-attention network for skeleton-based human action recognition. In: The IEEE Winter Conference on Applications of Computer Vision, pp. 635–644 (2020)

    Google Scholar 

  112. Jiang, M., Pan, N., Kong, J.: Spatial-temporal saliency action mask attention network for action recognition. J. Vis. Commun. Image Represent., p. 102846 (2020)

    Google Scholar 

  113. Yang, Z., Li, Y., Yang, J., Luo, J.: Action recognition with spatio-temporal visual attention on skeleton image sequences. IEEE Trans. Circuits Syst. Video Technol. 29(8), 2405–2415 (2019)

    CrossRef  Google Scholar 

  114. Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P. et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)

  115. Shahroudy, A., Liu, J., Ng, T.-T., Wang, G.: Ntu rgb+d: a large scale dataset for 3d human activity analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, June 2016

    Google Scholar 

  116. Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., Kot, A.C.: Ntu rgb+d 120: a large-scale benchmark for 3d human activity understanding. IEEE Trans. Pattern Anal. Mach, Intell (2019)

    Google Scholar 

  117. Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  118. Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308 (2017)

    Google Scholar 

  119. Dong, J., Gao, Y., Lee, H.J., Zhou, H., Yao, Y., Fang, Z., Huang, B.: Action recognition based on the fusion of graph convolutional networks with high order features. Appl. Sci. 10(4), 1482 (2020)

    CrossRef  Google Scholar 

  120. Liu, J., Wang, G., Hu, P., Duan, L.-Y., Kot, A.C.: Global context-aware attention lstm networks for 3d action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1647–1656 (2017)

    Google Scholar 

  121. Liu, M., Liu, H., Chen, C.: Enhanced skeleton visualization for view invariant human action recognition. Pattern Recogn. 68, 346–362 (2017)

    CrossRef  Google Scholar 

  122. Jian-Fang, H., Zheng, W.-S., Ma, L., Wang, G., Lai, J., Zhang, J.: Early action prediction by soft regression. IEEE Trans. Pattern Ana. Mach. Intell. 41(11), 2568–2583 (2018)

    Google Scholar 

  123. Liu, J., Shahroudy, A., Wang, G., Duan, L.-Y., Kot, A.C.: Skeleton-based online action prediction using scale selection network. IEEE Trans. Pattern Anal. Mach. Intell. 42(6), 1453–1467 (2019)

    CrossRef  Google Scholar 

  124. Papadopoulos, K., Ghorbel, E., Aouada, D., Ottersten, B.: Vertex feature encoding and hierarchical temporal modeling in a spatial-temporal graph convolutional network for action recognition. arXiv preprint arXiv:1912.09745, 2019

  125. Huynh-The, T., Hua, C.-H., Tu, N.A., Kim, D.-S.: Learning geometric features with dual–stream cnn for 3d action recognition. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2353–2357. IEEE (2020)

    Google Scholar 

  126. Zhang, X., Xu, C., Tian, X., Tao, D.: Graph edge convolutional neural networks for skeleton-based action recognition. IEEE Trans. Neural Networks Learn, Syst (2019)

    CrossRef  Google Scholar 

  127. Li, B., Li, X., Zhang, Z., Fei, W.: Spatio-temporal graph routing for skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence 33, 8561–8568 (2019)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh

    Sujan Sarker, Sejuti Rahman, Syeda Faiza Ahmed & Lafifa Jamal

  2. Department of Computer Science and Engineering, Ahsanullah University of Science and Technology, Dhaka, Bangladesh

    Tonmoy Hossain

  3. Department of Electrical and Electronic Engineering, University of Dhaka, Dhaka, Bangladesh

    Md Atiqur Rahman Ahad

  4. Department of Media Intelligent, Osaka University, Osaka, Japan

    Md Atiqur Rahman Ahad

Authors
  1. Sujan Sarker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sejuti Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tonmoy Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Syeda Faiza Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lafifa Jamal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Md Atiqur Rahman Ahad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sujan Sarker .

Editor information

Editors and Affiliations

  1. Department of Electrical and Electronic Engineering, University of Dhaka, Dhaka, Bangladesh

    Prof. Md Atiqur Rahman Ahad

  2. Multimedia R&D and Standards, Qualcomm Technologies Inc., San Diego, CA, USA

    Upal Mahbub

  3. College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, USA

    Tauhidur Rahman

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Sarker, S., Rahman, S., Hossain, T., Faiza Ahmed, S., Jamal, L., Ahad, M.A.R. (2021). Skeleton-Based Activity Recognition: Preprocessing and Approaches. In: Ahad, M.A.R., Mahbub, U., Rahman, T. (eds) Contactless Human Activity Analysis. Intelligent Systems Reference Library, vol 200. Springer, Cham. https://doi.org/10.1007/978-3-030-68590-4_2

Download citation