Human Action Recognition: A Survey

  • Meixia FuEmail author
  • Na Chen
  • Zhongjie Huang
  • Kaili Ni
  • Yuhao Liu
  • Songlin Sun
  • Xiaomei Ma
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 550)


In this paper, we provide a comprehensive survey in human action recognition and prediction, which has always been a universal and critical area in computer vision. Human action recognition is the first step for a machine to understand and percept the nature, which is small part in machine perception. Human action prediction is the higher layer than human action recognition that is small part in machine cognition, which would give the machine the ability of imagination and reasoning. Here, we only discuss human action recognition from two methodologies that is based on presentations and deep learning, separately. Then, 4 public datasets of human action recognition are descripted closely. Some challenges in dataset are also proposed because of the significance to the development of computer vision. Meanwhile, we compare and summarize recent-published research achievements under deep learning. In the end, we conclude about mentioned methods and future challenges to work on for computer vision.


Human action recognition Computer vision Machine perception Human action prediction Machine cognition Deep learning 



This work is supported by National Natural Science Foundation of China (Project61471066) and the open project fund (No. 201600017) of the National Key Laboratory of Electromagnetic Environment, China.


  1. 1.
    Moeslund, T.B., Granum, E.: A survey of computer vision-based human motion capture. Comput. Vis. Image Underst. 81(3), 231–268 (2001)CrossRefGoogle Scholar
  2. 2.
    Poppe, R.: A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)CrossRefGoogle Scholar
  3. 3.
    Vishwakarma, S.: A survey on activity recognition and behavior understanding in video surveillance. Vis. Comput. 29(10), 983–1009 (2013)CrossRefGoogle Scholar
  4. 4.
    Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE Computer Society (2001)Google Scholar
  5. 5.
    Laptev, I., Lindeberg, T.: On space-time interest points. Int. J. Comput. Vision 64(2–3), 107–123 (2005)CrossRefGoogle Scholar
  6. 6.
    Shechtman, E., Irani, M.: Space-time behavior based correlation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 405–412. IEEE Computer Society (2005)Google Scholar
  7. 7.
    Xu, W., Yang, M., et al.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)CrossRefGoogle Scholar
  8. 8.
    Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: International Conference on Neural Information Processing Systems, pp. 568–576. MIT Press (2014)Google Scholar
  9. 9.
    Wang, L., Qiao, Y., Tang, X.: Action recognition with trajectory-pooled deep-convolutional descriptors. In: Computer Vision and Pattern Recognition, pp. 4305–4314. IEEE (2015)Google Scholar
  10. 10.
    Varol, G., Laptev, I., Schmid, C.: Long-term temporal convolutions for action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1510–1517 (2018)CrossRefGoogle Scholar
  11. 11.
    Natarajan, P., Nevatia, R.: Coupled hidden semi Markov models for activity recognition. In: IEEE Workshop on Motion and Video Computing, p. 10. IEEE Computer Society (2007)Google Scholar
  12. 12.
    Kellokumpu, V., Zhao, G., Pietikäinen, M.: Human activity recognition using a dynamic texture based method. In: British Machine Vision Conference, Leeds. DBLP, September 2008Google Scholar
  13. 13.
    Gorelick, L., Blank, M., Shechtman, E., et al.: Actions as space-time shapes. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2247–2253 (2007)CrossRefGoogle Scholar
  14. 14.
    Donahue, J., Hendricks, L.A., Rohrbach, M., et al.: Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 677–691 (2014)CrossRefGoogle Scholar
  15. 15.
    Wang, L., Xiong, Y., Wang, Z., et al.: Temporal segment networks: towards good practices for deep action recognition. In: European Conference on Computer Vision, pp. 20–36. Springer, Cham (2016)CrossRefGoogle Scholar
  16. 16.
    Zhu, W., Hu, J., Sun, G., et al.: A key volume mining deep framework for action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1991–1999. IEEE Computer Society (2016)Google Scholar
  17. 17.
    Wu, C.Y., Zaheer, M., Hu, H., et al.: Compressed video action recognition. In: Computer Vision and Pattern Recognition. IEEE (2018)Google Scholar
  18. 18.
    Zang, J., Wang, L., Liu, Z., et al.: Attention-based temporal weighted convolutional neural network for action recognition. In: IFIP International Conference on Artificial Intelligence Applications and Innovations, pp. 97–108. Springer, Cham (2018)Google Scholar
  19. 19.
    Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: Computer Vision and Pattern Recognition, pp. 1110–1118. IEEE (2015)Google Scholar
  20. 20.
    Zhang, S., Xiao, J., Liu, X., et al.: Fusing geometric features for skeleton-based action recognition using multilayer LSTM networks. IEEE Trans. Multimed. 20, 2330–2343 (2018)CrossRefGoogle Scholar
  21. 21.
    Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition (2018)Google Scholar
  22. 22.
    Tang, Y., Tian, Y., Lu, J., et al.: Deep progressive reinforcement learning for skeleton-based action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5323–5332 (2018)Google Scholar
  23. 23.
    Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 3, pp. 32–36. IEEE (2004)Google Scholar
  24. 24.
    Kuehne, H., Jhuang, H., Garrote, E., et al.: HMDB: a large video database for human motion recognition. In: IEEE International Conference on Computer Vision (ICCV), pp. 2556–2563. IEEE (2011)Google Scholar
  25. 25.
    Soomro, K., Zamir, A.R., Shah, M.: UCF101: a dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)
  26. 26.
    Kay, W., Carreira, J., Simonyan, K., et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Meixia Fu
    • 1
    • 2
    • 3
    Email author
  • Na Chen
    • 1
    • 2
    • 3
  • Zhongjie Huang
    • 1
    • 2
    • 3
  • Kaili Ni
    • 1
    • 2
    • 3
  • Yuhao Liu
    • 1
    • 2
    • 3
  • Songlin Sun
    • 1
    • 2
    • 3
  • Xiaomei Ma
    • 4
  1. 1.National Engineering Laboratory for Mobile Network SecurityBeijing University of Posts and TelecommunicationsBeijingChina
  2. 2.Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of EducationBeijing University of Posts and TelecommunicationsBeijingChina
  3. 3.School of Information and Communication EngineeringBeijing University of Posts and TelecommunicationsBeijingChina
  4. 4.China United Network Communications Group Co., Ltd.BeijingChina

Personalised recommendations