Skip to main content
SpringerLink
Log in

A Very Deep Sequences Learning Approach for Human Action Recognition

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9517))

Included in the following conference series:

Abstract

Human action recognition is a popular study in computer vision. The most difficult challenge is capturing the movement features of image sequences or videos. In recent years, deep convolutional networks have achieved great success in many image classification and recognition tasks. But in videos interpretation tasks, the deep-learning has not done well. There were [18, 19] earlier models which were built on convolutional networks for human action recognition tasks. We propose an approach based on CNNs and RNN-like models which have abilities to extract spatial and temporal features both, a CNN model can get static scores, a LSTM or GRU layer which gets dynamic class scores of human action. In another side, compared to a two-stream ConvNet [18, 24], we do not need an optical-flow CNN stream that saves us considerable time, RNN-like models just need a few hours to get convergence. And, we have achieved a quite remarkable performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aggarwal, J.K., Cai, Q.: Human motion analysis: a review. Comput. Vis. Image Underst. (CVIU) 73(3), 428–440 (1999)

    Article  Google Scholar 

  2. Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983)

    Article  MATH  Google Scholar 

  3. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1106–1114 (2012)

    Google Scholar 

  4. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. CoRR, abs/1409.4842 (2014)

    Google Scholar 

  5. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)

    Google Scholar 

  6. Wang, H., Ullah, M.M., Kläser, A., Laptev, I., Schmid, C.: Evaluation of local spatio-temporal features for action recognition. In: BMVC (2009)

    Google Scholar 

  7. Kläser, A., Marszalek, M., Schmid, C.: A spatio-temporal descriptor based on 3D-gradients. In: BMVC (2008)

    Google Scholar 

  8. Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: International Conference on Pattern Recognition (ICPR), vol. 3, pp. 32–36 (2004)

    Google Scholar 

  9. Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: IEEE International Conference on Computer Vision (ICCV), pp. 1395–1402 (2005)

    Google Scholar 

  10. Laptev, I.: On space-time interest points. IJCV 64(2–3), 107–123 (2005)

    Article  Google Scholar 

  11. Wang, H., Kläser, A., Schmid, C., Liu, C.-L.: Dense trajectories and motion boundary descriptors for action recognition. IJCV 103(1), 60–79 (2005)

    Article  Google Scholar 

  12. Wang, H., Schmid, C.: Action recognition with improved trajectories. In: ICCV (2013)

    Google Scholar 

  13. Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: HMDB: a large video database for human motion recognition. In: ICCV (2011)

    Google Scholar 

  14. Soomro, K., Zamir, A.R., Shah, M.: UCF101: a dataset of 101 human actions classes from videos in the wild. CoRR, abs/1212.0402 (2012)

    Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part III. LNCS, vol. 8691, pp. 346–361. Springer, Heidelberg (2014)

    Google Scholar 

  16. Ji, S., Xu, W., Yang, M., Yu, K.: 3D convolutional neural networks for human action recognition. TPAMI 35(1), 221–231 (2013)

    Article  Google Scholar 

  17. Wang, L., Qiao, Y., Tang, X.: Action recognition with trajectory-pooled deep-convolutional descriptors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4305–4314 (2015)

    Google Scholar 

  18. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: NIPS (2014)

    Google Scholar 

  19. Donahue, J., Anne Hendricks, L., Guadarrama, S., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634 (2015)

    Google Scholar 

  20. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  21. Graves, A.: Supervised sequence labelling. In: Graves, A. (ed.) Supervised Sequence Labelling with Recurrent Neural Networks. SCI, vol. 385, pp. 5–13. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  22. Tan, C.C., Jiang, Y.-G., Ngo, C.-W.: Towards textually describing complex video contents with audio-visual concept classifiers. In: MM (2011)


    Google Scholar 

  23. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)


    Google Scholar 

  24. Wang, L., Xiong, Y., Wang, Z., et al.: Towards good practices for very deep two-stream ConvNets. arXiv preprint (2015). arXiv:1507.02159

  25. Chung, J., Gulcehre, C., Cho, K.H., et al.: Empirical evaluation of gated recurrent neural networks on sequence modeling (2014). arXiv preprint arXiv:1412.3555

  26. Vinyals, O., Ravuri, S.V., Povey, D.: Revisiting recurrent neural networks for robust ASR. In: ICASSP (2012)

    Google Scholar 

  27. Sutskever, I., Martens, J., Hinton, G.E.: Generating text with recurrent neural networks. In: ICML (2011)

    Google Scholar 

  28. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.S., Berg, A.C., Li, F.-F.: ImageNet large scale visual recognition challenge. CoRR abs/1409.0575 (2014). http://arxiv.org/abs/1409.0575

  29. Jiang, Y.-G., Liu, J., Roshan Zamir, A., Laptev, I., Piccardi, M., Shah, M., Sukthankar, R.: THUMOS challenge: action recognition with a large number of classes. In: ICCV 2013-Action-Workshop (2013)

    Google Scholar 

  30. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R.B., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. CoRR, abs/1408.5093

    Google Scholar 

  31. Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D., Bengio, Y.: Theano: new features and speed improvements. In: Proceedings NIPS 2012 Deep Learning Workshop (2012)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National High Technology Development Plan (863), under Grant No. 2011AA01A205, by the National Significant Science and Technology Projects of China, under Grant No. 2013ZX01039001-002-003; by the NSFC project under Grant Nos. U1433112, and 61170253.

Author information

Authors and Affiliations

  1. Graduate School at Shenzhen, Tsinghua University, Shenzhen, 518055, China

    Zhihui Lin & Chun Yuan

  2. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Chun Yuan

Authors
  1. Zhihui Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chun Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhihui Lin .

Editor information

Editors and Affiliations

  1. University of Texas at San Antonio, San Antonio, USA

    Qi Tian

  2. Dept. of Information Engineering, University of Trento, Povo, Trento, Italy

    Nicu Sebe

  3. EECS, University of Central Florida, Orlando, Florida, USA

    Guo-Jun Qi

  4. EURECOM, Sophia-Antipolis, France

    Benoit Huet

  5. Hefei University of Technology, Hefei, Anhui, China

    Richang Hong

  6. School of Computing and Information, Hefei University of Technology, Hefei, Anhui, China

    Xueliang Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Lin, Z., Yuan, C. (2016). A Very Deep Sequences Learning Approach for Human Action Recognition. In: Tian, Q., Sebe, N., Qi, GJ., Huet, B., Hong, R., Liu, X. (eds) MultiMedia Modeling. MMM 2016. Lecture Notes in Computer Science(), vol 9517. Springer, Cham. https://doi.org/10.1007/978-3-319-27674-8_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27674-8_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27673-1

  • Online ISBN: 978-3-319-27674-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation