Skip to main content
SpringerLink
Log in

Recognizing Human Actions Using 3D Skeletal Information and CNNs

  • Conference paper
  • First Online:
Engineering Applications of Neural Networks (EANN 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1000))

Abstract

In this paper we present an approach for the recognition of human actions targeting at activities of daily living (ADLs). Skeletal information is used to create images capturing the motion of joints in the 3D space. These images are then transformed to the spectral domain using 4 well-known image transforms. A deep Convolutional Neural Network is trained on those images. Our approach is thoroughly evaluated using a well-known, publicly available challenging dataset and for a set of actions that resembles to common ADLs, covering both cross-view and cross-subject cases.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.numpy.org/.

  2. 2.

    https://www.scipy.org/.

  3. 3.

    https://opencv.org/.

References

  1. Abadi, M., et al.: TensorFlow: a system for large-scale maching learning. In: Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI) (2016)

    Google Scholar 

  2. Berretti, S., Daoudi, M., Turaga, P., Basu, A.: Representation, analysis, and recognition of 3D humans: a survey. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 14(1S), 16 (2018)

    Google Scholar 

  3. Chollet, F.: Keras (2015). https://github.com/fchollet/keras

  4. Du, Y., Fu, Y., Wang, L.: Skeleton based action recognition with convolutional neural network. In: 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 579–583. IEEE (2015)

    Google Scholar 

  5. Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645–6649. IEEE (2013)

    Google Scholar 

  6. Hou, Y., Li, Z., Wang, P., Li, W.: Skeleton optical spectra-based action recognition using convolutional neural networks. IEEE Trans. Circuits Syst. Video Technol. 28(3), 807–811 (2018)

    Article  Google Scholar 

  7. Jiang, W., Yin, Z.: Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 1307–1310 (2015)

    Google Scholar 

  8. Ke, Q., An, S., Bennamoun, M., Sohel, F., Boussaid, F.: SkeletonNet: mining deep part features for 3-D action recognition. IEEE Signal Process. Lett. 24(6), 731–735 (2017)

    Article  Google Scholar 

  9. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  10. Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: HMDB: a large video database for human motion recognition. In: 2011 International Conference on Computer Vision, pp. 2556–2563. IEEE (2011)

    Google Scholar 

  11. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)

    Google Scholar 

  12. Lawton, M.P., Brody, E.M.: Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontol. 9(3 Part 1), 179–186 (1969)

    Article  Google Scholar 

  13. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  14. Li, C., Hou, Y., Wang, P., Li, W.: Joint distance maps based action recognition with convolutional neural networks. IEEE Signal Process. Lett. 24(5), 624–628 (2017)

    Article  Google Scholar 

  15. Liu, C., Hu, Y., Li, Y., Song, S., Liu, J.: PKU-MMD: a large scale benchmark for continuous multi-modal human action understanding. arXiv preprint arXiv:1703.07475 (2017)

  16. Liu, M., Liu, H., Chen, C.: Enhanced skeleton visualization for view invariant human action recognition. Pattern Recognit. 68, 346–362 (2017)

    Article  Google Scholar 

  17. Mathe, E., Mitsou, A., Spyrou, E., Mylonas, Ph.: Arm gesture recognition using a convolutional neural network. In: Proceedings of International Workshop Semantic and Social Media Adaptation and Personalization (SMAP) (2018)

    Google Scholar 

  18. Mathe, E., Maniatis, A., Spyrou, E., Mylonas, Ph.: A deep learning approach for human action recognition using skeletal information. In: Proceedings of World Congress “Genetics, Geriatrics and Neurodegenerative Diseases Research” (GeNeDiS) (2018)

    Google Scholar 

  19. Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), vol. 03, pp. 32–36. IEEE Computer Society (2004)

    Google Scholar 

  20. Shahroudy, A., Liu, J., Ng, T.T., Wang, G.: NTU RGB+D: a large scale dataset for 3D human activity analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1010–1019 (2016)

    Google Scholar 

  21. Soomro, K., Zamir, A.R., Shah, M.: UCF101: a dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)

  22. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  23. Wang, P., Li, W., Ogunbona, P., Wan, J., Escalera, S.: RGB-D-based human motion recognition with deep learning: a survey. Comput. Vis. Image Underst. 171, 118–139 (2018)

    Article  Google Scholar 

  24. Wang, P., Li, W., Li, C., Hou, Y.: Action recognition based on joint trajectory maps with convolutional neural networks. Knowl.-Based Syst. 158, 43–53 (2018)

    Article  Google Scholar 

Download references

Acknowledgment

We acknowledge support of this work by the project SYNTELESIS “Innovative Technologies and Applications based on the Internet of Things (IoT) and the Cloud Computing” (MIS 5002521) which is implemented under the “Action for the Strategic Development on the Research and Technological Sector”, funded by the Operational Programme “Competitiveness, Entrepreneurship and Innovation” (NSRF 2014-2020) and co-financed by Greece and the European Union (European Regional Development Fund).

Author information

Authors and Affiliations

  1. Department of Informatics and Telecommunications, University of Athens, Athens, Greece

    Antonios Papadakis

  2. Institute of Informatics and Telecommunications, National Center for Scientific Research - “Demokritos”, Athens, Greece

    Eirini Mathe & Evaggelos Spyrou

  3. Department of Computer Science and Telecommunications, University of Thessaly, Lamia, Greece

    Ioannis Vernikos

  4. General Department, University of Thessaly, Lamia, Greece

    Apostolos Maniatis & Evaggelos Spyrou

  5. Department of Informatics, Ionian University, Corfu, Greece

    Eirini Mathe & Phivos Mylonas

Authors
  1. Antonios Papadakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eirini Mathe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ioannis Vernikos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Apostolos Maniatis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Evaggelos Spyrou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Phivos Mylonas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Evaggelos Spyrou .

Editor information

Editors and Affiliations

  1. David Goldman Informatics Centre, University of Sunderland, Sunderland, UK

    John Macintyre

  2. Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  3. University of Piraeus, Piraeus, Greece

    Ilias Maglogiannis

  4. Oxford Brookes University, Oxford, UK

    Chrisina Jayne

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Papadakis, A., Mathe, E., Vernikos, I., Maniatis, A., Spyrou, E., Mylonas, P. (2019). Recognizing Human Actions Using 3D Skeletal Information and CNNs. In: Macintyre, J., Iliadis, L., Maglogiannis, I., Jayne, C. (eds) Engineering Applications of Neural Networks. EANN 2019. Communications in Computer and Information Science, vol 1000. Springer, Cham. https://doi.org/10.1007/978-3-030-20257-6_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20257-6_44

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20256-9

  • Online ISBN: 978-3-030-20257-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation