Skip to main content
SpringerLink
Log in

Single Run Action Detector over Video Stream - A Privacy Preserving Approach

  • Conference paper
  • First Online:
Deep Learning for Human Activity Recognition (DL-HAR 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1370))

Included in the following conference series:

Abstract

This paper takes initial strides at designing and evaluating a vision-based system for privacy ensured activity monitoring. The proposed technology utilizing Artificial Intelligence (AI)-empowered proactive systems offering continuous monitoring, behavioral analysis, and modeling of human activities. To this end, this paper presents Single Run Action Detector (S-RAD) which is a real-time privacy-preserving action detector that performs end-to-end action localization and classification. It is based on Faster-RCNN combined with temporal shift modeling and segment based sampling to capture the human actions. Results on UCF-Sports and UR Fall dataset present comparable accuracy to State-of-the-Art approaches with significantly lower model size and computation demand and the ability for real-time execution on edge embedded device (e.g. Nvidia Jetson Xavier).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/TeCSAR-UNCC/S-RAD-ActionLocalizationClassification.

References

  1. Alaoui, A.Y., El Fkihi, S., Thami, R.O.H.: Fall detection for elderly people using the variation of key points of human skeleton. IEEE Access 7, 154786–154795 (2019)

    Article  Google Scholar 

  2. Atallah, L., Lo, B., King, R., Yang, G.Z.: Sensor positioning for activity recognition using wearable accelerometers. IEEE Trans. Biomed. Circ. Syst. 5(4), 320–329 (2011)

    Article  Google Scholar 

  3. Cameiro, S.A., da Silva, G.P., Leite, G.V., Moreno, R., Guimarães, S.J.F., Pedrini, H.: Multi-stream deep convolutional network using high-level features applied to fall detection in video sequences. In: 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 293–298 (2019)

    Google Scholar 

  4. Chen, K., et al.: MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155 (2019)

  5. Duarte, K., Rawat, Y.S., Shah, M.: Videocapsulenet: a simplified network for action detection. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 2018, pp. 7621–7630. Curran Associates Inc., Red Hook (2018)

    Google Scholar 

  6. Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vision 88(2), 303–338 (2010). https://doi.org/10.1007/s11263-009-0275-40275-4

    Article  Google Scholar 

  7. Gkioxari et al., Georgia, J.M.: Finding action tubes. CoRR abs/1411.6031 (2014). http://arxiv.org/abs/1411.6031

  8. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385

  9. Hou, R., Chen, C., Shah, M.: Tube convolutional neural network (t-cnn) for action detection in videos. In: The IEEE International Conference on Computer Vision (ICCV), October 2017

    Google Scholar 

  10. Kalogeiton, V., Weinzaepfel, P., Ferrari, V., Schmid, C.: Action tubelet detector for spatio-temporal action localization. CoRR abs/1705.01861 (2017). http://arxiv.org/abs/1705.01861

  11. Kay, W., et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)

  12. Kwolek, B., Kepski, M.: Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput. Methods Programs Biomed. 117(3), 489–501 (2014)

    Article  Google Scholar 

  13. Leite, G., Silva, G., Pedrini, H.: Fall detection in video sequences based on a three-stream convolutional neural network. In: 2019 18th IEEE International Conference On Machine Learning and Applications (ICMLA), pp. 191–195 (2019)

    Google Scholar 

  14. Lin, J., Gan, C., Han, S.: Temporal shift module for efficient video understanding. CoRR abs/1811.08383 (2018). http://arxiv.org/abs/1811.08383

  15. Liu, W., et al.: SSD: single shot multibox detector. CoRR abs/1512.02325 (2015). http://arxiv.org/abs/1512.02325

  16. Lu, N., Wu, Y., Feng, L., Song, J.: Deep learning for fall detection: three-dimensional CNN combined with LSTM on video kinematic data. IEEE J. Biomed. Health Inform. 23(1), 314–323 (2019)

    Article  Google Scholar 

  17. Mirzadeh, S.I., Ghasemzadeh, H.: Optimal policy for deployment of machine learning models on energy-bounded systems. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI) (2020)

    Google Scholar 

  18. Neff, C., Mendieta, M., Mohan, S., Baharani, M., Rogers, S., Tabkhi, H.: Revamp2t: real-time edge video analytics for multicamera privacy-aware pedestrian tracking. IEEE Internet Things J. 7(4), 2591–2602 (2020)

    Article  Google Scholar 

  19. Pagan, J., et al.: Toward ultra-low-power remote health monitoring: an optimal and adaptive compressed sensing framework for activity recognition. IEEE Trans. Mobile Comput. (TMC) 18(3), 658–673 (2018)

    Article  Google Scholar 

  20. Peng et al., Xiaojiang, S.C.: Multi-region two-stream R-CNN for action detection. Lecture Notes in Computer Science, vol. 9908, pp. 744–759. Springer, Amsterdam, Netherlands, October 2016. https://doi.org/10.1007/978-3-319-46493-0_45, https://hal.inria.fr/hal-01349107

  21. Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015). http://arxiv.org/abs/1506.01497

  22. Saeedi, R., Purath, J., Venkatasubramanian, K., Ghasemzadeh, H.: Toward seamless wearable sensing: automatic on-body sensor localization for physical activity monitoring. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5385–5388. IEEE (2014)

    Google Scholar 

  23. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. CoRR abs/1406.2199 (2014). http://arxiv.org/abs/1406.2199

  24. Soomro, K., Zamir, A.R.: Action recognition in realistic sports videos (2014)

    Google Scholar 

  25. Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., Paluri, M.: C3D: generic features for video analysis. CoRR abs/1412.0767 (2014). http://arxiv.org/abs/1412.0767

  26. Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., Gool, L.V.: Temporal segment networks: Towards good practices for deep action recognition. CoRR abs/1608.00859 (2016). http://arxiv.org/abs/1608.00859

  27. Weinzaepfel, P., Harchaoui, Z., Schmid, C.: Learning to track for spatio-temporal action localization. CoRR abs/1506.01929 (2015). http://arxiv.org/abs/1506.01929

Download references

Author information

Authors and Affiliations

  1. University of North Carolina at Charlotte, Charlotte, NC, 28223, USA

    Anbumalar Saravanan, Justin Sanchez & Hamed Tabkhi

  2. Washington State University, Pullman, WA, 99164, USA

    Hassan Ghasemzadeh

  3. Azusa Pacific University, Azusa, CA, 91702, USA

    Aurelia Macabasco-O’Connell

Authors
  1. Anbumalar Saravanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Justin Sanchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hassan Ghasemzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aurelia Macabasco-O’Connell
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hamed Tabkhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Justin Sanchez .

Editor information

Editors and Affiliations

  1. Institute for Infocomm Research, A*STAR, Singapore, Singapore

    Xiaoli Li

  2. Institute for Infocomm Research, A*STAR, Singapore, Singapore

    Min Wu

  3. Institute for Infocomm Research, A*STAR, Singapore, Singapore

    Zhenghua Chen

  4. Institute for Infocomm Research, A*STAR, Singapore, Singapore

    Le Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saravanan, A., Sanchez, J., Ghasemzadeh, H., Macabasco-O’Connell, A., Tabkhi, H. (2021). Single Run Action Detector over Video Stream - A Privacy Preserving Approach. In: Li, X., Wu, M., Chen, Z., Zhang, L. (eds) Deep Learning for Human Activity Recognition. DL-HAR 2021. Communications in Computer and Information Science, vol 1370. Springer, Singapore. https://doi.org/10.1007/978-981-16-0575-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-0575-8_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-0574-1

  • Online ISBN: 978-981-16-0575-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation