Abstract
One of the most important and challenging research areas in the fields of Computer Vision is Human Activity Recognition (HAR). It has various applications including human-computer interaction, Intelligent driving. Intelligent video surveillance, human–robot interaction, ambient assisted living, etc. The major issues faced in this domain are feature extraction and feature selection which ends up in using handcrafted feature representation-based methods. This issue has been successfully addressed by Deep Learning techniques in various state-of-the-art classification methods for images and videos. But still which method is suited in which condition depends on the kind of dataset used for analysis. Deep Learning models like Convolutional Neural Networks (CNN), Variants of CNN, Recurrent Neural Networks (RNNs), and other fusion methods have shown promising results for a specific kind of activity related to HAR. Activity in HAR can be categorized as Gesture, Action, Interaction (Single-User, Multiple-User, Group). Each category has its own set of issues. This research aims to focus on Video Classification in bench marked datasets like UCF50, KTH, and Wizmann for Human gesture and Action Recognition which will then be confined to recognize the particular activity performed in the datasets. CNN and RNN fusion models such as Long Term Recurrent Convolutional Networks (LRCN) and ConvLSTM. These models performed efficiently of all the benchmarked datasets and hence can be utilized for Human Activity Recognition in arbitrary video dataset.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Underst 73(3):428–440
Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv 43(3):16:1–16:43
SchĂĽldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: IEEE ICPR
Ryoo M, Aggarwal J (2009) Spatio-temporal relationship match: Video structure comparison for recognition of complex human activities. In: ICCV, pp 1593–1600
Rodriguez MD, Ahmed J, Shah M (2008) Action mach: a spatiotemporal maximum average correlation height filter for action recognition. In: CVPR
Marszałek M, Laptev I, Schmid C (2009) Actions in context. In: IEEE CVPR
Xia L, Chen C, Aggarwal J (2012) View invariant human action recognition using histograms of 3d joints. In: IEEE-CVPRW, pp 20–27
Singh S, Velastin SA, Ragheb H (2010) Muhavi: a multicamera human action video dataset for the evaluation of action recognition methods. In: AVSS, 2010 seventh IEEE international conference on IEEE, pp 48–55
Aggarwal JK, Xia L (2014) Human activity recognition from 3D data: a review. Pattern Recogn Lett 48:70–80
Hubel DH, Wiesel TN (1959) Receptive fields of single neurons in the cat’s striate cortex. J Physiol
Mathworks Inc. Convolutional neural networks. https://in.mathworks.com/discovery/convolutional-neural-network.html
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553): 436–444
Al-Rfou R (2016) Theano: a python framework for fast computation of mathematical expressions. arXiv preprint arXiv:1605.02688
Ronan C et al (2011) Torch7: a Matlab-like environment for machine learning. BigLearn, NIPS Workshop. NO. EPFL-CONF-192376
Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093
MartĂn A et al (2016) TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467
https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
Jamil A et al (2017) Data augmentation-assisted deep learning of hand-drawn partially colored sketches for the visual search. PLoS ONE 12(8):e0183838
Le Cun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989)Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems (NIPS)
Girshick R, Donahue J, Darrell T, Malik J (2016) Region based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38(1)
Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6)
Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: IEEE conference on computer vision and pattern recognition (CVPR)
Zhang B, Wang L, Wang Z, Qiao Y, Wang H (2016) Real-time action recognition with enhanced motion vector CNNs. In: IEEE conference on computer vision and pattern recognition (CVPR)
Hou R, Chen C, Shah M (2017) Tube convolutional neural network (T-CNN) for action detection in videos. In: IEEE international conference on computer vision (ICCV)
Bux A, Angelov P, Habib Z (2017) A comprehensive review on handcrafted and learning-based action representation approaches for human activity recognition. Appl Sci 7(1): 110. https://doi.org/10.3390/app7010110
Martinez-del-Rincon J, Santofimia MJ, Nebel J (2013) Common sense reasoning for human action recognition. Pattern Recog Lett 34(15):1849–1860. https://doi.org/10.1016/j.patrec.2012.10.020
Haykin S (1994) Neural networks: a comprehensive foundation
Hochreiter S, Hochreiter S, Schmidhuber J, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Ordóñez FJ, Roggen D (2016) Deep convolutional and lSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1):115
Karpathy A, Johnson J, Fei-Fei L (2015) Visualizing and understanding recurrent networks. arXiv preprintarXiv:1506.02078
Reddy KK, Shah M (2012) Recognizing 50 human action categories of web videos, machine vision and applications journal (MVAP)
Schuldt C, Laptev I, Caputo B (2004)Recognizing human actions: a local SVM approach. Pattern recognition 2004, vol 3. ICPR, pp 32–36
Gorelick L, Blank M, Shechtman E, Irani M, Basri R (2007) Actions as space-time shapes. IEEE Trans Pattern Anal Mach Intell 29(12):2247–2253. https://doi.org/10.1109/TPAMI.2007.70711
Du Y, Wang W, Wang L (2015) Hierarchical recurrent neural network for skeleton based action recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1110–1118
Liu J, Shahroudy A, Xu D, Wang G (2016) Spatio-temporal LSTM with trust gates for 3D human action recognition. In: European conference on computer vision. Springer, pp 816–833
Shahroudy A, Liu J, Ng T-T, Wang G (2016) NTU RGB+D: a large scale dataset for 3D human activity analysis. In: IEEE conference on computer vision and pattern recognition (CVPR)
Zhu W, Lan C, Xing J, Zeng W, Li Y, Shen L, Xie X (2016) Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks. In: Thirtieth AAAI conference on artificial intelligence (AAAI)
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25. Curran Associates Inc., pp 1097–1105
Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Li F-F (2014) Large-scale video classification with convolutional neural networks. In: CVPR
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
Neverova N, Wolf C, Taylor G, Nebout F (2015a) Moddrop: adaptive multi-modal gesture recognition. Pre-print: arXiv:1501.00102
Taylor GW, Fergus R, LeCun Y, Bregler C (2010b) Convolutional learning of spatiotemporal features. In: European conference on computer vision. Springer, pp 140–153
Han C, Mei E, Wang C. hcs@stanford.edu, cwang17@stanford.edu, Evelyn66@stanford.edu
Yao A, Gall J, Fanelli G, Gool LV (2011) Does human action recognition benefit from pose estimation? In: BMVC
Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56(1):116–124
Donahue J, Hendricks LA, Rohrbach M, Venugopalan S, Guadarrama S, Saenko K, Darrell T (2017) Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans Pattern Anal Mach Intell 39(4):677–691. https://doi.org/10.1109/TPAMI.2016.2599174
Le QV (2013) Building high-level features using large scale unsupervised learning. In: 2013 IEEE international conference on acoustics, speech and signal processing, pp 8595–8598
Karpathy A, Fei-Fei L (2015) Deep visual-semantic alignments for generating image descriptions. In: The IEEE conference on computer vision and pattern recognition (CVPR)
Dean J, Corrado G, Monga R, Chen K, Devin M, Mao M, Aurelio Ranzato M, Senior A, Tucker P, Yang K, Le QV, Ng AY (2012) Large scale distributed deep networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc, pp 1223–1231
Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In ICML
Sanchez-Caballero A, Fuentes-Jimenez D, Losada-Gutiérrez C (2020) Exploiting the ConvLSTM: human action recognition using raw depth video-based recurrent neural networks
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Singh, U., Singhal, N. (2023). Exploiting Video Classification Using Deep Learning Models for Human Activity Recognition. In: Shukla, P.K., Singh, K.P., Tripathi, A.K., Engelbrecht, A. (eds) Computer Vision and Robotics. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-19-7892-0_14
Download citation
DOI: https://doi.org/10.1007/978-981-19-7892-0_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-7891-3
Online ISBN: 978-981-19-7892-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)