Human action recognition: a framework of statistical weighted segmentation and rank correlation-based selection
- 15 Downloads
Human action recognition from a video sequence has received much attention lately in the field of computer vision due to its range of applications in surveillance, healthcare, smart homes, tele-immersion, to name but a few. However, it is still facing several challenges such as human variations, occlusion, change in illumination, complex background. In this article, we consider the problems related to multiple human detection and classification using novel statistical weighted segmentation and rank correlation-based feature selection approach. Initially, preprocessing is performed on a set of frames to remove existing noise and to make the foreground maximal differentiable compared to the background. A novel weighted segmentation method is also introduced for human extraction prior to feature extraction. Ternary features are exploited including color, shape, and texture, which are later combined using serial-based features fusion method. To avoid redundancy, rank correlation-based feature selection technique is employed, which acts as a feature optimizer and leads to improved classification accuracy. The proposed method is validated on six datasets including Weizmann, KTH, Muhavi, WVU, UCF sports, and MSR action and validated based on seven performance measures. A fair comparison with existing work is also provided which proves the significance of proposed compared to other techniques.
KeywordsAction recognition Weighted segmentation Feature selection Rank correlation Weighted KNN
The authors would like to thank HEC Startup Research Grant Program (SRGP) Pakistan (Project# 1307).
MS generated this idea and developed a classification design and also identified the sole application. MAK performed the simulations by developing different patches of code with full integration. He is also responsible for this complete write-up. Different accuracy criteria are finalized and also simulated by this author. MF and JHS have given a complete shape to this article and identified several issues and helped the primary authors to overcome all those shortcomings. TS is responsible for the final proofreading along with the technical support in the classification step due to her research major. AR provided technical support in different sections which include feature extraction and fusion along with the issues raised in the development of selection approach. All authors read and approved the final manuscript.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no competing interest.
Availability of data and material
Six publicly available datasets are used in this research for validation of proposed method including Weizmann, KTH, Muhavi, WVU, UCF sports, and MSR action.
- 1.Khan MA, Akram T, Sharif M, Javed MY, Muhammad N, Yasmin M (2018) An implementation of optimized framework for action classification using multilayers neural network on selected fused features. Pattern Anal Appl 1–21. https://doi.org/10.1007/s10044-018-0688-1
- 5.Ogale NA (2006) A survey of techniques for human detection from video. Surv Univ Md 125:19Google Scholar
- 7.Mishra MSK, Jtmcoe F, Bhagat K (2015) A survey on human motion detection and surveillance. Int J Adv Res Electron Commun Eng 4(4):1044–1048Google Scholar
- 10.Dhulekar P, Gandhe S, Chitte H, Pardeshi K, (2017) Human action recognition: an overview. In: Proceedings of the international conference on data engineering and communication technology, pp 481–488Google Scholar
- 12.Ai S, Lu T, Xiong Y (2018) Improved dense trajectories for action recognition based on random projection and Fisher vectors. In: MIPPR 2017: pattern recognition and computer vision, p 1060915Google Scholar
- 13.Ming Y, Wang G, Hong X (2017) Spatial-temporal texture features for 3D human activity recognition using laser-based RGB-D videos. KSII Trans Internet Inf Syst 11(3):1595–1613Google Scholar
- 15.Jaouedi N, Boujnah N, Htiwich O, Bouhlel MS (2017) Human action recognition to human behavior analysis. In: 2017 International conference on information and digital technologies (IDT), pp 263–266Google Scholar
- 18.Akilandasowmya G, Sathiya P, AnandhaKumar P (2015) Human action analysis using K-NN classifier. In: 2015 Seventh international conference on advanced computing (ICoAC), pp 1–7Google Scholar
- 20.Uddin M, Kim J (2016) Human activity recognition using spatiotemporal 3-D body joint features with hidden Markov models. KSII Trans Internet Inf Syst 10(6):2767–2780Google Scholar
- 26.Azary S (2014) Grassmann learning for recognition and classification. Rochester Institute of TechnologyGoogle Scholar
- 31.Usha R, Perumal K (2014) Content based image retrieval using combined features of color and texture features with SVM classification. Int J Comput Sci Commun Netw 4:169–174Google Scholar
- 32.Hechenbichler K, Schliep K (2004) Weighted k-nearest-neighbor techniques and ordinal classificationGoogle Scholar
- 33.Bregonzio M, Gong S, Xiang T (2009) Recognising action as clouds of space-time interest points. In: IEEE conference on computer vision and pattern recognition, 2009 (CVPR 2009), pp 1948–1955Google Scholar
- 34.Kulathumani V, Kavi R, Ramagiri S (2011) WVU multi-view action recognition datasetGoogle Scholar
- 35.Kumar SS, John M (2016) Human activity recognition using optical flow based feature set. In: 2016 IEEE international Carnahan conference on security technology (ICCST), pp 1–5Google Scholar
- 36.Azary S (2014) Grassmann learning for recognition and classificationGoogle Scholar
- 37.Mahadeo NK, Papliński AP, Ray S (2012) Model-based pupil and iris localization. In: The 2012 International joint conference on neural networks (IJCNN), pp 1–7Google Scholar
- 39.Iosifidis A, Tefas A, Pitas I (2013) Multi-view human action recognition: a survey. In: 2013 Ninth international conference on intelligent information hiding and multimedia signal processing, pp 522–525Google Scholar
- 40.Singh S, Velastin SA, Ragheb H (2010) Muhavi: a multicamera human action video dataset for the evaluation of action recognition methods. In: 2010 Seventh IEEE international conference on advanced video and signal based surveillance (AVSS), pp 48–55Google Scholar
- 41.Cai J, Tang X, Zhang L, Feng G (2016) Learning zeroth class dictionary for human action recognition. In: 2016 IEEE international conference on image processing (ICIP), pp 4175–4179Google Scholar
- 43.Wu X, Jia Y (2012) View-invariant action recognition using latent kernelized structural SVM. In: European conference on computer vision, pp 411–424Google Scholar
- 44.Eweiwi A, Cheema S, Thurau C, Bauckhage C (2011) Temporal key poses for human action recognition. In: 2011 IEEE international conference on computer vision workshops (ICCV Workshops), pp 1310–1317Google Scholar
- 45.Rodriguez MD, Ahmed J, Shah M (2008) Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: IEEE conference on computer vision and pattern recognition, 2008 (CVPR 2008), pp 1–8Google Scholar
- 52.Soomro K, Zamir AR, Shah M (2012) UCF101: a dataset of 101 human actions classes from videos in the wild, arXiv preprint arXiv:1212.0402
- 53.Eltoukhy MM, Elhoseny M, Hosny KM, Singh AK (2018) Computer aided detection of mammographic mass using exact Gaussian–Hermite moments. J Ambient Intell Humaniz Comput 1–9. https://doi.org/10.1007/s12652-018-0905-1
- 55.Li J, Liu L, Zhou M, Yang J-J, Chen S, Liu H et al (2018) Feature selection and prediction of small-for-gestational-age infants. J Ambient Intell Humaniz Comput 1–15. https://doi.org/10.1007/s12652-018-0892-2
- 56.Li D, Qiu Z, Dai Q, Yao T, Mei T (2018) Recurrent tubelet proposal and recognition networks for action detection. In: Proceedings of the European conference on computer vision (ECCV), pp 303–318Google Scholar