Spatiotemporal features of human motion for gait recognition

  • Muhammad Hassan Khan
  • Muhammad Shahid Farid
  • Marcin Grzegorzek
Original Paper


Gait is a novel biometric feature that offers human identification at a distance and without physical interaction with the imaging device. Moreover, it performs well even in low resolution which makes it ideal for use in numerous human identification applications, e.g.,visual surveillance, monitoring and access control systems. Most existing gait-based human identification solutions extract human body silhouettes, contours or shapes from the images and construct gait features. Therefore, the performance of such algorithms highly depends upon the accuracy of human body segmentation, which is still a challenging problem in the literature. In this paper, we propose a new gait recognition algorithm which uses the spatial and temporal motion characteristics of human gait for individual identification without needing the silhouette extraction. The proposed algorithm extracts a set of spatiotemporal local descriptors from the gait video sequences. The extracted descriptors are encoded using the Fisher vector encoding and Gaussian mixture model-based codebook. The encoded features are classified using a simple linear support vector machine to recognize the individuals. The proposed gait recognition method is evaluated on five widely used gait databases, including indoor (CMU MoBo, CASIA-B) and outdoor (NLPR, CASIA-C, TUM GAID) gait databases. The results reveal that our method showed excellent performance on all five databases and outperformed the state-of-the-art gait recognition approaches.


Gait recognition Spatiotemporal features Fisher vector encoding Feature evaluation 


  1. 1.
    Alotaibi, M., Mahmood, A.: Reducing covariate factors of gait recognition using feature selection and dictionary-based sparse coding. Signal Image Video Process. 11(6), 1131–1138 (2017)CrossRefGoogle Scholar
  2. 2.
    Bashir, K., Xiang, T., Gong, S.: Gait recognition using gait entropy image. In: IET ICDP, pp. 1–6 (2009)Google Scholar
  3. 3.
    Bouchrika, I., Nixon, M.: Model-based feature extraction for gait analysis and recognition. In: IEEE ICCV, pp. 150–160 (2007)Google Scholar
  4. 4.
    Castro, F., Marín-Jiménez, M., Guil, N.: Multimodal features fusion for gait, gender and shoes recognition. Mach. Vis. Appl. 27, 1213–1228 (2016)CrossRefGoogle Scholar
  5. 5.
    Castro, F.: Fisher motion descriptor for multiview gait recognition. Int. J. Pattern Recognit. Artif. Intell. 31(1), 1756002 (2017)CrossRefGoogle Scholar
  6. 6.
    Castro, F.M., Marín-Jiménez, M.J., Guil, N., Pérez de la Blanca, N.: Automatic learning of gait signatures for people identification. In: Advances in Computational Intelligence, pp. 257–270 (2017)CrossRefGoogle Scholar
  7. 7.
    Chai, Y., et al.: A novel human gait recognition method by segmenting and extracting the region variance feature. Proc. Int. Conf. Pattern Recognit. (ICPR) 4, 425–428 (2006)Google Scholar
  8. 8.
    Chen, S., Gao, Y.: An invariant appearance model for gait recognition. In: Proc. IEEE Int. Conf. Multimed. and Expo (ICME), pp. 1375–1378. IEEE (2007)Google Scholar
  9. 9.
    Choudhury, S.D., Tjahjadi, T.: Silhouette-based gait recognition using procrustes shape analysis and elliptic fourier descriptors. Pattern Recognit. 45(9), 3414–3426 (2012)CrossRefGoogle Scholar
  10. 10.
    Cunado, D., Nixon, M.S., Carter, J.N.: Automatic extraction and description of human gait models for recognition purposes. Comput. Vis. Image Underst. 90(1), 1–41 (2003)CrossRefGoogle Scholar
  11. 11.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Stat. Methodol. 39, 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Dupuis, Y., Savatier, X., Vasseur, P.: Feature subset selection applied to model-free gait recognition. Image Vis. Comput. 31(8), 580–591 (2013)CrossRefGoogle Scholar
  13. 13.
    Fan, R.E.: Liblinear: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  14. 14.
    Goffredo, M., Carter, J.N., Nixon, M.S.: Front-view gait recognition. In: 2nd IEEE Int. Conf. on Biometrics: Theory, Applications and Systems, pp. 1–6. IEEE (2008)Google Scholar
  15. 15.
    Gross, R., Shi, J.: The CMU motion of body (MoBo) database. Carnegie Mellon University (2001)Google Scholar
  16. 16.
    Guan, Y., Li, C.T.: A robust speed-invariant gait recognition system for walker and runner identification. In: IEEE Int. Conf. on Biometrics (ICB), pp. 1–8 (2013)Google Scholar
  17. 17.
    Hofmann, M., Bachmann, S., Rigoll, G.: 2.5D gait biometrics using the depth gradient histogram energy image. In: Proc. IEEE BATS Conf., pp. 399–403 (2012)Google Scholar
  18. 18.
    Khan, M.H., et al.: Automatic recognition of movement patterns in the vojta-therapy using RGB-D data. In: Proc. Int. Conf. Image Process. (ICIP), pp. 1235–1239 (2016)Google Scholar
  19. 19.
    Khan, M.H., Li, F., Farid, M.S., Grzegorzek, M.: Gait recognition using motion trajectoryanalysis. In: Proc. 10th Int. Conf. on Computer Recognition Systems (CORES), pp. 73–82. Springer (2017)Google Scholar
  20. 20.
    Kusakunniran, W.: Attribute-based learning for gait recognition using spatio-temporal interest points. Image Vis. Comput. 32(12), 1117–1126 (2014)CrossRefGoogle Scholar
  21. 21.
    Kusakunniran, W., et al.: Automatic gait recognition using weighted binary pattern on video. In: Proc. 6th IEEE AVSS, pp. 49–54 (2009)Google Scholar
  22. 22.
    Lee, H., Hong, S., Kim, E.: An efficient gait recognition based on a selective neural network ensemble. Int. J. Imaging Syst. Technol. 18(4), 237–241 (2008)CrossRefGoogle Scholar
  23. 23.
    Lee, L., Grimson, W.E.L.: Gait analysis for recognition and classification. In: 5th IEEE Int. Conf. on Automatic Face and Gesture Recognit., pp. 155–162. IEEE (2002)Google Scholar
  24. 24.
    Lishani, A.O., Boubchir, L., Khalifa, E., Bouridane, A.: Human gait recognition based on haralick features. Signal Image Video Process. 11(6), 1123–1130 (2017)CrossRefGoogle Scholar
  25. 25.
    Little, R.J.A., Rubin, D.B.: Statistical Analysis with Missing Data. Wiley Series in Probability and Statistics. Wiley, New York (1987)Google Scholar
  26. 26.
    Liu, J., et al.: Skeleton-based action recognition using spatio-temporal LSTM network with trust gates. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1028–1039 (2017)CrossRefGoogle Scholar
  27. 27.
    Liu, J.: Skeleton-based human action recognition with global context-aware attention LSTM networks. IEEE Trans. Image Process. 27(4), 1586–1599 (2018)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Loula, F.: Recognizing people from their movement. J. Exp. Psychol. Hum. Percept. 31(1), 210 (2005)CrossRefGoogle Scholar
  29. 29.
    Lu, J., Zhang, E., Jing, C.: Gait recognition using wavelet descriptors and independent component analysis. In: Int. Symposium on Neural Networks, pp. 232–237. Springer (2006)Google Scholar
  30. 30.
    Man, J., et al.: Individual recognition using gait energy image. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 316–322 (2006)CrossRefGoogle Scholar
  31. 31.
    McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, Hoboken (2004)zbMATHGoogle Scholar
  32. 32.
    Nizami, I.F., et al.: Multi-view gait recognition fusion methodology. In: 3rd IEEE Conf. on Industrial Electronics and Applications, pp. 2101–2105. IEEE (2008)Google Scholar
  33. 33.
    Peng, X., Wang, L., Wang, X., Qiao, Y.: Bag of visual words and fusion methods for action recognition: comprehensive study and good practice. Comput. Vis. Image Underst. 150, 109–125 (2016)CrossRefGoogle Scholar
  34. 34.
    Peng, X., Zou, C., Qiao, Y., Peng, Q.: Action recognition with stacked fisher vectors. In: ECCV, pp. 581–595 (2014)Google Scholar
  35. 35.
    Perronnin, F., et al.: Improving the fisher kernel for large-scale image classification. In: ECCV, pp. 143–156 (2010)Google Scholar
  36. 36.
    Raheja, J.L., Chaudhary, A., Nandhini, K., Maiti, S.: Pre-consultation help necessity detection based on gait recognition. Signal Image Video Process. 9(6), 1357–1363 (2015)CrossRefGoogle Scholar
  37. 37.
    Rida, I., Almaadeed, S., Bouridane, A.: Gait recognition based on modified phase-only correlation. Signal Image Video Process. 10(3), 463–470 (2016)CrossRefGoogle Scholar
  38. 38.
    Sánchez, J.: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Shaikh, S.H., Saeed, K., Chaki, N.: Gait recognition using partial silhouette-based approach. In: 2014 Int. Conf. on Signal Processing and Integrated Networks (SPIN), pp. 101–106 (2014)Google Scholar
  40. 40.
    Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 568–576 (2014)Google Scholar
  41. 41.
    Sivapalan, S., et al.: Gait energy volumes and frontal gait recognition using depth images. In: Proc. Int. Joint Conf. Biometrics (IJCB), pp. 1–6 (2011)Google Scholar
  42. 42.
    Stevenage, S.V., Nixon, M.S., Vince, K.: Visual analysis of gait as a cue to identity. Appl. Cogn. Psychol. 13(6), 513–526 (1999)CrossRefGoogle Scholar
  43. 43.
    Sun, C., Nevatia, R.: Large-scale web video event classification by use of fisher vectors. In: Proc. IEEE Workshop on Applications of Computer Vision (WACV), pp. 15–22. IEEE (2013)Google Scholar
  44. 44.
    Tan, D., Huang, K., Yu, S., Tan, T.: Uniprojective features for gait recognition. In: Proc. Int. Conf. Biom., pp. 673–682 (2007)Google Scholar
  45. 45.
    Tan, D., et al.: Walker recognition without gait cycle estimation. In: Proc. Int. Conf. Biom., pp. 222–231 (2007)Google Scholar
  46. 46.
    Tan, D., et al.: Efficient night gait recognition based on template matching. Proc. ICPR 3, 1000–1003 (2006)Google Scholar
  47. 47.
    Vaidya, S., Shah, K.: Real time video surveillance system. Int. J. Comput. Appl. 86(14), 22–27 (2014)Google Scholar
  48. 48.
    Varol, G., Laptev, I., Schmid, C.: Long-term temporal convolutions for action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1510–1517 (2018)CrossRefGoogle Scholar
  49. 49.
    Veeraraghavan, A., Chowdhury, A.R., Chellappa, R.: Role of shape and kinematics in human movement analysis. In: Proc. IEEE CVPR, vol. 1, pp. I–730 (2004)Google Scholar
  50. 50.
    Veeraraghavan, A., et al.: Matching shape sequences in video with applications in human movement analysis. IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1896–1909 (2005)CrossRefGoogle Scholar
  51. 51.
    Wang, C., et al.: Human identification using temporal information preserving gait template. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2164–2176 (2012)CrossRefGoogle Scholar
  52. 52.
    Wang, H.: Dense trajectories and motion boundary descriptors for action recognition. Int. J. Comput. Vis. 103(1), 60–79 (2013)MathSciNetCrossRefGoogle Scholar
  53. 53.
    Wang, L., Ning, H., Tan, T., Hu, W.: Fusion of static and dynamic body biometrics for gait recognition. IEEE Trans. Circuits Syst. Video Technol. 14(2), 149–158 (2004)CrossRefGoogle Scholar
  54. 54.
    Wang, L., Tan, T., Hu, W., Ning, H.: Automatic gait recognition based on statistical shape analysis. IEEE Trans. Image Process. 12(9), 1120–1131 (2003)MathSciNetCrossRefGoogle Scholar
  55. 55.
    Wang, L., Tan, T., Ning, H., Hu, W.: Silhouette analysis-based gait recognition for human identification. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1505–1518 (2003)CrossRefGoogle Scholar
  56. 56.
    Whytock, T., et al.: Dynamic distance-based shape features for gait recognition. J. Math. Imaging Vis. 50(3), 314–326 (2014)CrossRefGoogle Scholar
  57. 57.
    Yang, Y., Tu, D., Li, G.: Gait recognition using flow histogram energy image. In: Proc. Int. Conf. Pattern Recognit. (ICPR), pp. 444–449 (2014)Google Scholar
  58. 58.
    Yu, S., et al.: A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. Proc. Int. Conf. Pattern Recognit. (ICPR) 4, 441–444 (2006)Google Scholar
  59. 59.
    Zeng, W., et al.: Silhouette-based gait recognition via deterministic learning. Pattern Recognit. 47(11), 3568–3584 (2014)CrossRefGoogle Scholar
  60. 60.
    Zhang, E., Zhao, Y., Xiong, W.: Active energy image plus 2DLPP for gait recognition. Signal Process. 90(7), 2295–2302 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  • Muhammad Hassan Khan
    • 1
  • Muhammad Shahid Farid
    • 2
  • Marcin Grzegorzek
    • 1
  1. 1.Research Group for Pattern RecognitionUniversity of SiegenSiegenGermany
  2. 2.College of Information TechnologyUniversity of the PunjabLahorePakistan

Personalised recommendations