Skip to main content

Advertisement

SpringerLink
Log in

Non-local gait feature extraction and human identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As a new human identification technology, gait recognition is receiving more and more attention in recent years. However, traditional gait recognition techniques are limited by the challenges of feature representation and extraction algorithms. In this paper, by utilizing the self-attention mechanism, we propose a novel gait-based human identification solution. Firstly, we utilize non-local neural networks (NLNN) to extract non-local features from a pair of randomly selected gait energy maps (GEIs). Secondly, based on the relationship between GEIs and various parts of the human body, the output of NLNN is horizontally segmented into three sections, i.e., strong-dynamic region, weak-dynamic region and micro-dynamic region, respectively. Thirdly, the segmented gait features are weighted ensembled by three two-class classifiers. Finally, two experiments are carried out with the OU-ISIR large population dataset and the CASIA dataset B to evaluate the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Proceedings of the 34th international conference on machine learning, vol 70

  2. Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In: Advances in neural information processing systems, vol 29

  3. Chen Q, Wang Y, Liu Z, Liu Q, Huang D (2017) Feature map pooling for cross-view gait recognition based on silhouette sequence images. In: IEEE international joint conference on biometrics (IJCB), pp 54–61

  4. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems 27. Curran Associates, Inc, pp 2672–2680

  5. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

    MATH  Google Scholar 

  6. Hagui M, Mahjoub MA (2016) Hidden conditional random fields for gait recognition. In: International image processing, applications and systems, pp 1–6

  7. Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(02):316–323

    Article  Google Scholar 

  8. Hanon AlAsadi A (2014) Gait recognition using support vector machine and neural network. J Basrah Res 40:68–78

    Google Scholar 

  9. He Y, Zhang J (2018) Deep learning for gait recognition: a survey. Pattern Recognit Artif Intell 31(05):442–451

    Google Scholar 

  10. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  11. Iwama H, Okumura M, Makihara Y, Yagi Y (2012) The OU-ISIR gait database comprising the large population dataset and performance evaluation of gait recognition. IEEE Trans Inf Forensics Secur 7(5):1511–1521

    Article  Google Scholar 

  12. Kanwar A, Upadhyay P (2014) An appearance based approach for gait identification using infrared imaging. In: International conference on issues and challenges in intelligent computing techniques (ICICT), pp 719–724

  13. Kingma D P, Welling M (2014) Auto-encoding variational bayes. In: 2nd International conference on learning representations, vol 1

  14. Kozlow P, Abid N, Yanushkevich S N (2018) Gait type analysis using dynamic bayesian networks. Sensors 18(10):3329–3338

    Article  Google Scholar 

  15. Krajushkina A, Nõmm S, Toomela A, Medijainen K, Tamm E, Vaske M, Uvarov D, Kahar H, Nugis M, Taba P (2018) Gait analysis based approach for parkinson’s disease modeling with decision tree classifiers. In: IEEE International conference on systems, man, and cybernetics, vol 10, pp 3720–3725

  16. Krizhevsky A, Sutskever I, Hinton G E (2017) ImageNet classification with deep convolutional neural networks. CACM 60(6):84–90

    Article  Google Scholar 

  17. Kusakunniran W, Wu Q, Li H, Zhang J (2010) Multiple views gait recognition using view transformation model based on optimized gait energy image. In: IEEE International conference on information and automation, pp 1058–1064

  18. Lam T, Cheung K H, Liu J (2011) Gait flow image: a silhouette-based gait representation for human identification. Pattern Recognit 44:973–987

    Article  Google Scholar 

  19. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7):436–445

    Article  Google Scholar 

  20. Manap HH, Tahir NM, Abdullah R (2012) Anomalous gait detection using naive bayes classifier. In: IEEE symposium on industrial electronics and applications, pp 378–381

  21. Mao X, Li Q, Xie H, Lau RYK, Wang Z, Smolley SP (2017) Least squares generative adversarial networks. In: 2017 IEEE international conference on computer vision (ICCV), pp 2813–2821

  22. Muramatsu D, Makihara Y, Yagi Y (2015) Cross-view gait recognition by fusion of multiple transformation consistency measures. IET Biom 4 (2):62–73

    Article  Google Scholar 

  23. Muramatsu D, Makihara Y, Yagi Y (2016) View transformation model incorporating quality measures for cross-view gait recognition. IEEE Trans Cybern 46(7):1602–1615

    Article  Google Scholar 

  24. Pan Z, Yu W, Yi X, Khan A, Yuan F, Zheng Y (2019) Recent progress on generative adversarial networks (GANs): a survey. IEEE Access 7:36322–36333

    Article  Google Scholar 

  25. San-Segundo R, Cordoba R, Ferreiros J, D’Haro-Enríquez LF (2016) Frequency features and GMM-UBM approach for gait-based person identification using smartphone inertial signals. Pattern Recogn Lett 73(C):60–67

    Article  Google Scholar 

  26. Sarkar S, Phillips P, Liu Z (2005) The humanid gait challenge problem: data sets, performance, and analysis. IEEE Trans Pattern Anal Mach Intell 27 (02):162–177

    Article  Google Scholar 

  27. Shiraga K, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2016) Geinet: view-invariant gait recognition using a convolutional neural network. In: International conference on biometrics (ICB), vol 1, pp 1–8

  28. Sonderby CK, Raiko T, Maaloe L, Sonderby S K, Winther O (2016) Ladder variational autoencoders. In: Advances in neural information processing systems, vol 29

  29. Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Trans Circ Syst Video Technol 1(1):1–1

    Google Scholar 

  30. Tong S, Fu Y, Yue X, Ling H (2018) Multi-view gait recognition based on a spatial-temporal deep neural network. IEEE Access 6:57583–57596

    Article  Google Scholar 

  31. Tsunashima H, Hoshi T, Chen Q (2018) DzGAN: improved conditional generative adversarial nets using divided Z-vector. In: 2018 International conference on computing and big data. International conference on computing and big data, Coll Charleston, Charleston, SC, SEP 08-10, 2018, pp 52–55

  32. Wang X, Yan W Q (2019) Cross-view gait recognition through ensemble learning. In: Neural computing and applications

  33. Wang X, Yan W Q (2020) Human gait recognition based on frame-by-frame gait energy images and convolutional long short term memory. Int J Neural Syst 30(1):1–12

    Article  Google Scholar 

  34. Wang X, Wang J, Yan K (2018) Gait recognition based on Gabor wavelets and (2D)2PCA. Multimed Tools Appl 77(10):12545–12561

    Article  Google Scholar 

  35. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: IEEE/CVF conference on computer vision and pattern recognition, pp 7794–7803

  36. Wang X, Feng S, Yan W Q (2019) Human gait recognition based on self-adaptive hidden Markov model. In: IEEE transactions on computational biology and bioinformatics, pp 1–10

  37. Wang X, Zhang J, Yan W Q (2019) Gait recognition using multichannel convolution neural networks. In: Neural computing and applications, pp 532–539

  38. Wu Z, Huang Y, Wang L, Wang X, Tan T (2017) A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Trans Pattern Anal Mach Intell 39(02):209–226

    Article  Google Scholar 

  39. Wu H, Weng J, Chen X, Lu W (2018) Feedback weight convolutional neural network for gait recognition. J Vis Commun Image Represent 55:424–432

    Article  Google Scholar 

  40. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: International conference on pattern recognition, pp 441–444

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant No.61602431 and Zhejiang Provincial Natural Science Foundation of China under Grant No.Y20F020113, as well as a scholarship from the China Scholarship Council.

Author information

Authors and Affiliations

  1. Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, No. 258, Xueyuan Street, Hangzhou, 310018, China

    Xiuhui Wang

  2. Auckland University of Technology, No. 2-14, Wakefiled Street, Auckland, 1010, New Zealand

    Wei Qi Yan

Authors
  1. Xiuhui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Qi Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiuhui Wang.

Ethics declarations

Conflict of interest

We declare that we have not financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled “Non-local Gait Feature Extraction and Human Identification”.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Yan, W.Q. Non-local gait feature extraction and human identification. Multimed Tools Appl 80, 6065–6078 (2021). https://doi.org/10.1007/s11042-020-09935-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09935-x

Keywords

Search

Navigation