Journal of Central South University

, Volume 26, Issue 10, pp 2759–2770 | Cite as

Gait recognition based on Wasserstein generating adversarial image inpainting network

  • Li-min Xia (夏利民)Email author
  • Hao Wang (王浩)
  • Wei-ting Guo (郭炜婷)


Aiming at the problem of small area human occlusion in gait recognition, a method based on generating adversarial image inpainting network was proposed which can generate a context consistent image for gait occlusion area. In order to reduce the effect of noise on feature extraction, the stacked automatic encoder with robustness was used. In order to improve the ability of gait classification, the sparse coding was used to express and classify the gait features. Experiments results showed the effectiveness of the proposed method in comparison with other state-of-the-art methods on the public databases CASIA-B and TUM-Gaid for gait recognition.

Key words

gait recognition image inpainting generating adversarial network stacking automatic encoder 



针对步态识别中小面积人体遮挡问题,提出了一种基于Wasserstein GAN 的图像补全网。该网 络能够为图像中遮挡区域生成上下文一致的补全图像。为了减少噪声对特征提取的影响,采用具有鲁 棒性的堆叠自动编码器进行特征提取。为了提高分类的能力,采用稀疏编码的方法对步态特征进行表 示与分类。在公共数据集CASIA-B 和TUM-GAID 上对该方法进行了验证,并与其他方法进行了对比 试验,结果表明了该方法的有效性。


步态识别 图像补全 生成式对抗网络 堆叠自动编码器 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    DENG M, WANG C, CHENG F. Fusion of spatial-temporal and kinematic features for gait recognition with deterministic learning [J]. Pattern Recognition, 2017, 67: 186–200.CrossRefGoogle Scholar
  2. [2]
    LU W, ZONG W, XING W, BAO E. Gait recognition based on joint distribution of motion angles [J]. Journal of Visual Languages & Computing, 2014, 25(6): 754–763.CrossRefGoogle Scholar
  3. [3]
    FRANCESCO B, ALFREDO P. TGLSTM: A time based graph deep learning approach to gait recognition [J]. Pattern Recognition letters, 2019, 126(4): 132–138.Google Scholar
  4. [4]
    SUN B L, ZHANG Z, LIU X Y, HU B, ZHU T S. Self-esteem recognition based on gait pattern using Kinect [J]. Gait & Posture, 2017, 58(3): 428–432.CrossRefGoogle Scholar
  5. [5]
    YANG K, DOU Y, LV S, ZHANG F, LV Q. Relative distance features for gait recognition with Kinect [J]. Journal of Visual Communication and Image Representation, 2016, 39: 209–217.CrossRefGoogle Scholar
  6. [6]
    LÓPEZ-FERNÁNDEZ D, MADRID-CUEVAS F J, CARMONA-POYATO A, MUÑOZ-SALINA S, MEDINA-CARNICER R. A new approach for multi-view gait recognition on unconstrained paths [J]. Journal of Visual Communication and Image Representation, 2016, 38(7): 396–406.CrossRefGoogle Scholar
  7. [7]
    LUO J, TANG J, TJAHJADI T, XIAO X. Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis [J]. Pattern Recognition, 2016, 60: 361–377.CrossRefGoogle Scholar
  8. [8]
    LI W, KUO J, PENG J. Gait recognition via GEI subspace projections and collaborative representation classification [J]. Neurocomputing, 2018, 275: 1932–1945.CrossRefGoogle Scholar
  9. [9]
    DAS CHOUDHURY S, TJAHJADI T. Clothing and carrying condition invariant gait recognition based on rotation forest [J]. Pattern Recognition Letters, 2016, 80: 1–7.CrossRefGoogle Scholar
  10. [10]
    AMER A T, KHALED A, SHANABLEH T. Decision-level fusion for single-view gait recognition with various carrying and clothing conditions [J]. Image and Vision Computing, 2017, 61: 54–69.CrossRefGoogle Scholar
  11. [11]
    YU S, CHEN H, WANG Q, SHEN L, HUANG Y. Invariant feature extraction for gait recognition using only one uniform model [J]. Neurocomputing, 2017, 239: 81–93.CrossRefGoogle Scholar
  12. [12]
    WU Z, HUANG Y, WANG L. A comprehensive study on cross-view gait based human identification with deep CNNs [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(2): 209–226.CrossRefGoogle Scholar
  13. [13]
    PRATIK C, SHAMIK S, JAYANTA M. Frontal gait recognition from occluded scenes [J]. Pattern Recognition Letters, 2015, 63: 9–15.CrossRefGoogle Scholar
  14. [14]
    XIE J, XU L, CHEN E. Image denoising and inpainting with deep neural networks [C]// 26th Annual Conference on Neural Information Processing Systems 2012. Lake Tahoe, Nevada: IEEE, 2012: 341–349.Google Scholar
  15. [15]
    PATHAK D, KRAHENBUHL P, DONAHUE J, DARRELL T, EFROS A A. Context encoders: feature learning by inpainting [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 2536–2544.Google Scholar
  16. [16]
    YEH R A, CHEN C, LIM T Y, SCHWING A G, HASEGAWA-JOHNSON M, DO M N. Semantic Image Inpainting with Deep Generative Models [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 6882–6890.Google Scholar
  17. [17]
    GOODFELLOW I J, POUGET A J, MIRZA M. Generative ad-versarial nets [C]// Proceedings of the 26th Annual Conference on Neural Information Processing Systems. Montreal, Canada: IEEE, 2014: 2672–2680.Google Scholar
  18. [18]
    ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN [EB/OL]. [2018-07-18]. Google Scholar
  19. [19]
    LUO J, XU Y, TANG C, LV J. Learning inverse mapping by autoencoder based generative adversarial nets [C]// International Conference on Neural Information Processing. Germany: Springer, 2017: 207–216.Google Scholar
  20. [20]
    MOHAMMAD A K, DENNIS S M, MINHO L. Coupled generative adversarial stacked auto-encoder: CoGASA [J]. Neural Networks, 2018, 100: 1–9.CrossRefGoogle Scholar
  21. [21]
    HINTON G, SRIVASTAVA N, SWERSKY K. Neural networks for machine learning: overview of mini-batch gradient descent [EB/OL]. [2018-07-18]. Scholar
  22. [22]
    WANG Jun, ZHOU Si-chao, XIA Li-min. Human interaction recognition based on sparse representation of feature covariance matrices [J]. Journal of Central South University, 2018, 25(2): 304–314.CrossRefGoogle Scholar
  23. [23]
    AHARON M, ELAD M, BRUCKSTEIN A. k-SVD: An algorithm for designing overcomplete dictionaries for sparse representation [J]. IEEE Transactions on Signal Processing, 2006, 54(11): 4311–4322.CrossRefGoogle Scholar
  24. [24]
    LI Y, LIU S, YANG J, YANG M. Generative face completion [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017: 5892–5900.Google Scholar
  25. [25]
    LISHANI A O, BOUBCHIR L, KHALIFA E, BOURIDANE A. Human gait recognition based on Haralick features [J]. Signal Image and Video Processing, 2017, 11(6): 1123–1130.CrossRefGoogle Scholar
  26. [26]
    ALOTAIBI M, MAHMOOD A. Improved Gait recognition based on specialized deep convolutional neural networks [J]. Computer Vision and Image Understanding, 2017, 164: 103–110.CrossRefGoogle Scholar
  27. [27]
    KAREN S, ANDREW Z. Very deep convolutional networks for large-scale image recognition [C]// 3rd International Conference on Learning Representations. San Diego: ICLR, 2015: 178–184.Google Scholar
  28. [28]
    SHI D K. Gait recognition algorithm research and application platform design based on human geometric characteristics [D]. Zhejiang: Zhejiang University, 2017. (in Chinese)Google Scholar
  29. [29]
    ZHANG L, ZHANG L, TAO D, DU B. A sparse and discriminative tensor to vector projection for human gait feature representation [J]. Signal Processing, 2015, 106(C): 245 A.252.CrossRefGoogle Scholar
  30. [30]
    RIDA I, MAADEED S A, BOURIDANE A. Unsupervised feature selection method for improved human gait recognition [C]// 2015 23rd European Signal Processing Conference (EUSIPCO). Nice, New York, USA: IEEE, 2015: 1128–1132.CrossRefGoogle Scholar
  31. [31]
    CHEN X, XU J. Uncooperative gait recognition: Re-ranking based on sparse coding and multi-view hypergraph learning [J]. Pattern Recognition, 2016, 53:116–129.CrossRefGoogle Scholar

Copyright information

© Central South University Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of AutomationCentral South UniversityChangshaChina

Personalised recommendations