Multimodal Feature Learning for Gait Biometric Based Human Identity Recognition

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8227)


In this paper we propose a novel multimodal feature learning technique based on deep learning for gait biometric based human-identification scheme from surveillance videos. Experimental evaluation of proposed learning features based on novel deep learning and standard (PCA/LDA) features in combination with classifier techniques (NN/MLP/SVM/SMO) on different datasets from two gait databases (the publicly available CASIA multiview multispectral database, and the UCMG multiview database), show a significant improvement in recognition accuracies with proposed fused deep learning features.


multimodal multiview gait vPCA Deep Learning identification fusion 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zheng, S.: CASIA Gait Database collected by Institute of Automation, Chinese Academy of Sciences, CASIA Gait Database,
  2. 2.
    Huang, L.: Person Recognition By Feature Fusion, Dept. of Engineering Technology Metropolitan State College of Denver Denver. IEEE, USA (2011)Google Scholar
  3. 3.
    Bringer, J., Chabanne, H.: Biometric Identification Paradigm Towards Privacy and Confidentiality Protection, Biometric: Theory. In: Nichols, E.R. (ed.) Application and Issues, pp. 123–141 (2011)Google Scholar
  4. 4.
    Jain, A.K.: Next Generation Biometrics, Department of Computer Science & Engineering. Michigan State University, Department of Brain & Cognitive Engineering, Korea University (2009)Google Scholar
  5. 5.
    Yampolskiy, R.V., Govindaraja, V.: Taxonomy of Behavioral Biometrics. In: Behavioral Biometrics for Human Identification, pp. 1–43 (2010)Google Scholar
  6. 6.
    Meraoumia, A., Chitroub, S., Bouridane, A.: Fusion of Finger-Knuckle-Print and Palmprint for an Efficient Multi-biometric System of Person Recognition. In: IEEE Communications Society Subject Matter Experts for Publication in the IEEE ICC (2011)Google Scholar
  7. 7.
    Berretti, S., Bimbo, A., Pala, P.: 3D face recognaition using isogeodesic stripes. IEEE Transaction on Pattern Analysis and Machine Intelligence 32(12) (2010)Google Scholar
  8. 8.
    Yuan, L., Mu, Z., Xu, Z.: Using Ear Biometrics for Personal Recognition. School of Information Engineering, Univ. of Science and Technology Beijing, Beijing 100083, yuanli64@hotmail.comGoogle Scholar
  9. 9.
    Ross, A., Jain, A.K.: Information fusion in biometrics. Pattern Recognition Letters 24, 2115–2125 (2003)CrossRefGoogle Scholar
  10. 10.
    Jain, A.K., Hong, L., Kulkarni, Y.: A multimodal biometric system using fingerprints, face and speech. In: 2nd Int’l Conf. AVBPA, vol. 10, pp. 182–187 (1999)Google Scholar
  11. 11.
    Wang, Y., Tan, T., Jain, A.K.: Combining face and iris biometrics for identity verification. In: Int’l Conf. AVBPA, pp. 805–813 (2003)Google Scholar
  12. 12.
    Chang, K., et al.: Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics. IEEE Trans. PAMI 25, 1160–1165 (2003)CrossRefGoogle Scholar
  13. 13.
    Kittler, J., et al.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20, 226–239 (1998)CrossRefGoogle Scholar
  14. 14.
    Smith, L.I.: A tutorial on Principal Components AnalysisGoogle Scholar
  15. 15.
    Linear discriminant analysis, Wikipedia,
  16. 16.
    Multi Layer Perceptron,
  17. 17.
    Platt, J.C.: Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines, Microsoft Research,, Technical Report MSR-TR-98-14 (17) (1998)Google Scholar
  18. 18.
    Shlizerman, I.K., Basri, R.: 3D Face Reconstruction from a Single Image Using a Single Reference Face Shape. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(2) (2011)Google Scholar
  19. 19.
    Hossain, E., Chetty, G.: Multimodal Identity Verification Based on Learning Face and Gait Cues. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) ICONIP 2011, Part III. LNCS, vol. 7064, pp. 1–8. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  20. 20.
    Chin, Y.J., Ong, T.S., Teoh, A.B.J., Goh, M.K.O.: Multimodal Biometrics based Bit Extraction Method for Template Security, Faculty of Information Science and Technology, Multimedia University, Malaysia, School of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea. IEEE (2011)Google Scholar
  21. 21.
    Multilayer Perceptron Neural Networks, The Multilayer Perceptron Neural Network Model,
  22. 22.
    UCMG Gait Database (to be made publicly available),
  23. 23.
    Hinton, G.E.: To recognize shapes, first learn to generate images. Progress in Brain Research 165, 535–547 (2007)CrossRefGoogle Scholar
  24. 24.
    Huo, X., Ni, X., Smith, A.K.: A survey of manifold-based learning methods. In: Recent Advances in Datamining of Enterprise Data Algorithms and Applications, pp. 691–745 (2004)Google Scholar
  25. 25.
    LeCun, Y., Chopra, S., Hadsell, R.: A tutorial on energy-based learning. In: Predicting Structured Data. MIT Press (2006)Google Scholar
  26. 26.
    Kusakunniran, W., Zhang, J., Wu, Q., Li, H.: Multiple Views Gait Recognition using View Transformation Model of Gait Energy Image. In: Second IEEE International Workshop on Tracking Humans for the Evaluation of their Motion in Image Sequences (THEMIS 2009): a workshop in ICCV 2009, pp. 1058–1064 (2009)Google Scholar
  27. 27.
    Chetty, G., Wagner, M.: Liveness Detection Using Cross Modal Correlations in Face-Voice Person Authentication. In: Proceedings Interspeech Conference, pp. 2181–2184 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.IT&E, Faculty of ESTeMUniversity of CanberraAustralia

Personalised recommendations