Advertisement

Deep Learning in Gait Analysis for Security and Healthcare

  • Omar Costilla-ReyesEmail author
  • Ruben Vera-Rodriguez
  • Abdullah S. Alharthi
  • Syed U. Yunas
  • Krikor B. Ozanyan
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 865)

Abstract

Human motion is an important spatio-temporal pattern as it can be a powerful indicator of human well-being and identity. In particular, human gait offers a unique motion pattern of an individual. Gait refers to the study of locomotion in both humans and animals. It involves the coordination of several parts of the human body: the brain, the spinal cord, the nerves, muscles, bones, and also joints. Gait analysis has been studied for a variety of applications including healthcare, biometrics, sports, and many others. Until recently, the analysis has been done mainly by human observation, using parameters and features established in existing practice and therefore limited by the nature of measurements captured by the gait sensing modalities. In this chapter, we reviewed key conceptual and algorithmic facets of deep learning applied to gait analysis in two important contexts: security and healthcare.

Keywords

Deep learning Gait analysis Biometrics Dual-task Machine learning 

Notes

Acknowledgements

We express our gratitude to the participants for taking the time to participate in this research and to David H. Foster for useful discussions. This work was supported by the U.K. Engineering and Physical Sciences Research Council EP/K005294/1 EP/K503447/1, in part by CONACyT (Mexico), grant 467373 and in part by the University of Manchester Data Science Institute. O. Costilla-Reyes would like to acknowledge CONACyT (Mexico) for a studentship. We acknowledge NVIDIA for the donation of the GPU used to perform some of the experiments of this research.

References

  1. 1.
    Michael, W., Whittle. Gait Analysis: An Introduction. Butterworth, Heinemann, (2014)Google Scholar
  2. 2.
    Nadkarni, N.K., Mawji, E., McIlroy W.E, Black S.E.: Spatial and temporal gait parameters in Alzheimer’s disease and aging. Gait and Posture 30(4), 452–454 (2009)Google Scholar
  3. 3.
    Hollman, J.H., Kovash, F.M., Kubik, J.J., Linbo, R.A.: Age-related differences in spatiotemporal markers of gait stability during dual task walking. Gait and Posture 26(1), 113–119 (2007)CrossRefGoogle Scholar
  4. 4.
    Beurskens, R., Bock, O. Age-related deficits of dual-task walking: a review. Neural Plast. 2012 (2012)Google Scholar
  5. 5.
    Hausdorff, J.M., Schweiger, A., Herman, T., Yogev-Seligmann, G., Giladi, N.: Dual-task decrements in gait: contributing factors among healthy older adults. J. Gerontol. Ser. A: Biol. Sci. Med. Sci.63(12), 1335–1343 (2008)Google Scholar
  6. 6.
    Barnes, D.E., Yaffe, K.: The projected effect of risk factor reduction on Alzheimer’s disease prevalence. Lancet Neurol. 10(9), 819–828 (2011)Google Scholar
  7. 7.
    Lundin-Olsson, L., Nyberg, L., Gustafson, Y.: Stops walking when talking as a predictor of falls in elderly people. Lancet 349(9052), 617 (1997)Google Scholar
  8. 8.
    Costilla-Reyes, O., Vera-Rodriguez, R., Scully, P., Ozanyan, K.B.: Analysis of Spatio-temporal representations for robust footstep recognition with deep residual neural networks. IEEE Trans. Pattern Anal. Mach. Intell.41(2), 285–296 (2018)Google Scholar
  9. 9.
    Vacca, J.R.. Biometric Technologies and Verification Systems. Butterworth, Heinemann (2007)Google Scholar
  10. 10.
    P. Daphne Tsatsoulis, Jaech, A., Batie, R., Savvides, M.. Continuous authentication using biometrics. IGI Global, 68–88 (2012)Google Scholar
  11. 11.
    Muro-de-la Herran, A., Garcia-Zapirain, B., Mendez-Zorrilla, A.: Gait analysis methods: an overview of wearable and non-wearable systems, highlighting clinical applications. Sensors 14(2), 3362–3394 (2014)Google Scholar
  12. 12.
    Eric Mason, J., Traoré, I., Woungang, I.: Machine Learning Techniques for Gait Biometric Recognition. Springer (2016)Google Scholar
  13. 13.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  14. 14.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M. et al.: Imagenet large scale visual recognition challenge. arXiv preprint arXiv:1409.0575 (2014)
  15. 15.
    J.M. Montepare, Goldstein, S.B.: The identification of emotions from gait information. J. Nonverbal Behav. 11(1), 33–42 (2016)Google Scholar
  16. 16.
    Martina, Z., Simon,H., Wilkowska, W.: When Your Living Space Knows What You Do: Acceptance of Medical Home Monitoring by Different Technologies. Springer (2011)Google Scholar
  17. 17.
    Alharthi, A.S., Yunas, S.U., Ozanyan, K.B.: Deep learning for monitoring of human gait: a review. IEEE Sens. J. (submitted) (2019)Google Scholar
  18. 18.
    Connor, P., Ross, A.: Biometric recognition by gait: a survey of modalities and features. Comput. Vis. Image Underst. 167, 1–27 (2018)CrossRefGoogle Scholar
  19. 19.
    El-Alfy, H., Mitsugami, I., Yagi, Y.: A new gait-based identification method using local Gauss maps. In: Asian Conference on Computer Vision, pp. 3–18. Springer (2014)Google Scholar
  20. 20.
    Ioannidis, D., Tzovaras, D., Damousis, I.G., Argyropoulos, S., Moustakas, K.: Gait recognition using compact feature extraction transforms and depth information. IEEE Trans. Inf. Forensics Secur. 2.3, 623–630 (2007)Google Scholar
  21. 21.
    Arora, P., Srivastava, S., Arora, K., Bareja, S.: Improved gait recognition using gradient histogram Gaussian image. Procedia Comput. Sci. 58, 408–413 (2015)CrossRefGoogle Scholar
  22. 22.
    Liu, Y., Zhang, J., Wang, C. ,Wang, L.: Multiple HOG templates for gait recognition.2012 21st International Conference on Pattern Recognition (ICPR), pp. 2930–2933. IEEE (2012)Google Scholar
  23. 23.
    Sivapalan, S., Chen, D., Denman, S., Sridharan, S., Fookes, C.: Gait energy volumes and frontal gait recognition using depth images. In: 2011 International Joint Conference on Biometrics (IJCB), pp. 1–6. IEEE (2011)Google Scholar
  24. 24.
    Costilla-Reyes, O., Scully, P., Ozanyan, K.B.: Temporal pattern recognition in gait activities recorded with a footprint imaging sensor system. IIEEE Sens. J. 16(24), 8815–8822 (2016)Google Scholar
  25. 25.
    Costilla-Reyes, O., Scully, P., Ozanyan, K.B.: Deep neural networks for learning spatio-temporal features from tomography sensors. IEEE Trans. Ind. Electron. 65(1), 645–653 (2018)Google Scholar
  26. 26.
    Cattin, P.C.: Biometric authentication system using human gait. PhD thesis. Diss., ETH Zurich, Nr. 14603, pp. 1–140 (2002)Google Scholar
  27. 27.
    Stevenson, J.P., Firebaugh, S.L., Charles, H.K.. Biometric identification from a floor based PVDF sensor array using hidden Markov models. Proc. SAS 7 (2007)Google Scholar
  28. 28.
    Mostayed, A., Kim, S., Mazumder, M.M.G., Park, S.J.: Foot step based person identification using histogram similarity and wavelet decomposition. In: Proceedings of the 2nd International Conference on Information Security and Assurance, , pp. 307–311. IEEE (2008)Google Scholar
  29. 29.
    Vera-Rodriguez, R., Mason, J.S.D., Fierrez, J., Ortega-Garcia, J.: Comparative analysis and fusion of spatiotemporal information for footstep recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(4), 823–834 (2013)Google Scholar
  30. 30.
    Zhong, Y., Deng, Y.: Sensor orientation invariant mobile gait biometrics. In: 2014 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–8. IEEE (2014)Google Scholar
  31. 31.
    Zhang, Y., Pan, G., Jia, K., Lu, M., Wang, Y., Wu, Z.: Accelerometer based gait recognition by sparse representation of signature points with clusters’. IEEE Trans. Cybern. 45(9), 1864–1875 (2015)Google Scholar
  32. 32.
    Bours, P., Shrestha, R.: Eigensteps: a giant leap for gait recognition. In: 2010 2nd International Workshop on Security and Communication Networks (IWSCN), pp. 1–6. IEEE (2010)Google Scholar
  33. 33.
    Gafurov, D., Snekkenes, E., Bours, P.: Improved gait recognition performance using cycle matching. In: 2010 IEEE 24th International Conference on Advanced Information Networking and Applications Workshops (WAINA), pp. 836–841. IEEE (2010)Google Scholar
  34. 34.
    Rong, L., Jianzhong, Z., Ming, L., Xiangfeng, H.: A wearable acceleration sensor system for gait recognition. In: 2007 2nd IEEE Conference on Industrial Electronics and Applications, ICIEA 2007, pp. 2654–2659. IEEE (2007)Google Scholar
  35. 35.
    Zifeng, W., Huang, Y., Wang, L., Wang, X., Tan, T.: A comprehensive study on cross-view gait based human identification with deep CNNs. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 209–226 (2017)CrossRefGoogle Scholar
  36. 36.
    Costilla-Reyes, O., Vera-Rodriguez, R., Scully, P., Ozanyan, K.B.: Spatial footstep recognition by convolutional neural networks for biometric applications. In: Proceedings of IEEE SENSORS 2016. IEEE (2016)Google Scholar
  37. 37.
    Yun, J.: User identification using gait patterns on UbiFloorII. Sensors 11(3), 2611–2639 (2011)CrossRefGoogle Scholar
  38. 38.
    Claude Cattin, P.: Biometric authentication system using human gait. PhD thesis. PhD dissertation Technische Wissenschaften ETH Zurich Nr. 14603 (2002)Google Scholar
  39. 39.
    Kanth Saripalle, S.: Classification of human postural and gestural movements using center of pressure parameters derived from force platforms. PhD thesis. University of Missouri- Kansas City (2010)Google Scholar
  40. 40.
    Headon, R., Curwen, R.: Recognizing movements from the ground reaction force. In: Proceedings of the 2001 Workshop on Perceptive User Interfaces, pp. 1–8. ACM (2001)Google Scholar
  41. 41.
    Simeone, O.: A very brief introduction to machine learning with applications to communication systems. In: IEEE Transactions on Cognitive Communications and Networking (2018)Google Scholar
  42. 42.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web (1999)Google Scholar
  43. 43.
    Zhao, W., Krishnaswamy, A., Chellappa, R., Swets, D.L., Weng, J.: Discriminant analysis of principal components for face recognition. In: Face Recognition, pp. 73–85. Springer (1998)Google Scholar
  44. 44.
    Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)Google Scholar
  45. 45.
    Yi, N., Li, C., Feng, X., Shi, M.: Research and improvement of convolutional neural network. In: 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS), pp. 637–640. IEEE (2018)Google Scholar
  46. 46.
    Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. Adv. Neural Inf. Process. Syst., 568–576 (2014)Google Scholar
  47. 47.
    Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634 (2015)Google Scholar
  48. 48.
    Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3d convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)Google Scholar
  49. 49.
    Feichtenhofer, C., Pinz, A., Wildes, R.: Spatiotemporal residual networks for video action recognition. Adv. Neural Inf. Process. Syst., 3468–3476 (2016)Google Scholar
  50. 50.
    Ji, S., Wei, X., Yang, M., Kai, Y.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)CrossRefGoogle Scholar
  51. 51.
    Wang, X., Farhadi, A., Gupta, A.: Actions—transformations. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2658–2667 (2016)Google Scholar
  52. 52.
    Gafurov, D.: A survey of biometric gait recognition: approaches, security and challenges. In: Annual Norwegian Computer Science Conference, pp. 19–21. Citeseer (2007)Google Scholar
  53. 53.
    Montero-Odasso, M., Muir, S.W., Speechley, M.: Dual-task complexity affects gait in people with mild cognitive impairment: the interplay between gait variability, dual tasking, and risk of falls. Arch. Phys. Med. Rehabil. 93(2), 293–299 (2012)Google Scholar
  54. 54.
    Owings, T.M., Grabiner, M.D.: Measuring step kinematic variability on an instrumented treadmill: how many steps are enough? J. Biomech. 36(8), 1215–1218 (2003)Google Scholar
  55. 55.
    Atkinson, H.H., Rosano, C., Simonsick, E.M., Williamson, J.D., Davis, C., Ambrosius, W.T., S.R. Rapp, Cesari, M., Newman, A.B., Harris, T.B.: Cognitive function, gait speed decline, and comorbidities: the health, aging and body composition study. J. Gerontol. Ser. A: Biol. Sci. Med. Sci. 62(8), 844–850 (2007)Google Scholar
  56. 56.
    Powers, D.M.W.: Evaluation: from precision, recall and F-measure to Roc, informedness, markedness & correlation. J. Mach. Learn. Technol. 2(1), 37–63 (2011)MathSciNetGoogle Scholar
  57. 57.
    Gunn-Moore, D., Kaidanovich-Beilin, O., Iradi, M.C.G., Gunn-Moore, F., Lovestone, S.: Alzheimer’s disease in humans and other animals: a conseREFERENCES 33 quence of postreproductive life span and longevity rather than aging. Alzheimer’s Dement.: J. Alzheimer’s Assoc. 14(2), 195–204 (2018)Google Scholar
  58. 58.
    Allan, L.M., Ballard, C.G., Burn, D.J., Anne Kenny, R.: Prevalence and severity of gait disorders in Alzheimer’s and non-Alzheimer’s dementias. J. Am. Geriatr. Soc. 53(10), 1681–1687 (2005)Google Scholar
  59. 59.
    Middleton, L., Buss, A., Bazin, A., Nixon, M.: A floor sensor system for gait recognition. In: Proceedings of Fourth IEEE Workshop on Automatic Identification Advanced Technologies, pp. 171–176 (2005)Google Scholar
  60. 60.
    Vera-Rodriguez, R., Fierrez, J., Mason, J.S.D., Ortega-Garcia, J.: A novel approach of gait recognition through fusion with footstep information. In: Proceedings IAPR International Conference on Biometrics, ICB (2013)Google Scholar
  61. 61.
    Qian, G., Zhang, J., Kidane, A.: People identification using floor pressure sensing and analysis. IEEE Sens. J. 10(9), 1447–1460 (2010)CrossRefGoogle Scholar
  62. 62.
    He, K., Zhang, X., Ren, S., Sun,J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  63. 63.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 448–456 (2015)Google Scholar
  64. 64.
    Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, vol. 15, pp. 315–323 (2011)Google Scholar
  65. 65.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Proceedings of European Conference on Computer Vision, pp. 630–645. Springer (2016)Google Scholar
  66. 66.
    Dauphin, Y., de Vries, H., Bengio, Y.: Equilibrated adaptive learning rates for non-convex optimization. Adv. Neural Inf. Process. Syst. , 1504–1512 (2015)Google Scholar
  67. 67.
    Russakovsky, O., Deng, J., Hao, S.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  68. 68.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICRL), pp. 1–14 (2015)Google Scholar
  69. 69.
    Martin, A., Doddington, G., Kamm, T., Ordowski, M., Przybocki, M.: The DET curve in assessment of detection task performance. Tech. Rep, DTIC (1997)Google Scholar
  70. 70.
    Lawson, J., Murray, M., Zamboni, G., Koychev, I.G., Ritchie, C.W., Ridha, B.H., Rowe, J.B., Thomas, A., Ffytche, D.H., Howard, R.J.: Deep and frequent phenotyping: a feasibility study for experimental medicine in dementia. Alzheimer’s Dement.: J. Alzheimer’s Assoc. 13(7), pp. 1268–1269 (2017)Google Scholar
  71. 71.
    Gledson, A., Asfiandy, D., Mellor, J., Ba-Dhfari, T.O.F., Stringer, G., Couth, S., Burns, A., Leroi, I., Zeng, X., Keane, J.: Combining mouse and keyboard events with higher level desktop actions to detect mild cognitive impairment. In: 2016 IEEE International Conference on Healthcare Informatics (ICHI), pp. 139–145. IEEE (2016)Google Scholar
  72. 72.
    Aharon Satt, A., Sorin, A., Hoory, R., Toledo-Ronen, O., Derreumaux, A., Manera, V., Verhey, F., Aalten, P., Robert, P.H.: Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimer’s Dement.: Diagn., Assess. Dis. Monit. 1(1), 112–124 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Omar Costilla-Reyes
    • 1
    Email author
  • Ruben Vera-Rodriguez
    • 2
  • Abdullah S. Alharthi
    • 3
  • Syed U. Yunas
    • 3
  • Krikor B. Ozanyan
    • 3
  1. 1.Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUSA
  2. 2.Biometrics and Data Pattern Analytics (BiDA) Lab - ATVSUniversidad Autonoma de MadridMadridSpain
  3. 3.School of Electrical and Electronic EngineeringThe University of ManchesterManchesterUK

Personalised recommendations