Deep Learning and Sensor Fusion Methods for Studying Gait Changes Under Cognitive Load in Males and Females

  • Abdullah S. AlharthiEmail author
  • Krikor B. Ozanyan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)


Human gait is the manner of walking in people. It is influenced by weight, age, health condition or the interaction with the surrounding environment. In this work, we study gait changes under cognitive load in healthy males and females, using machine learning methods. A deep learning model with multi-processing pipelining and back propagation techniques, is proposed for cognitive load gait analysis. The IMAGiMAT floor system enabling sensor fusion from plastic optical fiber (POF) elements, is utilized to record gait raw data on spatiotemporal ground reaction force (GRF). A deep parallel Convolutional Neural Network (CNN) is engineered for POF sensors fusion, and gait GRF classification. The Layer-Wise Relevance Propagation (LRP), is applied to reveal which gait events are relevant towards informing the parallel CNN prediction. The CNN differentiates between males and females with 95% weighted average precision, and cognitive load gait classification with 93% weighted average precision. These findings present a new hypothesis, whereas larger dataset holds promise for human activity analysis.


Convolutional Neural Networks (CNN) Cognitive load gait Ground Reaction Force (GRF) Layer-Wise Relevance Propagation (LRP) 


  1. 1.
    Yogev, G., Hausdorff, J.M., Giladi, N.: The role of executive function and attention in gait. Official J. Mov. Disord. Soc. 23(3), 329–342 (2008)Google Scholar
  2. 2.
    Woollacott, M., Shumway-Cook, A.: Attention and the control of posture and gait: a review of an emerging area of research. Gait Posture 16(1), 1–14 (2002)CrossRefGoogle Scholar
  3. 3.
    Bertram, J.E.A., Ruina, A.: Multiple walking speed–frequency relations are predicted by constrained optimization. J. Theor. Biol. 209(4), 445–453 (2001)CrossRefGoogle Scholar
  4. 4.
    Holt, K.G., Jeng, S.F., Ratcliffe, R., Hamill, J.: Energetic cost and stability during human walking at the preferred stride frequency. J. Motor Behav. 27(2), 164–178 (1995)CrossRefGoogle Scholar
  5. 5.
    Selinger, J.C., O’Connor, S.M., Wong, J.D., Donelan, J.M.: Humans can continuously optimize energetic cost during walking. Curr. Biol. 25(18), 2452–2456 (2015)CrossRefGoogle Scholar
  6. 6.
    Adenzato, M., Brambilla, M., Manenti, R., et al.: Gender differences in cognitive theory of mind revealed by transcranial direct current stimulation on medial prefrontal cortex. Sci. Rep. 7(41219) (2017).
  7. 7.
    Sherwin, B.B.: Estrogen and cognitive functioning in women. Endocr. Rev. 24(2), 133–151 (2003)CrossRefGoogle Scholar
  8. 8.
    Zaidi, Z.F.: Gender differences in human brain: a review. Open Anat. J. 2, 37–55 (2010)CrossRefGoogle Scholar
  9. 9.
    Bruening, D.A., Frimenko, R.E., Goodyear, C.D., Bowden, D.R., Fullenkamp, A.M.: Sex differences in whole body gait kinematics at preferred speeds. Gait Posture 41(2), 540–545 (2015)CrossRefGoogle Scholar
  10. 10.
    Laws, K.R., Irvine, K., Gale, T.M.: Sex differences in cognitive impairment in alzheimer’s disease. World J. Psychiatry 22(1), 54–65 (2016)CrossRefGoogle Scholar
  11. 11.
    McPherson, S., Back, C., Buckwalter, J.G., Cummings, J.L.: Gender-related cognitive deficits in alzheimer’s disease. Int. Psychogeriatr. 11(2), 117–122 (1999)CrossRefGoogle Scholar
  12. 12.
    Alharthi, A.S., Yunas, S.U., Ozanyan, K.B.: Deep learning for monitoring of human gait: a review. IEEE Sens. J. (2019). Scholar
  13. 13.
    Cantoral-Ceballos, J., et al.: Intelligent carpet system, based on photonic guided-path tomography, for gait and balance monitoring in home environments. IEEE Sens. J. 15(1), 279–289 (2015)CrossRefGoogle Scholar
  14. 14.
    Suutala, J., Fujinami, K., Röning, J.: Gaussian process person identifier based on simple floor sensors. In: Roggen, D., Lombriser, C., Tröster, G., Kortuem, G., Havinga, P. (eds.) EuroSSC 2008. LNCS, vol. 5279, pp. 55–68. Springer, Heidelberg (2008). Scholar
  15. 15.
    Yun, J., Woo, W., Ryu, J.: User identification using user’s walking pattern over the ubiFloorII. In: Hao, Y., et al. (eds.) CIS 2005. LNCS (LNAI), vol. 3801, pp. 949–956. Springer, Heidelberg (2005). Scholar
  16. 16.
    Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R.: Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recogn. 65, 211–222 (2017)CrossRefGoogle Scholar
  17. 17.
    Bharadhwaj, H.: Layer-wise relevance propagation for explainable deep learning based speech recognition. In: 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), pp. 168–174 (2018)Google Scholar
  18. 18.
    Maximilian, A., et al.: iNNvestigate neural networks (2018).
  19. 19.
    Isaac, E.R.H.P., Elias, S., Rajagopalan, S., Easwarakumar, K.S.: Multiview gait-based gender classification through pose-based voting. Pattern Recogn. Lett. 126, 41–50 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Electrical and Electronic EngineeringThe University of ManchesterManchesterUK

Personalised recommendations