Abstract
Rehabilitation robots are gaining significant popularity for impaired gait rehabilitation. However, to make the recovering individual feel natural while walking and restore their original gait pattern, adapting the rehabilitation system according to the individual’s need and walking characteristics becomes imperative. In this paper, we have compared four deep learning models for their ability to generate a personalized gait trajectory at different gait speeds. The first three models are primitive and are the basic implementations of long short term memory (LSTM), convolutional neural network (CNN) and gated recurrent unit (GRU). The fourth model is our proposed model, which is a sequential combination of LSTM and CNN. We considered hip, knee and ankle joints data as human gait is represented as the joint angle trajectories of these joints in the sagittal plane. We trained these models on a benchmark public human walking dataset consisting of treadmill walking data of 42 healthy individuals at eight different walking speeds. Anthropometric and demographic data along with gait speeds were given as input to the models. Our proposed LSTM-CNN sequential model is able to generate stable gait trajectories in the speed range of 0.49-1.76 m/s with a high correlation of 0.98 between the actual and the predicted trajectories, and an R2 Score of 0.94 is obtained. This work can be utilized for providing personalized gait reference trajectories for the rehabilitation of amputees and stroke patients using rehabilitation systems such as exoskeleton robots and prosthetic legs. Also, this work can be utilized for generating stable walking trajectories for bipedal robots.
Similar content being viewed by others
Data Availability
The dataset analysed during the current study is available in the figshare repository, https://doi.org/10.6084/m9.figshare.5722711. No dataset is generated in this study.
References
Aertbeliën E, De Schutter J (2014) Learning a predictive model of human gait for the control of a lower-limb exoskeleton. In: 5th IEEE RAS/EMBS international conference on biomedical robotics and biomechatronics. IEEE, pp 520–525
Banbury CR, Reddi VJ, Lam M, Fu W, Fazel A, Holleman J, Huang X, Hurtado R, Kanter D, Lokhmotov A et al (2020) Benchmarking tinyml systems: challenges and direction. arXiv:2003.04821
Chao H, He Y, Zhang J, Feng J (2019) Gaitset: regarding gait as a set for cross-view gait recognition. Proc AAAI Conf Artif Intell 33(01):8126–8133
Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555
Fang B, Zhou Q, Sun F, Shan J, Wang M, Xiang C, Zhang Q (2020) Gait neural network for human-exoskeleton interaction. Frontiers in Neurorobotics, pp 58
Findlow A, Goulermas J, Nester C, Howard D, Kenney L (2008) Predicting lower limb joint kinematics using wearable motion sensors. Gait & Posture 28(1):120–126
Fukuchi CA, Fukuchi RK, Duarte M (2018) A public dataset of overground and treadmill walking kinematics and kinetics in healthy individuals. PeerJ 6:e4640
Fukuchi CA, Fukuchi RK, Duarte M (2019) Test of two prediction methods for minimum and maximum values of gait kinematics and kinetics data over a range of speeds. Gait & Posture 73:269–272
Gholami M, Napier C, Menon C (2020) Estimating lower extremity running gait kinematics with a single accelerometer: a deep learning approach. Sensors 20(10):2939
Hochreiter S (1998) The vanishing gradient problem during learning recurrent neural nets and problem solutions. Internat J Uncertain Fuzziness Knowledge-Based Systems 6(02):107–116
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Holanda LJ, Silva PM, Amorim TC, Lacerda MO, Simão CR, Morya E (2017) Robotic assisted gait as a tool for rehabilitation of individuals with spinal cord injury: a systematic review. J Neuroeng Rehabili 14(1):1–7
Horst F, Lapuschkin S, Samek W, Müller K-R, Schöllhorn WI (2019) Explaining the unique nature of individual gait patterns with deep learning. Sci Rep 9(1):1–13
Jain R (2022) Stride segmentation of inertial sensor data using statistical methods for different walking activities. Robotica 40(8):2567–2580
Jain R, Semwal VB, Kaushik P (2022) Deep ensemble learning approach for lower extremity activities recognition using wearable sensors. Expert Syst 39(6):e12743. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.12743
LeCun Y, Bengio Y., Hinton G (2015) Deep learning. Nature 521(7553):436–444
Liang F-Y, Zhong C-H, Zhao X, Castro DL, Chen B, Gao F, Liao W-H (2018) Online adaptive and lstm-based trajectory generation of lower limb exoskeletons for stroke rehabilitation. In: 2018 IEEE international conference on robotics and biomimetics (ROBIO), pp 27–32
Louie DR, Eng JJ (2016) Powered robotic exoskeletons in post-stroke rehabilitation of gait: a scoping review. J Neuroeng Rehabili 13(1):1–10
Luu TP, Low K, Qu X, Lim H, Hoon K (2014) An individual-specific gait pattern prediction model based on generalized regression neural networks. Gait & Posture 39(1):443–448
McGrath RL, Pires-Fernandes M, Knarr B, Higginson JS, Sergi F (2017) Toward goal-oriented robotic gait training: The effect of gait speed and stride length on lower extremity joint torques. In: 2017 international conference on rehabilitation robotics (ICORR), IEEE, pp 270–275
Moissenet F, Leboeuf F, Armand S (2019) Lower limb sagittal gait kinematics can be predicted based on walking speed, gender, age and bmi. Sci Rep 9 (1):1–12
Morone G, Paolucci S, Cherubini A, De Angelis D, Venturiero V, Coiro P, Iosa M (2017) Robot-assisted gait training for stroke patients: current state of the art and perspectives of robotics. Neuropsychiatr Dis Treat 13:1303
Muro-De-La-Herran A, Garcia-Zapirain B, Mendez-Zorrilla A (2014) Gait analysis methods: an overview of wearable and non-wearable systems, highlighting clinical applications. Sensors 14(2):3362–3394
Ren S, Wang W, Hou Z. -G., Chen B, Liang X, Wang J, Peng L (2019) Personalized gait trajectory generation based on anthropometric features using random forest. Journal of Ambient Intelligence and Humanized Computing, pp 1–12
Su B, Gutierrez-Farewik EM (2020) Gait trajectory and gait phase prediction based on an lstm network. Sensors 20(24):7127
Vallery H, Van Asseldonk EH, Buss M, Van Der Kooij H (2008) Reference trajectory generation for rehabilitation robots: complementary limb motion estimation. IEEE Trans Neural Syst Rehabil Eng 17(1):23–30
Wu X, Liu D. -X., Liu M, Chen C, Guo H (2018) Individualized gait pattern generation for sharing lower limb exoskeleton robot. IEEE Trans Autom Sci Eng 15(4):1459–1470
Yun Y, Kim H-C, Shin SY, Lee J, Deshpande AD, Kim C (2014) Statistical method for prediction of gait kinematics with gaussian process regression. J Biomechan 47(1):186–192
Zaroug A, Lai DT, Mudie K, Begg R (2020) Lower limb kinematics trajectory prediction using long short-term memory neural networks. Front Bioeng Biotechnol 8:362
Zhou Z, Liang B, Huang G, Liu B, Nong J, Xie L (2020) Individualized gait generation for rehabilitation robots based on recurrent neural networks. IEEE Trans Neural Syst Rehabil Eng 29:273–281
Acknowledgements
The authors would like to thank the Ministry of Education, Govt. of India for funding the project under HEFA CSR grant SAN/CSR/08/2021-22. The authors also like to thank SERB, DST Govt. of India, for funding the project to Dr. Vijay Bhaskar Semwal under the Early career award (ECR) scheme, DST No: ECR/2018/000203.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
All the ethical issues have been taken care of while writing the manuscript, and we have complied with all the standards to the best of our knowledge.
Conflict of Interests
The author(s) proclaim no conflict of interest regarding this research paper with any person or organization. This manuscript is based on original research findings done by the authors themselves.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The work is funded by SERB, DST, Govt. of India to Dr. Vijay Bhaskar Semwal under Early Career Award(ECR) with DST No: ECR/2018/000203 dated 04-June-2019.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Semwal, V.B., Jain, R., Maheshwari, P. et al. Gait reference trajectory generation at different walking speeds using LSTM and CNN. Multimed Tools Appl 82, 33401–33419 (2023). https://doi.org/10.1007/s11042-023-14733-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-14733-2