Skip to main content
Log in

Gait reference trajectory generation at different walking speeds using LSTM and CNN

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Rehabilitation robots are gaining significant popularity for impaired gait rehabilitation. However, to make the recovering individual feel natural while walking and restore their original gait pattern, adapting the rehabilitation system according to the individual’s need and walking characteristics becomes imperative. In this paper, we have compared four deep learning models for their ability to generate a personalized gait trajectory at different gait speeds. The first three models are primitive and are the basic implementations of long short term memory (LSTM), convolutional neural network (CNN) and gated recurrent unit (GRU). The fourth model is our proposed model, which is a sequential combination of LSTM and CNN. We considered hip, knee and ankle joints data as human gait is represented as the joint angle trajectories of these joints in the sagittal plane. We trained these models on a benchmark public human walking dataset consisting of treadmill walking data of 42 healthy individuals at eight different walking speeds. Anthropometric and demographic data along with gait speeds were given as input to the models. Our proposed LSTM-CNN sequential model is able to generate stable gait trajectories in the speed range of 0.49-1.76 m/s with a high correlation of 0.98 between the actual and the predicted trajectories, and an R2 Score of 0.94 is obtained. This work can be utilized for providing personalized gait reference trajectories for the rehabilitation of amputees and stroke patients using rehabilitation systems such as exoskeleton robots and prosthetic legs. Also, this work can be utilized for generating stable walking trajectories for bipedal robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The dataset analysed during the current study is available in the figshare repository, https://doi.org/10.6084/m9.figshare.5722711. No dataset is generated in this study.

References

  1. Aertbeliën E, De Schutter J (2014) Learning a predictive model of human gait for the control of a lower-limb exoskeleton. In: 5th IEEE RAS/EMBS international conference on biomedical robotics and biomechatronics. IEEE, pp 520–525

  2. Banbury CR, Reddi VJ, Lam M, Fu W, Fazel A, Holleman J, Huang X, Hurtado R, Kanter D, Lokhmotov A et al (2020) Benchmarking tinyml systems: challenges and direction. arXiv:2003.04821

  3. Chao H, He Y, Zhang J, Feng J (2019) Gaitset: regarding gait as a set for cross-view gait recognition. Proc AAAI Conf Artif Intell 33(01):8126–8133

    Google Scholar 

  4. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555

  5. Fang B, Zhou Q, Sun F, Shan J, Wang M, Xiang C, Zhang Q (2020) Gait neural network for human-exoskeleton interaction. Frontiers in Neurorobotics, pp 58

  6. Findlow A, Goulermas J, Nester C, Howard D, Kenney L (2008) Predicting lower limb joint kinematics using wearable motion sensors. Gait & Posture 28(1):120–126

    Article  Google Scholar 

  7. Fukuchi CA, Fukuchi RK, Duarte M (2018) A public dataset of overground and treadmill walking kinematics and kinetics in healthy individuals. PeerJ 6:e4640

    Article  Google Scholar 

  8. Fukuchi CA, Fukuchi RK, Duarte M (2019) Test of two prediction methods for minimum and maximum values of gait kinematics and kinetics data over a range of speeds. Gait & Posture 73:269–272

    Article  Google Scholar 

  9. Gholami M, Napier C, Menon C (2020) Estimating lower extremity running gait kinematics with a single accelerometer: a deep learning approach. Sensors 20(10):2939

    Article  Google Scholar 

  10. Hochreiter S (1998) The vanishing gradient problem during learning recurrent neural nets and problem solutions. Internat J Uncertain Fuzziness Knowledge-Based Systems 6(02):107–116

    Article  MATH  Google Scholar 

  11. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  12. Holanda LJ, Silva PM, Amorim TC, Lacerda MO, Simão CR, Morya E (2017) Robotic assisted gait as a tool for rehabilitation of individuals with spinal cord injury: a systematic review. J Neuroeng Rehabili 14(1):1–7

    Google Scholar 

  13. Horst F, Lapuschkin S, Samek W, Müller K-R, Schöllhorn WI (2019) Explaining the unique nature of individual gait patterns with deep learning. Sci Rep 9(1):1–13

    Article  Google Scholar 

  14. Jain R (2022) Stride segmentation of inertial sensor data using statistical methods for different walking activities. Robotica 40(8):2567–2580

    Article  Google Scholar 

  15. Jain R, Semwal VB, Kaushik P (2022) Deep ensemble learning approach for lower extremity activities recognition using wearable sensors. Expert Syst 39(6):e12743. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.12743

    Article  Google Scholar 

  16. LeCun Y, Bengio Y., Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  17. Liang F-Y, Zhong C-H, Zhao X, Castro DL, Chen B, Gao F, Liao W-H (2018) Online adaptive and lstm-based trajectory generation of lower limb exoskeletons for stroke rehabilitation. In: 2018 IEEE international conference on robotics and biomimetics (ROBIO), pp 27–32

  18. Louie DR, Eng JJ (2016) Powered robotic exoskeletons in post-stroke rehabilitation of gait: a scoping review. J Neuroeng Rehabili 13(1):1–10

    Google Scholar 

  19. Luu TP, Low K, Qu X, Lim H, Hoon K (2014) An individual-specific gait pattern prediction model based on generalized regression neural networks. Gait & Posture 39(1):443–448

    Article  Google Scholar 

  20. McGrath RL, Pires-Fernandes M, Knarr B, Higginson JS, Sergi F (2017) Toward goal-oriented robotic gait training: The effect of gait speed and stride length on lower extremity joint torques. In: 2017 international conference on rehabilitation robotics (ICORR), IEEE, pp 270–275

  21. Moissenet F, Leboeuf F, Armand S (2019) Lower limb sagittal gait kinematics can be predicted based on walking speed, gender, age and bmi. Sci Rep 9 (1):1–12

    Article  Google Scholar 

  22. Morone G, Paolucci S, Cherubini A, De Angelis D, Venturiero V, Coiro P, Iosa M (2017) Robot-assisted gait training for stroke patients: current state of the art and perspectives of robotics. Neuropsychiatr Dis Treat 13:1303

    Article  Google Scholar 

  23. Muro-De-La-Herran A, Garcia-Zapirain B, Mendez-Zorrilla A (2014) Gait analysis methods: an overview of wearable and non-wearable systems, highlighting clinical applications. Sensors 14(2):3362–3394

    Article  Google Scholar 

  24. Ren S, Wang W, Hou Z. -G., Chen B, Liang X, Wang J, Peng L (2019) Personalized gait trajectory generation based on anthropometric features using random forest. Journal of Ambient Intelligence and Humanized Computing, pp 1–12

  25. Su B, Gutierrez-Farewik EM (2020) Gait trajectory and gait phase prediction based on an lstm network. Sensors 20(24):7127

    Article  Google Scholar 

  26. Vallery H, Van Asseldonk EH, Buss M, Van Der Kooij H (2008) Reference trajectory generation for rehabilitation robots: complementary limb motion estimation. IEEE Trans Neural Syst Rehabil Eng 17(1):23–30

    Article  Google Scholar 

  27. Wu X, Liu D. -X., Liu M, Chen C, Guo H (2018) Individualized gait pattern generation for sharing lower limb exoskeleton robot. IEEE Trans Autom Sci Eng 15(4):1459–1470

    Article  Google Scholar 

  28. Yun Y, Kim H-C, Shin SY, Lee J, Deshpande AD, Kim C (2014) Statistical method for prediction of gait kinematics with gaussian process regression. J Biomechan 47(1):186–192

    Article  Google Scholar 

  29. Zaroug A, Lai DT, Mudie K, Begg R (2020) Lower limb kinematics trajectory prediction using long short-term memory neural networks. Front Bioeng Biotechnol 8:362

    Article  Google Scholar 

  30. Zhou Z, Liang B, Huang G, Liu B, Nong J, Xie L (2020) Individualized gait generation for rehabilitation robots based on recurrent neural networks. IEEE Trans Neural Syst Rehabil Eng 29:273–281

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Ministry of Education, Govt. of India for funding the project under HEFA CSR grant SAN/CSR/08/2021-22. The authors also like to thank SERB, DST Govt. of India, for funding the project to Dr. Vijay Bhaskar Semwal under the Early career award (ECR) scheme, DST No: ECR/2018/000203.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rahul Jain.

Ethics declarations

All the ethical issues have been taken care of while writing the manuscript, and we have complied with all the standards to the best of our knowledge.

Conflict of Interests

The author(s) proclaim no conflict of interest regarding this research paper with any person or organization. This manuscript is based on original research findings done by the authors themselves.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The work is funded by SERB, DST, Govt. of India to Dr. Vijay Bhaskar Semwal under Early Career Award(ECR) with DST No: ECR/2018/000203 dated 04-June-2019.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Semwal, V.B., Jain, R., Maheshwari, P. et al. Gait reference trajectory generation at different walking speeds using LSTM and CNN. Multimed Tools Appl 82, 33401–33419 (2023). https://doi.org/10.1007/s11042-023-14733-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14733-2

Keywords

Navigation