Skip to main content
SpringerLink
Log in

Adaptive Gait Trajectory and Event Prediction of Lower Limb Exoskeletons for Various Terrains Using Reinforcement Learning

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The recent development of lower limb exoskeletons has focus on assisting walking on a variety of terrains, such as level ground, stairs, and ramps. However, smooth transitions between different locomotion modes are challenging due to unknown future information. In this paper, we propose a method that adaptively generates a complete gait cycle of trajectory and event prediction for lower limb exoskeletons during the transition stage. First, we design a generative model with the variational autoencoder (VAE) structure to parameterize the gait cycles of trajectory and events into low-dimensional latent representations. Then, the gait trajectory and event prediction are generated by VAE reconstruction, stride time and gait phase alignment. Lastly, reinforcement learning (RL) is adopted to minimize the process error of continuous gait trajectory and event prediction by finding the optimal latent representations of a generative model. The experiments collected gait data from passive and active exoskeletons. The results show that our method significantly outperforms the extend Kalman filter (EKF)-based method and moving horizon estimation (MHE)-based method for both gait trajectory and event prediction tasks during transitions between and within locomotion modes. The capability of our method gives the opportunity to enrich the control mode by providing future information and contributes a valuable tool for natural and smooth transitions among different terrains for the lower limb exoskeleton.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data availibility

The data generated during the current study are available in the repository https://github.com/fengye4242/RLData.

References

  1. Sun, Y., Tang, Y., Zheng, J., Dong, D., Chen, X., Bai, L.: From sensing to control of lower limb exoskeleton: A systematic review. Annu. Rev. Control 53, 83–96 (2022)

    Article  MathSciNet  Google Scholar 

  2. Wang, D., Gu, X., Li, W., Jin, Y., Yang, M., Yu, H.: Evaluation of safety-related performance of wearable lower limb exoskeleton robot (wller): A systematic review. Robot. Auton. Syst., 104308 (2022)

  3. Sun, L., Jing, J., Li, C., Lu, R.: Multi-terrains assistive force parameter optimization method for soft exoskeleton. IEEE Trans. Neural Syst. Rehabil. Eng., (2023)

  4. Wu, X., Ma, Y., Yong, X., Wang, C., He, Y., Li, N.: Locomotion mode identification and gait phase estimation for exoskeletons during continuous multilocomotion tasks. IEEE Trans. Cogn. Develop. Syst. 13(1), 45–56 (2019)

    Article  Google Scholar 

  5. Xue, T., Wang, Z., Zhang, T., Zhang, M.: Adaptive oscillator-based robust control for flexible hip assistive exoskeleton. IEEE Robot. Autom. Lett. 4(4), 3318–3323 (2019)

    Article  Google Scholar 

  6. Pais-Vieira, C., Allahdad, M., Neves-Amado, J., Perrotta, A., Morya, E., Moioli, R., Shapkova, E., Pais-Vieira, M.: Method for positioning and rehabilitation training with the exoatlet\(^{\circledR} \) powered exoskeleton. MethodsX 7, 100849 (2020)

    Article  Google Scholar 

  7. Zhou, J., Li, Z., Li, X., Wang, X., Song, R.: Human-robot cooperation control based on trajectory deformation algorithm for a lower limb rehabilitation robot. IEEE/ASME Trans. Mechatron. 26(6), 3128–3138 (2021)

    Article  Google Scholar 

  8. Aertbeliën, E., De Schutter, J.: Learning a predictive model of human gait for the control of a lower-limb exoskeleton. In: 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 520–525. (2014)

  9. Oh, S., Baek, E., Song, S.-K., Mohammed, S., Jeon, D., Kong, K.: A generalized control framework of assistive controllers and its application to lower limb exoskeletons. Robot. Auton. Syst. 73, 68–77 (2015)

    Article  Google Scholar 

  10. Chen, Z., Guo, Q., Li, T., Yan, Y., Jiang, D.: Gait prediction and variable admittance control for lower limb exoskeleton with measurement delay and extended-state-observer. IEEE Trans. Neural Netw. Learn. Syst., (2022)

  11. Yun, Y., Kim, H.-C., Shin, S.Y., Lee, J., Deshpande, A.D., Kim, C.: Statistical method for prediction of gait kinematics with gaussian process regression. J. Biomech. 47(1), 186–192 (2014)

    Article  Google Scholar 

  12. Wu, X., Liu, D.-X., Liu, M., Chen, C., Guo, H.: Individualized gait pattern generation for sharing lower limb exoskeleton robot. IEEE Trans. Autom. Sci. Eng. 15(4), 1459–1470 (2018)

    Article  Google Scholar 

  13. Singh, B., Vijayvargiya, A., Kumar, R.: Kinematic modeling for biped robot gait trajectory using machine learning techniques. J. Bionic Eng. 19(2), 355–369 (2022)

    Article  Google Scholar 

  14. Song, W., Zhao, P., Li, X., Deng, X., Zi, B.: Data-driven design of a six-bar lower-limb rehabilitation mechanism based on gait trajectory prediction. IEEE Trans. Neural Syst. Rehabil. Eng. 31, 109–118 (2022)

    Article  Google Scholar 

  15. Liu, D.-X., Wu, X., Wang, C., Chen, C.: Gait trajectory prediction for lower-limb exoskeleton based on deep spatial-temporal model (dstm). In: 2017 2nd International Conference on Advanced Robotics and Mechatronics (ICARM), pp. 564–569. (2017)

  16. Dey, S., Schilling, A.F.: A function approximator model for robust online foot angle trajectory prediction using a single imu sensor: Implication for controlling active rosthetic feet. IEEE Trans. Ind. Inform. 19(2), 1467–1475 (2022)

    Article  Google Scholar 

  17. Jia, L., Ai, Q., Meng, W., Liu, Q., Xie, S.Q.: Individualized gait trajectory prediction based on fusion lstm networks for robotic rehabilitation training. In: 2021 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), pp. 988–993. (2021)

  18. Zaroug, A., Lai, D.T., Mudie, K., Begg, R.: Lower limb kinematics trajectory prediction using long short-term memory neural networks. Front. bioeng. biotechnol. 8, 362 (2020)

    Article  Google Scholar 

  19. Zaroug, A., Garofolini, A., Lai, D.T., Mudie, K., Begg, R.: Prediction of gait trajectories based on the long short term memory neural networks. Plos One 16(8), e0255597 (2021)

    Article  Google Scholar 

  20. Su, B., Gutierrez-Farewik, E.M.: Gait trajectory and gait phase prediction based on an lstm network. Sensors 20(24), 7127 (2020)

    Article  Google Scholar 

  21. Shushtari, M., Nasiri, R., Arami, A.: Online reference trajectory adaptation: A personalized control strategy for lower limb exoskeletons. IEEE Robot. Autom. Lett. 7(1), 128–134 (2021)

    Article  Google Scholar 

  22. Semwal, V.B., Jain, R., Maheshwari, P., Khatwani, S.: Gait reference trajectory generation at different walking speeds using lstm and cnn. Multimed. Tools Appl., 1–19 (2023)

  23. Martinez, A., Lawson, B., Goldfarb, M.: A controller for guiding leg movement during overground walking with a lower limb exoskeleton. IEEE Trans Robot 34(1), 183–193 (2017)

    Article  Google Scholar 

  24. Tanghe, K., De Groote, F., Lefeber, D., De Schutter, J., Aertbeliën, E.: Gait trajectory and event prediction from state estimation for exoskeletons during gait. IEEE Trans. Neural Syst. Rehabil. Eng. 28(1), 211–220 (2019)

    Article  Google Scholar 

  25. Wei, B., Yi, C., Zhang, S., Guo, H., Zhu, J., Ding, Z., Jiang, F.: Taking locomotion mode as prior: One algorithm-enabled gait events and kinematics prediction on various terrains. IEEE Sensors J, (2023)

  26. Zhou, X., Yu, Z., Chen, D., Ye, X.: Experimental research on follow-up control of lower limb exoskeleton robot. In: ISMSEE 2022; The 2nd International Symposium on Mechanical Systems and Electronic Engineering, pp. 1–5. VDE, (2022)

  27. Taborri, J., Scalona, E., Rossi, S., Palermo, E., Patanè, F., Cappa, P.: Real-time gait detection based on hidden markov model: Is it possible to avoid training procedure? In: 2015 IEEE International Symposium on Medical Measurements and Applications (MeMeA) Proceedings, pp. 141–145. (2015)

  28. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, (2014)

  29. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, vol. 15, pp. 315–323. Fort Lauderdale, FL, USA: PMLR, 11–13, (2011)

  30. Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: Proceedings of The 33rd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 48, pp. 1928–1937. New York, New York, USA: PMLR, 20–22 Jun (2016)

  31. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [Online]. Available: arXiv:1412.6980 (2015)

  32. Burgess, C.P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., Lerchner, A.: Understanding disentangling in \(\beta \)-vae. arXiv preprint arXiv:1804.03599 (2018)

  33. Hebenstreit, F., Leibold, A., Krinner, S., Welsch, G., Lochmann, M., Eskofier, B.M.: Effect of walking speed on gait sub phase durations. Hum Mov Sci 43, 118–124 (2015)

  34. Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 80, pp. 1861–1870. PMLR, 10–15 Jul (2018)

  35. Lobo, J.L., Del Ser, J., Bifet, A., Kasabov, N.: Spiking neural networks and online learning: An overview and perspectives. Neural Netw 121, 88–100 (2020)

    Article  Google Scholar 

  36. Liu, D.-X., Xu, J., Chen, C., Long, X., Tao, D., Wu, X.: Vision-assisted autonomous lower-limb exoskeleton robot. IEEE Trans. Syst. Man Cybern. Syst. 51(6), 3759–3770 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported in part by The Key Research and Development Plan of Jiangsu Province under Grant BE2017007-1.

Author information

Authors and Affiliations

  1. Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, 230031, China

    Zhipeng Yu & Xiaojie Wang

  2. Department of Science Island, University of Science and Technology of China, Hefei, 230026, China

    Zhipeng Yu

  3. Department of Automation, University of Science and Technology of China, Hefei, 230026, China

    Jianghai Zhao

  4. Changzhou Institute of Advanced Manufacturing Technology, Chinese Academy of Sciences, Changzhou, 213164, China

    Danhui Chen & Shuyan Chen

Authors
  1. Zhipeng Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianghai Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danhui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuyan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaojie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Zhipeng Yu, Jianghai Zhao, Danhui Chen and Shuyan Chen. The first draft of the manuscript was written by Zhipeng Yu, and reviewed and revised by Xiaojie Wang. All authors commented on previous versions of the manuscript, and all read and approved the final manuscript.

Corresponding author

Correspondence to Xiaojie Wang.

Ethics declarations

Ethics approval

This research were approved by the Ethics Committee of Hefei institutes of Physical Science, Chinese Academy of Sciences (No. SWYX-Y-2022-36).

Consent to participate

All the participants gave their informed consent.

Consent to publish

The authors declare no interest conflict regarding the publication of this paper.

Conflict of Interests

The authors declare that they have no conflict of interests with others.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, Z., Zhao, J., Chen, D. et al. Adaptive Gait Trajectory and Event Prediction of Lower Limb Exoskeletons for Various Terrains Using Reinforcement Learning. J Intell Robot Syst 109, 23 (2023). https://doi.org/10.1007/s10846-023-01963-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-023-01963-7

Keywords

Search

Navigation