Skip to main content

Advertisement

SpringerLink
Log in

Artificial Neural Network-Based Activities Classification, Gait Phase Estimation, and Prediction

  • Original Article
  • Published:
Annals of Biomedical Engineering Aims and scope Submit manuscript

Abstract

Gait patterns are critical to health monitoring, gait impairment assessment, and wearable device control. Unrhythmic gait pattern detection under community-based conditions is a new frontier in this area. The present paper describes a high-accuracy gait phase estimation and prediction algorithm built on a two-stage artificial neural network. This work targets to develop an algorithm that can estimate and predict the gait cycle in real time using a portable controller with only two IMU sensors (one on each thigh) in the community setting. Our algorithm can detect the gait phase in unrhythmic conditions during walking, stair ascending, and stair descending, and classify these activities with standing. Moreover, our algorithm is able to predict both future intra- and inter-stride gait phases, offering a potential means to improve wearable device controller performance. The proposed data-driven algorithm is based on a dataset consisting of 5 able-bodied subjects and validated on 3 different able-bodied subjects. Under unrhythmic activity situations, validation shows that the algorithm can accurately identify multiple activities with 99.55% accuracy, and estimate (\({\mathrm{rRMSE}}_{0}\): 6.3%) and predict 200-ms-ahead (\({\mathrm{rRMSE}}_{200\mathrm{ ms}}\): 8.6%) the gait phase percentage in real time, which are on average 57.7 and 54.0% smaller than the error from the event-based method in the same conditions. This study showcases a solution to estimate and predict gait status for multiple unrhythmic activities, which may be deployed to controllers for wearable robots or health monitoring devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8

Similar content being viewed by others

References

  1. Attal, F., S. Mohammed, M. Dedabrishvili, F. Chamroukhi, L. Oukhellou, and Y. Amirat. Physical human activity recognition using wearable sensors. Sensors. 15(12):31314–31338, 2015

    Article  PubMed  PubMed Central  Google Scholar 

  2. Bartlett, H. L., and M. Goldfarb. A phase variable approach for IMU-based locomotion activity recognition. IEEE Trans. Biomed. Eng. 65(6):1330–1338, 2017

    Article  PubMed  Google Scholar 

  3. Camargo, J., W. Flanagan, N. Csomay-Shanklin, B. Kanwar, and A. Young. A machine learning strategy for locomotion classification and parameter estimation using fusion of wearable sensors. IEEE Trans. Biomed. Eng. 68(5):1569–1578, 2021

    Article  PubMed  Google Scholar 

  4. Caramia, C., D. Torricelli, M. Schmid, A. Muñoz-Gonzalez, J. Gonzalez-Vargas, F. Grandas, and J. L. Pons. IMU-based classification of Parkinson’s disease from gait: a sensitivity analysis on sensor location and feature selection. IEEE J. Biomed. Health Inform. 22(6):1765–1774, 2018

    Article  PubMed  Google Scholar 

  5. Carcreff, L., C. N. Gerber, A. Paraschiv-Ionescu, G. De Coulon, C. J. Newman, S. Armand, and K. Aminian. What is the best configuration of wearable sensors to measure spatiotemporal gait parameters in children with cerebral palsy? Sensors. 18(2):394, 2018

    Article  PubMed  PubMed Central  Google Scholar 

  6. Choi, H., Y. J. Park, K. Seo, J. Lee, S.-E. Lee, and Y. Shim. A multifunctional ankle exoskeleton for mobility enhancement of gait-impaired individuals and seniors. IEEE Robot. Autom. Lett. 3(1):411–418, 2017

    Article  Google Scholar 

  7. Culver, S., H. Bartlett, A. Shultz, and M. Goldfarb. A stair ascent and descent controller for a powered ankle prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 26(5):993–1002, 2018

    Article  PubMed  Google Scholar 

  8. Elery, T., S. Rezazadeh, C. Nesler, and R. D. Gregg. Design and validation of a powered knee–ankle prosthesis with high-torque, low-impedance actuators. IEEE Trans. Robot. 36(6):1649–1668, 2020

    Article  PubMed  PubMed Central  Google Scholar 

  9. Filli, L., T. Sutter, C. S. Easthope, T. Killeen, C. Meyer, K. Reuter, L. Lörincz, M. Bolliger, M. Weller, and A. Curt. Profiling walking dysfunction in multiple sclerosis: characterisation, classification and progression over time. Sci. Rep. 8(1):1–13, 2018

    Article  CAS  Google Scholar 

  10. Gao, F., G. Liu, F. Liang, and W.-H. Liao. IMU-based locomotion mode identification for transtibial prostheses, orthoses, and exoskeletons. IEEE Trans. Neural Syst. Rehabil. Eng. 28(6):1334–1343, 2020

    Article  PubMed  Google Scholar 

  11. Gieseke, F. and C. Igel. Training big random forests with little resources. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2018, pp. 1445–1454.

  12. Hu, B., E. Rouse, and L. Hargrove. Fusion of bilateral lower-limb neuromechanical signals improves prediction of locomotor activities. Front. Robot. AI. 5:78, 2018

    Article  PubMed  PubMed Central  Google Scholar 

  13. Huang, H., T. A. Kuiken, and R. D. Lipschutz. A strategy for identifying locomotion modes using surface electromyography. IEEE Trans. Biomed. Eng. 56(1):65–73, 2008

    Article  Google Scholar 

  14. Huang, H., F. Zhang, L. J. Hargrove, Z. Dou, D. R. Rogers, and K. B. Englehart. Continuous locomotion-mode identification for prosthetic legs based on neuromuscular–mechanical fusion. IEEE Trans. Biomed. Eng. 58(10):2867–2875, 2011

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hutabarat, Y., D. Owaki, and M. Hayashibe. Quantitative gait assessment with feature-rich diversity using two IMU sensors. IEEE Trans. Med. Robot. Bionics. 2(4):639–648, 2020

    Article  Google Scholar 

  16. Jang, J., K. Kim, J. Lee, B. Lim, and Y. Shim. Online gait task recognition algorithm for hip exoskeleton. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015. IEEE, pp. 5327–5332.

  17. Kang, I., P. Kunapuli, and A. J. Young. Real-time neural network-based gait phase estimation using a robotic hip exoskeleton. IEEE Trans. Med. Robot. Bionics. 2(1):28–37, 2019

    Article  Google Scholar 

  18. Kang, I., D. D. Molinaro, G. Choi, and A. J. Young. Continuous locomotion mode classification using a robotic hip exoskeleton. In: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), 2020. IEEE, pp. 376–381.

  19. Kim, J., G. Lee, R. Heimgartner, D. A. Revi, N. Karavas, D. Nathanson, I. Galiana, A. Eckert-Erdheim, P. Murphy, and D. Perry. Reducing the metabolic rate of walking and running with a versatile, portable exosuit. Science. 365(6454):668–672, 2019

    Article  CAS  PubMed  Google Scholar 

  20. Lewis, C. L., and D. P. Ferris. Invariant hip moment pattern while walking with a robotic hip exoskeleton. J. Biomech. 44(5):789–793, 2011

    Article  PubMed  PubMed Central  Google Scholar 

  21. Li, H., S. Derrode, and W. Pieczynski. An adaptive and on-line IMU-based locomotion activity classification method using a triplet Markov model. Neurocomputing. 362:94–105, 2019

    Article  Google Scholar 

  22. Malcolm, P., W. Derave, S. Galle, and D. De Clercq. A simple exoskeleton that assists plantarflexion can reduce the metabolic cost of human walking. PLoS ONE.8(2):e56137, 2013

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Rumelhart, D. E., G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Nature. 323(6088):533–536, 1986

    Article  Google Scholar 

  24. Sánchez Manchola, M. D., M. J. P. Bernal, M. Munera, and C. A. Cifuentes. Gait phase detection for lower-limb exoskeletons using foot motion data from a single inertial measurement unit in hemiparetic individuals. Sensors. 19(13):2988, 2019

    Article  PubMed  PubMed Central  Google Scholar 

  25. Seo, K., K. Kim, Y. J. Park, J.-K. Cho, J. Lee, B. Choi, B. Lim, Y. Lee, and Y. Shim. Adaptive oscillator-based control for active lower-limb exoskeleton and its metabolic impact. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018. IEEE, pp. 6752–6758.

  26. Seo, K., Y. J. Park, J. Lee, S. Hyung, M. Lee, J. Kim, H. Choi, and Y. Shim. RNN-based on-line continuous gait phase estimation from shank-mounted IMUs to control ankle exoskeletons. In: 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), 2019. IEEE, pp. 809–815.

  27. Sugar, T. G., A. Bates, M. Holgate, J. Kerestes, M. Mignolet, P. New, R. K. Ramachandran, S. Redkar, and C. Wheeler. Limit cycles to enhance human performance based on phase oscillators. J. Mech. Robot.7(1):011001, 2015

    Article  Google Scholar 

  28. Swami, C. P., N. Lenhard, and J. Kang. A novel framework for designing a multi-DoF prosthetic wrist control using machine learning. Sci. Rep. 11(1):1–13, 2021

    Article  Google Scholar 

  29. Wang, F., L. Yan, and J. Xiao. Recognition of the gait phase based on new deep learning algorithm using multisensor information fusion. Sens. Mater. 31(10):3041–3054, 2019

    Google Scholar 

  30. Yang, J., T.-H. Huang, S. Yu, X. Yang, H. Su, A. M. Spungen, and C.-Y. Tsai. Machine learning based adaptive gait phase estimation using inertial measurement sensors. In: 2019 Design of Medical Devices Conference, 2019. American Society of Mechanical Engineers Digital Collection.

  31. Yap, H. K., H. Y. Ng, and C.-H. Yeow. High-force soft printable pneumatics for soft robotic applications. Soft Robot. 3(3):144–158, 2016

    Article  Google Scholar 

  32. Yu, S., T.-H. Huang, D. Wang, B. Lynn, D. Sayd, V. Silivanov, Y. S. Park, Y. Tian, and H. Su. Design and control of a high-torque and highly backdriveable hybrid soft exoskeleton for knee injury prevention during squatting. IEEE Robot. Autom. Lett. 4(4):4579–4586, 2019

    Article  Google Scholar 

  33. Zhang, J., P. Fiers, K. A. Witte, R. W. Jackson, K. L. Poggensee, C. G. Atkeson, and S. H. Collins. Human-in-the-loop optimization of exoskeleton assistance during walking. Science. 356(6344):1280–1284, 2017

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

This work is supported in part by the National Science Foundation (NSF) CAREER Award CMMI 1944655, NSF 2026622, NIH R01EB029765, NIDILRR 90DPGE0011. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author (s) and do not necessarily reflect the views of the funding organizations.

Conflict of interest

No benefits in any form have been or will be received from a commercial party related directly or indirectly to the subject of this manuscript.

Author information

Authors and Affiliations

  1. Lab of Biomechatronics and Intelligent Robotics (BIRO), Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC, 27695, USA

    Shuangyue Yu, Jianfu Yang, Tzu-Hao Huang, Junxi Zhu & Hao Su

  2. Department of Rehabilitation and Regenerative Medicine, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, 10032, USA

    Christopher J. Visco, Farah Hameed & Joel Stein

  3. Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, 07102, USA

    Xianlian Zhou

Authors
  1. Shuangyue Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianfu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tzu-Hao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junxi Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christopher J. Visco
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Farah Hameed
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joel Stein
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xianlian Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Hao Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Su.

Additional information

Associate Editor Stefan M. Duma oversaw the review of this article.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, S., Yang, J., Huang, TH. et al. Artificial Neural Network-Based Activities Classification, Gait Phase Estimation, and Prediction. Ann Biomed Eng 51, 1471–1484 (2023). https://doi.org/10.1007/s10439-023-03151-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-023-03151-y

Keywords

Search

Navigation