Abstract
In recent years, gait analysis outside the laboratory attracts more and more attention in clinical applications as well as in life sciences. Wearable sensors such as inertial sensors show high potential in these applications. Unfortunately, they can only measure kinematic motions patterns indirectly and the outcome is currently jeopardized by measurement discrepancies compared with the gold standard of optical motion tracking. The aim of this study was to overcome the limitation of measurement discrepancies and the missing information on kinetic motion parameters using a machine learning application based on artificial neural networks. For this purpose, inertial sensor data—linear acceleration and angular rate—was simulated from a database of optical motion tracking data and used as input for a feedforward and long short-term memory neural network to predict the joint angles and moments of the lower limbs during gait. Both networks achieved mean correlation coefficients higher than 0.80 in the minor motion planes, and correlation coefficients higher than 0.98 in the sagittal plane. These results encourage further applications of artificial intelligence to support gait analysis.
Similar content being viewed by others
Notes
The input data is augmented by randomly rotating the relative orientation of the virtual sensor. Thus, the same motion is recorded, but distributed differently over the sensor axes.
References
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/. Software available from tensorflow.org
Aljaaf AJ, Hussain AJ, Fergus P, Przybyla A, Barton G (2016) Evaluation of machine learning methods to predict knee loading from the movement of body segments. In: Proceedings of the international joint conference on neural networks, vol 2016, pp 5168–5173, DOI https://doi.org/10.1109/IJCNN.2016.7727882
Ardestani MM, Chen Z, Wang L, Lian Q, Liu Y, He J, Li D, Jin Z (2014) Feed forward artificial neural network to predict contact force at medial knee joint: application to gait modification. Neurocomputing 139:114–129. https://doi.org/10.1016/j.neucom.2014.02.054
Ardestani MM, Moazen M, Jin Z (2014) Gait modification and optimization using neural network-genetic algorithm approach: application to knee rehabilitation. Expert Syst Appl 41(16):7466–7477. https://doi.org/10.1016/j.eswa.2014.06.034
Ardestani MM, Zhang X, Wang L, Lian Q, Liu Y, He J, Li D, Jin Z (2014) Human lower extremity joint moment prediction: a wavelet neural network approach. Expert Syst Appl 41(9):4422–4433. https://doi.org/10.1016/j.eswa.2013.11.003
Argent R, Drummond S, Remus A, O’Reilly M, Caulfield B (2019) Evaluating the use of machine learning in the assessment of joint angle using a single inertial sensor. J Rehabil Assist Technol Eng 6:1–10. https://doi.org/10.1177/2055668319868544
Atkinson K, Han W (2005) Theoretical numerical analysis, vol 39. Springer
Brennan A, Zhang J, Deluzio K, Li Q (2011) Quantification of inertial sensor-based 3d joint angle measurement accuracy using an instrumented gimbal. Gait Posture 34(3):320–323
Brunner T, Lauffenburger JP, Changey S, Basset M (2015) Magnetometer-augmented IMU simulator: in-depth elaboration. Sensors (Switzerland) 15(3):5293–5310. https://doi.org/10.3390/s150305293
Caldas R, Mundt M, Potthast W, Buarque de Lima Neto F, Markert B (2017) A systematic review of gait analysis methods based on inertial sensors and adaptive algorithms. Gait Posture 57:204–210. https://doi.org/10.1016/j.gaitpost.2017.06.019
Favre J, Hayoz M, Erhart-Hledik JC, Andriacchi TP (2012) A neural network model to predict knee adduction moment during walking based on ground reaction force and anthropometric measurements. J Biomech 45 (4):692–698. https://doi.org/10.1016/j.jbiomech.2011.11.057
Ferreira JP, Vieira A, Ferreira P, Crisóstomo M, Coimbra AP (2018) Human knee joint walking pattern generation using computational intelligence techniques. Neural Comput Appl 30(6):1701–13
Findlow AH, Goulermas JY, Nester CJ, Howard D, Kenney LPJ (2008) Predicting lower limb joint kinematics using wearable motion sensors. Gait Posture 28(1):120–126. https://doi.org/10.1016/j.gaitpost.2007.11.001
Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning. Springer series in statistics
Gers FA, Schmidhuber J, Cummins F (1999) Learning to forget: Continual prediction with LSTM. Proc ICANN’99 Int Conf on Artificial Neural Networks, 2471:850–855. https://doi.org/10.1162/089976600300015015
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press. http://www.deeplearningbook.org
Goulermas JY, Findlow AH, Nester CJ, Liatsis P, Zeng XJ, Kenney LPJ, Tresadern P, Thies SB, Howard D (2008) An instance-based algorithm with auxiliary similarity information for the estimation of gait kinematics from wearable sensors. IEEE Trans Neural Netw 19(9):1574–1582. https://doi.org/10.1109/TNN.2008.2000808
Goulermas JY, Howard D, Nester CJ, Jones RK, Ren L (2005) Regression techniques for the prediction of lower limb kinematics. J Biomech Eng 127(6):1020. https://doi.org/10.1115/1.2049328
Hahn ME, O’Keefe KB (2008) A neural network model for estimation of net joint moments during normal gait. J Musculoskelet Res 11(03):117–126. https://doi.org/10.1142/S0218957708002036
Harrington M, Zavatsky A, Lawson S, Yuan Z, Theologis T (2007) Prediction of the hip joint centre in adults, children, and patients with cerebral palsy based on magnetic resonance imaging. J Biomech 40(3):595–602
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780
Hu B, Dixon PC, Jacobs JV, Dennerlein JT, Schiffman JM (2018) Machine learning algorithms based on signals from a single wearable inertial sensor can detect surface- and age-related differences in walking. J Biomech 71:37–42. https://doi.org/10.1016/j.jbiomech.2018.01.005
Huang Y, Planck M, Systems I (2018) Deep inertial poser : learning to reconstruct human pose from sparse inertial measurements in real time. SIGGRAPH Asia, 37(6). https://doi.org/10.1145/3272127.3275108
Johnson WR, Mian A, Donnelly CJ, Lloyd D, Alderson J (2018) Predicting athlete ground reaction forces and moments from motion capture. Med Biol Eng Comput 56(10):1781–1792. https://doi.org/10.1007/s11517-018-1802-7
Kay RM, Dennis S, Rethlefsen S, Reynolds RA, Skaggs DL, Tolo VT (2000) The effect of preoperative gait analysis on orthopaedic decision making. Clin Orthop Relat Res 372:217–222. https://doi.org/10.1097/00003086-200003000-00023
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization CoRR
Koeppe A, Bamer F, Hernandez Padilla CA, Markert B (2017) Neural network representation of a phase-field model for brittle fracture. PAMM 17(1):253–254
Komnik I, Peters M, Funken J, David S, Weiss S, Potthast W (2016) Non-sagittal knee joint kinematics and kinetics during gait on level and sloped grounds with unicompartmental and total knee arthroplasty patients. PLoS ONE 11(12):1–18
Komnik I, Weiss S, Fantini Pagani C, Potthast W (2015) Motion analysis of patients after knee arthroplasty during activities of daily living – a systematic review. Gait Posture 41(2):370–377
Maiwald C, Sterzing T, Mayer T, Milani T (2009) Detecting foot-to-ground contact from kinematic data in running. Footwear Sci 1(2):111–118
Mundt M, Koeppe A, Bamer F, Potthast W (2018) Prediction of joint kinetics based on joint kinematics using neural networks. In: 36th Conference of the international society of biomechanics in sports, pp 7–10
Mundt M, Thomsen W, David S, Dupré T, Bamer F, Potthast W, Markert B (2019) Assessment of the measurement accuracy of inertial sensors during different tasks of daily living. J Biomech 84:81–86. https://doi.org/10.1016/j.jbiomech.2018.12.023
Nüesch C., Roos E, Pagenstert G, Mündermann A. (2017) Measuring joint kinematics of treadmill walking and running: comparison between an inertial sensor based system and a camera-based system. J Biomech 57:32–38. https://doi.org/10.1016/j.jbiomech.2017.03.015
Oh SE, Choi A, Mun JH (2013) Prediction of ground reaction forces during gait based on kinematics and a neural network model. J Biomech 46(14):2372–2380. https://doi.org/10.1016/j.jbiomech.2013.07.036
Osateerakun P, Barton G, Foster R, Bennett S, Lakshminarayan R (2018) P 037 – Prediction of moments from movements without force platforms using artificial neural networks: a pilot test. Gait Posture 65:299–300. https://doi.org/10.1016/j.gaitpost.2018.06.194
Palermo E, Rossi S, Marini F, Patané F., Cappa P (2014) Experimental evaluation of accuracy and repeatability of a novel body-to-sensor calibration procedure for inertial sensor-based gait analysis. Measur J Int Measur Confed 52(1):145–155. https://doi.org/10.1016/j.measurement.2014.03.004
Richards JD (2008) Biomechanics in clinic and research, 1st edn. Churchill Livingstone, London
Robert-Lachaine X, Mecheri H, Larue C, Plamondon A (2016) Validation of inertial measurement units with an optoelectronic system for whole-body motion analysis. Med Biol Eng Comput 55(4):609–619. https://doi.org/10.1007/s11517-016-1537-2
Robert-Lachaine X, Mecheri H, Larue C, Plamondon A (2017) Accuracy and repeatability of single-pose calibration of inertial measurement units for whole-body motion analysis. Gait Posture 54:80–86. https://doi.org/10.1016/j.gaitpost.2017.02.029
Robertson G, Caldwell G, Hamill J, Kamen G, Whittlesey S (2013) Research methods in biomechanics, 2nd edn. Human Kinetics
Samà A., Angulo C, Pardo D, Català A., Cabestany J (2011) Analyzing human gait and posture by combining feature selection and kernel methods. Neurocomputing 74(16):2665–2674. https://doi.org/10.1016/j.neucom.2011.03.028
Seel T, Raisch J, Schauer T (2014) IMU-based joint angle measurement for gait analysis. Sensors 14 (4):6891–6909. https://doi.org/10.3390/s140406891
Sivakumar S, Gopalai AA, Lim KH, Gouwanda D (2019) Artificial neural network based ankle joint angle estimation using instrumented foot insoles. Biomed Signal Process Control 54:1–9
Solà J. (2017) Quaternion kinematics for the error-state Kalman filter. arXiv:https://arxiv.org/abs/1711.02508
Struzik A, Juras G, Pietraszewski B, Rokita A (2016) Effect of drop jump technique on the reactive strength index. Journal of Human Kinetics
Umer W, Li H, Szeto GPY, Wong AYL (2017) Identification of biomechanical risk factors for the development of lower-back disorders during manual rebar tying. J Construct Eng Manag 143(1):04016080. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001208
Vienne A, Barrois RP, Buffat S, Ricard D, Vidal PP (2017) Inertial sensors to assess gait quality in patients with neurological disorders: a systematic review of technical and analytical challenges. Front Psychol 8 (MAY):1–12. https://doi.org/10.3389/fpsyg.2017.00817
de Vries WHK, Veeger HEJ, Baten CTM, van der Helm FCT (2009) Magnetic distortion in motion labs, implications for validating inertial magnetic sensors. Gait Posture 29(4):535–541. https://doi.org/10.1016/j.gaitpost.2008.12.004
Wu G, Siegler S, Allard P, Kirtley C, Leardini A, Rosenbaum D, Whittle M, D’Lima DD, Cristofolini L, Witte H, Schmid O, Stokes I (2002) ISB recommendation on definitions of joint coordinate system of various joints for the reporting of human joint motion—part I: ankle, hip, and spine. J Biomech 35(4):543–548. https://doi.org/10.1016/S0021-9290(01)00222-6
Xiao X, Zarar S (2018) Machine learning for placement-insensitive inertial motion capture. Proc - IEEE Int Conf Robot Autom, 6716–21
Young AD, Ling MJ, Arvind DK (2011) IMUSim: a simulation environment for inertial sensing algorithm design and evaluation. In: Proceedings of the 10th ACMIEEE international conference on information processing in sensor networks, pp 199–210
Zimmermann T, Taetz B, Bleser G (2018) IMU-to-segment assignment and orientation alignment for the lower body using deep learning. Sensors (Switzerland) 18(1):1–35. https://doi.org/10.3390/s18010302
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mundt, M., Thomsen, W., Witter, T. et al. Prediction of lower limb joint angles and moments during gait using artificial neural networks. Med Biol Eng Comput 58, 211–225 (2020). https://doi.org/10.1007/s11517-019-02061-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-019-02061-3