Abstract
In this work, learning-based inverse dynamics algorithms are proposed for the analysis of human motion. Immeasurable joint torques and exterior contact forces are directly estimated from motions by machine learning techniques including deep neural networks, random forests and Ridge regression. A multistage subclass approach is introduced. The method recovers occluded motion data and generates meaningful features, as well as gait phase labels to restrict and facilitate the regression of forces and moments. In contrast to the state-of-the-art inverse dynamics optimization, the learning-based methods are independent of ground reaction force measurements and the global position and orientation of the human body. These properties make the application to reconstructed poses from videos or inertial measurements possible, creating fast and simple access to the underlying dynamics of recorded human motions. The performance of the proposed methods is evaluated on a self-recorded data set including walking and running motions and on a publicly available gait data set by Fukuchi et al. (PeerJ 6:e4640, 2018). Furthermore, the applicability to reconstructed gait sequences taken from the well-known CMU database (Human motion capture database, 2014. http://mocap.cs.cmu.edu/) is investigated. Finally, the method is tested as a tool to detect abnormal torque distributions in gait, based on a reconstructed 3D motion of a limping subject.
Similar content being viewed by others
References
Cao Z, Simon T, Wei SE, Sheikh Y (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: CVPR
Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27
Choi A, Lee JM, Mun JH (2013) Ground reaction forces predicted by using artificial neural network during asymmetric movements. Int J Precis Eng Manuf 14(3):475–483
CMU (2014) Human motion capture database. http://mocap.cs.cmu.edu/. Accessed: 08 Dec 2019
Diba A, Fayyaz M, Sharma V, Karami AH, Arzani MM, Yousefzadeh R, Gool LV (2017) Temporal 3d convnets: New architecture and transfer learning for video classification. CoRR arXiv:abs/1711.08200
Ding Y, Galiana I, Asbeck AT, Rossi SMMD, Bae J, Santos TRT, de Araujo VL, Lee S, Holt KG, Walsh C (2017) Biomechanical and physiological evaluation of multi-joint assistance with soft exosuits. IEEE Trans Neural Syst Rehabil Eng 25(2):119–130
Fluit R, Andersen M, Kolk S, Verdonschot N, Koopman H (2014) Prediction of ground reaction forces and moments during various activities of daily living. J Biomech 47(10):2321–2329
Forner-Cordero A, Koopman H, van der Helm F (2006) Inverse dynamics calculations during gait with restricted ground reaction force information from pressure insoles. Gait Posture 23(2):189–199
Fukuchi CA, Fukuchi RK, Duarte M (2018) A public dataset of overground and treadmill walking kinematics and kinetics in healthy individuals. PeerJ 6:e4640
Ghochani A, Ghomshe FT, Nejad SR, Rahimnejad M (2010) Analysis of torques and forces applied on limbs and joints of lower extremities in free kick in football. Procedia Eng 2(2):3269–3274
Guo M, Chou E, Huang DA, Song S, Yeung S, Fei-Fei L (2018) Neural graph matching networks for fewshot 3d action recognition. In: The European Conference on Computer Vision (ECCV)
Johnson L, Ballard DH (2014) Efficient codes for inverse dynamics during walking. In: Proceedings of the 28th AAAI conference on artificial intelligence. AAAI Press, pp 343–349
Johnson W, Alderson J, Lloyd D, Mian A (2018) Predicting athlete ground reaction forces and moments from spatio-temporal driven cnn models. IEEE Trans Biomed Eng 66:1
Jung Y, Jung M, Lee K, Koo S (2014) Ground reaction force estimation using an insole-type pressure mat and joint kinematics during walking. J Biomech 47(11):2693–2699
Karatsidis A, Bellusci G, Schepers HM, de Zee M, Andersen MS, Veltink PH (2017) Estimation of ground reaction forces and moments during gait using only inertial motion capture. Sensors 17(1):75:1–75:22
King MG, Lawrenson PR, Semciw AI, Middleton KJ, Crossley KM (2018) Lower limb biomechanics in femoroacetabular impingement syndrome: a systematic review and meta-analysis. Br J Sports Med 52(9):566–580
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. http://arxiv.org/abs/1412.6980, cite arxiv:1412.6980Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015
Li C, Tong R, Tang M (2018) Modelling human body pose for action recognition using deep neural networks. Arab J Sci Eng 43:7777–7788
Li Y, Xie Y, Zhang Q (2018) 3d gesture recognition based on handheld smart terminals. Int J Ambient Comput Intell 9(4):96–111
Lv X, Chai J, Xia S (2016) Data-driven inverse dynamics for human motion. ACM Trans Graph 35(6):163:1–163:12
von Marcard T, Rosenhahn B, Black M, Pons-Moll G (2017) Sparse inertial poser: Automatic 3d human pose estimation from sparse imus. Computer Graphics Forum 36(2). In: Proceedings of the 38th annual conference of the European Association for computer graphics (Eurographics)
von Marcard T, Henschel R, Black MJ, Rosenhahn B, Pons-Moll G (2018) Recovering accurate 3D human pose in the wild using IMUs and a moving camera. In: European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, vol 11214. Springer, Cha, pp 614–631
Martinez J, Hossain R, Romero J, Little JJ (2017) A simple yet effective baseline for 3d human pose estimation. In: ICCV, IEEE Computer Society, pp 2659–2668
Mehta D, Sridhar S, Sotnychenko O, Rhodin H, Shafiei M, Seidel HP, Xu W, Casas D, Theobalt C (2017) Vnect: Real-time 3d human pose estimation with a single rgb camera. ACM Trans Gr 36:44
Muntoni F, Domingos J, Manzur A, Mayhew A, Guglieri M, Signorovitch J, Ward S (2017) Clustering trajectories of ambulatory function in the north star clinical network database. Neuromuscul Disord 27:S99
Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: ECCV (8) Lecture Notes in Computer Science, vol 9912. Springer, pp 483–499
Oh SE, Choi A, Mun JH (2013) Prediction of ground reaction forces during gait based on kinematics and a neural network model. J Biomech 46(14):2372–2380
Park S, Hwang J, Kwak N (2016) 3d human pose estimation using convolutional neural networks with 2d pose information. In: Hua G, Jégou H (eds) Computer Vision—ECCV 2016 Workshops. Springer, Cham, pp 156–169
Pavei G, Seminati E, Storniolo JL, Peyré-Tartaruga LA (2017) Estimates of running ground reaction force parameters from motion analysis. J Appl Biomech 33(1):69–75
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830
Pizzolato C, Reggiani M, Modenese L, Lloyd DG (2016) Real-time inverse kinematics and inverse dynamics for lower limb applications using opensim. Comput Methods Biomech Biomed Eng 20(4):436–445
Powers CM (2010) The influence of abnormal hip mechanics on knee injury: a biomechanical perspective. J Orthopaed Sports Phys Therapy 40(2):42–51
Rifkin RM, Lippert RA (2003) Notes on regularized least-squares. Tech. Rep. MIT-CSAIL-TR-2007-025. Computer Science and Artificial Intelligence Laboratory, MIT
Rouhani H, Favre J, Crevoisier X, Aminian K (2010) Ambulatory assessment of 3d ground reaction force using plantar pressure distribution. Gait Posture 32(3):311–316
Schwab AL, Delhaes GMJ (2009) Lecture Notes Multibody Dynamics B, wb1413
Steinparz F (1985) Co-ordinate transformation and robot control with Denavit-Hartenberg matrices. J Microcomput Appl 8(4):303–316
Waltz R, Morales J, Nocedal J, Orban D (2006) An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math Program 107(3):391–408
Wandt B, Ackermann H, Rosenhahn B (2016) 3d reconstruction of human motion from monocular image sequences. Trans Pattern Anal Machine Intell 38(8):1505–1516
Wandt B, Ackermann H, Rosenhahn B (2018) A kinematic chain space for monocular motion capture. In: ECCV Workshops
Wei X, Chai J (2010) Videomocap: modeling physically realistic human motion from monocular video sequences. ACM Trans Graph 29(4):42:1–42:10
Winter D (2009) Biomechanics and motor control of human movement. Wiley, Hoboken
Xiang Y, Arora JS, Rahmatalla S, Abdel-Malek K (2010) Physics-based modeling and simulation of human walking: a review of optimization-based and other approaches. Struct Multidiscip Optim 42(1):1–23
Xiang Y, Chung HJ, Kim JH, Bhatt R, Rahmatalla S, Yang J, Marler T, Arora JS, Abdel-Malek K (2010) Predictive dynamics: an optimization-based novel approach for human motion simulation. Struct Multidiscip Optim 41(3):465–479
Xu W, Chatterjee A, Zollhöfer M, Rhodin H, Mehta D, Seidel HP, Theobalt C (2018) Monoperfcap: human performance capture from monocular video. ACM Trans Graph 37(2):27:1–27:15
Yang W, Ouyang W, Wang X, Ren JSJ, Li H, Wang X (2018) 3d human pose estimation in the wild by adversarial learning. In: CVPR. IEEE Computer Society, pp 5255–5264
Zanfir A, Marinoiu E, Sminchisescu C (2018) Monocular 3d pose and shape estimation of multiple people in natural scenes - the importance of multiple scene constraints. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Zell P, Rosenhahn B (2017) Learning-based inverse dynamics of human motion. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, pp 842–850
Zell P, Wandt B, Rosenhahn B (2017) Joint 3d human motion capture and physical analysis from monocular videos. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors of this paper have no financial or personal relationships with other people or organizations that could inappropriately influence this work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by the European Research Council (ERC) under the ERC-2013-PoC Grant (No. 1924-2013).
Appendix 1: Implementation details concerning [20]
Appendix 1: Implementation details concerning [20]
We compare the performance of our learning-based inverse dynamics methods to a data-driven maximum a posteriori approach by [20]. For this purpose, the referenced method is implemented with a few modifications to allow a fair comparison between the two methods. These modifications were made to achieve a better operation using our specific data and physical model.
To find z(t) at each frame, the following weighted sum of energy terms is minimized:
with the weights \((\lambda _1,\lambda _2,\lambda _3,\lambda _4) = (2,2,100,1)\). In consistence with our physical model, the friction term \(E_{\mathrm {friction}}\) is left out, assuming a negligible effect for the considered movements. The state and control parameters are
where \({\varvec{F}}_c\) consists of the GRF and the GRM. In contrast to [20], the center of pressure and the torsional torque are replaced by the resulting GRM.
Similar to [20], principle component analysis is used to linearize the local parameter space at each frame. The local environment is built of the 200 next neighbors of \({\varvec{z}}(t)\). We only optimize the scores \({\varvec{s}}\) of the first n principle components stacked in the matrix \({\varvec{K}}\). These n components constitute 95 % of the overall variability of the local data. This way the number of optimization variables is drastically reduced. The optimization problem becomes
with the mean \(\varvec{\mu }\) of the neighbouring parameter vectors.
As indicated before, the state vector \({\varvec{z}}\) is adapted to fit the applied physical model, which has an immediate effect on the physical term \(E_{\mathrm {phys}}\). This term describes the deviation of the kinematic state \(({\varvec{q}}(t), \dot{{\varvec{q}}}(t), \ddot{{\varvec{q}}}(t))\) given by \({\varvec{z}}(t)\) from the kinematics arising from the acting forces and torques via the EOM. Based on Eq. (4), the energy term is defined as
The changed physical model further necessitates a slight modification of the smoothness term:
The remaining energy terms \(E_{\mathrm {prior}}\) and \(E_{\mathrm {data}}\) can be employed without adaptation.
Rights and permissions
About this article
Cite this article
Zell, P., Rosenhahn, B. Learning inverse dynamics for human locomotion analysis. Neural Comput & Applic 32, 11729–11743 (2020). https://doi.org/10.1007/s00521-019-04658-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-019-04658-z