Musculoskeletal Model
To perform the proposed study, we started with a simple and a complex musculoskeletal model taken from the Models folder installed with OpenSim 3.2.9 The simple model (gait10dof18musc) contained three degrees of freedom (hip, knee, and ankle angle in the sagittal plane) and nine muscles per leg, while the complex model (gait2392) contained five degrees of freedom (three at the hip and one at the knee and ankle) and 43 muscles per leg.
Each muscle in the model was represented as a Hill-type muscle-tendon unit35 (Fig. 1). Muscle dynamics was described by two nonlinear, first order differential equations—activation and contraction dynamics—that relate the control—muscle excitation—to the states—muscle activation and either normalized fiber length or normalized tendon force. Activation dynamics was modeled based on Winters,27,33 using a tanh function to smoothly transition between activation and deactivation:
$$f = 0.5 \tanh (b(e - a)),$$
(1)
$$\frac{da}{dt} = \left[ {\frac{1}{{\tau_{a} \left( {0.5 + 1.5a} \right)}}\left( {f + 0.5} \right) + \frac{0.5 + 1.5a}{{\tau_{d} }}\left( { - f + 0.5} \right)} \right]\left( {e - a} \right),$$
(2)
where e is muscle excitation, a is muscle activation, τ
a
= 0.015 s is the activation time constant, τ
d
= 0.060 s is the deactivation time constant, and b = 0.1 is a parameter determining transition smoothness. Contraction dynamics was described based on Hill’s model35 (Fig. 1). The muscle-tendon actuator consisted of a tendon with length l
T in series with a muscle with fiber length l
M, where the pennation angle α defines the angle between the tendon and the muscle fibers. Properties of muscle and tendon were described by dimensionless characteristics (Fig. 2). Five parameters scaled these generic characteristics for a specific muscle: optimal fiber length \(l_{\text{M}}^{0}\), maximal muscle fiber velocity \(v_{\text{M}}^{\hbox{max} }\), peak isometric muscle force \(F_{\text{M}}^{0}\), tendon slack length \(l_{\text{T}}^{s}\), and pennation angle at optimal fiber length α
0. Values for these five parameters were taken from the OpenSim models described above. Tendon was modeled by a nonlinear spring:
$$F_{\text{T}} = F_{\text{M}}^{0} f_{\text{t}} \left( {\tilde{l}_{\text{T}} } \right),$$
(3)
where F
T is tendon force, \(\tilde{l}_{\text{T}} = l_{\text{T}} /l_{\text{T}}^{s}\) is normalized tendon length and f
t is the tendon force-length characteristic (see online supplement for mathematical expression). Muscle was modeled by a contractile element in parallel with a passive element:
$$F_{\text{M}} = F_{\text{M}}^{0} \left[ {af_{\text{act}} \left( {\tilde{l}_{\text{M}} } \right)f_{\text{v}} \left( {\tilde{v}_{\text{M}} } \right) + f_{\text{pas}} \left( {\tilde{l}_{\text{M}} } \right)} \right],$$
(4)
where F
M is muscle force, \(\tilde{l}_{\text{M}} = l_{\text{M}} /l_{\text{M}}^{o}\) is normalized fiber length, \(\tilde{v}_{\text{M}} = v_{\text{M}} /v_{\text{M}}^{\hbox{max} }\) is normalized fiber velocity, and f
act, f
pas, and f
v are the active muscle force-length, passive muscle force-length, and muscle force-velocity characteristics, respectively (see online supplement for mathematical expressions). The interaction between muscle and tendon was described by (Fig. 1):
$$l_{\text{MT}} = l_{\text{T}} + l_{\text{M}} \cos \alpha ,$$
(5)
$$l_{\text{M}} \sin \alpha = l_{\text{M}}^{0} \sin \alpha_{0} ,$$
(6)
$$F_{\text{T}} = F_{\text{M}} \cos \alpha .$$
(7)
The five Eqs. (3)–(7) determine the five unknowns F
T, F
M, l
T, l
M, α, given the input a and the muscle-tendon length l
MT. The dynamic nature of the Hill model results from the fiber velocity dependence of Eq. (4). Note, however, that under the assumption of a rigid tendon and hence constant tendon length \(l_{T} = l_{T}^{s}\), muscle fiber length and velocity are completely determined by muscle-tendon length l
MT and velocity v
MT (Eqs. 5, 6), which can be computed from skeletal kinematics, thereby allowing algebraic solution of Eqs. (3)–(7).
Given the algebraic relationship between muscle fiber length and tendon force, it is equally valid to choose muscle length or tendon force as the state variable when solving Eqs. (3)–(7).25 All characteristics are at least second order continuous and f
t
is at least third order continuous (Fig. 2). For numerical reasons, f
t
is allowed to be less than zero instead of equal to zero when the tendon is slack. Negative tendon forces are non-physiological but will never occur when muscle and tendon force are equilibrated (Eq. 7), since muscle force cannot drop below zero (Eq. 4). This modification of f
t
makes the solution of Eqs. (3)–(7) better conditioned when the muscle-tendon actuator is slack (zero tendon force corresponds to normalized tendon length of 1 rather than a whole range of tendon lengths).
Experimental Data and Data Processing
Experimental data for one walking cycle were taken from the Models folder installed with OpenSim 3.2, since the availability of this dataset allows other researchers to compare their methods to the one presented in this paper. Experimental marker trajectories were sampled at 60 Hz. The exact same experimental data were used for the simple and complex model. The muscle force distribution underlying this walking motion was computed for the right limb of both models by combining dynamic optimization with an inverse dynamics analysis of skeletal motion where measured joint kinematics and external (ground reaction) forces were inputs and the joint reaction torques were outputs.7,15 The inverse dynamics joint torques along with the muscle-tendon lengths and velocities and the muscle moment arms were calculated using the standard workflow in OpenSim 3.2 and used as inputs for the dynamic optimization problems described below (see Fig. 3 for more details). These problems were solved for the controls and states (see below for a formulation-dependent definition) over the motion cycle. The initial and final states, however, are unknown. We found that the initial and final states only influenced the optimal controls and states over a period of about 50 ms at the beginning and end of the time interval over which the dynamic optimization problem was solved. Therefore, problems were solved for a time interval containing five additional data points at the beginning and end of the motion cycle to limit the influence of the unknown initial and final state (the final state influences the optimal control at preceding time instants) on the solution for the motion cycle under consideration and results for these additional data points were not reported.
Problem Formulations and Solution Method
The goal of each optimization problem was to find muscle excitations bounded between 0 and 1 that produced the specified inverse dynamics joint torques while minimizing the integral of the sum of squared excitations for all muscles over the duration of the motion. The use of a quadratic cost functional was first proposed by Pedotti at al.24 and is a measure of muscular effort. Activation and contraction dynamics relate muscle excitations to muscle forces whereas the pre-computed muscle moment arms relate muscle forces to joint torques. For each degree of freedom, an ideal actuator that can produce torque instantaneously was added to the model to guarantee problem feasibility in the presence of modeling and measurement errors. The use of these non-physiological actuators was discouraged by weighting their contribution heavily in the cost function. This approach resulted in the following dynamic optimization problems.
Cost Functional
The cost functional consisted of two terms. The first term represented muscular effort modeled by the integral of the sum of squared muscle excitations, whereas the second term penalized the use of the non-physiological ideal torque actuators:
$$\mathop \smallint \limits_{{t_{0} }}^{{t_{f} }} \left( {\mathop \sum \limits_{m = 1}^{M} e_{m}^{2} + w\mathop \sum \limits_{k = 1}^{K} e_{Tk}^{2} } \right)dt$$
(8)
where t is time, t
0 and t
f are the initial and final time, respectively, m = 1,…,M indicates the different muscles, e
Tk
are the inputs for the ideal actuators, k = 1,…,K indicates the different degrees of freedom, and w = 1000 is a weight penalizing the use of the non-physiological ideal actuators. This weight was chosen such that the contribution of the ideal torque actuators is below 1 Nm for walking, which we think is acceptable given measurement and modeling uncertainties.
Bounds
Muscle excitations were bounded between 0 and 1 whereas the ideal torque actuators could generate both positive and negative torques:
$$0 \le e_{m} \le 1$$
(9)
$$- 1 \le e_{{{\text{T}}k}} \le 1$$
(10)
for m = 1,…, M and k = 1,…, K, respectively.
Path Constraints
The pre-computed muscle moment arms related the muscle forces to the inverse dynamics joint reaction torques:
$$T_{{{\text{ID}}k}} = \mathop \sum \limits_{m = 1}^{M} d_{mk} F_{{{\text{T}}m}} + e_{{{\text{T}}k}} T_{\hbox{max} }$$
(11)
for k = 1,…,K, where T
IDk
is the inverse dynamics joint torque, d
mk
is the moment arm of muscle m with respect to the kth degree of freedom, and T
max = 150 Nm is the maximal torque output of the ideal actuators. T
max was chosen to have the same order of magnitude as the maximal joint torques exerted during the motion to guarantee feasibility of the dynamic optimization problem.
Constraints Imposing Muscle Dynamics
Activation dynamics was imposed using Eqs. (1)–(2). Contraction dynamics was imposed using four different formulations as described below:
-
1.
Using normalized tendon force \(\tilde{F}_{\text{T}} = \frac{{F_{\text{T}} }}{{F_{\text{M}}^{0} }}\) as a state:
$$\frac{{d\tilde{F}_{\text{T}} }}{dt} = f_{1} (a,\tilde{F}_{\text{T}} ).$$
(12)
-
2.
Using normalized muscle fiber length as a state:
$$\frac{{d\tilde{l}_{\text{M}} }}{dt} = f_{2} (a,\tilde{l}_{\text{M}} ).$$
(13)
This formulation of contraction dynamics was typically used in previous methods (e.g.,9,29).
-
3.
Using normalized tendon force as a state and introducing u
F, the scaled time derivative of the normalized tendon force, as a new control simplifying the contraction dynamic equations:
$$\frac{{d\tilde{F}_{\text{T}} }}{dt} = s_{\text{F}} u_{\text{F}} ,$$
(14)
where s
F = 10 is a scaling factor. The scaling factor was chosen such that the controls u
F had the same order of magnitude as the other controls and the states. The Hill model was then imposed as a path constraint:
$$f_{3} \left( {a,\tilde{F}_{\text{T}} ,u_{\text{F}} } \right) = 0.$$
(15)
-
4.
Using normalized muscle fiber length as a state and introducing u
v, the scaled time derivative of the normalized muscle length, as a new control simplifying the contraction dynamic equations:
$$\frac{{d\tilde{l}_{\text{M}} }}{dt} = \frac{{v_{\text{M}}^{\hbox{max} } }}{{l_{\text{M}}^{O} }}u_{\text{v}} .$$
(16)
where \(v_{\text{M}}^{ \hbox{max} } /l_{\text{M}}^{0}\) is a scaling factor that converts u
v
, normalized muscle fiber velocity \(\tilde{v}_{\text{M}} = \frac{{v_{\text{M}} }}{{v_{\text{M}}^{\hbox{max} } }}\), into the first time derivative of normalized muscle fiber length. Note that normalized muscle fiber velocity is not the first time derivative of normalized muscle length unless normalized time is being used. The Hill model was then imposed as a path constraint:
$$f_{4} \left( {a,\tilde{l}_{\text{M}} ,u_{\text{v}} } \right) = 0.$$
(17)
All functions f
i
, i = 1,…,4, were derived from the Hill model described by Eqs. (3)–(7) (see online supplement for full-form expressions). In formulation 2 and 4, which use normalized muscle fiber length as a state, F
T was computed from \(\tilde{l}_{\text{M}}\) based on Eqs. (5), (6), and (3) to evaluate joint torques (Eq. 11). Evaluating f
1 and f
2 required dividing by muscle activation. Muscle activation was bounded between 0.01 and 1 for all formulations to allow comparison of the solutions obtained with the different formulations. The optimal controls and cost function are only expected to be identical if the optimization problems are equivalent, which would not be the case if the states were bounded differently. Normalized muscle forces were bounded between 0 and 3, normalized muscle fiber lengths were bounded between 0.4 and 1.6, controls u
F were bounded between −50 and 50, and controls u
v
were bounded between −1 and 1. At the optimal solution, only the bounds on muscle excitations and muscle activations were active. The feasible set of formulations 3 and 4 differs from the feasible set of formulations 1 and 2 due to the bounds on the additional controls. However, unless these bounds are active at the optimal solution, all formulations have the same globally optimal muscle excitations. Initial and final states were constrained to be within the bounds specified for the states but were not prescribed.
The four muscle dynamic optimization problems were solved numerically through direct collocation using GPOPS-II optimal control software.22 GPOPS-II is a MATLAB program that transcribes the dynamic optimization problem to a NLP using a Legendre-Guass-Radau (LGR) quadrature collocation method. All problems were solved on a mesh with 100 equally spaced intervals using third order LGR collocation. Analysis of the mesh accuracy (see below) showed that a further increase in the number of mesh intervals had only a small influence on the optimal solution. The interior point solver IPOPT30 was used to solve the resulting large-scale NLPs using second derivative information with a NLP relative error tolerance of 1e−6 and a maximum of 2000 iterations. The open-source automatic differentiation software ADiGator23 was used to generate derivative source code for use by IPOPT. Automatic differentiation generates analytic derivatives of general functions defined by computer code by applying differentiation rules (e.g., product, quotient, and chain rules) on the elementary function operations that underlie the code.23 All computations were performed on an Intel Core i7-4600U 2.1 GHz processor with 16 GB RAM. This computation process is illustrated by the block diagram in Fig. 3.
Analysis of Results
The four problem formulations were evaluated by estimating muscle forces over one walking cycle using both the simple and complex musculoskeletal model. Convergence, optimal cost function values, mesh accuracy, and CPU times for the different formulations were compared. Mesh accuracy was studied by calculating the root mean square (RMS) difference between the excitations calculated using 100 and 200 mesh intervals, respectively. Solution robustness against changes in the initial guess for the controls and the states was also investigated. Robustness was defined as the RMS difference between excitations calculated using a hot start and an arbitrary initial guess. The hot start was obtained from muscle activations calculated using a previously proposed approach that accounted for activation dynamics but not contraction dynamics.8 These activations were used as the initial guess for both the muscle excitations and activations. Dynamically consistent muscle fiber lengths and muscle forces as well as muscle velocities and time derivatives of muscle forces were computed based on contraction dynamics using the initial guess for the activations as an input. These quantities were used as the initial guess for the other controls and states. The arbitrary initial guess consisted of constant values for all controls and states (initial guess of 0.2 for excitations, activations, and normalized tendon force; initial guess of 1 for normalized fiber lengths, initial guess of 0 for all other controls and states). In addition, the effect of bounding muscle activations between 0 and 1 instead of between 0.01 and 1 on the CPU time and mesh accuracy for the third and fourth formulations, which do not require division by muscle activation, was investigated.
Optimality of the results was verified two ways. First, a post-optimality analysis as described in detail by Graham and Rao11 was performed to investigate the proximity of the numerical solution to the true optimal solution of the dynamic optimization problem. To this end, the first-order optimality condition that the costate is the sensitivity of the cost with respect to the state along the optimal solution was verified based on the equivalence between the NLP and calculus of variations optimality conditions for LRG collocation methods. A discrete approximation of the costate of the dynamic optimization problem was obtained by a linear transformation of the Lagrange multipliers of the NLP arising from LGR collocation,10 and this computation is automatically performed by GPOPS-II when solving an optimization problem. To perform this post-optimality analysis, the dynamic optimization problem was resolved over the walking cycle imposing the previously obtained solution at the beginning of the walking cycle as the initial state. The sensitivity of the cost with respect to the initial state was approximated by resolving the dynamic optimization problem using a perturbed initial state and computing the ratio of the change in cost to the change in initial state. By comparing the costate approximations at the initial time with the sensitivity of the cost to changes of 0.001 in the initial activation of each muscle, we evaluated the optimality of the obtained solutions. For this analysis, GPOPS-II’s mesh refinement algorithm was used. Since contraction dynamics was imposed as a path constraint in formulations 3 and 4 and GPOPS-II’s mesh refinement algorithm does not account for path constraints, the mesh was refined based on activation dynamics accuracy only. This post-optimality analysis was performed for formulations 3 and 4 only, since formulations 1 and 2 did not always converge, and only for the simple model, since problem formulation and solution methods are not model-specific and computation times were much lower for the simple model.
Second, a less formal verification was performed by using the equivalence between static and dynamic optimization in the limit of zero activation and deactivation time constants and infinite tendon stiffness. Since the static optimization problem is quadratic, the global optimality of its solution can be guaranteed. Close proximity of the solution of the dynamic optimization problem with small activation and deactivation time constants and high tendon stiffness to the solution of the static optimization problem is therefore an indication of the optimality of the dynamic optimization solution. It is important to note here that optimality of the dynamic optimization solution of the problem with modified parameters does not guarantee optimality of the solution of the problem with original parameters. We resolved the dynamic optimization problem with activation and deactivation time constants of 5 ms instead of 15 and 60 ms, respectively, and by increasing the value of parameter k
T
determining the steepness of the tendon force length characteristic from 35 to 1000 (see also online supplement). We then compared the solution of this limit problem to the solution of a corresponding static optimization problem. The static optimization problem was configured to match the dynamic optimization problem as closely as possible. The cost function was the integrand of the cost functional of the dynamic optimization problem evaluated at each time instant i, where muscle excitation was replaced by muscle activation. Muscle activations were bounded between 0 and 1 whereas the inputs for the ideal torque actuators—e
Tk
—were bounded between -1 and 1. Pre-computed muscle moment arms were used to relate the tendon forces and ideal torques to the inverse dynamics joint reaction torques (Eq. 11). Muscle activation and tendon force were linearly related through Eqs. (4) and (7) using a rigid tendon with length \(l_{\text{T}}^{s}\). The static optimization problem was solved using MATLAB’s lsqlin.