Analysis of Motor Synergies Utilization for Optimal Movement Generation for a Human-like Robotic Arm

Abstract

Controlling human-like robots with musculoskeletal structure has been a challenging problem in engineering. In biological studies, motor synergy hypothesis has been proposed as a solution in order to control high degree-of-freedom and complex human body. In this paper, we focus on exploring the applicability of motor synergies in generating goal-directed movements by optimal control in a human-like robotic arm. We focus on three problems: 1) Can motor synergies facilitate the solving of optimal control problem? 2) What properties should motor synergies have in order to achieve tasks? 3) How should motor synergies be utilized better? For the first problem we show that goal-directed movements can be achieved by utilizing motor synergies which have properties of achieving the goals. For the second problem, we testify motor synergies which have different properties and discover that energy efficiency is an important aspect in motor synergies which can also be utilized to achieve goal-directed movements. This discovery also implies that we can obtain motor synergies by other ways rather than from goal-directed optimal control signals. For the third problem, we show that the control complexity can be further reduced by utilizing a subset of motor synergies which are effective to achieve goals.

1 Introduction

Musculoskeletal robots are expected to have the ability to behave like biological creatures due to their mechanical structure. However, as the degrees of freedom within a mechanical structure increases, the controlling aspect becomes more complex.

Within the field of human motor control research, the hypothesis of motor synergies[1–3] has been proposed as a solution to the degree-of-freedom problem. Among many interpretations of motor synergies, one suggests that many muscles are not controlled individually and that a few groups of muscles with specific activation patterns, namely motor synergies, are coordinated. Control can then be simplified by coordinating fewer control variables in terms of motor synergies.

However, it has been shown that human-like goal-directed movements can be predicted by optimal control[4–6]. But the curse of dimensionality[7] due to solving the optimal control problem is still one of the main difficulties to be overcome in engineering.

In this paper, we explore motor synergies in controlling musculoskeletal robotic system by optimal control. We pay special attention to the following problems: 1) Can motor synergies facilitate the solving of optimal control problem? 2) What properties should motor synergies have in order to achieve tasks? 3) How should motor synergies be utilized better? Note that [8] includes some previous results for the first and second problems.

In addition to the control complexity reduction by utilizing motor synergies, the main contributions of this research are the understanding of properties of motor synergies and the ways to obtain the motor synergies. We analyze the performance by utilizing two classes of motor synergies, namely achieving-goal synergies and fitness-optimized synergies. The former ones are extracted from solutions optimized according to the cost-to-goal function specified as task goals, whereas the latter ones are extracted from solutions optimized according to different fitness criteria having different meanings of “movement fitness” instead of explicit task goals. Moreover, since the fitness-optimized synergies are obtained according to different optimization criteria, successful achievement of some particular tasks utilizing the fitness-optimized synergies would imply that motor synergies can be extracted by other methods (e.g. genetic algorithm (GA)), rather than from solutions of the optimal control problem. Another contribution is about how to utilize a given set of motor synergies better by using an effective synergies subset. This concept of the effective synergies is similar to the one described in [9], where their effective synergies correspond to the subset of muscles that best exploit the musculoskeletal dynamics. In contrast, ours specifically means a subset of motor synergies that effectively achieve a goal specified in the cost-to-go function of the optimal control problem.

Similar studies on motor synergies properties can be found in [10], where the motor synergies were obtained from optimal control solutions only. Moreover, they studied the “time-varying synergies” which represent the spatiotemporal actuation pattern. In contrast, we study simpler “time-invariant synergies” which represent the spatial actuation pattern, the optimal controller then determines the corresponding synergy activations. In order to achieve a novel task, “time-invariant” motor synergies might be better, because they are task-independent. To achieve a (novel) task, the “time-invariant” synergies are coordinated by the corresponding time-varying synergy activations, which possess task-related information and are determined by (optimal) controller in use.

The rest of the paper starts with a brief introduction about motor synergies and optimal control in Section 2. The procedures of extracting the two classes of motor synergies are described in Section 3. In Section 4, performance of reaching movements of utilizing different motor synergies is analyzed. Section 4.3 describes the idea of effective synergies. The human-like robotic arm model is provided in Appendix.

2 Optimal control and motor synergies

In this section, we give a brief review of optimal control, which is used to generate goal-directed movements. We first give the standard formulation. Then the concept of motor synergies is briefly introduced. Finally, we demonstrate how to utilize synergies in optimal control.

2.1 Standard optimal control formulation

A standard optimal control problem for a deterministic continuous system is to find an optimal control u ^p* (t ) to minimize the cost-to-go function $\tilde J({x^P}({t_0}))$:

$$\begin{array}{*{20}{l}} {\quad \;\;{{{\mathbf{\dot x}}}^p}(t) = f({{\mathbf{x}}^p}(t),{{\mathbf{u}}^p}(t))} \\ {\quad \;\;{{\mathbf{y}}^p}(t) = h({{\mathbf{x}}^p}(t),{{\mathbf{u}}^p}(t))} \\ {\tilde J({{\mathbf{x}}^p}({t_0})) = \tilde l({{\mathbf{y}}^p}(\bar T)) + \int_{{t_0}}^{\bar T} {l({{\mathbf{y}}^p}} (t),{{\mathbf{u}}^p}(t))\operatorname{d} t} \end{array}$$

(1)

where x ^p,u ^p,y ^p, t ₀, $\bar T,\;\tilde l$ and l denote robot states, control input, observation, start time, final time, final state cost and instantaneous cost, respectively. The deterministic functions f and h describe the robot dynamics and observer, respectively. In this paper, we use a 3-joint planar robotic arm with 10 muscles to conduct our simulations. The details about f and h are described in Appendix.

The curse of dimensionality occurs in musculoskeletal robots (systems) because the number of muscles (actuators) is usually large. With the aid of motor synergies, the dimensionality can be reduced. In this research, we adopted the nonlinear optimal control framework iterative linear quadratic Gaussian (iLQG)[11] to solve for optimal solutions.

2.2 Motor synergies

There are many interpretations about motor synergies. In this paper, the motor synergies correspond to the “synchronous synergies”. A synchronous motor synergy is the spatial representation of a synchronous activation pattern of a group of actuators (muscles). Control signals of n actuators can be black approximated as linear combinations of M (M ⩽ n) motor synergies:

$${\mathbf{u}}(t) \approx \sum\limits_{i = 1}^M {{{\mathbf{w}}_i}{a_i}(t) = {\mathbf{Wa}}(t)} $$

(2)

where u (t) ∈ R ⁿ is the actuator input at time t, w _i ∈ R ⁿ and a _i (t ) ∈ R ¹ are synergy i and the corresponding synergy activation, respectively. W is an n x M matrix that collects all M synergies, and is constant black for all time t.

In contrast to the time-varying synergies with the form $u(t) = \Sigma _i^M{a_i}{w_i}(t)$ studied in [10,12], the time-invariant synergy w _i is constant throughout all the time. In the case that u consists of control signals of many different tasks, the extracted synergies w _i are task-independent, because all the task-related information is retained as temporal information in the corresponding time-varying synergy activations a _i (t ). Therefore, in order to achieve various and novel tasks, the time-invariant synergy is a more generic approach, where the controller uses the synergies as a basis and coordinates them to achieve a (novel) task.

Because biological muscles can only provide contraction forces and have non-negative control signals, all the actuator inputs u (t ), the synergies W and synergy activations a (t ) are constrained to be non-negative for better understanding of the muscles (actuators) activations in solutions and mimic biological control system. The non-negative motor synergies are extracted by using non-negative matrix factorization (NMF). The extraction will be explained later in Section 3.1. The term synergies is used for short to the non-negative synchronous motor synergies. Fig. 1 depicts an example of a set of synergies consisting of 3 synergies. Readers can refer to [3] for more details about motor synergies.

Fig. 1

A set of 3 synergies extracted by NMF (M = 3). Each synergy is time invariant, specifying synchronous activations of all the 10 muscles

2.3 Utilizing motor synergies

Given a set of synergies W ∈ R ^{n ×M} where M < n , the control space is transformed into one with lower dimensionality by (2), i.e., u ^p = W a ^p. Then the optimal control problem can be solved in lower dimensionality with respect to a ^p ∈ R ^M :

$$\begin{array}{*{20}{l}} {\quad \;\;\,{{{\mathbf{\dot x}}}^p}(t) = f({{\mathbf{x}}^p}(t),{\mathbf{W}}{{\mathbf{a}}^p}(t))} \\ {\quad \;\;\,{{\mathbf{y}}^p}(t) = h({{\mathbf{x}}^p}(t),{\mathbf{W}}{{\mathbf{a}}^p}(t))} \\ {J({{\mathbf{x}}^p}({t_0})) = \tilde l({{\mathbf{y}}^p}(\bar T)) + \int_0^{\bar T} {l({{\mathbf{y}}^p}(t),{\mathbf{W}}{{\mathbf{a}}^p}(t))\operatorname{d} t} } \end{array}$$

(3)

which is to find the optimal synergy activation a ^p *(t ) such that the cost-to-go function J (x ^p (t ₀ )) is minimized.

3 Synergies with different properties

Synergies inherit the properties from movements. There are two classes of motor synergies, achieving-goal synergies and fitness-optimized synergies, which will be utilized in this research. The two classes of synergies have different properties because they are extracted from movements which are generated by different approaches. In this section, we explain how to obtain the two classes of synergies in detail. We first describe how to extract motor synergies from a set of control signals. Then the construction of control signals with different properties will be explained.

3.1 Extracting motor synergies

Synergies can be extracted by various common pattern recognition tools as demonstrated in [13]. In this paper, NMF is adopted to extract synergies, such that the root-mean squared residual (‖U ⁰ − U ^M‖ _f /(Nu)) ^1/2 between the original signals U ⁰ and the reconstructed signals U ^M = W ^M A ^M is minimized, where N ^u is the number of elements in the matrix U ⁰ (or U ^M ), W ^M ∈ R ^{n x M} is the set of M synergies extracted, A ^M ∈ R ^{M x (N} "^{/n )} is the corresponding synergy activations. Extraction is performed by function nnmf in the Matlab statistic toolbox. For example, a synergies set W^M ∈ R^{n xM} can be extracted from r movement, by applying NMF to a control signal matrix concatenated from all r movements which are gathered based on a certain criterion, where $\tilde u_j^0,j = 1, \cdots$ , r is the series of control signals of movement with index j. All synergies extracted by NMF are then normalized as follows

$${w_{ij}} \leftarrow \frac{{{w_{ij}}}}{{\sqrt {\sum\limits_{k = 1}^n {{{({w_{kj}})}^2}} } }}$$

(4)

where W = {w _j , j = 1,••• , M }, and w _j = {w _ij , i = 1, • • • , n }. In the following sections it will be described in detail how to gather movements and obtain different synergies by using different criteria.

3.2 Class 1: Achieving-goal synergies

Achieving-goal synergies are extracted from the optimal control solutions of (1), that is, the optimal solutions without utilizing synergies. It is named as “achieving-goal” because they are extracted from the optimal solutions of (1), having the properties of achieving goal which is explicitly specified in the cost-to-goal function (e.g. reaching targets in Cartesian coordinates). It is reasonable that the same goals can be achieved by utilizing achieving-goal synergies. The achieving-goal synergies are denoted as W ^M ∈ R ^{n x M} where n and M are the dimension of control signals and the number of synergies extracted, respectively.

3.3 Class 2: Fitness-optimized synergies

Fitness-optimized synergies are extracted from control signals of movements which are generated by optimizing movements according to specific movement fitness criteria. The movement fitness is not related to any explicit task goal such as targets in Cartesian coordinates. Therefore, the resulting movements are different from the goal-directed movements generated by solving the optimal control problem. In this paper, fitness-optimized synergies are extracted from some “good” movements which are generated by using GA with three different movement fitness criteria, namely energy-efficient, curvature and jerk. Details of generating such movements are as follows.

A movement ξ = {x _t , u _t} , t = 1, • • • , $\bar T$ is generated from a given configuration x ₀ and final time $\bar T$ using the arm dynamics described in Appendix. The activation of a muscle (actuator) i at time t is defined by a Gaussian function with 3 parameters controlling the amplitude A _i , the width σ _i and the center c _i of the Gaussian:

$${u_i}(t) = {A_i}\exp \left( {\frac{{{{(t - {c_i})}^2}}}{{2{\sigma _i}}}} \right),\quad t = 1, \cdots ,\bar T.$$

(5)

A “good” movement ξ _q is generated by optimizing the activation parameters Ξ _q = [A _i , σ _i , c _i ] _q , i = 1, • • • , 10 of each muscle such that the value of the movement according to the following movement fitness functions V _q , q = {en, cur, jerk} are minimized:

(6)

where V _en , V _cur and V _jerk relate to input-output efficiency, smoothness of trajectory in the Cartesian plane, and jerk of end-effector trajectory in the Cartesian plane, respectively. ∠ _t is the angle between successive end-effector velocities: ∠ _t = | cos⁻¹ (〈û (t − 1), û (t)〉)| where 〈 û (t −1),û (t) 〉 denotes the dot product of successive normalized end-effector velocities with û = u /(||u ||₂).

The reciprocal of V _en measures the ratio between the total kinetic energy output and the total force input to the robot. Therefore, a smaller value of V _en implies a better energy efficiency. V _cur measures the smoothness by the maximum change of trajectory tangent. V _jerk is the modified version of measuring trajectory smoothness defined in [14], where $\dddot p$ denotes the rate of change of acceleration. Detailed description of other symbols can be found in Appendix.

Each movement ξ _q is optimized with given random final time $\bar T$ and random resting (zero velocities) configuration x ₀ with respect to each movement fitness criterion V _q by adopting GA to search for values of Ξ _q = [A _i , σ _i , c _i ] _q , i = ⋯ 10, such that V _q is minimized. The GA optimization is performed by using function ga in Matlab. For each movement, the population size of Ξ _q is set to 20 and then 5 generations are carried out before the termination of evolution.

${\text{W}}_{en}^M,{\text{W}}_{cur}^M$ and ${\text{W}}_{jerk}^M$ are used to denote the energy-efficient synergies, curvature synergies, the jerk synergies, respectively, where M is the number of synergies extracted.

3.4 Class 3: Random synergies

In order to verify the necessity of the inherent properties in motor synergies, we construct non-negative random synergies which are drawn from identical distribution and normalized by using (4). We use Ŵ ^M to denote the random synergies.

4 Performance analysis

Reaching task is a common goal-directed movement in human motion. In this research, simulations of reaching movements of the human-like robotic arm are analyzed with focuses on the following issues:

• 1) Whether motor synergies can facilitate the solving of the optimal control problem?

• 2) What properties should motor synergies have?

The reaching movements are achieved by defining the cost-to-go function:

(7)

where x ^p and u ^p are the robot states and muscle inputs. y ^p is a vector containing positions and velocities of the end effectors in the Cartesian coordinates and y ^p* denotes the target positions and velocities. The movement time T is determined by Fitts’s law $\bar T = a + b{\log _2}(1 + 2d/w)$ to mimic human motion behavior, where d is the distance from the target and w is the tolerance. The values of parameters P = diag{10000, 1000}, Q = 0, R = I , a = 0, b = 0.08, w = 0.02(≈ 2 cm) are used. With this setting, the cost-to-go value J approximately equals the Euclidean distance d from desired targets in centimeter (J = 1 implies d≈1cm in our case).

In this section, we first outline our experiments which correspond to the above two objectives respectively. Then results of the reaching movements are presented accordingly. For simplicity, here we define:

• 1) Original solutions: Solutions without utilizing synergies by solving (1) directly.

• 2) Synergies-goal solutions: Solutions utilizing achieving-goal synergies by solving (3).

• 3) Synergies-en/cur/jerk solutions: Solutions utilizing fitness-optimized synergies by solving (3), in which the synergies are extracted from movements generated by GA according to movement fitness functions V _en , V _cur or V _jerk .

• 4) Random solutions: Solutions utilizing random synergies by solving (3).

4.1 Experiment 1: Feasibility of motor synergies approach

The objective of this experiment is to verify the feasibility of motor synergies approach in solving the optimal control problem. Achieving-goal synergies are utilized in this experiment. Because it is reasonable that achieving-goal synergies have properties to achieve the same goals. We would say that the motor synergies approach is feasible if its (synergies-goal solutions)

• 1) success rate of reaching target has similar level, and

• 2) computation expense is reduced

when compared with original solutions. In the next experiment, the properties of motor synergies will be explored.

The experiment procedure is summarized in Fig. 2 (a). In this experiment, 400 pairs of initial and target position of the end effector {x ₀ ,y ^p*} were uniformly distributed over the workspace such that there were 100 pairs in each of the four target distance $d = ||y_0^p - {y^{{p^ * }}}||_2^2$ ranges (reaching distance): [0–10] cm, [10–20] cm, [20–30] cm and [30–40] cm. Firstly, the 400 original solutions were simulated. Then achieving-goal synergies ${\text{W}}_*^M$ were extracted from the control signals in the original solutions by NMF as described in Section 3.1. Eight sets of synergies ${\text{W}}_*^M$ were extracted where M = 3, • • • , 10. Secondly, for each M = 3, • • • , 10, the 400 synergies-goal solutions were simulated utilizing ${\text{W}}_*^M$ using the same 400 pairs of {x ₀, y ^p* }. Finally, the 400 random solutions were simulated for verifying the necessity of the inherent characteristics in motor synergies.

Fig. 2

Summary of (a) Experiment 1 and (b) Experiment 2. All reaching movements were carried out using the same 400 pairs of initial and target positions {x ₀ , y ^p*}

The optimal control problem solutions were solved by adopting the iLQG[11]. Iteration of iLQG was terminated if improvement of cost-to-go value is smaller than 10–³ or if the cost-to-go value is smaller than 1 (i.e., distance from target < 1cm).

The box-plots of cost-to-go values (log₁₀ scaled) and total computation time spent of utilizing synergies ${\text{W}}_*^M$ for different reaching distances are depicted in Figs. 3 (a) and (b), respectively. Because a smaller value of cost-to-go value means a closer distance from the desired target, a lower position of the percentiles markers implies a better success rate. In Fig. 3 (a), it is observed that when more than 3 synergies are utilized (M > 3), the cost-to-go values are not much different from the original solutions (M = 0) for all the four reaching distance ranges. On the other hand, it is observed in Fig. 3 (b) that the computation expense is reduced as expected. The computation time spent is less than the original solutions and lesser time is spent when fewer synergies are utilized. In particular, from careful inspection at the data’s percentile markers it is observed that the success rate is better in the synergies-goal solutions, especially for reaching further targets, since the percentiles markers has lower positions in the box-plots.

Fig. 3

Performances of utilizing different numbers of synergies for different reaching distances. The cost-to-go values of utilizing achieving-goal synergies W ^M and energy-efficient synergies ${\text{W}}_{en}^M$ are depicted in (a) and (c); the total computation times spent are depicted in (b) and (d), respectively. Each column represents the results of the 100 movements. “M =0” and “M =[3–10]” refer to the original solutions utilizing no synergies, and the number of synergies utilized. The two edges and the horizontal line in each box indicate the 25th, 75th and the median (50th) percentile of the data. The 60th, 70th, 80th, 90th and 100th percentiles are indicated by markers “+”, “x”, “∇”, “◊” and “□”, respectively. It can be observed that utilizing more than 3 achieving-goal synergies M > 3 has a similar success rate compared with the original solutions, with remarkable reduction in computation expense. Similar trend can also be observed when energy-efficient synergies are utilized

Fig. 4 depicts the performance of utilizing random synergies Ŵ ^M . In contrast to the synergies-goal solutions, it is obvious that targets could not be reached in random solutions. This implies that synergies must possess inherent properties in some sense. For better illustration, the end effector trajectories of a movement are depicted in Fig. 5. It is obvious that the resulting trajectories are closed to the original solutions (dash line in the figure) when more synergies are utilized. Moreover, the success rate is better when synergies are utilized in this reaching task because the resulting cost-to-go values are lower.

Fig. 4

Performance of utilizing random synergies Ŵ ^M . It can be observed that goals could not be achieved

Fig. 5

Trajectories of a reaching task: A reaching movement of the original solutions is shown in dash line (M =0). Movements utilizing different numbers of achieving-goal synergies are shown in different lines (M = K indicates K synergies are utilized). The corresponding cost-to-go values are also shown (smaller values for better trajectory). It is obvious that when more than 3 synergies are utilized, the resulting trajectories are closed to the original solutions (M =0)

Fig. 6 depicts the induced accelerations by synergies of the trajectory depicted in Fig. 5 (M = 5). The induced acceleration vector ${\tilde a^i}(k)$ at time step k is computed from the difference of the end effector velocities in the task space: ${\tilde a^i}(k) = {\tilde v_i}(k + 1) - v(k)$, where ${\tilde v_i}(k + 1)$ is the task space velocities after applying only a synergy w _i with maximum activation $a_i^{\max }$, i.e., $u(t) = {w_i}a_i^{\max }$, at current state x _k . It can be observed that the directions of the induced accelerations are quite constant, and spread towards different directions to span the task space. This activation pattern can be interpreted as follows: At the beginning, Synergy 3 brings the end effector towards the target and Synergy 1 acts in the opposite direction to control the speed. Near the end of the trajectory, the acting of Synergy 3 decreases and the other four synergies increase in order to stop at the target. This “sparse” synergies activation pattern implies that there exists a subset of synergies which always act to achieve a task, while some synergies do not contribute. We will discuss more about this in Section 4.3.

Fig. 6

The trajectory shown is the same as the one depicted in Fig. 6 when 5 synergies (M = 5) are utilized. The arrows along the trajectory denote the induced accelerations of individual synergies. (b) The achieving-goal synergies utilized and the corresponding synergy activations of generating the trajectory in (a). It is observed that Synergy 3 and Synergy 1 dominate the control signals. Synergy 3 and Synergy 1 act the opposite and the target, whereas the other synergies act to other directions. Near the end of the movement the activation of Synergy 3 decreases, while the other 4 synergy activations increase in order to decelerate the movement

4.2 Experiment 2: Utilizing synergies with different properties

In the previous experiment, we verified that the optimal control problem could be solved by utilizing achieving-goal synergies. In this experiment, we extent our focus to investigate what properties of motor synergies should have in order to solve the optimal control problem. From this experiment, we can also obtain some hints about how to obtain motor synergies. Because obtaining achieving-goal synergies requires solving for original solutions in advance, it is better to discover a way to obtain motor synergies and then to solve the optimal control problem directly.

The experiment procedure is summarized in Fig. 3(b). We first obtain three sets of fitness-optimized synergies, namely energy-efficient synergies ${\text{W}}_{en}^M$ curvature synergies ${\text{W}}_{cur}^M$ and jerk synergies ${\text{W}}_{jerk}^M$. Within each of the three sets, M = 5, • • • , 10 synergies are extracted from 2000 “good” movements generated by GA according to the movement fitness functions V _en , V _cur and V _jerk , respectively, as described in Section 3.3. Then simulations of the reaching movements utilizing ${\text{W}}_{en}^M{\text{, W}}_{cur}^M$ and ${\text{W}}_{jerk}^M$ were carried out, using the same 400 initial and target pairs {x ₀ ,y ^p* } of Experiment 1. (Synergies M = 3, 4 were not tested because of large reconstruction error in NMF extraction.)

Fig. 7 depicts the comparison of the original solutions, the synergies-goal solutions and the synergies-en/cur/jerk solutions. For better illustration, only the results of reaching 100 targets [30–40] cm away while utilizing 6 synergies (M = 6) are shown. The results of reaching the remaining 300 targets and utilizing different numbers of synergies also have the same trend as depicted in Figs. 3 (c) and (d). These results show that utilizing energy-efficient synergies can also achieve similar performance compared with utilizing achieving-goal synergies. In contrast, utilizing curvature synergies and jerk synergies performed badly. Fig. 8 depicts a similarity matrix to measures the difference between the achieving-goal synergies ${\text{W}}_*^M$ and the energy-efficient synergies ${\text{W}}_{en}^M$. We found that the two sets of synergies are quite different. The maximum value of all dot product measures between ${\text{W}}_*^M$ and ${\text{W}}_{en}^M$ is about 0.25 for all M . Nevertheless, the energy-efficient synergies can still be utilized to achieve the reaching tasks.

Fig. 7

Performance of the reaching task (target range: d = [30 – 40] cm) utilizing 6 synergies (M = 6). Distribution of (a) cost-to-go values and (b) total computation time spent are shown. The columns from the left represent results of original solutions (Org nosyn), synergies-goal solutions (Syn* M6), synergiesen solutions (En. eff.), synergies-cur solutions (Curvature) and synergies-jerk solutions (Jerk), respectively. It can be observed that utilizing energy-e

Fig. 8

Similarity matrix of achieving-goal synergies and energy-efficient synergies (M = 6). The values of the matrix are the dot product of an achieving-goal synergy and an energy-efficient synergy. Lighter color has closer value of 1. efficient synergy. Lighter color has closer value of 1. The synergy 2 of the achieving goal synergies and the synergy 4 of the energy are the most similar with dot product value = 0.2489.

4.3 Experiment 3: Utilizing effective synergies

In this section, we explore more about motor synergies in the sense of how to utilize them better. Given a set of synergies, usually not all synergies contribute to achieving goal at the same time. This can be explained by using a linear dynamics system x _k+1 = A _k x _k + B _k u _k to approximate the nonlinear dynamic. The existence of null space in the control dependent term B _k enables the selection of some synergies to achieve a specific goal in solving the optimal solution. Only some synergies in a given set of synergies are acting to achieve goal and therefore we call them effective synergies. The existence of effective synergies can be observed in synergies-goal solutions, that is, only a subset of synergies are with nonzero activation. Therefore, if we can predict those effective synergies in advance, the control complexity and thus computation expense might possibly be further reduced in the sense of solving for solutions in a lower dimensional space spanned by the effective synergies. In this section, we explore the possibility of such idea in order to further improve the utilization of motor synergies.

In optimal control, a cost-to-go function in the form J _k = a(x _k ) + b(x _k ,π _k ) estimates the cost-to-go value using current state x _k and control law π _k . One can then use the control-dependent term b( x _k ,π _k ) (or a modified one which can approximate b) to estimate the contribution of each synergy. The set of effective synergies can then be collected by ignoring synergies which have no, small or negative (acting away from goal) contribution. Fig. 9 depicts the results of reaching targets at [30–40] cm, utilizing 8 achieving-goal synergies ${\text{W}}_*^M,\,M = 8$. It can be observed that for the case utilizing effective synergies in ${\text{W}}_*^8$ (the leftmost column, eff8syn), the computation time spent is reduced, although with a penalization of slight decrease of the success rate. Comparing with the case utilizing effective muscles within 10 muscles (effnosyn, W = I _{10 × 10}), the success rates are at the same level but more computation time is spent. These results show that by predicting and utilizing the “effective synergies” subset, faster computation can be achieved, although accompanied with a slight decrease of success rate.

Fig. 9

Results of utilizing effective synergies of 100 movements reaching targets [30–40] cm away. The 4 sets of data from the left of (a) correspond to utilizing “eff8syn”: effective synergies in ${\text{W}}_*^8$; “syn8”: ${\text{W}}_*^8$ only; “org”: all muscles (no synergies, i.e., original solution) and “effnosyn”: effective muscles from all 10 muscles. It can be observed that utilization of effective synergies reduce the computation expense, with a penalization of slight decrease of the success rate

5 Discussions and future work

Time-invariant vs. time-varying synergies: In this paper we adopted “synchronous synergies”, which specify the spatial activation pattern of all actuators at a time. In contrast, “time-varying synergies” have both the spatial component specifying the actuation pattern, and temporal component specifying the time profile of synergy activation. For “synchronous synergies”, application is easier because synergies can be extracted by common pattern recognition tools[13]. As shown in this paper, solving for optimal solution utilizing “synchronous synergies” is equivalent to finding the corresponding temporal profiles of synergy activations. For “time-varying synergies”, the synergies are more difficult to be extracted[15]. But utilizing “time-varying synergies” might further simplify the control problem because it requires much fewer time profile parameters to be determined at the beginning of motion (t = 0) instead of computing solution at each time step. Moreover, the time-delay information of system is already possessed in the temporal profile which may also simplify the control. Research of combining the two kinds of synergies is also a prospective direction.

Properties of motor synergies: In this paper, two types of synergies, namely achieving-goal synergies and fitness-optimized synergies, which possess different properties were examined. It is reasonable that the achieving-goal synergies W ^M can solve the same goal-directed problem. The success of reaching task utilizing energy-efficient synergies suggests that “energy efficient” may be one of the common properties which is an important aspect of motor synergies. This inherent energy-related property may be sufficient and versatile for achieving other novel tasks. One intuitive reason is that both the synergies W ^M and may possess common properties such as mechanical or structural characteristics of the robot itself. Exploring the relationship between the achieving-goal synergies and energy-efficient synergies will be one of the future topics.

Learning/Obtaining motor synergies: In practice, it is better to consider how to obtain suitable synergies sets for noisy and changing environment. The satisfactory results of utilizing the energy-efficient synergies extracted from rather “good energy efficient” movements suggests that synergies may also be obtained during a trial-by-trial process, where synergies can be “improved” from “good trials” possessing energy-efficient properties. For example, one may collect segments of many “good energy efficient” random movements, and extract the energy-efficient synergies. Investigation of such synergies improvement in trial-by-trial process is also a prospective research direction.

6 Conclusions

In this paper, we investigated the applicability of motor synergies in controlling musculoskeletal robotic system by optimal control. The simulations of reaching movements of a human-like robotic arm were analyzed. Two types of motor synergies having different properties were utilized in simulations, namely achieving-goal synergies and fitness-optimized synergies. The former ones have the properties of achieving-goal because they are extracted from the optimal solutions of achieving some task goals. The latter ones are fitness-optimized because they are extracted from some “good” movements which are optimized according to three specific movement fitness criteria, namely energy-efficient synergies, curvature synergies and jerk synergies.

Firstly, we verified that motor synergies can facilitate the solving of the optimal control problem utilizing achieving-goal synergies. Simulations show that the reaching task can still be achieved, with the same level of success rate and less computation expense. Furthermore, we investigated what properties motor synergies should have by analyzing the performance of utilizing fitness-optimized synergies. The reaching task could still be achieved when energy-efficient synergies are utilized. These results imply that in order to achieve goal-directed task, synergies do not necessarily need to have achieving-goal properties. Synergies having other properties such as energy-efficient can also be utilized to achieve goal-directed task. Our results also suggest that it is possible to obtain synergies by other methods rather than from solving optimal control problem. Finally, we investigated how to utilize motor synergies better. We found that by predicting and subsequently utilizing the effective synergies, which is a subset of synergies contributing to achieve the goal in solving the optimal control problem, the control complexity and thus computation expense can be further reduced, with a penalization of slight decrease of the success rate.

Appendix: Human-like arm model

A 3-joint planar robotic arm with 10 linear actuators was constructed as the common simulation platform as shown in Fig. 10. The structural parameters such as mass, length, and muscle attachment positions of the 3-joint robotic arm are designed by referring to human anatomical data. The 10 muscles consist of two Biceps (1.BIC_ln , 2.BIC _sh ), Brachialis (3.BRA), Brachioradialis (4.BRD), Pronator teres (5.PT), three Triceps (6.TRI_ln , 7.TRI _lt , 8.TRI _m ), a hand flexor (9. Hand _flx ) and a hand extensor (10.Hand _ext ). Because our main focus is on the dimensionality problem, the inherent nonlinear time-varying dynamics of human muscle[16] are simplified as linear actuators without time delay. Thus this 3-joint robotic arm model provides enough complexity to mimic human structure and simplicity for research investigation. The dynamics of the arm is modeled in a standard form:

$$H({\mathbf{\theta }}){\mathbf{\ddot \theta }} + V({\mathbf{\theta }},{\mathbf{\dot \theta }}){\mathbf{\dot \theta }} + G({\mathbf{\theta }}) = \tau ({\mathbf{\theta }},{{\mathbf{u}}^p})$$

(8)

where θ = [θ ₁ θ ₂ θ₃ ]^T and u ^p = [u₁ … u₁₀ ]^T are the joint angles and the control signals, respectively. The joint torque τ (θ, u ^p) is the sum of resultant joint torques of individual muscle forces:

$$\tau ({\mathbf{\theta }},{\mathbf{u}}) = \sum\limits_i {J_i^{\mathbf{T}}({\mathbf{\theta }})} {\hat f^i}({\mathbf{u}})$$

(9)

where ${\hat f^i}(u)$ is the muscle force i and $J_i^T(\theta )$ is the corresponding Jacobian transformation matrix. The numbers of ${\hat f^i}$ and $J_i^T$ depend on whether the wrapping of muscles happens. In the case of no wrapping of muscles at joints, a muscle is considered as two identical external forces but opposite in the direction acting at the two corresponding attachment points. In the case of wrapping at q joints, a muscle is considered as 2(q + 1) external force pairs acting at the corresponding attachment and wrapping points. The amplitude of muscle force ${\hat f^i}$ is modeled as linear mapping from the corresponding control input u:

$${\hat f^i}({\mathbf{u}}) = \left| {f_{\max }^i \cdot {u^i}} \right|$$

(10)

where $f_{\max }^i$ is the maximum force of muscle i .

Fig. 10

A set of 3 synergies extracted by NMF (M = 3). Each synergy is time invariant, specifying synchronous activations of all the 10 muscles

By expressing the robot state as the joint angles and angular velocities x ^p = [θ ₁ θ ₂ θ ₃ θ ₁ θ 2 θ ₃]^T , the control input as the muscle input u ^p = [u ₁ … u 10]^T, and the end effector’s Cartesian positions and velocities as y ^p = [p _x py p ˙x p ˙y ]^T, the standard forms (1) and (3) can be obtained as

$$\begin{array}{*{20}{l}} {{{{\mathbf{\dot x}}}^p}(t) = f({{\mathbf{x}}^p}(t),{{\mathbf{u}}^p}(t))} \\ {{{{\mathbf{\dot y}}}^p}(t) = h({{\mathbf{x}}^p}(t)).} \end{array}$$

(11)

Computer simulation over period t $\in \;[0,\bar T]$ is carried out by Euler integration by discretization with t = kΔt:

$$\begin{array}{*{20}{l}} {{\mathbf{x}}_{k + 1}^p = {\mathbf{x}}_k^p + \Delta tf({\mathbf{x}}_k^p,{\mathbf{u}}_k^p),k = 0, \cdots ,N} \\ {{\mathbf{y}}_{k + 1}^p = h({\mathbf{x}}_{k + 1}^p)} \end{array}$$

(12)

where $x_k^p = {x^p}(k\Delta t)$ and $u_k^p = {u^p}(k\Delta t)$. In our arm model with physical properties based on human anatomical data, it was found that the simulation becomes unstable with a large time step and thus Δt = 0.001 second was used.