Fuzzy neuronal model of motor control inspired by cerebellar pathways to online and gradually learn inverse biomechanical functions in the presence of delay

Abstract

Contrary to forward biomechanical functions, which are deterministic, inverse biomechanical functions are generally not. Calculating an inverse biomechanical function is an ill-posed problem, which has no unique solution for a manipulator with several degrees of freedom. Studies of the command and control of biological movements suggest that the cerebellum takes part in the computation of approximate inverse functions, and this ability can control fast movements by predicting the consequence of current motor command. Limb movements toward a goal are defined as fast if they last less than the total duration of the processing and transmission delays in the motor and sensory pathways. Because of these delays, fast movements cannot be continuously controlled in a closed loop by use of sensory signals. Thus, fast movements must be controlled by some open loop controller, of which cerebellar pathways constitute an important part. This article presents a system-level fuzzy neuronal motor control circuit, inspired by the cerebellar pathways. The cerebellar cortex (CC) is assumed to embed internal models of the biomechanical functions of the limb segments. Such neural models are able to predict the consequences of motor commands and issue predictive signals encoding movement variables, which are sent to the controller via internal feedback loops. Differences between desired and expected values of variables of movements are calculated in the deep cerebellar nuclei (DCN). After motor learning, the whole circuit can approximate the inverse function of the biomechanical function of a limb and acts as a controller. In this research, internal models of direct biomechanical functions are learned and embedded in the connectivity of the cerebellar pathways. Two fuzzy neural networks represent the two parts of the cerebellum, and an online gradual learning drives the acquisition of the internal models in CC and the controlling rules in DCN. As during real learning, exercise and repetition increase skill and speed. The learning procedure is started by a simple and slow movement, controlled in the presence of delays by a simple closed loop controller comparable to the spinal reflexes. The speed of the movements is then increased gradually, and output error signals are used to compute teaching signals and drive learning. Repetition of movements at each speed level allows to properly set the two neural networks, and progressively learn the movement. Finally, conditions of stability of the proposed model as an inverter are identified. Next, the control of a single segment arm, moved by two muscles, is simulated. After proper setting by motor learning, the circuit is able to reject perturbations.

Appendices

Appendix 1: Stability

The conditions of stability of the system are hereafter studied by the linearization method. The motor command signal $\alpha$ and the forward biomechanical function F are approximated by Taylor’s expansions. The results show that the stability domain depends on the current state of the system x(t), the current motor command $\alpha (t)$, and the value of an adjustable coefficient K. Stability can be preserved, provided the value of K is dynamically set at any time.

At time t, the system is assumed to be in equilibrium and the motor command signal to be $\alpha _{{0}}$. Linearization of the function of the motor order by Taylor’s expansion around the equilibrium gives:

$$\begin{aligned} \alpha (t+1)-\alpha _{0}=\left( \alpha (t)-\alpha _{0} \right) \frac{\partial \alpha (t+1)}{\partial \alpha (t)}\vert _{\alpha (t)=\alpha _{0}} \end{aligned}$$

(A-1)

The equation can be derived for a next step:

$$\begin{aligned} \alpha \left( t+2 \right) -\alpha _{0}=\left( \alpha \left( t+1 \right) -\alpha _{0} \right) \frac{\partial \alpha \left( t+1 \right) }{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}} \end{aligned}$$

(A-2)

From (A-1) and (A-2), it comes:

$$\begin{aligned} \alpha \left( t+2 \right) -\alpha _{0}=\left( \alpha \left( t \right) -\alpha _{0} \right) {\left( \frac{\partial \alpha \left( t+1 \right) }{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}\right) }^{2} \end{aligned}$$

(A-3)

This relation can be recurrently generalized for any step n to the next one:

$$\begin{aligned} \alpha \left( t+n \right) -\alpha _{0}\!=\!\left( \alpha \left( t \right) -\alpha _{0} \right) {\left( \frac{\partial \alpha \left( t+1 \right) }{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) \!=\!\alpha _{0}}\right) }^{n}\quad \end{aligned}$$

(A-4)

According to (A-4), after a little deviation from the equilibrium point $\alpha _{{0}}$ at time t, the system can return to equilibrium if, after n steps, the following equation is satisfied:

$$\begin{aligned} \alpha \left( t+n \right) =\alpha _{0} \end{aligned}$$

(A-5)

So, based on (A-4) and (A-5), the following equation must be satisfied:

$$\begin{aligned} \left( \alpha \left( t \right) -\alpha _{0} \right) {\left( \frac{\partial \alpha \left( t+1 \right) }{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}\right) }^{n}=0 \end{aligned}$$

(A-6)

Because of the deviation from the equilibrium point at time t, the value $\alpha (t)$ is not equal to $\alpha _{{0}}$; so the following equation must be satisfied:

$$\begin{aligned} {\left( \frac{\partial \alpha \left( t+1 \right) }{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}\right) }^{n}=0 \end{aligned}$$

(A-7)

and considering n as a positive integer number, the following condition is derived:

$$\begin{aligned} -1<\frac{\partial \alpha \left( t+1 \right) }{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}<1 \end{aligned}$$

(A-8)

Considering the structure of the circuit, this equation can be rewritten as:

$$\begin{aligned}&-1<1-K\frac{\partial F^{*}}{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}<1\nonumber \\&\mathrm {so}{:}\, 0<K\frac{\partial F^{*}}{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}<2 \end{aligned}$$

(A-9)

where $F^{*}$ approximates F, the forward biomechanical function of the limb.

If $K = 1$, the following condition is derived from (A-9):

$$\begin{aligned} 0<\frac{\partial F^{*}}{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}<2 \end{aligned}$$

(A-10)

Appendix 2: Value of coefficient K

Assume that the biomechanical function F of the limb can be approximated at any time by a linear system and that a neural network embeds an inner representation $F^{*}$. Assume also that the feedback and motor pathways have some small, but not negligible transmission delays: ${D}_{{1}}$ in the feedback pathway, which transmits an efference copy of the motor order, and ${D}_{{2}}$ in the motor pathway from the CNS to the motoneurones, which innervate the muscles of the limb. The circuit shown in Fig. 1 translates into a structure the equations in the Laplace space (Fig. 8). A Taylor’s expansion of ${D}_{{i}}=\mathrm{e}^{-Ds}$, the Laplace transform of a time delay ${D}_{{i}}$, allows to approximate ${D}_{{i}}$ by ${1-}{D}_{{i}}{\cdot }{s}$.

Fig. 8

The circuit representation in Laplace space

From the desired position $\theta ^{\mathrm{D}}$, two equations determine the motor command $\alpha$ and the actual position $\theta ^{\mathrm {A}}$:

$$\begin{aligned}&\theta ^{\mathrm{A}}=\alpha {\cdot }\left( 1-D_{2}{\cdot } s \right) {\cdot } F \end{aligned}$$

(A-11)

$$\begin{aligned}&\alpha =\alpha {\cdot }\left( 1-D_{1}{\cdot } s \right) +Ks{\cdot }\left[ \theta ^{\mathrm{D}}-\alpha {\cdot }\left( 1-D_{1}{\cdot } s \right) F^{*} \right] \nonumber \\&\qquad +\,K_{0}{\cdot } s{\cdot }\theta ^{\mathrm{D}} \end{aligned}$$

(A-12)

From (A-12), the desired angular position $\theta ^{\mathrm{D}}$ can be calculated as:

$$\begin{aligned} \theta ^{\mathrm{D}}=\frac{\alpha {\cdot }\left[ D_{1}+K\cdot \left( 1-D_{1}{\cdot } s \right) {\cdot } F^{*} \right] }{K+ K_{0}} \end{aligned}$$

(A-13)

If an approximate inverse function is actually computed, the desired and actual angular positions are equal:

$$\begin{aligned} \theta ^{\mathrm{D}}=\theta ^{\mathrm{A}} \end{aligned}$$

(A-14)

So, based on (A-11), (A-13) and (A-14), the following equation is derived:

$$\begin{aligned} \frac{\alpha {\cdot }\left[ D_{1}+K{\cdot }\left( 1-D_{1}{\cdot } s \right) {\cdot } F^{*} \right] }{K+ K_{0}}=\alpha {\cdot } \left( 1-D_{2}{\cdot } s \right) {\cdot } F \end{aligned}$$

(A-15)

A value of K, suitable to approximate the inverse function; can be deduced:

$$\begin{aligned} K=\frac{D_{1}-K_{0}{\cdot } \left( 1-D_{2}{\cdot } s \right) {\cdot } F}{\left( 1-D_{2}{\cdot } s \right) {\cdot } F-\left( 1-D_{1}{\cdot } s \right) {\cdot } {F}^{*}} \end{aligned}$$

(A-16)

By considering the error E between F and $F^{*}$ equal to $E = F - F^{*}$, the above equation can be rewritten as:

$$\begin{aligned} K=\frac{D_{1}-K_{0}{\cdot } \left( 1-D_{2}{\cdot } s \right) {\cdot } F}{\left( D_{1}-D_{2} \right) {\cdot } s{\cdot } F+\left( 1-D_{1}{\cdot } s \right) {\cdot } E} \end{aligned}$$

(A-17)

So the proposed structure can compute an approximate inverse function of the biomechanical function, provided that the time-dependent coefficient K be dynamically set to a suitable value.

Appendix 3: Structure of fuzzy neural networks

FNNs with a 5-layer structure stand for the neuronal networks of the cerebellar cortex (CC) and deep cerebellar nuclei (DCN) (Fig. 9).

Fig. 9

A fuzzy neural network with dynamic structure shaping (Grown and Prune). The inputs to the Rule Generator neuron are the signals from the input layer (via track no. 1), the output signals of the fuzzy rule layer (via track no. 1), and the error signal measuring the difference between the desired and the actual outputs (via track no. 3). If the coverage of the incoming information by the fuzzy rules is insufficient (this being measured by comparison to a predefined threshold value), the Rule Generator neuron adds new fuzzy rules. This can be interpreted as activating inactive neurons (granular cells in the cerebellar cortex)

The layers of such FNNs process signals according to precise rules:

• 1. Input layer This first layer receives an input vector with n dimensions ${X} = [{x}_{{1}}, {x}_{{2}}, \ldots , {x}_{{n}}]^{\mathrm{T}}$, which it projects to the neurons of the second layer (Fuzzy Sets layer) and to the rule generator neuron.

• 2. Fuzzy sets layer The neurons of this layer compute the membership function of input vector received from the Input Layer for different fuzzy sets defined in this layer. Therefore, their output, $\mu _{ij}$ is defined as follows, for the ith rule and jth input dimension:

  $$\begin{aligned} \mu _{ij}=\mathrm{e}^{-\left( \frac{x_{j}-m_{ij}}{\sigma _{ij}} \right) ^{2}} \end{aligned}$$

  (A-18)

  where $m_{ij}$ is the center and $\sigma _{ij}$ is the width parameter of the ith rule for the jth input dimension.

• 3. Fuzzy rules layer The neurons of this layer combine the inputs from the Fuzzy Sets Layer and project the final value as the membership value of the input vector to a fuzzy rule. Considering R as the number of fuzzy rules the output of this layer is defined as below:

  $$\begin{aligned} \varphi _{i}=\prod \limits _{j=1}^n \mu _{ij} =\mathrm{e}^{-\sum \nolimits _{j=1}^n {\, \left( \frac{x_{j}-m_{ij}}{\sigma _{ij}} \right) }^{2} } \end{aligned}$$

  (A-19)

• 4. Fuzzy rule generator layer The unique neuron of this layer receives inputs from three origins: 1. From the input layer, a copy of the current sample of input signals.

2. From the fuzzy rules layer, copies of signals which re-code, by fuzzy rules, the current sample of input signals.

3. From a comparing element, a signal coding the movement error, and used as a teaching signal.

In the fuzzy rules layer, input signals are recoded into “fuzzy signals”, which are compared to a suitable threshold in the Generator neuron. If the “fuzzy signals” are lower than the threshold, this means that the current input vector is not completely represented by the current set of fuzzy rules. If at the same time, the error is significant, this coincidence shows that the coverage of the “inputs space” by the “fuzzy space” is insufficient. To improve this coverage, adding new fuzzy rules is necessary. Thus, the generator neuron sends teaching signals to the neurons of the Membership Function Layer. New fuzzy rules are added by a process comparable to an activation of granular cells, or useless rules are removed by a process comparable to a deactivation. This procedure of structure learning ensures an appropriate coverage of the space of input vectors by the fuzzy rules so that the “inputs space” is well represented in the “fuzzy space” (Rubio 2009; Wang et al. 2009; Han and Qiao 2010; Malek et al. 2012).

Table 3 Parameter values in the model of biomechanical function

These three entries to the generator neuron are shown in Fig. 9:

• 1. Input sample (Blue lines, with number 1) The input sample is sent to the generator neuron.

• 2. Outputs of the Fuzzy Rules Layer (Green lines, with number 2) The coverage of the current input sample $\varPhi$ by the fuzzy rules is calculated as:

  $$\begin{aligned} \varPhi =\sum \limits _{i=1}^R \varphi _{i} \end{aligned}$$

  (A-20)

  A fuzzy rule is to be added if, simultaneously, the coverage of the input signals by the fuzzy rules $\varPhi$ is insufficient and a significant error is measured.

• 3. The error signal (Red line with number 3) A comparing element calculates the movement error. If this error is significant, and the coverage of the current input sample by the fuzzy rules is insufficient, then adding a supplementary fuzzy rule could help to reduce the error.

A new rule is added to cover the part of the input space which is not represented precisely enough. A logical choice to cover this part of the input space, is to set the center of the new fuzzy rule—the mean value of the Gaussian curve—at the mean of the input sample which is insufficiently coded (Huang et al. 2005; Rubio 2009; Wang et al. 2009; Han and Qiao 2010; Malek et al. 2012) (Fig. 7).

The available input sample is used to calculate the center of the new fuzzy rule (($R+{1})$th fuzzy rule), and its width is a fixed vector as follows:

$$\begin{aligned}&{m}_{{R+1}}=\left[ {\begin{array}{llll} {x}_{{1}} &{}\quad {x}_{{2}} &{}\ldots &{} {x}_{{n}}\\ \end{array} } \right] ^{\mathrm {T}} \end{aligned}$$

(A-21)

$$\begin{aligned}&\sigma _{{R+1}}={{\sigma }\left[ {\begin{array}{llll} 1 &{}\quad 1 &{}\quad \cdots &{}\quad 1\\ \end{array} } \right] }^{\mathrm {T}} \end{aligned}$$

(A-22)

5. Output layer This layer consists in only one linear neuron, which receives inputs signals from the fuzzy rules layer. It calculates their weighted linear sum, which is the final output of the network:

$$\begin{aligned} \phi= & {} \left[ {\begin{array}{llll} \varphi _{1} &{}\quad {\cdots } &{}\quad \varphi _{R}\\ \end{array} } \right] ^{\mathrm{T}}, \end{aligned}$$

(A-23)

$$\begin{aligned} y= & {} {W}^{\mathrm {T}}{\cdot }\phi = \sum \limits _{i=1}^R {w_{i}{\cdot }\varphi _{i}} \end{aligned}$$

(A-24)

where the vector of synaptic weights is ${W} = [{w}_{{1}},{w}_{{2}}, \ldots , {w}_{{R}}]^{\mathrm {T}}$, and y is the final output.

Appendix 4: Range of values of K ensuring stability

When the learning of $F^{*}$ is achieved and the parameters of this function remain constant, based on the definition of the output of the FNN and on the result exposed in “Appendix 1”, it comes:

$$\begin{aligned} F^{*}\left( \alpha _{t},x_{t} \right) = {W}^{\mathrm {T}} {\cdot } \phi = \sum \limits _{i=1}^R {w_{i}{\cdot }\varphi _{i}} \end{aligned}$$

(A-25)

where $\alpha _{t}$ and $x_{t}$ are respectively the input motor order and the state of the system at time t.

Now, based on the definition of the FNN, the following equation can be calculated:

$$\begin{aligned} \frac{\partial F^{*}}{\partial \alpha \left( t \right) }\vert _{\alpha \left( t \right) =\alpha _{0}}= \sum \limits _{i=1}^R {w_{i}{\cdot }\varphi _{i}} \left( -\frac{2\left( \alpha _{0}-m_{i\alpha } \right) }{\sigma _{i\alpha }^{2}} \right) \end{aligned}$$

(A-26)

where $m_{i\alpha }$ and $\sigma _{i\alpha }$ are the center and width of the membership function related to the ith rule.

Stability is ensured if K remains in a range of values depending on the state of the system at any time t, according to the rules:

$$\begin{aligned} \left\{ \begin{array}{ll} 0<K<\frac{1}{\sum \nolimits _{i=1}^R {w_{i}{\cdot }{\varphi }_{i}} \left( \frac{\left( m_{i\alpha }-\alpha _{0} \right) }{\sigma _{i\alpha }^{2}} \right) }, &{}\quad \mathrm {if}\, \sum \limits _{i=1}^R {w_{i}{\cdot }{\varphi }_{i}} \left( \frac{2\left( m_{i\alpha }-\alpha _{0} \right) }{\sigma _{i\alpha }^{2}} \right) >0\\ -\frac{1}{\sum \nolimits _{i=1}^R {w_{i}{\cdot }{\varphi }_{i}} \left( \frac{2\left( \alpha _{0}-m_{i\alpha } \right) }{\sigma _{i\alpha }^{2}} \right) }<K<0, &{}\quad \mathrm {if}\, \sum \limits _{i=1}^R {w_{i}{\cdot } \varphi _{i}} \left( \frac{2\left( m_{i\alpha }-\alpha _{0} \right) }{\sigma _{i\alpha }^{2}} \right) <0.\\ \end{array} \right. \nonumber \\ \end{aligned}$$

(A-27)

Appendix 5: Biomechanical function

The simulated limb is a one-segment arm with two artificial muscles, based on the Hill model of muscle, inspired by a model proposed for the eye (Koene and Erkelens 2002). Its state equations are written in the discrete time and its biomechanical function is controlled in discrete time. Index 1 denotes the variables of the biomechanics of the agonist muscle and index 2 those of the biomechanics of the antagonistic muscle.

Notations $\alpha _{{i}}$ is the motor order sent to the $i{\mathrm {th}}$ muscle, $l_{{i}}$ is the length of the $i\mathrm{{th}}$ muscle, $F_{\mathrm{ai}}$ is the active force for the $i\mathrm{{th}}$ muscle, $F_{r}$ is the resistance force, Ks and $K_{r}$ are constants and $\tau _{1}$, $\tau _{2}$ and $\tau _{3}$ are time constants, $\sigma$ is the recruitment and activation time constant of the low-pass filter producing $F_{a}$, H is the Hill constant, $\delta$ is the sampling period, J is the moment inertia, $\alpha _{0}$ is the rest-activity of the motor neurones, $\varOmega$ is the torque and $\ddot{\theta }$ is the angular acceleration.

The equations of the biomechanical system are:

The values used in this study are presented in Table 3.