1 Introduction

The objective of iterative learning control (ILC) is to follow a motion profile defined over a fixed finite time interval, 0 ≤ t ≤ T, using data from previous executions of this task to suitably update the control input until perfect tracking of a defined reference, r(t), t ∈ [0, T]is achieved. Often in combination with current-trial information, the aim is to sequentially improve the performance of the operation as the trial/iteration/repetition index, k, increases. Since its formal conception 30 years ago[1], ILC has been an area of intense research interest in both theoretical and application domains[2,3]. A body of recent work has succeeded in removing the stipulation that tracking of a reference signal is required at all points of [0, T]. The framework that has been developed enforces tracking at only a subset of points {t j }1≤j≤M [3–7] and hence addresses applications such as production line automation, crane control, satellite positioning, and robotic “pick and place” tasks in which the system output (e.g., payload position) is only critical at a finite set of prescribed time instants. It has also been used to model human motor control[8] and within stroke rehabilitation to assist patients performing reaching and object manipulation tasks[9].

This paper expands the framework to tackle more general tracking problems that encompass the requirement

  1. 1)

    that a defined output signal passes through specified points at a finite number of times {t j }⊂ [0, T],

  2. 2)

    that, on defined subintervals \([{\hat t_j},{\hat t_{j + 1}})\), defined combinations of subsets of the outputs track required signals,

  3. 3)

    that, on defined subintervals, the output vector is unspecified but lies on a plane in the output space and,

  4. 4)

    no tracking of any kind is required elsewhere on [0, T].

These tasks fit many requirements needed in automation tasks which include welding or cutting movements, or human motion control where the movement is restricted by the task to straight line and/or planar segments. This expansion of the framework also enables it to tackle a special case of the “spatial ILC” problem[10–12] in which a path is specified in space (or more generally, as a functional constraint between outputs), but no temporal constraint is imposed on how the system moves along this path (or plane). Spatial ILC presents substantial novelty, but no framework currently exists which provides widely applicable algorithmic solutions and performance analysis. The inclusion of aspects of spatial ILC hence addresses applications requiring a predefined path (or plane) to be specified between intermediate point locations, but opens up the possibility of moving along the path (or plane) in a way that achieves some other objective (e.g., control effort minimisation).

Solutions are developed in this paper using the framework of norm optimal ILC (NOILC), which is a prominent member of the class of gradient based ILC algorithms whose convergence and robustness properties have been extensively studied by many groups [13–15]. Appearing originally in [16], NOILC embeds precise control over error and input norm evolution and has received significant attention in the ILC community[17–19], with experimental applications including an accelerator based free electron laser[20], multi-axis laser facility[19] and within stroke rehabilitation[20]. Theoretical developments to NOILC include extension to discrete time[21], an N-iteration ahead predictive solution[22], acceleration mechanisms[23], the inclusion of convex input constraints[24] and analysis of the effect of non-minimum phase zeros[25, 26]. To address 1)–4), a substantial generalization of NOILC is required to combine a general point-to-point tracking problem with partial tracking requirements on subintervals. It is achieved in this paper using an operator formulation of the plant dynamics and the application of NOILC concepts to construct an iterative algorithm that converges to a control input signal that ensures that specified values of system outputs y(t j ) or linear combinations of outputs F j y(t j ) are achieved at specified intermediate times {t j }1≤j≤M, whilst simultaneously ensuring that specified, and possibly different, linear combinations \({P_j}y(t),1 \le j \le \hat M\) of outputs track specified signals on (possibly different) subintervals \([{\hat t_{j - 1}},{\hat t_j}]\). Such combinations could be simple tracking tasks for a subset of the outputs or they could represent constraints on output behaviour that are necessary for effective operation. If no continuous tracking is required on an interval [t q −1, t q ], then this is simply described by setting P q = 0. The problem therefore combines the requirement to pass through defined points with the need to track defined signals on disjoint sub-intervals. The formulation is extremely general but can be realized by implementable feedback/feed forward ILC algorithms.

This paper is structured as follows. In Section 2, the intermediate weighting problem with partial tracking requirements for continuous state space systems is modelled in a general product Hilbert space setting and applied to formulate the ILC problem in Section 3 as a NOILC problem. Experimental validation results are given in Section 4 on a non-minimum-phase electromechanical test facility and their relationship to theoretical predictions is discussed. Conclusions and future work appear in Section 5.

2 Modelling in a product Hilbert space

The problem can be interpreted as a NOILC objective with emphasis placed on rapid and accurate convergence of all (or a subset) of outputs (or linear combination of outputs) at specified intermediate times {t j }1≤j≤M with effective tracking of defined signals constructed from (linear combinations of) outputs on subintervals \([{\hat t_{j - 1}},{\hat t_j}],1 \le j \le \hat M\). Through the introduction of parameters P j , Q j , 1 ≤ j ≤ M and \({\hat Q_j},1 \le j \le \hat M\), the problem can hence be expressed as a combination of the tracking of a standard reference r(t), t ∈ [0, T], and intermediate point tracking. This enables NOILC solutions to be derived using the relevant operators, signals and underlying signal spaces.

2.1 Input/output notation

Let S(A, B, C)be an m-output, l-input, state dimension n linear, time-invariant system of the form

$$\matrix{{\dot x(t) = Ax(t) + Bu(t),\quad x(0) = {x_0}} \hfill \cr{y(t) = Cx(t),\quad t \in [0,T]} \hfill \cr}$$
(1)

written in the operator form

$$\matrix{{y = Gu + d,\quad G:L_2^\ell [0,T] \rightarrow L_2^m[0,T],} \hfill \cr{\quad \;\;y,d \in L_2^m[0,T],u \in L_2^\ell [0,T]} \hfill \cr}$$
(2)

where convolution operator G and signal d are defined by

$$(Gu)(t) = \int_o^t {C{{\rm{e}}^{A(t - s)}}Bu(s){\rm{d}}s,\;\;d(t) = C{{\rm{e}}^{At}}{x_0},\;\;t \in [0,T]}$$
(3)

and T < ∞ is fixed. Note that all functions in the range of G are continuous and differentiable.

Let 0 = t0 < t1 < ⋯ < T M = T (respectively \(0 = {\hat t_0} < {\hat t_1} < \cdots < {\hat T_{\hat M}} = T\)) be M (respectively \(\hat M\)) distinct points in [0, T] and, for any \(f \in L_2^m[0,t]\), consider the linear map f ↦ fe

$$f^e = \left[ {\begin{array}{*{20}c} {F_1 f(t_1 )} \\ \vdots \\ {F_M f(t_M )} \\ {Pf} \\ \end{array} } \right] \in R^{f_1 } \times \cdots \times R^{f_1 } \times L_2^{p_1 } \left[ {0,\hat t_1 } \right] \times \cdots \times L_2^{p_{\hat M} } \left[ {\hat t_{\hat M - 1} ,T} \right]$$
(4)

where F j is f j × m and of full row rank, 1 ≤ j ≤ M. The inclusion of F j is a generalization that enables only selected elements or linear combinations of elements of f that are important in the problem to be specified at t = t j .

The operator P is defined by the functional relations:

$$Pf = \left[ {\matrix{{{{(Pf)}_1}} \cr\vdots \cr{{{(Pf)}_{\hat M}}} \cr}} \right],\;{(Pf)_j} \in L_2^{{p_j}}[{\hat t_{j - 1}},{\hat t_j}],\;1 \leq j \leq \hat M$$

where

$${(Pf)_j}(t) = {P_j}f(t),\;t \in [{\hat t_{j - 1}},{\hat t_j}],\;1 \leq j \leq \hat M$$

and either

  1. 1)

    each P j is a p j × m matrix of full row rank (representing the tracking requirement \({P_j}y(t) \equiv r_j^P(t)\) on [t j −1, t j ])

  2. 2)

    or P j = 0 (no tracking requirement on that interval).

They can be regarded as “partial projections” of the signal f on the domain \([{\hat t_{j - 1}},{\hat t_j}]\) representing combinations of f of control interest in these intervals.

With this notation, the “extended output” ye from the plant is defined to be a “partially projected” output signal (P y )(t) plus the values F j y(t j ), 1 ≤ j ≤ M, at intermediate points, i.e., dynamics can be modelled by the form

$${y^e} = {G_e}u + {d_e},\;{G_e}u = {(Gu)^e} = \left[ {\matrix{{{G_1}u} \cr\vdots \cr{{G_M}u} \cr{PGu} \cr}} \right],{d_e} = \left[ {\matrix{{{F_1}d({t_1})} \cr\vdots \cr{{F_M}d({t_M})} \cr{Pd} \cr}} \right]$$
(5)

with \({G_e}:L_2^\ell [0,T] \to {R^{{f_1}}} \times \cdots \times {R^{{f_M}}} \times L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{\hat M - 1}},{\hat t_{\hat M}}]\) as a linear bounded operator.

A technical note: If G e does not have range dense in \(L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{\hat M - 1}},{\hat t_{\hat M}}]\), then \(L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{\hat M - 1}},{\hat t_{\hat M}}]\) is replaced by the closure of the range of G e as the output Hilbert space with the natural inherited topology. This has no effect on the theory which incorporates this easily by using the virtual reference \(r \in L_2^m[0,T]\).

Here, \({G_j}:L_2^\ell [0,T] \to {R^{{f_j}}}\) is defined by the relation

$${G_j}u = {F_j}\int_0^{{t_j}} {C{{\rm{e}}^{A({t_j} - t)}}Bu(t){\rm{d}}t}$$
(6)

and \({(PG)_j}:L_2^\ell [0,T] \to L_2^{{p_j}}[{\hat t_{j - 1}},{\hat t_j}]\) is defined by

$$({(PG)_j}u)(t) = {P_j}\int_0^t {C{{\rm{e}}^{A(t - {t^{\prime}})}}Bu({t^{\prime}}){\rm{d}}{t^{\prime}},\quad t \in [{{\hat t}_{j - 1}},{{\hat t}_j}].}$$
(7)

A reference signal re is constructed by specifying the desired values of F j y(t j ), 1 ≤ j ≤ M and the desired signals \(r_j^P(t)\) for \({P_j}y(t),t \in [{\hat t_{j - 1}},{\hat t_j}],1 \le j \le \hat M\). The attainability of this reference by the system requires consistency between these characterizations, and is addressed as follows.

Tracking signal characterization: It is assumed that re can be generated using a (possibly non-unique, but continuous) virtual reference r for the plant output y such that r − d lies in the range of G. In this case, the tracking error ee = re − ye can be associated with the “usual” tracking error e(t)= r(t) − y(t) used in NOILC as

$${e^e}(t) = {r^e} - {y^e} = {(r - y)^e} = \left[ {\matrix{{{F_1}e({t_1})} \cr\vdots \cr{{F_M}e({t_M})} \cr{Pe} \cr}} \right].$$
(8)

It is useful to note the following:

  1. 1)

    The virtual reference r and the associated virtual error e will play a role in the simplification of the theory and relating the optimization problems used in the algorithm to, more traditional, optimal control problems. Note that neither r nor e are needed in the final computations as they are always multiplied by operations that convert them into components of the extended signals such as P j e that can be computed from the specified signal P j r and the measured plant output.

  2. 2)

    The pure intermediate point problem is obtained by removing P f from the definition of fe whilst NOILC is obtained by removing the intermediate points and looking at the case of \(\hat M = 1,{\hat t_1} = T\) and P = I. In terms of defining fe, this is just (respectively)

    $${f^e} = \left[ {\matrix{{{F_1}f({t_1})} \cr\vdots \cr{{F_M}f({t_M})} \cr}} \right] \in {R^{{f_1}}} \times \cdots \times {R^{{f_1}}},\;{f^e} = f \in L_2^m[0,T].$$
    (9)

    It is noted that the first is obtained in what follows by setting \({P_j} = 0,1 \le j \le \hat M\) whilst the second is obtained using F j = 0, 1 ≤ j ≤ M and \({\hat Q_j} = Q,1 \le j \le \hat M\) and P j = I.

  3. 3)

    The extension of the problem to the case of all t j < T is achieved by increasing M by one and then defining an additional point at t M = T with the corresponding F M = 0.

  4. 4)

    If there are no tracking requirements on some subinterval \([{\hat t_{q - 1}},{\hat t_q}]\), then simply set P q = 0.

To illustrate the wide range of tracking signals that the construction allows, consider the following example of a reference signal for the case of m = 3 and control over a time interval [0, 12]:

  1. 1)

    The intermediate points are defined by the requirement that output y1(t) passes through the points y1 = 1, 2, 3, 2, 1, 0attimest = 2, 4, 6, 8, 10, 12 respectively.

  2. 2)

    Dynamics of all outputs are constrained during the subinterval [4, 8] ⊂ [0, 12], to satisfy the planar/spatial constraint y1(t)+ y2(t)+ y3(t) ≡ 2.

The resultant data set is t1 = 2, t2 =4, t3 = 6, t4 = 8, t5 = 10, t6 = 12 with M = 6 and F1 = F2 = ⋯ = F6 = [1, 0, 0]. The remaining data sets are \({\hat t_1} = 4,{\hat t_2} = 8,{\hat t_3} = 12\) with \(\hat M = 3\). Hence, P1 = P3 =0 and P2 = [1, 1, 1] with reference signals \(r_1^P(t) \equiv 0,{r^P}(t) \equiv 0\) and \(r_2^P(t) \equiv 2\) on the intervals [0, 4), [4, 8], (8, 12].

2.2 Hilbert space structures and operators

The product space \({R^{{f_1}}} \times \cdots \times {R^{{f_1}}} \times L_2^{{p_1}}[0,{\hat t_1}] \times \cdots \times L_2^{{p_{\hat M}}}[{\hat t_{M - 1}},T]\) is a Hilbert space with inner product

$$\matrix{{{{\langle {({v_1}, \cdots ,{v_M},{f_1}, \cdots ,{f_{\hat M}}),({w_1}, \cdots ,{w_M},{g_1}, \cdots ,{g_{\hat M}})} \rangle}_{[Q]}} =} \hfill \cr{\quad \quad \quad \;\sum\limits_{j = 1}^M {v_j^{\rm{T}}{Q_j}{w_j} + \sum\limits_{j = 1}^{\hat M} {\int_{{{\hat t}_{j - 1}}}^{{{\hat t}_j}} {{f_j}{{(t)}^{\rm{T}}}{{\hat Q}_j}{g_j}(t){\rm{d}}t}}}} \hfill \cr}$$
(10)

where the f j × f j matrices Q j , 1 ≤ j ≤ M and the p j × p j matrices \({\hat Q_j},1 \le j \le \hat M\) are all symmetric and positive definite. The simple notation [Q] is used to denote the data set \([Q] = \{{Q_1}, \cdots ,{Q_M},{\hat Q_1}, \cdots ,{\hat Q_{\hat M}}\}\).

Using the normal definition of the associated norm, namely \(q = \sqrt {\langle q,q\rangle}\), the squared norm of the error ee, expressed in terms of the “underlying” error e, is simply

$$\matrix{{\Vert {e^e}\Vert_{[Q]}^2 = \sum\limits_{j = 1}^M {{e^{\rm{T}}}({t_j})F_j^{\rm{T}}{Q_j}{F_j}e({t_j}) +}} \hfill \cr{\quad \quad \quad \;\;\sum\limits_{j = 1}^{\hat M} {\int_{{{\hat t}_{j- 1}}}^{{{\hat t}_j}} {{e^{\rm{T}}}(t)P_j^{\rm{T}}{{\hat Q}_j}{P_j}e(t){\rm{d}}t.}}} \hfill \cr}$$
(11)

Finally, note that

  1. 1)

    \(L_2^\ell [0,T]\) is also a Hilbert space with inner product and norm defined by

    $$\matrix{{{{\langle {{u_1},{u_2}} \rangle}_R} = \int_0^{\rm{T}} {u_1^{\rm{T}}(t)R{u_2}(t){\rm{d}}t}} \hfill \cr{\quad \Vert u\Vert_R^2 = \int_0^{\rm{T}} {{u^{\rm{T}}}(t)Ru(t){\rm{d}}t}} \hfill \cr}$$
    (12)

    where ℓ × ℓ matrix R is symmetric and positive definite.

  2. 2)

    Equation (11) suggests that any objective function containing the term

    $$\sum\limits_{j = 1}^M {{e^{\rm{T}}}({t_j}){H_j}e({t_j}) + \sum\limits_{j = 1}^{\hat M} {\int_{{{\hat t}_{j - 1}}}^{{{\hat t}_j}} {{e^{\rm{T}}}(t)Q(t)e(t){\rm{d}}t}}}$$

    with Q(t) symmetric, positive semi-definite and piece-wise constant on [0, T] and each H j symmetric and positive semi-definite, can be associated with an ILC problem of the type discussed in this paper by identifying the intervals \(({\hat t_{j - 1}},{\hat t_j}),1 \le j \le \hat M\) where either Q(t)=0or Q(t)is constant with rank p j ≤ m and setting

    $$\matrix{{Q(t) = P_j^{\rm{T}}{{\hat Q}_j}{P_j},\;t \in [{{\hat t}_{j - 1}},{{\hat t}_j}],{{\hat Q}_j} = \hat Q_j^{\rm{T}} > 0,\;1 \leq j \leq \hat M} \hfill \cr{\;{H_j} = F_j^{\rm{T}}{Q_j}{F_j},\;\;{Q_j} = Q_j^{\rm{T}} > 0,\;1 \leq j \leq \hat M} \hfill \cr}$$
    (13)

    where each F j has full row rank and either P j has full row rank or is zero.

3 Formulation of the ILC problem

In its original form, NOILC for linear, time-invariant, state-space systems is based on the minimization of an objective function

$$\matrix{{{u_{k + 1}} = \arg \min \{J({u_{k + 1}}) = \Vert{e_{k + 1}}\Vert{^2} + \Vert{u_{k + 1}} - {u_k}\Vert{^2}:} \hfill \cr{\quad \quad \quad e = r - y,y = Gu + d\} ,k \geq 0.} \hfill \cr}$$
(14)

It is clear that the new mixed problem introduced in the previous section can be expressed in this form by identifying G with G e and the norms with the new norms defined. It follows that the iterative procedure defined by constructing u k +1 on iteration k + 1 by minimizing the objective function

$$\matrix{{J({u_{k + 1}}) = \sum\limits_{j = 1}^M {e_{k + 1}^{\rm{T}}({t_j})} F_j^{\rm{T}}{Q_j}{F_j}{e_{k + 1}}({t_j}) +} \hfill \cr{\quad \quad \quad \quad \sum\limits_{j = 1}^{\hat M} {\int_{{{\hat t}_{j - 1}}}^{{{\hat t}_j}} {e_{k + 1}^{\rm{T}}(t)Q(t){e_{k + 1}}(t){\rm{d}}t +}}} \hfill \cr{\quad \quad \quad \quad \int_0^{\rm{T}} {{{\left({{u_{k + 1}}(t) - {u_k}(t)} \right)}^{\rm{T}}}R\left({{u_{k + 1}}(t) - {u_k}(t)} \right){\rm{d}}t}} \hfill \cr{\quad Q(t) + P_j^{\rm{T}}{{\hat Q}_j}{P_j},\quad t \in ({{\hat t}_{j - 1}},{{\hat t}_j}],\;\;1 \leq j \leq M} \hfill \cr}$$
(15)

subject to the state space dynamics of S(A, B, C) is precisely a NOILC iteration for G e . Note that the criterion is expressed in terms of the virtual error ek+1 rather than the extended error \(e_{k + 1}^e\).

In the next sections, the solution is found in its Hilbert space setting and then formulated as the solution of a two-point boundary value problem with M “jump” conditions.

3.1 Formal solution of the intermediate point algorithm with subinterval weighting

The new algorithm can be written, in the notation of the previous section, as

$$\matrix{{{u_{k + 1}} = \arg \min \{J({u_{k + 1}}) = \Vert e_{k + 1}^e\Vert _{[Q]}^2 + \Vert {u_{k + 1}} - {u_k}\Vert_R^2:} \hfill \cr{\quad \quad \quad {e^e} = {r^e} - {y^e},{y^e} = {G_e}u + {d_e}\} .} \hfill \cr}$$
(16)

This has identical structure to the NOILC problem originally proposed in [16], the only differences being the definition of operators, signals and the underlying Hilbert spaces. The iterative solution to (16) is hence written formally as

$${u_{k + 1}} = {u_k} + G_e^{\ast}e_{k + 1}^e \Rightarrow e_{k + 1}^e = {(I + {G_e}G_e^{\ast})^{- 1}}e_k^e$$
(17)

where \(G_e^*\) is the adjoint operator of G e in the topology introduced in Section 2. This yields the monotonicity and convergence properties summarised in the following theorem. Note that this guarantees monotonic convergence of \(e_k^e{_{[Q]}}\) for all possible choices of symmetric positive definite R, [Q], and furthermore that \(e_k^e{_{[Q]}} \to 0\) implies \(e_k^e \to 0\).

  • Theorem 1. Under the conditions defined above and given an initial choice of input u0, the extended error norm is monotonically decreasing

    $$\Vert e_{k + 1}^e\Vert{_{[Q]}}\;\; \leq \;\;\Vert e_k^e\Vert{_{[Q]}},\quad \forall k \geq 0$$
    (18)

    with equality holding if and only if u k +1 = u k .As a consequence, there exists a real number E∞ ≤ 0 such that

    $$\mathop {\lim}\limits_{k \rightarrow \infty} \Vert e_k^e\Vert{_{[Q]}} = {E_\infty}$$
    (19)

    and E∞ = 0 if and only if

    $$\mathop {\lim}\limits_{k \rightarrow \infty} e_k^e = 0.$$
    (20)

    This condition is satisfied if the extended reference re lies in the closure of the range of G e . Finally,

    1. 1)

      the resultant input sequence satisfies

      $$\sum\limits_{k = 1}^\infty {\Vert{u_{k + 1}} - {u_k}\Vert_R^2 < \infty}$$
      (21)
    2. 2)

      if the kernel of \(G_e^*\) is {0}, the initial input u0 = 0 and the signal re − d e lies in the range of \({G_e}G_e^*\), then the input sequence {u k }k≥0 converges to the unique stationary point, u∞, of the Lagrangian associated with the optimization problem

      $${u_\infty} = \arg \min \{J(u) = \Vert u\Vert{^2}:{r^e} = {G_e}u + {d_e}\} .$$
      (22)

      In other word, it converges to the input achieving the tracking requirement that has the smallest energy as represented by its norm in \({\mathcal U}\).

  • Proof. The result follows from the work in [16] apart from the last statement. To show this, note that the Lagrangian associated with the minimum energy problem is just, with Lagrange multiplier \(\lambda \in {{\cal R}_e}\),

    $${{\cal L}}(u,\lambda) = \Vert u\Vert_{{\cal U}}^2 + 2 < \lambda ,\quad {r^e} - {G_e}u - {d_e} > {{{\cal R}}_e}$$
    (23)

    which has a stationary point when \({u_\infty} = G_e^*\lambda\) and re = G e u∞ + d e , i.e., \({r^e} - {d_e} = {G_e}G_e^*\lambda\). The stationary point solution for λ exists and is unique as is the defined input solution u∞. This solution solves the minimum energy problem follows from the projection theorem in Hilbert space and the fact that the linear variety {u: re = G e u + d e } is non-empty and closed. Next, we write the iterations for the ILC algorithm in the form

    $$\matrix{{{u_{k + 1}} = {u_k} + G_e^{\ast}{e_{k + 1}},\quad {e_{k + 1}} = {{(I + {G_e}G_e^{\ast})}^{- 1}}e_k^e,\quad k \geq 0} \hfill \cr{\quad \quad \quad \quad \;\;\mathop {\lim}\limits_{k \rightarrow \infty} e_k^e = 0.} \hfill \cr}$$
    (24)

    Using the notation \(X = {(I + {G_e}G_e^*)^{- 1}}\), implies that \({u_k} = G_e^*\sum _{j = 1}^k{X^j}e_0^e\). Writing \({r^e} - {d_e} = e_0^e = {G_e}G_e^*{\lambda _0}\) and noting that \(X{G_e}G_e^* = I - X\) gives

    $$\matrix{{{u_k} = G_e^{\ast}\left({\sum\limits_{j = 1}^k {{X^j}e_0^e}} \right) = G_e^{\ast}\left({\sum\limits_{j = 0}^{(k - 1)} {{X^j}(I - X){\lambda _0}}} \right)} \hfill \cr{\quad \quad \quad \quad \quad \quad \quad \;\;G_e^{\ast}(I - {X^k}){\lambda _0}.} \hfill \cr}$$
    (25)

    It follows that u k converges in norm to an input û∞ satisfying \({\hat u_\infty} = G_e^*{\lambda _0}\) and re = G e û∞ + d e , i.e., it converges to a stationary point of the Lagrangian which is unique. Clearly û∞ = u∞ and λ0 = λ. □

It is useful to note the following observations:

  1. 1)

    Convergence to the minimum energy controller distinguishes the proposed solution from any solution computed using NOILC and an arbitrarily chosen reference signal r(t) defined on the whole time interval [0, T] but satisfying the intermediate point and partial tracking requirements.

  2. 2)

    If all P j = 0, the output space is finite dimensional and \({G_e}G_e^*\) is a square matrix, self adjoint in the defined topology. The approach then reduces to the intermediate point tracking problem[5]. In addition, G is right-invertible, then the convergence is geometric as

    $$\Vert e_{k + 1}^e\Vert{_{[Q]}} \leq {1 \over {1 + {{\underline \sigma}^2}}}\Vert e_k^e\Vert{_{[Q]}},\quad \forall k \geq 0$$
    (26)

    where \(\underline {{\sigma ^2}}\) is the smallest eigenvalue of the matrix \({G_e}G_e^*\). In this case, invertibility of \({G_e}G_e^*\). In this case, invertibility of \({G_e}G_e^*\) ensures convergence to zero error.

  3. 3)

    In addition, taking the case of f j = m and F j = I m for example, the choice of Q j is available to the designer to influence the nature and form of the convergence, e.g., choosing Q j = λj−1Q0 for some symmetric positive definite matrix Q0 and some choice of λ ∈ (0, 1) will prioritise the reduction of errors at t j −1 before t j .

  4. 4)

    In a similar manner, choosing different values for \({\hat Q_j},1 \le j \le \hat M\) places different priorities on convergence on the subintervals \([{\hat t_j},{\hat t_{j + 1}})\).

In the discussion that follows, it is assumed that at least one \({\hat Q_j}\) is non-zero.

3.2 The two-point boundary value problem on iteration k + 1:

The solution (17) of NOILC problem (16) requires operator \(G_e^*\), which is computed in the Appendix. Substituting this operator into (17) converts the solution, u k +1, into the form of a two-point boundary value problem (TPBVP) with jump conditions, given by

$$\matrix{{{{\dot x}_{k + 1}}(t) = A{x_{k + 1}}(t) + B{u_{k + 1}}(t)} \hfill \cr{{x_{k + 1}}(0) = {x_0}} \hfill \cr{{u_{k + 1}}(t) = {u_k}(t) + {R^{- 1}}{B^{\rm{T}}}p(t)} \hfill \cr{\dot p(t) = - {A^{\rm{T}}}p(t) - {C^{\rm{T}}}Q(t){e_{k + 1}}(t),\quad t \in [0,T]} \hfill \cr}$$
(27)

where

$$\matrix{{p(T) = {C^{\rm{T}}}F_M^{\rm{T}}{Q_M}{F_M}{e_{k + 1}}(T)} \hfill \cr{p({t_j} -) = p({t_j} +) + {C^{\rm{T}}}F_j^{\rm{T}}{Q_j}{F_j}{e_{k + 1}}({t_j}),\;1 \leq j < M.} \hfill \cr}$$

. The next subsection converts this characterization into a feedback/feedforward realization using Riccati techniques.

3.3 A causal algorithm

The formal solution as a boundary value problem can be expressed either as a causal feedforward solution when Q(t)=0[5] or more generally, as a causal feedback plus feedforward solution. The details are described below.

The approach to constructing a state feedback plus predictive/feedforward term causal solution is familiar and closely related to the classical linear quadratic tracking problem. The solution provided below is not unique but all solutions are equivalent in the sense that they generate the same u k +1. The approach is illustrated by writing

$$p(t) = - K(t){x_{k + 1}}(t) + {\xi _{k + 1}}(t)$$
(28)

where K(t), ξ k +1(t) are assumed to be continuous and differentiable in each open segment (t j , t j +1) but possibly discontinuous at the points t = t j and K(t) is symmetric. The state vector is continuous on [0, T].

A causal feedback solution for u k +1 is constructed as follows:

  • Step 1. Compute off-line the Riccati feedback matrix from the matrix differential equation, terminal condition and jumps

    $$\matrix{{Q(t) = P_j^{\rm{T}}{{\hat Q}_j}{P_j},\;\;t \in ({{\hat t}_{j - 1}},{{\hat t}_j}],\quad 1 \leq j \leq \hat M} \hfill \cr{\dot K(t) + {A^{\rm{T}}}K(t) + K(t)A - K(t)B{R^{- 1}}{B^{\rm{T}}}K(t) +} \hfill \cr{\quad {C^{\rm{T}}}Q(t)C = 0} \hfill \cr}$$
    (29)

    with

    $$\left\{{\matrix{{K(T) = {C^{\rm{T}}}F_M^{\rm{T}}{Q_M}{F_M}C} \hfill \cr{K({t_j} -) = K({t_j} +) + {C^{\rm{T}}}F_j^{\rm{T}}{Q_j}{F_j}C,\quad 1 \leq j \leq M.} \hfill \cr}} \right.$$

    .

  • Step 2. On the (k + 1)-th iteration, compute off-line the predictive feedforward term from the differential equation, terminal condition and jumps

    $$\matrix{{{{\dot \xi}_{k + 1}}(t) + {A^{\rm{T}}}{\xi _{k + 1}}(t) - K(t)B{R^{- 1}}{B^{\rm{T}}}{\xi _{k + 1}}(t) -} \hfill \cr{\quad K(t)B{u_k}(t) + {C^{\rm{T}}}Q(t)r(t) = 0} \hfill \cr{{\xi _{k + 1}}(T) = {C^{\rm{T}}}F_M^{\rm{T}}{Q_M}{F_M}r(T)} \hfill \cr{{\xi _{k + 1}}({t_j} -) = {\xi _{k + 1}}({t_j} +) + {C^{\rm{T}}}F_j^{\rm{T}}{Q_j}{F_j}r({t_j}),1 \leq j \leq M.} \hfill \cr}$$
    (30)

    Although these equations contain the virtual reference r(t), note that F j r(t j ), 1 ≤ j ≤ M is specified in the tracking problem and that the product \(Q(t)r(t) = P_j^{\rm{T}}{\hat Q_j}{P_j}r(t),t \in [{t_{j - 1}},{t_j}]\) is known as P j r(t) is specified on [tj−1, t j ].

  • Step 3. Implement the control law

    $${u_{k + 1}}(t) = {u_k}(t) + {R^{- 1}}{B^{\rm{T}}}[{\xi _{k + 1}}(t) - K(t){x_{k + 1}}(t)]$$
    (31)

    to achieve the predicted monotonic convergence.

3.4 Performance and design observations

Given the tracking objectives, the choice of R,\(R,{\hat Q_j},1 \le j \le \hat M,{Q_j},1 \le j \le M\), and t j , 1 ≤ j < M are open to the designer of the system. As with the choice of Q and R in the traditional linear quadratic regulator (LQR) problem, there are no strict rules for their selection but the following comments can be made.

  1. 1)

    The case when \({\hat Q_j} \ne 0\) is equivalent, intuitively, to a requirement that the output tracks reference P j r(t) on subintervals of [0, T]. Changing the relative values of \({\hat Q_j}\) will change the relative convergence rates on the intervals with the general rule that, increasing \({\hat Q_q}\) will improve convergence rates on the subinterval \([{\hat t_{q - 1}},{\hat t_q}]\). Also decreasing the values of the \({\hat Q_j}\) relative to the Q j will place increasing emphasis on the specific intermediate points at times t j , 1 ≤ j ≤ M leading to an expected faster reduction of errors at the intermediate points and slower convergence elsewhere. Details will depend upon plant dynamics and the choice of R.

  2. 2)

    In general terms, reducing the value of R is expected, intuitively to tend to accelerate algorithm convergence.

4 Experimental results

The approach has been tested on a six degree of freedom anthropomorphic robotic arm (see Fig. 1) whose five rotary joints are composed of PowerCubes (Schunk GmbH & Co.) incorporating brushless servomotors with integrated power electronics and transmission. These communicate with a dSPACE ds1103 control board via a CAN bus at a rate of 500 kbit/s, and each is controlled in torque mode. The task considered involves planar movement using only two actuated joints. Starting from an initial location, the end-effector moves to a specified location (e.g., to pick up a tool head), moving along a path (replicating a cutting or welding action), returning to another specified location (e.g., to replace/clean a tool head), and returning to the initial location.

Fig. 1
figure 1

Robot showing actuated joints

Full size image

The plant model has been identified using frequency response tests and is given by transfer function matrix (32). Cascaded current and velocity control loops are used, with inputs and outputs measured in radians.

The task requires the output to pass through selected points at specified times plus the additional requirement that a linear combination of outputs takes a specified functional value on a proper subinterval. More precisely, zero initial conditions are assumed for each iteration and tracking is represented by using intermediate point locations, together with an output constraint that enforces a path following component on a subinterval. The path following component does not specify output values but does specify their functional relationship.

The intermediate output values are given by the M =5 time points

$${t_1} = 3,\quad {t_2} = 4,\quad {t_3} = 10,\quad {t_4} = 11,\quad {t_5} = T = 14$$

with the corresponding outputs

$$r(t_1 ) = \left[ {\begin{array}{*{20}c} {0.368 9} \\ {1.548 0} \\ \end{array} } \right],r(t_2 ) = \left[ {\begin{array}{*{20}c} {0.957 0} \\ {0.388 3} \\ \end{array} } \right],r(t_3 ) = \left[ {\begin{array}{*{20}c} {1.566 0} \\ {1.260 9} \\ \end{array} } \right],r(t_4 ) = \left[ {\begin{array}{*{20}c} {0.419 1} \\ {1.672 3} \\ \end{array} } \right],r(t_5 ) = \left[ {\begin{array}{*{20}c} 0 \\ 0 \\ \end{array} } \right]$$

where F j = I, 1 ≤ j ≤ 5. In addition, tracking is also defined by the constraint in the interval [4, 10].

$$\matrix{{{y_2} - 1.433{y_1} = - 0.9832\;({\rm{i}}.{\rm{e}}{\rm{.,}}\;\hat M = 3,\;{{\hat t}_1} = 4,{{\hat t}_2} = 10,{{\hat t}_3} = T),} \hfill \cr{{P_1} = [0,\;0],\;{P_2} = [ - 1.433,\;1.0],\;{P_3} = [0,\;0],} \hfill \cr{r_1^P(t) \equiv 0,\;r_2^P(t) = - 0.9832,\;\;r_3^P(t) \equiv 0} \hfill \cr}$$
(32)

representing a coordinated spatial movement of the two joints in terms of relative values. Note that this does not specify numerical values of y1(t) or y2(t) individually at any point in [4, 10]. To achieve the task, the control action (31) has been implemented using the state feedback matrix (29) and predictive feedforward term (30). As the states are not measured directly, a Kalman filter provides the state estimates using process and measurement disturbance co-variance weights of 10I and I respectively.

$$G(s) = \left[ {\begin{array}{*{20}c} {\frac{{3.34e^5 s^3 + 2.01e^7 s^2 + 3.87e^8 s + 2.34e^9 }} {{s^8 + 217.8s^7 + 1.9e^4 s^6 + 8.42e^5 s^5 + 2e^7 s^4 + 2.42e^8 s^3 + 1.17e^9 s^2 }}} & {\frac{{720}} {{s^3 + 95s^2 + 1300s + 8000}}} & {\frac{{600}} {{s^3 + 89s^2 + 1150s + 7500}}} & {\frac{{4.5e^4 s^3 + 1.71e^7 s^2 + 1.72e^9 s + 1.85^{10} }} {{s^9 + 203s^8 + 1.72e^4 s^7 + 7.82e^5 s^6 + 2.06e^7 s^5 + 3.14e^8 s^4 + 2.55e^9 s^3 + 8.50e^9 s^2 }}} \\ \end{array} } \right]$$
(33)

.

To provide a comparative approach to quantify convergence rates, the uniquely defined iteration index, k*, corresponding to the first index where the tracking criterion

$$\Vert e_k^e\Vert = \Vert {r^e} - y_k^e\Vert < \epsilon \Vert {r^e}\Vert$$
(34)

is met is noted. A value of ε = 0.01 is used in all the results that follow corresponding to an improvement in tracking by a factor of 100 relative to the extended reference.

Fig. 2 shows the converged paths after 200 trials using R = I, Q j = 50I, 1 ≤ j ≤ 5 and \({\hat Q}_{j} = 2, 1 \leq j \leq 3\). Corresponding output angles and control input signals are shown in Fig. 3, together with signals on trials 1, 3 and 5. Norm results over all trials are shown in Fig. 4 for a variety of Q j and \({\hat Q_j}\).

Fig. 2
figure 2

Observed paths on trial 200 in world space (corresponding to path of robot end-effector in Cartesian space) and joint space, using Q j = 50I and \({\hat Q} = 2\)

Full size image
Fig. 3
figure 3

Output angles, y1,k(t), y2,k(t) and corresponding control inputs u1,k(t), u2,k(t) over 200 trials, using Q j = 50I, \({\hat Q} = 2\). The minimum energy solution, calculated using the nominal plant model, is shown for comparison and labeled \(\tilde r(t)\)

Full size image
Fig. 4
figure 4

Error norms (combined intermediate point tracking and spatial tracking) and input norms over 200 trials, for a variety of Q j and \({\hat Q_j}\)

Full size image

Table 1 shows summary data, confirming that the both the intermediate point tracking and the spatial tracking components are achieved to high accuracy. As Q j increases more emphasis is placed on minimizing the intermediate point error, compared to the spatial tracking error. In practice, an optimum compromise can be found between Q j and \({\hat Q_j}\) values which provides convergence to low values of each objective. At the same time, there is clearly a trade-off between rapid convergence to low error, and the control effort required to achieve it. Lower values of Q j and \({\hat Q_j}\) typically enable less energy to be expended, but k* increases.

Table 1 Summary norm data, where \(||Pe_{{k}^{\ast}}||\) is the spatial error norm, and the intermediate point (IP) tracking norm is \(||e_{{k}^{\ast}}^{e}|| - ||Pe_{{k}^{\ast}}||\)
Full size table

The combined intermediate point and spatial tracking approach is now compared to the standard ILC framework. As is shown in Theorem 1, in the absence of plant model mismatch this method converges to the minimum energy solution to the problem, and hence the most appropriate comparison is to use standard NOILC with r(t) calculated in advance using the nominal plant model to achieve the intermediate point and spatial tracking tasks whilst also requiring minimum input energy. This signal is shown in Fig. 3 where it is labeled \(\tilde r(t)\). Using this predefined reference, standard NOILC corresponds to removing the intermediate points and using \(\hat M = 1,{\hat t_1} = T,P = I\).

Results in Table 2 show that standard NOILC requires a significantly larger control effort norm to obtain similar levels of error, while requiring a greater number of trials.

Table 2 Standard NOILC using a reference r(t) equal to the minimum energy solution
Full size table

5 Conclusions

A well-defined NOILC problem which places specific emphasis on two objectives, namely, simultaneous fast accurate tracking at a finite number of intermediate points of time within the interval [0, T] plus tracking of defined linear combinations of outputs on sub-intervals of [0, T], can be approached and solved using the operator methods underpinning those of NOILC in [16]. The resultant algorithm inherits the original convergence properties and monotonicity properties that makes NOILC so attractive. The formulation is quite general for linear time invariant state space systems and can be represented in a feedforward structure (when Q(t) ≡ 0), but more generally, has the form of familiar current trial Riccati state feedback and predictive/feedforward terms generated from off-line, reverse-time simulations of differential equations with defined jump conditions at the intermediate points selected. An analysis of the effect of the choice of weighting matrices provides guidelines to their selection in terms of convergence rates and the effect of removing intermediate points from the problem.

Experimental results indicate that the predictions of error monotonicity and rapid convergence can apply in practice with great accuracy and with robustness to modelling errors. Further research is needed to address more general issues of parameter choice, a formal characterization of the algorithm robustness to modelling errors and the effects of varying initial condition.