Identification of the Dynamics in the Singular Vectors of the System Toeplitz Matrix of Markov Parameters

Abstract

The lower triangular Toeplitz matrix of Markov parameters is the finite time, time domain equivalent of the z-transfer function of digital systems. Its properties are therefore of interest, and this paper discovers previously unknown properties. The singular value decomposition (SVD) of the Toeplitz matrix contains the matrix of the singular values, and two orthonormal matrices of input and output singular vectors. An understanding of these three matrices in terms of system frequency response has been available for the limiting case when the Toeplitz matrix tends to infinity. Here we obtain an understanding of what happens to the SVD as the dimension is reduced. A system identification algorithm uses the entries in the input/output singular vectors as samples of a continuous time linear system under free response, with the result that such identified dynamics represent the data. Numerical examples for several different system orders are used to show the characteristics of identified eigenvalues embedded in the singular vectors. Each singular vector contains at least a pure frequency that is close to but not the same as the frequency when the dimension tends to infinity. Other eigenvalues help the output singular vectors to end at zero, and the input singular vectors to start at zero, a property of the finite time matrix. Singular vector dynamics in various Iterative Learning Control (ILC) laws are discussed here and numerically investigated, and the dynamics are relevant to basis function ILC, to finite time low pass filtering design, and to new versions of stable inverse theory, among other applications within the theory of control.

Appendix: : On the Relationship of the SVD of P to Frequency Response in the Limit as Dimension Tends to Infinity

Reference [7] developed this relationship showing that in the limit, singular values correspond to steady state magnitude frequency response, and singular vectors indicate steady state phase frequency response. An outline of the derivation is presented here, the reader is referred to [7] for details. The difference between this result and SVD of P for fixed finite dimension p is of interest in evaluating the applications discussed in the Introduction.

1.1 z-Transform Based Input-Output Model

Consider a z-transform based system model

$$Y(z)=G(z)U(z)$$

(A1)

which can represent the command U(z) to response Y (z) of a feedback control system. Note that the use of a z-transfer function indicates that the response given corresponds to zero initial conditions which is also true of Toeplitz matrix P. Given sinusoidal inputs $u(kT)={\cos \limits } (\omega kT)$, $u(kT)=\sin \limits (\omega kT)$, u(kT) = e^iωkT, the steady state frequency responses are given as

$$\begin{array}{@{}rcl@{}} y(kT) &=&M(\omega) \cos(\omega kT+\theta(\omega))\\ y(kT) &=&M(\omega) \sin(\omega kT+\theta(\omega)) \\ y(kT) &=&M(\omega) e^{i[\omega kT+\theta(\omega)]} \end{array}$$

(A2)

where

$$M(\omega)=|G\left( e^{i\omega T}\right)|; \ \ \ \theta(\omega)=\measuredangle G\left( e^{i\omega T} \right)$$

(A3)

and T is the time interval between sample times, and k is the sample time index. This solution is the steady state frequency response, meaning that it is the response for all time after any initial condition effects on the solution are negligible.

1.2 Toeplitz Based Input-Output Model

The input-output relationship in the time domain corresponding to the z-transfer function model above, for a given input history vector $\underline {u}$ and resulting output history vector $\underline {y}$ is

$$\underline{y}=P\underline{u}$$

(A4)

This is derived as Eq. 3 from a state space system model. It is specialized here to use zero initial conditions x(0) = 0, with disturbance $\underline {v}$ eliminated, and subscript j not included - it is used only for indicating the iteration number in ILC applications.

Referring to Eq. 2, the number of time steps of input history vector $\underline {u}$ is p, starting from time zero and ending at time p − 1, and the resulting output history vector $\underline {y}$ starts at step 1 and goes to step p, corresponding to a one time-step delay from input to output. Define a Discrete Fourier Transform (DFT) matrix in a balanced form for input and output as

$$\mathscr{F} = {1\over \sqrt{p}} \left[\begin{array}{cccc} ({z^{0}_{o}})^{0} & ({z^{0}_{o}})^{-1} & {\cdots} & ({z^{0}_{o}})^{-(p-1)} \\ ({z^{1}_{o}})^{0} & ({z^{1}_{o}})^{-1} & {\cdots} & ({z^{1}_{o}})^{-(p-1)} \\ {\vdots} & {\vdots} & {\ddots} & \vdots \\ (z^{p-1}_{o})^{0} & (z^{p-1}_{o})^{-1} & {\cdots} & (z^{p-1}_{o})^{-(p-1)} \end{array}\right]$$

(A5)

where ${z^{k}_{o}}=\exp (ik\omega _{o} )$, ω_o = (2π/p), and k = 0, 1,…,p − 1, which specifies the discrete frequencies that can be seen in a p time step data set. The inverse is given in terms of the conjugate transpose as ${\mathscr{F}}^{-1}=({\mathscr{F}}^{*} )^{T}$. Premultiply both sides of Eq. A4 by ${\mathscr{F}}$, and insert ${\mathscr{F}}^{-1}{\mathscr{F}}$ before $\underline {u}$

$$\begin{array}{@{}rcl@{}} \mathscr{F}\underline{y} &=& [\mathscr{F}P\mathscr{F}^{-1}]\mathscr{F}\underline{u} \\ Y_{DFT} &=& [\mathscr{F}P\mathscr{F}^{-1}]U_{DFT} \end{array}$$

(A6)

Letting p tend to infinity must produce the steady state frequency response of the system in the complex form of the third of Eq. A2, so that

$$\text{diag}\left[M_{0} e^{i\theta_{o}},\ M_{1} e^{i\theta_{1}},\ \cdots,\ M_{p-1}e^{i\theta_{p-1}}\right] =[\mathscr{F}P\mathscr{F}^{-1}] = \mathscr{F}U{\Sigma} V^{T} (\mathscr{F}^{*})^{T}$$

(A7)

where M_j = M(jω₀) and 𝜃_j = 𝜃(jω₀) defined in Eq. A3 for j = 0, 1,…,p − 1. The Singular Value Decomposition (SVD) of P = UΣV^T has been inserted on the right. The right hand side must be diagonal, so the i th row of ${\mathscr{F}}U$ times singular value σ_i times the i th column of $V^{T} ({\mathscr{F}}^{*})^{T}$ must equal the corresponding entry on the diagonal on the left of this equation. Observe that the i^th column of ${\mathscr{F}}U$ (or ${\mathscr{F}}V$) represents the DFT of the ith singular vector of U (or V). The U and V are orthonormal, and so is ${\mathscr{F}}U$ but in complex space. So all of the magnitude information on the left is contained in the M_j and all the magnitude information on the right is contained in the magnitude σ_j. We conclude that in the limit as $p\rightarrow \infty$ considered here, the σ_j give the magnitude frequency response for the associated discrete frequency, and the M(jω₀) and σ_j cancel in the above equation, i.e.

$$\text{diag}\left[e^{i\theta_{o}},\ e^{i\theta_{1}},\ \cdots,\ e^{i\theta_{p-1}}\right] = \mathscr{F}U V^{T} (\mathscr{F}^{*})^{T}$$

(A8)

Noting that the ${\mathscr{F}}$ and $({\mathscr{F}}^{*})^{T}$ are inverses of each other, it is evident that the phase change 𝜃_j from input to output must be totally contained in the product UV^T of output and input singular vectors. To proceed further, one needs to develop and equate the real parts and imaginary parts on each side of the equation. There are two σ_j related to each frequency, one is needed to span each of the sine and cosine inputs for that frequency, and one notes that 𝜃₁ = 𝜃_p− 1 so the entries on the left form complex conjugate pairs. The column vectors in U and V can then be determined as sinusoids (of the discrete frequencies associated with a p time step record), and the input and corresponding output singular vectors have a phase angle difference as specified on the left hand side of the equation.