Spatio-Temporal Koopman Decomposition

Abstract

This paper deals with a new purely data-driven method, called the spatio-temporal Koopman decomposition, to approximate spatio-temporal data as a linear combination of (possibly growing or decaying exponentially) standing or traveling waves. The method combines (i) either standard singular value decomposition (SVD) or higher-order SVD and (ii) either standard dynamic mode decomposition (DMD) or an extension of this method by the authors, called higher-order DMD. In particular, for periodic or quasiperiodic attractors, the method gives the spatio-temporal pattern as a superposition of standing and/or traveling waves, which are identified in an efficient and robust way. Such superposition may give the whole pattern as a modulated, periodic or quasiperiodic, standing or traveling wave. The method is illustrated in some simple toy-model dynamics, and its performance is tested in the identification of standing and traveling waves in the Ginzburg–Landau equation and of azimuthal waves in a rotating spherical shell with thermal convection.

Appendices

Appendix A. HODMD: The DMD-d Algorithm

The aim is to obtain the reduced spatial and temporal DMD expansions appearing in (2.6)–(2.7), (4.7)–(4.8), and (4.19)–(4.21), whose computation is completely similar. Thus, we only consider the temporal expansion (2.6), which can be written in vector form as

$$\begin{aligned}&\hat{\varvec{t}}_{k}\simeq \sum _{n=1}^{N}\hat{a}_{n} \hat{\varvec{q}}_{n}\hbox {e}^{(\delta _{n}+\small {\hbox {i}}\omega _{n})t_k} \quad \text {for }r=1,\ldots ,R'\text { and }k=1,\ldots ,K, \end{aligned}$$

(A.1)

where the index k denotes the involved equispaced values of time,

$$\begin{aligned} t_k=(k-1)\Delta t, \end{aligned}$$

(A.2)

and notation has been simplified by omitting the superscript t appearing in (2.6), (4.7), and (4.19). By construction, the reduced snapshots $(\hat{\varvec{t}}_{k})$ are mutually orthogonal and thus linearly independent. The computation is to be performed using the reduced snapshot matrix

$$\begin{aligned} \widehat{\varvec{T}}_1^K=[\hat{\varvec{t}}_1,\ldots ,\hat{\varvec{t}}_K]. \end{aligned}$$

(A.3)

Standard DMD (Schmid 2010) relies on the assumption

$$\begin{aligned} \widehat{\varvec{T}}_2^K\simeq \widehat{\varvec{R}}\, \widehat{\varvec{T}}_1^{K-1}\quad \text {or }\quad \hat{\varvec{t}}_{k+1}\simeq \widehat{\varvec{R}}\,\hat{\varvec{t}}_k \text { for }k=1,\ldots ,K-1, \end{aligned}$$

(A.4)

for a certain linear operator $\widehat{\varvec{R}}$ (Koopman operator), which is a $R'\times R'$ matrix. HODMD with index $d\ge 1$, instead, is based on the more general DMD-d assumption

$$\begin{aligned} \widetilde{\varvec{T}}_2^{K-d}\simeq \widetilde{\varvec{R}}\, \widetilde{\varvec{T}}_1^{K-d-1} \quad \text {or }\quad \tilde{\varvec{t}}_{k+1}\simeq \widetilde{\varvec{R}}\,\tilde{\varvec{t}}_k \text { for }k=1,\ldots ,K-d, \end{aligned}$$

(A.5)

for some $(R'd\times R'd$)-modified Koopman matrix $\widetilde{\varvec{R}}$, where the modified snapshots and the modified snapshot matrix are defined as

$$\begin{aligned}&\tilde{\varvec{t}}_k=\left[ \begin{array}{c}\hat{\varvec{t}}_{k}\\ \hat{\varvec{t}}_{k+1}\\ \ldots \\ \hat{\varvec{t}}_{k+d-2} \\ \hat{\varvec{t}}_{k+d-1} \end{array}\right] ,\quad \widetilde{\varvec{T}}_{1}^{K-d}= [\tilde{\varvec{t}}_1,\ldots ,\tilde{\varvec{t}}_{k-d}] = \left[ \begin{array}{c}\widehat{\varvec{T}}_{1}^{K-d}\\ \widehat{\varvec{T}}_{2}^{K-d+1}\\ \ldots \\ \widehat{\varvec{T}}_{d-1}^{K-1} \\ \widehat{\varvec{T}}_{d}^{K} \end{array}\right] . \end{aligned}$$

(A.6)

Note that HODMD includes standard DMD as a particular case because (A.5) reduces to (A.4) for $d=1$. As further discussed in Le Clainche and Vega (2017) by means of several counterexamples, if (A.4) holds, then the reduced snapshots are readily seen to satisfy (A.1), but the converse is generally untrue, namely not all expansions of the form (A.1) can be obtained using the assumption (A.4). Specifically, standard DMD cannot give the DMD expansion when the number of terms appearing in this expansion, N, is larger $R'$. Instead, the expansion (A.1) can always be obtained from the snapshots under the DMD-d assumption (A.5) with appropriate d. For the particular case in which (A.1) and (A.5) are both exact, the equivalence of these two assumptions has been established in Le Clainche and Vega (2017, Appendix A, theorem D).

On the other hand, the modified Koopman matrix $\widetilde{\varvec{R}}$ will not be calculated below because (A.5) is further dimension-reduced to take advantage of the redundancies that are present among the modified snapshots. Such redundancies are expected because of the correlations among the snapshots and the delayed snapshots.

With the above in mind, the DMD-d algorithm proceeds in three steps:

• Step 1: Dimension reduction As anticipated, a new dimension reduction (in addition to that already performed in Sects. 2 and 4) is convenient. To this end, we apply truncated SVD to the modified snapshot matrix, defined in (A.6), which yields

  $$\begin{aligned} \widetilde{\varvec{T}}_{1}^{K-d}\simeq \widetilde{\varvec{U}}\,\widetilde{\varvec{\Sigma }}\,\widetilde{\varvec{V}}^\top \equiv \widetilde{\varvec{U}}\,\varvec{W}_1^{K-d}, \quad \text {with }\varvec{W}_1^{K-d}=\widetilde{\varvec{\Sigma }}\,\widetilde{\varvec{V}}^\top , \end{aligned}$$

  (A.7)

  where the $R'd\times \widetilde{R}'$-matrix $\widetilde{\varvec{U}}$ and the $(K-d)\times \widetilde{R}'$-matrix $\widetilde{\varvec{V}}$ are such that $\widetilde{\varvec{U}}^\top \widetilde{\varvec{U}}=\widetilde{\varvec{V}}^\top \widetilde{\varvec{V}}=$ the $\widetilde{R}'\times \widetilde{R}'$-unit matrix; $\widetilde{\varvec{\Sigma }}$ is the $\widetilde{R}'\times \widetilde{R}'$-diagonal matrix containing the retained singular values $\tilde{\sigma }_1,\tilde{\sigma }_2,\ldots ,\tilde{\sigma }_{\widetilde{R}'}$, which are sorted in decreasing order). The number of retained modes, $\widetilde{R}'$, is chosen through the standard SVD-error estimate,

  $$\begin{aligned} \frac{\tilde{\sigma }_{\widetilde{R}'+1}^2+\cdots + \tilde{\sigma }_{\widetilde{R}}^2}{\tilde{\sigma }_{1}^2+\cdots +\tilde{\sigma }_{\widetilde{R}}^2}\le \varepsilon _1, \quad \text {with }\,\, \widetilde{R}=\min \{R' d,K-d\}, \end{aligned}$$

  (A.8)

  where, consistently with the primary dimension reduction already performed in Sects. 2 and 4, the threshold $\varepsilon _1$ coincides its counterparts in the various applications of HODMD, in Eqs. (2.5), (4.5), (4.17), and (4.28).

  To complete the step, we note that the columns of the matrices $\tilde{\varvec{T}}_1^{K-d}$ and $\varvec{W}_1^{K-d}$ appearing in (A.7) are such that

  $$\begin{aligned} \tilde{\varvec{t}}_k=\tilde{\varvec{U}}\varvec{w}_k, \end{aligned}$$

  (A.9)

  pre-multiply (A.5) by $\varvec{U}^\top$, and invoke (A.7), which leads to

  $$\begin{aligned} \varvec{W}_2^{K-d}\simeq \varvec{R}\,\varvec{W}_1^{K-d-1},\quad \text {or }\,\, \varvec{w}_{k+1}\simeq \varvec{R}\,\varvec{w}_{k}. \end{aligned}$$

  (A.10)

  Here, the involved Koopman matrices $\varvec{R}$ and $\tilde{\varvec{R}}$ are related to each other as $\varvec{R}=\widetilde{\varvec{U}}^\top \widetilde{\varvec{R}} \widetilde{\varvec{U}}$, but this equation will not be needed below.

• Step 2: Computation of the DMD modes, growth rates, and frequencies The dimension-reduced Koopman matrix $\varvec{R}$ appearing in (A.10) is calculated via de pseudoinverse as follows. Standard SVD (no truncation) applied to the $\tilde{R}'\times (K-d)$ matrix $\varvec{W}_1^{K-d-1}$ yields

  $$\begin{aligned} \varvec{W}_1^{K-d-1}=\varvec{U}\varvec{\Sigma }\varvec{V}^\top , \end{aligned}$$

  (A.11)

  where the $\widetilde{R}'\times \widetilde{R}'$ non-singular matrices $\varvec{U}$ and $\varvec{\Sigma }$ and the $(K-d)\times \widetilde{R}'$ rectangular matrix $\varvec{V}$ are such that $\varvec{U}\,\varvec{U}^\top =\varvec{U}^\top \varvec{U}=\varvec{V}^\top \varvec{V}=$ the $\widetilde{R}'\times \widetilde{R}'$ unit matrix and $\varvec{\Sigma }$ is diagonal. Thus, substituting (A.11) into (A.10), pre-multiplying the resulting equation by $\varvec{U}^\top$, and postmultiplying by $\varvec{V}\varvec{\Sigma }^{-1}$ yield

  $$\begin{aligned} \varvec{R}=\varvec{U}^\top \varvec{W}_2^{K-d}\varvec{V}\varvec{\Sigma }^{-1}. \end{aligned}$$

  (A.12)

  Once the matrix $\varvec{R}$ has been calculated, (A.10) implies that the dimension-reduced snapshots $\varvec{w}_k$ can be written as

  $$\begin{aligned}&\varvec{w}_{k}\simeq \sum _{n=1}^{\tilde{R}'} a_{n}\varvec{p}_n\mu _n^{k-1}, \end{aligned}$$

  (A.13)

  where $\varvec{p}_n$ are the eigenvectors of $\tilde{R}'\times \tilde{R}'$ matrix $\varvec{R}$, $\mu _n$ are the associated eigenvalues, and the amplitudes $a_n$ remain undetermined. Substituting this into (A.9) yields the following expression for the modified snapshots appearing in (A.6)

  $$\begin{aligned} \tilde{\varvec{t}}_{k}\simeq \sum _{n=1}^{\tilde{R}'} a_{n}\tilde{\varvec{q}}_n\mu _n^{k-1},\quad \text {with }\tilde{\varvec{q}}_n=\tilde{\varvec{U}}\varvec{p}_n. \end{aligned}$$

  (A.14)

  According to (A.6), the first $R'$ components of $\tilde{\varvec{t}}_k$ give the reduced snapshots $\hat{\varvec{t}}_k$, which can thus be written as

  $$\begin{aligned} \hat{\varvec{t}}_{k}\simeq \sum _{n=1}^{\tilde{R}'} a_{n}\hat{\varvec{q}}_n\mu _n^{k-1}, \end{aligned}$$

  (A.15)

  where, in principle, the reduced modes $\hat{\varvec{q}}_n$ collect the first $R'$ components of $\tilde{\varvec{q}}_n$. However, for convenience, the reduced modes are rescaled as unit vectors, such that

  $$\begin{aligned} \Vert \hat{\varvec{q}}_n\Vert _2=1\quad \text {for }n=1,\ldots ,R', \end{aligned}$$

  (A.16)

  which modifies the (still undetermined) amplitudes $a_n$. Invoking the expression (A.2) for the considered values of time, (A.15) readily yields the required DMD expansion (A.1), with

  $$\begin{aligned} \delta _n+\hbox {i}\omega _n=\frac{1}{\Delta t}\log \mu _n. \end{aligned}$$

  (A.17)

  which completes the step.

• Step 3: Computation of the mode amplitudes The mode amplitudes are calculated using (A.15), which is highly overdetermined vector equation if $K\gg R'$ (the usual case). Here, it is solved by least squares fitting via approximation via the pseudoinverse, namely by first rewriting (A.15) in matrix form as

  $$\begin{aligned} \varvec{L}\,\varvec{a}=\varvec{b}, \end{aligned}$$

  (A.18)

  where $\varvec{a}=[a_1,\ldots ,a_n]^\top$ is the unknown amplitude vector and the $(R' K\times R')$-matrix $\varvec{L}$ and the forcing term $\varvec{b}$ are readily calculated from (A.15). The pseudoinverse of $\varvec{L}$ is calculated by applying standard SVD (no truncation) to the matrix $\varvec{L}$, as

  $$\begin{aligned} \varvec{L}= \varvec{U}_1\varvec{\Sigma }_1\varvec{V}_1^\top , \end{aligned}$$

  (A.19)

  with the $(R'K\times R')$-matrix $\varvec{U}_1$ and the $R'\times R'$-matrices $\varvec{\Sigma }_1$ and $\varvec{V}_1$ such that

  $$\begin{aligned} \varvec{U}_1^\top \varvec{U}_1=\varvec{V}_1^\top \varvec{V}_1=\varvec{V}_1\,\varvec{V}_1^\top = \text {the}\,\, R'\times R' \text { unit matrix}. \end{aligned}$$

  (A.20)

  Substituting (A.19) into (A.18), pre-multiplying the resulting equation by $\varvec{V}_1\varvec{\Sigma }_1^{-1}\varvec{U}_1^\top$, and invoking (A.20) yield

  $$\begin{aligned} \varvec{a}=\varvec{V}_1\varvec{\Sigma }^{-1}\varvec{U}_1^\top \,\varvec{b}. \end{aligned}$$

  (A.21)

  As a final step, the DMD modes can be reordered sorting the absolute value of the mode amplitudes in decreasing order, and a smaller number of modes, $N\le R'$, is selected imposing that

  $$\begin{aligned} |a_{N+1}/a_1|<\varepsilon _2, \end{aligned}$$

  (A.22)

  for some tunable (small) threshold $\varepsilon _2$. This completes the derivation of the DMD expansion (A.1).

As described above, the DMD-d algorithm involves the following tunable parameters: the index d and the thresholds $\varepsilon _1$ and $\varepsilon _2$, appearing in (A.8) and (A.22). The parameters $\varepsilon _1$ and $\varepsilon _2$ coincide with their counterparts selected in Sects. 2 and 4. Concerning the index d, when $d>1$ because the number of involved frequencies is larger than the reduced dimension $R'$, the size of the modified snapshot matrix defined in (A.6) is $R'd\times (K-d)$, meaning that the number of terms appearing (A.14) (which coincides with the maximum number of terms appearing in the DMD expansion (A.1)) is such that

$$\begin{aligned} \tilde{R}'\le \min \{R'd,K-d\}. \end{aligned}$$

(A.23)

Thus, d must be neither too small nor too large. Usually, the optimal number of d is very easily found because the results are fairly insensitive to d in a wide interval around d. It must be also noted that, for a given sampled timespan, d scales with the considered number of snapshots, namely, e.g., d must be doubled when K is doubled.

Appendix B. Higher-Order Singular Value Decomposition

The description below is made for a P-th order tensor $\varvec{T}$, whose components depend on P indexes, as

$$\begin{aligned} T_{j_1j_2\dots j_P}\quad \text {for }\,j_p=1,\ldots ,J_P\,\text { and }\,p=1,\ldots ,P. \end{aligned}$$

(B.1)

For $P=2$, the tensor is a matrix and can be treated via standard SVD, which reads in this context as (Golub and van Loan 1996)

$$\begin{aligned} T_{j_1 j_2}=\sum _{r=1}^R\sigma _r U_{j_1r}V_{j_2r}, \end{aligned}$$

(B.2)

where R is the rank of the matrix. Higher-order tensors require more general decompositions (Kolda and Bader 2009). In particular, the HOSVD was introduced by Tucker (1966) and has been more recently popularized by de Lathauwer et al. (2000). HOSVD is very robust and computationally efficient decomposition, which has been used in a number of applications, including genomics (Omberg et al. 2007), pattern recognition (Savas and Eden 2007), aerodynamics (Lorente et al. 2008), dynamic control (Benito et al. 2011), and aeronautic design (de Lucas et al. 2011). In HOSVD, the tensor (B.1) is decomposed as

$$\begin{aligned} T_{j_1j_2\ldots j_P}=\sum _{j_1'=1}^{R_1}\sum _{j_2'=1}^{R_2}\cdots \sum _{j_P'=1}^{R_P} S_{j_1'j_2'\ldots j_P'}U^{(1)}_{j_1j_1'}U_{j_2j'_2}^{(2)}\dots U_{j_Pj'_P}^{(P)}, \end{aligned}$$

(B.3)

where, for $p=1,\ldots ,P$,

$$\begin{aligned} R_p\le min\{J_p,H_p\},\quad \text {with }\,\,H_p=J_1\dots J_{p-1}J_{p+1}\dots J_P, \end{aligned}$$

(B.4)

is defined below. Note that S is another P-th order tensor, known as the core tensor, and the columns of the matrices $U_{j_pj'_p}^{(p)}$ for $p=1,\ldots ,P$, are known as the modes of the decomposition. The modes are the proper orthogonal decomposition (POD) modes of the associated tensor fibers (namely, the vectors that are obtained fixing all but one of the indexes of the tensor, see Fig. 16). In other words, for each index $j_p$, the remaining indexes can be unfolded into a single index, which gives a matrix of order $J_p\times H_p$, with $H_p=J_1\dots J_{p-1}J_{p+1}\dots J_P$, whose rank is precisely the maximum index $R_P$ appearing in (B.3). Applying standard SVD to this matrix, the resulting left-SVD modes are precisely the associated HOSVD modes appearing in (B.3) along this direction, $U^{(p)}$. The singular values calculated via standard SVD are sorted in a decreasing order and denoted as

$$\begin{aligned} \sigma ^{(p)}_1\ge \ldots \ge \sigma ^{(p)}_{J_p}\ge 0\quad \text {for }p=1,\ldots ,P. \end{aligned}$$

(B.5)

Noting that the HOSVD-modes calculated that way are orthonormal with the Euclidean inner product, the core tensor is readily calculated in terms of the exact tensor and the modes as

$$\begin{aligned} S_{j_1'\ldots j_n'}= \sum _{j_1=1}^{J_1}\cdots \sum _{j_P=1}^{J_P}T_{j_1\ldots j_P}U_{j_1j'_1}^{(1)}\dots U_{j_Pj'_P}^{(P)}. \end{aligned}$$

(B.6)

The core tensor, the HOSVD modes, and the singular values along the various tensor fibers are very efficiently calculated using the MATLAB routine ‘hosvd’ (provided in the file exchange of MATLAB community¹ as toolbox TP Tool), which is able to treat tensors of any order. It must be noted that this is a strict extension of SVD since, for matrices, it reduces to the standard SVD (B.2). In other words, if $P=2$, then the core tensor (which is a matrix) is diagonal, and singular values along the two directions exactly coincide.

Fig. 16

The fibers of a third-order tensor

Now, as in standard SVD, when the elements of the tensor $\varvec{T}$ show redundancies along the various tensor directions, truncation of (B.3), as

$$\begin{aligned} T_{j_1j_2\ldots j_P}^\text {trunc}=\sum _{j_1'=1}^{J'_1}\sum _{j_2'=1}^{J'_2}\cdots \sum _{j_p'=1}^{J'_P} S_{j_1'j_2'\ldots j_n'}U^{(1)}_{j_1j_1'}U_{j_2j'_2}^{(2)}\dots U_{j_Pj'_P}^{(P)}, \end{aligned}$$

(B.7)

with

$$\begin{aligned} J'_p\le R_p\,\text { for }\,p=1,\ldots ,P, \end{aligned}$$

(B.8)

may still give a good approximation. It turns out that the relative RMS error of the truncated approximation is bounded as (Kolda and Bader 2009)

$$\begin{aligned} \text {RRMSE}\equiv \frac{\Vert \varvec{T}-\varvec{T}^\text {trunc}\Vert _\text {Fro}}{\Vert \varvec{T}\Vert _\text {Fro}} \le \text {EE}\equiv \frac{\sqrt{\sum _{p=1}^P[(\sigma ^{(p)}_{J'_p+1})^2+\cdots +(\sigma ^{(p)}_{J_p})^2]}}{\sqrt{\sum _{p=1}^P[(\sigma ^{(p)}_1)^2+\cdots +(\sigma ^{(p)}_{J_p})^2] }}, \end{aligned}$$

(B.9)

where $\Vert \cdot \Vert$ stands for the Frobenius norm of the tensors (namely, the square root of the sum of the squares of its components) and the singular values $\sigma ^{(p)}_1,\ldots ,\sigma ^{(p)}_{J_p}$ are as defined above, in (B.5). Thus, for a given required accuracy in terms of the RRMSE, $\varepsilon _1$, the numbers of modes retained along the various directions may be selected such that

$$\begin{aligned} \text {EE}\le \varepsilon _1. \end{aligned}$$

(B.10)

However, this is only one condition that (for $P\ge 3$) does not determine the individual numbers of modes to be retained. The required additional conditions may be selected in various ways. A reasonable strategy is to equidistribute the error bound EE defined in (B.9) along the various dimensions, but requiring that the P terms in brackets in the numerator of (B.9) all be equal. This selects $J'_1$,...,$J'_P$ as the minimum values such that

$$\begin{aligned} \frac{\sigma ^{(p)}_{J'_p+1}}{\Vert \varvec{T}\Vert _\text {Fro}}\le \frac{\varepsilon _1}{\sqrt{P}}\text {for }p=1,\ldots ,P. \end{aligned}$$

(B.11)