Extracting parametric dynamics from time-series data

Abstract

In this paper, we present a data-driven regression approach to identify parametric governing equations from time-series data. Iterative computations are performed for each time stamp to first determine if the governing equations to be recovered are time dependent. The results are then used as input data to extract the parametric equations. A combination of the constrained $\ell ^1$ and $\ell ^0+\ell ^2$ optimization problems are used to ensure parsimonious representation of the learned dynamics in the form of parametric differential equations. The method is demonstrated on three canonical dynamics. We show that the proposed method outperforms other sparse-promoting algorithms in identifying parametric differential equations in the low-noise regime in the aspect of accuracy and computation time.

Appendices

Appendix A: The Douglas-Rachford algorithm

Consider the minimization problem (1.2), where $G_1$ and $G_2$ are functions for which one can compute the proximal mappings $\text {prox}_{\gamma G_i}$, $i=1,2$, via Eq. (1.3). The DR algorithm was introduced in [20] as a generalization of an algorithm introduced by Douglas and Rachford in the case of quadratic minimization [53]. Under certain conditions, the DR algorithm have the following convergence property [22, 23, 43, 44].

Theorem A.1

Let $G_1$ and $G_2$ be proper, closed, and convex functions. For any $\gamma >0$, any $\mu \in (0,2)$, and any initial point $\tilde{x}^0$, the iterates $x^k$ generated by Eq. (1.4) converges linearly to a minimizer of the minimization problem (1.2).

For the constrained $\ell ^1$ minimization problem (P$_{1,\epsilon }$) of main interest in this paper, we define

$$\begin{aligned}&G_{1}(\omega , x) := \Vert x \Vert _1 + \text {Ind}_{\mathcal {B}} (\omega ),\\&G_{2}(\omega , x) := \text {Ind}_{\mathcal {K}} (\omega , x), \end{aligned}$$

where

$$\begin{aligned}&\mathcal {K} := \{(\omega , x): \ \omega = A x \}, \\&\mathcal {B} :=B_{\epsilon }(b) = \{ \omega : \ \Vert \omega -b \Vert _2 \le \epsilon \}, \end{aligned}$$

and $\text {Ind}$ denotes the indicator function. The proximal operators $\text {prox}_{\gamma G_i}$, $i=1,2$, are then given by:

$$\begin{aligned} \text {prox}_{\gamma G_1}(\omega , x)&:= \begin{pmatrix} S_{\gamma }(x),&{} \text {Proj}_{\mathcal {B}}(\omega )\\ \end{pmatrix},\\ \text {prox}_{\gamma G_2}(\omega , x)&:= \begin{pmatrix} y,&{} Ay\\ \end{pmatrix}, \end{aligned}$$

where

$$\begin{aligned} y = (I + A^T A)^{-1} (x + A^T \omega ). \end{aligned}$$

The function S is the soft-thresholding function which is defined component-wise as follows:

$$\begin{aligned}{}[S_{\gamma }(x)]_j =\left\{ \begin{array}{ll} x_j - \gamma \ \frac{x_j}{|x_j|}, &{} \quad |x_j| \ge \gamma , \\ 0, &{} \quad \text {otherwise}, \end{array} \right. \end{aligned}$$

and $\text {prox}_{\mathcal {B}}$ is the projection operator onto the ball $\mathcal {B}$:

$$\begin{aligned} \text {prox}_{\mathcal {B}}(\omega ) =\left\{ \begin{array}{ll} b + \epsilon \ \frac{\omega - b}{\Vert \omega - b\Vert _2}, &{} \quad \omega \notin \mathcal {B}, \\ \omega , &{} \quad \omega \in \mathcal {B}, \end{array} \right. \end{aligned}$$

For the constrained $\ell ^0$ minimization problem (P$_{0,\epsilon }$) considered for comparison in Sect. 5, redefine $G_1$ as follows:

$$\begin{aligned} {\tilde{G}}_{1}(\omega , x)&:= \Vert x \Vert _0 + \text {Ind}_{\mathcal {B}} (\omega ). \end{aligned}$$

The proximal operator $\text {prox}_{\gamma {\tilde{G}}_1}$ is given by:

$$\begin{aligned} \text {prox}_{\gamma {\tilde{G}}_1}(\omega , x)&:= \begin{pmatrix} H_{\gamma }(x),&{} \text {Proj}_{\mathcal {B}}(\omega )\\ \end{pmatrix}, \end{aligned}$$

where the function H is the hard-thresholding function which is defined component-wise as follows:

$$\begin{aligned}{}[H_{\gamma }(x)]_j =\left\{ \begin{aligned} x_j&,&\quad |x_j|\ge \sqrt{\gamma },\\ 0&,&\quad \text {otherwise}. \end{aligned} \right. \end{aligned}$$

In Sect. 3, we use the basic form of the DR algorithm, i.e. Eq. (1.4). In [54,55,56], accelerated variants of the DR algorithm are presented. To solve Eq. (P$_{1,\epsilon }$), one can also choose other algorithms for $\ell ^1$ minimization [27], for example ADMM, SPGL1, and the split Bregman algorithm [57].

Appendix B: Hyperparameters and supplementary figures

In this section, we provide the hyperparameters and some supplementary figures related to the computational experiments in Sect. 4.

Table 12 The dictionary matrices in Sect. 4

Table 13 The Lorenz 96 system: Hyperparameters used in the STRidge algorithm

Table 14 The Fisher-KPP equation: Hyperparameters used in the STRidge algorithm

Table 15 The Burgers’ equation: Hyperparameters used in the STRidge algorithm

Table 16 Dictionary sizes for one-step methods

Fig. 12

The parametric Burgers’ equation, Example 4.8. Coefficients of the nonzero terms in Eq. (4.20). The subvector $\textbf{t}$ used in the second learning is indicated by the dashed lines