Empirical evaluated SDE modelling for dimensionality-reduced systems and its predictability estimates

Abstract

This paper develops and validates a method of empirical modelling for a dimensionality-reduced system of a nonlinear dynamical system based on the framework of the stochastic differential equation (SDE). Following the mathematical theorem corresponding to some inverse problem of the probability theory, we derive the empirically evaluating formulae for the drift vector and diffusion matrix. Focusing on a low-dimensional dynamical system of the Lorenz system, we empirically reconstruct an SDE that approximates the original time-series on the projected 2-dimensional plane. The distribution of the ensemble variance of solutions generated by the numerical SDE well agrees with that of the trajectories of the projected time-series, which indicates the ability of the SDE modelling to represent local predictability. Moreover, we also compare our SDE constructing method with the conventional Mori–Zwanzig projected operator method, which is used to derive a generalised Langevin equation for dimensionality-reduced systems, to assess the applicability of the obtained SDE model derived by the presented method.

Appendix: A Error estimates for the evaluating formulae for the coefficients

1.1 A.1 Geometric Brownian motion

Here, we consider the error estimates of the geometric Brownian motion

$$\begin{aligned} \mathrm dX_t = \alpha X_t \mathrm dt + \beta X_t \mathrm dW_t, \quad X_0=x. \end{aligned}$$

(39)

Here, $\alpha$ and $\beta$ are constants, $W_t$ is the 1-dimensional Wiener process and x is the initial condition.

As shown in Sect. 2.2, a solution of (39) is given by

$$\begin{aligned} X_t = x \exp \left( \left( \alpha -\frac{\beta ^2}{2}\right) t +\beta W_t\right) . \end{aligned}$$

It is well-known that

$$\begin{aligned} {\mathrm E}_{{\varvec{x}}}[ \exp \left( \gamma t +\beta W_t\right) ]=\exp \left( \left( \gamma +\frac{\beta ^2}{2}\right) t\right) \end{aligned}$$

for constant $\gamma$, therefore $X_t$ satisfies

$$\begin{aligned} \frac{{\mathrm E}_{{\varvec{x}}}[ X_t-x ]}{t} = \frac{x (\exp ( \alpha t ) - 1 )}{t} =\alpha x\cdot \frac{\exp ( \alpha t ) - 1 }{\alpha t} =\alpha x+O(t). \end{aligned}$$

(40)

This implies the result given in (9).

Similarly, one can calculate the second order variation as follows.

$$\begin{aligned}&\frac{{\mathrm E}_{{\varvec{x}}}[ (X_t-x)^2 ]}{2t} = \frac{ x^2\{\exp ((2\alpha +\beta ^2)t) -2\exp ( \alpha t ) + 1 \}}{2t} \nonumber \\&\phantom {\frac{{\mathrm E}_{{\varvec{x}}}[ (X_t-x)^2 ]}{2t} } = \frac{ x^2\{\exp ((2\alpha \!+\!\beta ^2)t) \!-\! \exp ( 2\alpha t ) \}}{2t} \!+\! \frac{ x^2\{\exp (2\alpha t)\! -\!2\exp ( \alpha t )\! +\! 1 \}}{2t} \nonumber \\&\phantom {\frac{{\mathrm E}_{{\varvec{x}}}[ (X_t-x)^2 ]}{2t} } = \frac{(\beta x)^2}{2}\cdot \frac{\exp (2\alpha t) (\exp (\beta ^2t) -1)}{\beta ^2t} + \frac{ {\mathrm E}_{{\varvec{x}}}[ X_t-x ]^2}{2t} \nonumber \\&\phantom {\frac{{\mathrm E}_{{\varvec{x}}}[ (X_t-x)^2 ]}{2t} } = \frac{(\beta x)^2}{2} + \frac{t}{2}\cdot \left( \frac{ {\mathrm E}_{{\varvec{x}}}[ X_t-x ]}{t} \right) ^2 + O(t). \end{aligned}$$

(41)

Therefore the variance form

$$\begin{aligned}&\frac{{\mathrm E}_{{\varvec{x}}}[ (X_t-x)^2 ] \!-\! {\mathrm E}_{{\varvec{x}}}[ X_t\!-\!x ]^2}{2t} = \frac{(\beta x)^2}{2}\cdot \frac{ \exp (2\alpha t) (\exp (\beta ^2t) \!-\!1 )}{\beta ^2t} = \frac{(\beta x)^2}{2}\! +\! O(t) \nonumber \\ \end{aligned}$$

(42)

leads better estimate than (41). It supports the covariance form (5) for the estimate of the diffusion matrix instead of the moment form (6).

1.2 A.2 General cases

Here we calculate the error estimate of the approximation formulae given in (12) and (13) in Sec. 2.2. Let us consider an SDE in a general case

$$\begin{aligned} \mathrm d{\varvec{X}}_t = {\varvec{A}}({\varvec{X}}_t) \mathrm dt + \mathbb {S}({\varvec{X}}_t) \mathrm d{\varvec{W}}_t. \end{aligned}$$

(43)

Similarly to the error estimates (40) and (42), we can calculate the error estimates analytically for the drift vector and the diffusion matrix of (43) in a general case. For the estimate of the drift vector, the solution of (43) satisfies

$$\begin{aligned} {\mathrm E}_{{\varvec{x}}}[X^i_t - x^i ] = \int _0^t {\mathrm E}_{{\varvec{x}}}[A^i({\varvec{X}}_s)]\mathrm ds, \end{aligned}$$

(44)

therefore we obtain

$$\begin{aligned} \frac{{\mathrm E}_{{\varvec{x}}}[X^i_t - x^i ]}{t} = \frac{1}{t}\int _0^t {\mathrm E}_{{\varvec{x}}}[A^i({\varvec{X}}_s)]\mathrm ds = A^i({\varvec{x}}) + \frac{1}{t}\int _0^t ( {\mathrm E}_{{\varvec{x}}}[A^i({\varvec{X}}_s)] - A^i({\varvec{x}}) )\mathrm ds. \end{aligned}$$

(45)

For the estimate of the second term of the right-hand side of (45), Itô’s formula derives that

$$\begin{aligned} \left| A^i({\varvec{x}}) - {\mathrm E}_{{\varvec{x}}} [A^i({\varvec{X}}_s)] \right| \le \left( \sum _{k=1}^N \left\| \partial _k A^i\right\| _{\infty } \left\| A^k\right\| _{\infty } + \frac{1}{2} \sum _{k,l=1}^N \left\| \partial _k \partial _l A^i\right\| _{\infty } \left\| B^{kl}\right\| _{\infty } \right) \cdot s, \end{aligned}$$

(46)

where $\left\| \cdot \right\| _{\infty }$ and $\partial _k$ denote the $L^{\infty }$ norm and the derivative with respect to $x^k$, respectively. Consequently, from (45) we obtain the error estimate for the drift vector as follows.

$$\begin{aligned} \frac{{\mathrm E}_{{\varvec{x}}}[X^i_t - x^i ]}{t} = A^i({\varvec{x}}) + O(t). \end{aligned}$$

(47)

This results supports (12) in Sect. 2.2.

For the estimate of the diffusion matrix, from Itô’s formula, the expectation of the second order variation becomes

$$\begin{aligned}&\phantom {=} {\mathrm E}_{{\varvec{x}}}\left[ (X^i_\tau -x^i)(X^j_\tau -x^j) \right] \nonumber \\&=2\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ B^{ij}({\varvec{X}}_s)]\mathrm ds +\int _0^\tau {\mathrm E}_{{\varvec{x}}} \left[ (X^i_s-x^i) A^j({\varvec{X}}_s) + A^i({\varvec{X}}_s) (X^j_s-x^j) \right] \mathrm ds. \nonumber \\ \end{aligned}$$

(48)

By using (44) and Leibnitz’s rule, the second term in the right-hand side in (48) holds that

$$\begin{aligned}&\phantom {=} \int _0^\tau {\mathrm E}_{{\varvec{x}}} [ (X^i_s-x^i) A^j({\varvec{X}}_s) + A^i({\varvec{X}}_s) (X^j_s-x^j) ] \mathrm ds - E_{{\varvec{x}}} [ X^i_\tau -x^i ] E_{{\varvec{x}}} [ X^j_\tau -x^j ] \nonumber \\&\quad =\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ (X^i_s-x^i) A^j({\varvec{X}}_s) + A^i({\varvec{X}}_s) (X^j_s-x^j) ] \mathrm ds \nonumber \\&\qquad \phantom {=} - \int _0^\tau ( {\mathrm E}_{{\varvec{x}}} [ X^i_s-x^i ] {\mathrm E}_{{\varvec{x}}} [ A^j({\varvec{X}}_s) ] + {\mathrm E}_{{\varvec{x}}} [ A^i({\varvec{X}}_s) ] {\mathrm E}_{{\varvec{x}}} [ X^j_s-x^j ] ) \mathrm ds \nonumber \\&\quad =\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ (X^i_s-x^i) (A^j({\varvec{X}}_s) -A^j({\varvec{x}}) ) + (A^i({\varvec{X}}_s) -A^i({\varvec{x}}) ) (X^j_s-x^j) ] \mathrm ds \nonumber \\&\quad \phantom {=} +\!\!\int _0^\tau \!( {\mathrm E}_{{\varvec{x}}} [ X^i_s-x^i ] {\mathrm E}_{{\varvec{x}}} [ A^j({\varvec{x}}) \!-\! A^j({\varvec{X}}_s)]\! +\! {\mathrm E}_{{\varvec{x}}} [ A^i({\varvec{x}})\! -\! A^i({\varvec{X}}_s)] {\mathrm E}_{{\varvec{x}}} [ X^j_s\!-\!x^j ] ) \mathrm ds.\nonumber \\ \end{aligned}$$

(49)

The estimates of the difference of the drift vector (46) and that of the state

$$\begin{aligned} \left| {\mathrm E}_{{\varvec{x}}} [ X^i_s-x^i ]\right| \le s\left\| A^i\right\| _{\infty }, \end{aligned}$$

which is derived from (44), imply that the third and the fourth terms of the right-hand side in (49) become

$$\begin{aligned}&\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ X^i_s-x^i ] ( A^j({\varvec{x}}) - {\mathrm E}_{{\varvec{x}}} [A^j({\varvec{X}}_s)] ) \mathrm ds \\&\quad + \int _0^\tau ( A^i({\varvec{x}}) - {\mathrm E}_{{\varvec{x}}} [A^i({\varvec{X}}_s)] ) {\mathrm E}_{{\varvec{x}}} [ X^j_s-x^j ] \mathrm ds =O(\tau ^3). \end{aligned}$$

Hence, this result and (48) derive that

$$\begin{aligned}&{\mathrm E}_{{\varvec{x}}} [ (X^i_\tau -x^i)(X^j_\tau -x^j) ] - E_{{\varvec{x}}} [ X^i_s-x^i ] E_{{\varvec{x}}} [ X^j_s-x^j ] \nonumber \\&\quad =2\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ B^{ij}({\varvec{X}}_s)]\mathrm ds +\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ (X^i_s-x^i) (A^j({\varvec{X}}_s) -A^j({\varvec{x}}) )] \mathrm ds \nonumber \\&\qquad \phantom {=} +\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ (A^i({\varvec{X}}_s) -A^i({\varvec{x}}) ) (X^j_s-x^j) ] \mathrm ds +O(\tau ^3). \end{aligned}$$

(50)

Using Itô’s formula again, the integrand of the second term of the right-hand side in (50) takes the following forms

$$\begin{aligned}&\phantom {=} {\mathrm E}_{{\varvec{x}}} [ (X^i_s-x^i) (A^j({\varvec{X}}_s) -A^j({\varvec{x}}) ) ] \nonumber \\&\quad = \int _0^s {\mathrm E}_{{\varvec{x}}} [ A^i({\varvec{X}}_u) (A^j({\varvec{X}}_u)-A^j({\varvec{x}})) ] \mathrm du \nonumber \\&\qquad + \sum _{k=1}^N \int _0^s {\mathrm E}_{{\varvec{x}}} [ (X^i_u-x^i) (\partial _k A^j) ({\varvec{X}}_u) A^k({\varvec{X}}_u) ] \mathrm du \nonumber \\&\qquad + \frac{1}{2} \sum _{k=1}^N \int _0^s {\mathrm E}_{{\varvec{x}}} [ (\partial _k A^j) ({\varvec{X}}_u) B^{ik}({\varvec{X}}_u) ] \mathrm du \nonumber \\&\qquad + \frac{1}{2} \sum _{k,l=1}^N \int _0^s {\mathrm E}_{{\varvec{x}}} [ (X^i_u-x^i) (\partial _k \partial _l A^j) ({\varvec{X}}_u) B^{kl}({\varvec{X}}_u) ] \mathrm du \nonumber \\&\quad = \int _0^s {\mathrm E}_{{\varvec{x}}} [ A^i({\varvec{X}}_u) (A^j({\varvec{X}}_u)-A^j({\varvec{x}})) ] \mathrm du \nonumber \\&\qquad + \sum _{k=1}^N \int _0^s {\mathrm E}_{{\varvec{x}}} [ (X^i_u-x^i) (\partial _k A^j) ({\varvec{X}}_u) A^k({\varvec{X}}_u) ] \!\!\mathrm du \nonumber \\&\quad \phantom {=} + \frac{1}{2} \sum _{k=1}^N \int _0^s ( {\mathrm E}_{{\varvec{x}}} [ (\partial _k A^j) ({\varvec{X}}_u) B^{ik}({\varvec{X}}_u) ] -(\partial _k A^j)({\varvec{x}}) B^{ik}({\varvec{x}}) )\mathrm du \nonumber \\&\quad \phantom {=} + \frac{1}{2} \sum _{k,l=1}^N \int _0^s {\mathrm E}_{{\varvec{x}}} [ (X^i_u-x^i) (\partial _k \partial _l A^j) ({\varvec{X}}_u) B^{kl}({\varvec{X}}_u) ] \mathrm du \nonumber \\&\qquad + \frac{1}{2} \sum _{k=1}^N (\partial _k A^j) ({\varvec{x}}) B^{ik}({\varvec{x}})\cdot s. \end{aligned}$$

(51)

By virtue of the similar estimates to (46) calculated from Itô’s formula, the integrands of the right-hand side in (53) satisfy the following estimates:

$$\begin{aligned} \left\{ \begin{array}{l} \left| {\mathrm E}_{{\varvec{x}}} [ A^i({\varvec{X}}_u) (A^j({\varvec{X}}_u)-A^j({\varvec{x}})) ] \right| \le C_1 u,\\ \left| {\mathrm E}_{{\varvec{x}}} [ (X^i_u-x^i) (\partial _k A^j) ({\varvec{X}}_u) A^k({\varvec{X}}_u) ] \right| \le C_2 u, \\ \left| {\mathrm E}_{{\varvec{x}}} [ (\partial _k A^j) ({\varvec{X}}_u) B^{ik}({\varvec{X}}_u) ] -(\partial _k A^j) ({\varvec{x}}) B^{ik}({\varvec{x}}) \right| \le C_3 u, \\ \left| {\mathrm E}_{{\varvec{x}}} [ (X^i_u-x^i) (\partial _k \partial _l A^j) ({\varvec{X}}_u) B^{kl}({\varvec{X}}_u) ] \right| \le C_4 u, \end{array} \right. \end{aligned}$$

(52)

where $C_1$, $C_2$, $C_3$ and $C_4$ are constants depending on the $L^\infty$ norms of the components of ${\varvec{A}}$ and $\mathbb {B}$ and their derivatives of order up to 4. From these estimates, (53) becomes

$$\begin{aligned} {\mathrm E}_{{\varvec{x}}} \left[ (X^i_s-x^i) (A^j({\varvec{X}}_s) -A^j({\varvec{x}}) ) \right] \mathrm ds =\frac{1}{2} \sum _{k=1}^N (\partial _k A^j) ({\varvec{x}}) B^{ik}({\varvec{x}})\cdot s + O(s^2). \end{aligned}$$

Applying the similar estimates carried out above to the integrand of the third term of the right-hand side in (50), we consequently obtain

$$\begin{aligned}&{\mathrm E}_{{\varvec{x}}}\left[ (X^i_\tau -x^i)(X^j_\tau -x^j) \right] - E_{{\varvec{x}}} [ X^i_\tau -x^i ] E_{{\varvec{x}}} [ X^j_\tau -x^j ]\\&\quad =2\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ B^{ij}({\varvec{X}}_s)]\mathrm ds +\frac{1}{4} \sum _{k=1}^N \left( (\partial _k A^j) ({\varvec{x}}) B^{ik}({\varvec{x}}) +(\partial _k A^i) ({\varvec{x}}) B^{jk}({\varvec{x}})\right) \tau ^2 \\&\qquad +O(\tau ^3). \end{aligned}$$

Hence, the approximation error of $\mathbb {B}$ can be estimated as follows.

$$\begin{aligned}&\frac{{\mathrm E}_{{\varvec{x}}}\left[ (X^i_\tau -x^i)(X^j_\tau -x^j) \right] - {\mathrm E}_{{\varvec{x}}}\left[ X^i_\tau -x^i \right] {\mathrm E}_{{\varvec{x}}}\left[ X^j_\tau -x^j\right] }{2\tau } - B^{ij}({\varvec{x}}) \nonumber \\&\quad =\frac{1}{\tau }\int _0^\tau {\mathrm E}_{{\varvec{x}}} [ B^{ij}({\varvec{X}}_s) - B^{ij}({\varvec{x}}) ] \mathrm ds \nonumber \\&\qquad +\frac{1}{8} \sum _{k=1}^N \left( (\partial _k A^j) ({\varvec{x}}) B^{ik}({\varvec{x}}) +(\partial _k A^i) ({\varvec{x}}) B^{jk}({\varvec{x}})\right) \tau +O(\tau ^2). \end{aligned}$$

(53)

This leads the error estimate (13) in Sect. 2.2.