Data-driven modeling of the chaotic thermal convection in an annular thermosyphon

Abstract

Identifying accurate and yet interpretable low-order models from data has gained a renewed interest over the past decade. In the present work, we illustrate how the combined use of dimensionality reduction and sparse system identification techniques allows us to obtain an accurate model of the chaotic thermal convection in a two-dimensional annular thermosyphon. Taking as guidelines the derivation of the Lorenz system, the chaotic thermal convection dynamics simulated using a high-fidelity computational fluid dynamics solver are first embedded into a low-dimensional space using dynamic mode decomposition. After having reviewed the physical properties the reduced-order model should exhibit, the latter is identified using SINDy, an increasingly popular and flexible framework for the identification of nonlinear continuous-time dynamical systems from data. The identified model closely resembles the canonical Lorenz system, having the same structure and exhibiting the same physical properties. It moreover accurately predicts a bifurcation of the high-dimensional system (corresponding to the onset of steady convection cells) occurring at a much lower Rayleigh number than the one considered in this study.

Appendices

Derivation of the optimal solution to the DMD problem

1.1 Equivalence between norm-minimization and trace-maximization formulation of the DMD problem

This section aims at deriving the equivalence between the norm-minimization and trace-maximization formulations of the DMD problem and the corresponding solution. Given two data sequences ${\varvec{X}}$ and ${\varvec{Y}}$ related by

$$\begin{aligned} {\varvec{y}}_k = {\varvec{f}}({\varvec{x}}_k), \end{aligned}$$

the aim of DMD is to find the low-rank operator ${\varvec{A}}$ that best approximates the possibly nonlinear function ${\varvec{f}}$ in the least-squares sense. As discussed in Sect. 4.1, this can be formulated as the following rank-constrained minimization problem

$$\begin{aligned} \begin{aligned} {{\,\mathrm{minimize~}\,}}_{{\varvec{P}}, {\varvec{Q}}}&\quad \Vert {\varvec{Y}} - {\varvec{PQ}}^H {\varvec{X}} \Vert _F^2 \\ {{\,\mathrm{subject~to~}\,}}&\quad \text {rank } {\varvec{P}} = \text {rank } {\varvec{Q}} = r \\&\quad {\varvec{P}}^H {\varvec{P}} = {\varvec{I}}, \end{aligned} \end{aligned}$$

(4)

where we use the fact the unknown low-rank operator can be factorized as ${\varvec{A}} = {\varvec{PQ}}^H$. Introducing the cost function

$$\begin{aligned} \begin{aligned} \mathcal {J}({\varvec{P}}, {\varvec{Q}})&= \Vert {\varvec{Y}} - {\varvec{PQ}}^H {\varvec{X}} \Vert _F^2 \\&= \text {Tr} \left( ({\varvec{Y}} - {\varvec{PQ}}^H {\varvec{X}})({\varvec{Y}} - {\varvec{PQ}}^H {\varvec{X}})^H \right) \end{aligned} \end{aligned}$$

for the sake of notational simplicity, the solution $\left( {\varvec{P}}^*, {\varvec{Q}}^* \right)$ of Eq. (4) is given by the stationary points of $\mathcal {J}({\varvec{P}}, {\varvec{Q}})$, i.e.,

$$\begin{aligned} \begin{aligned} \frac{\partial \mathcal {J}}{\partial {\varvec{P}}}&= -\,2 {\varvec{YX}}^H {\varvec{Q}} + 2 {\varvec{PQ}}^H {\varvec{XX}}^H {\varvec{Q}}&= 0\\ \frac{\partial \mathcal {J}}{\partial {\varvec{Q}}}&= -\,2 {\varvec{XY}}^H {\varvec{P}} + 2 {\varvec{XX}}^H {\varvec{QP}}^H {\varvec{P}}&= 0 \end{aligned} \end{aligned}$$

Introducing the orthogonality constraint on ${\varvec{P}}$ in the second equation above yields the following expression for ${\varvec{Q}}$

$$\begin{aligned} {\varvec{Q}}^H = {\varvec{P}}^H {\varvec{C}}_{yx} {\varvec{C}}_{xx}^{-1}, \end{aligned}$$

where ${\varvec{C}}_{yx} = {\varvec{YX}}^H$ and ${\varvec{C}}_{xx} = {\varvec{XX}}^H$ are the empirical variance–covariance matrices introduced in Sect. 4.1. Inserting this expression for ${\varvec{Q}}$ in Eq. (4) yields

$$\begin{aligned} \begin{aligned} {{\,\mathrm{minimize~}\,}}_{{\varvec{P}}}&\quad \Vert {\varvec{Y}} - {\varvec{PP}}^H {\varvec{C}}_{yx} {\varvec{C}}_{xx}^{-1} {\varvec{X}} \Vert _F^2 \\ {{\,\mathrm{subject~to~}\,}}&\quad \text {rank } {\varvec{P}} = r \\&\quad {\varvec{P}}^H {\varvec{P}} = {\varvec{I}}, \end{aligned} \end{aligned}$$

(5)

From this point, it is straightforward to show that the above problem can be rewritten as

$$\begin{aligned}&{{\,\mathrm{minimize~}\,}}_{{\varvec{P}}} \text { Tr }\left( {\varvec{C}}_{yy}\right) -2 \text {Tr} \left( {\varvec{P}}^H{\varvec{C}}_{yx}{\varvec{C}}_{xx}^{-1}{\varvec{C}}_{xy}{\varvec{P}}\right) \nonumber \\&{{\,\mathrm{subject~to~}\,}}\text { rank } {\varvec{P}}=r\nonumber \\&{\varvec{P}}^{H}{\varvec{P}}={\varvec{I}} \end{aligned}$$

(6)

$\text {Tr}\left( {\varvec{C}}_{yy}\right)$ being constant, we thus established the equivalence between the norm minimization problem (4) and the following trace maximization problem

$$\begin{aligned} \begin{aligned} {{\,\mathrm{maximize~}\,}}_{{\varvec{P}}}&\quad \text {Tr}\left( {\varvec{P}}^H {\varvec{C}}_{yx} {\varvec{C}}_{xx}^{-1} {\varvec{C}}_{xy} {\varvec{P}}\right) \\ {{\,\mathrm{subject~to~}\,}}&\quad \text {rank } {\varvec{P}}=r \\&\quad {\varvec{P}}^H {\varvec{P}} = {\varvec{I}}. \end{aligned} \end{aligned}$$

(7)

Finally, recalling that the trace of a matrix is the sum of its eigenvalues, basic linear algebra is sufficient to prove that the solution of the above optimizations problem is given by the first r leading eigenvectors of the symmetric positive definite matrix ${\varvec{C}}_{yx} {\varvec{C}}_{xx}^{-1} {\varvec{C}}_{xy}$. Once ${\varvec{P}}$ has been computed, the matrix ${\varvec{Q}}$ is simply given by ${\varvec{Q}}^H = {\varvec{P}}^H {\varvec{C}}_{yx} {\varvec{C}}_{xx}^{-1}$ as stated previously.

Interestingly, it can be noted that if ${\varvec{X}}$ is a full rank $m \times m$ matrix (i.e., as many linearly independent samples as degrees of freedom), then

$$\begin{aligned} \begin{aligned} {\varvec{C}}_{yx} {\varvec{C}}_{xx}^{-1} {\varvec{C}}_{xy}&= {\varvec{Y}} {\varvec{X}}^H \left( {\varvec{XX}}^H \right) ^{-1} {\varvec{XY}}^H \\&= {\varvec{YY}}^H \\&= {\varvec{C}}_{yy}. \end{aligned} \end{aligned}$$

In this particular case (hardly encountered in fluid dynamics due to the memory footprint of storing m snapshots of a m-dimensional vector), the columns of ${\varvec{P}}$ are simply the first r leading eigenvectors of the variance–covariance matrix ${\varvec{C}}_{yy}$, i.e., the POD modes of the output matrix ${\varvec{Y}}$. Alternatively, if ${\varvec{X}}$ is a skinny $m \times n$ matrix (with $m > n$) or a rank-deficient $m \times m$ matrix, one can introduce its economy-size singular value decomposition

$$\begin{aligned} {\varvec{X}} = {\varvec{U}}_{{\varvec{X}}} \varvec{\Upsigma }_{{\varvec{X}}} {\varvec{V}}^H_{{\varvec{X}}}. \end{aligned}$$

The columns of ${\varvec{P}}$ are then given by the first r left singular vectors of the matrix ${\varvec{YV}}_{{\varvec{X}}}{\varvec{V}}_{{\varvec{X}}}^H$ where ${\varvec{V}}_{{\varvec{X}}} {\varvec{V}}_{{\varvec{X}}}^H$ is the projector onto the rowspan of ${\varvec{X}}$.

1.2 Computing the DMD modes and eigenvalues from the low-rank factorization of ${\varvec{A}}$

It must be noted that the columns of ${\varvec{P}}$ are not the so-called DMD modes although they span the same subspace. The DMD modes can nonetheless be easily computed from the low-rank factorization ${\varvec{A}} = {\varvec{PQ}}^H$ of the DMD operator. The $i{\mathrm{th}}$ DMD mode is solution to the eigenvalue problem

Since needs to be in the columnspan of ${\varvec{A}}$, it can be expressed as a linear combination of the columns of ${\varvec{P}}$, i.e.,

where ${\varvec{b}}_i$ is a small r-dimensional vector. Introducing this expression into the DMD eigenvalue problem yields after some simplifications

$$\begin{aligned} {\varvec{b}}_i \lambda _i = {\varvec{Q}}^H {\varvec{P}} {\varvec{b}}_i. \end{aligned}$$

Although ${\varvec{A}}$ is a high-dimensional $m \times m$ matrix, ${\varvec{Q}}^H {\varvec{P}}$ is a small $r \times r$ matrix whose eigenvectors ${\varvec{b}}_i$ and eigenvalues $\lambda _i$ can easily be computed using direct solvers thus enabling the computations of the eigenvalues and eigenvectors of ${\varvec{A}}$ at a reduced cost.

Similarly, the adjoint DMD modes $\varvec{\Upphi }_i$ are solution to the adjoint DMD eigenvalue problem

$$\begin{aligned} \varvec{\Upphi }_i \bar{\lambda }_i = {\varvec{A}}^H \varvec{\Upphi }_i, \end{aligned}$$

where the overbar denotes the complex conjugate. These now live in the rowspan of ${\varvec{A}}$ and can thus be expressed as

$$\begin{aligned} \varvec{\Upphi }_i = {\varvec{Qc}}_i \end{aligned}$$

where once again ${\varvec{c}}_i$ is a small r-dimensional vector. The corresponding low-dimensional eigenvalue problem then reads

$$\begin{aligned} {\varvec{c}}_i \bar{\lambda }_i = {\varvec{P}}^H {\varvec{Q}} {\varvec{c}}_i \end{aligned}$$

with ${\varvec{P}}^H {\varvec{Q}}$ a small $r \times r$ matrix. It must be noted however that, from a practical point of view, converging the adjoint DMD modes $\varvec{\Upphi }_i$ may require significantly more data than needed to the converge the direct DMD modes .

Broomhead–King embedding for attractor reconstruction

Attractor reconstruction from a single time-series has been instrumental in highlighting the connection between the dynamics of the thermosyphon and that of the canonical Lorenz system (see Figs. 3, 4). The aim of this section is to briefly introduce inexperienced readers to the Broomhead–King attractor reconstruction technique used herein. For more details, please refer to the original paper [20]. For the sake of simplicity and reproducibility, we will apply this attractor reconstruction technique using the Lorenz system

$$\begin{aligned} \begin{aligned} \dot{x}&= \sigma \left( y - x \right) \\ \dot{y}&= x \left( \rho - z \right) - y \\ \dot{z}&= xy - \beta z \end{aligned} \end{aligned}$$

with $\left( \sigma , \rho , \beta \right) = \left( 10, 28, {8}\big /{3} \right)$ so that it exhibits chaotic dynamics. The properties of this dynamical system have been discussed in Sect. 3. Figure 17 depicts its well-known strange attractor along with representative time-series of x(t) , y(t) and z(t) . In the rest of this appendix, we will work exclusively with the time-series of x(t) . The whole dataset consists of the evolution of x(t) from $t = 0$ to $t = 400$ with a sampling period $\Delta t = 0.001$ resulting in 400,000 samples.

Although the Takens embedding theorem [19] provides theoretical guarantees to reconstruct a multidimensional attractor from a single time-series, this approach might break when applied to real data that may be contaminated by measurement noise or it might be quite sensitive on the sampling period used to acquire the signal. Additionally, once the lag $\tau$ has been determined, the number of time-lagged versions of x(t) needed to reconstruct the attractor needs to be determined using technique-based nearest neighbors [52,53,54] which might provide conflicting estimates of the attractor’s dimension and different reconstructions. As to overcome these limitations, Broomhead and King [20] have proposed instead to reconstruct the attractor based on the singular value decomposition of the Hankel matrix constructed from time-lagged version of x(t)

$$\begin{aligned} {\varvec{H}} = \frac{1}{\sqrt{N}} \begin{bmatrix} x_1 &{}\quad x_2 &{}\quad x_3 &{}\quad \cdots &{}\quad x_d \\ x_2 &{}\quad x_3 &{}\quad x_4 &{}\quad \cdots &{}\quad x_{d+1} \\ \vdots &{}\quad \vdots &{}\quad \vdots &{}\quad \cdots &{}\quad \vdots \\ x_{N-d+1} &{}\quad x_{N-d+2} &{}\quad x_{N-d+3} &{}\quad \cdots &{}\quad x_{N} \\ \end{bmatrix} \end{aligned}$$

where $x_k = x(t_k)$, N is the total length of the time-series (i.e., 400,000 in our case) and d is the maximum number of time-lagged considered. The choice of d will be discussed later on. Decomposing this Hankel matrix as ${\varvec{H}} = {\varvec{U}} \varvec{\Upsigma } {\varvec{V}}^H$, the dimension of the attractor can then be inferred from the distribution of the singular values $\sigma _i$. The columns of ${\varvec{U}}$ then provide an orthonormal basis for the embedding space, while the columns of ${\varvec{V}}$ describe the evolution of the system within this embedding space, thus enabling the reconstruction we aimed for.

The choice of the number of lags d needed to construct the Hankel matrix is of crucial importance as it will determine the window length $\tau _w = d \Delta t$ used to embed the dynamics which, in turn, will determine the structure of the singular value spectrum. In practice, the window length $\tau _w$ needs to be large enough to average out the possible noise contaminating the data. Note however that, in the limit $\tau _w \rightarrow \infty$, the singular value decomposition of the Hankel matrix converges to the discrete Fourier transform of x(t) . Chaotic systems being characterized by a continuous spectrum, an infinite number of singular components would thus be needed to reconstruct the signal, grossly overestimating the dimension of the attractor. In their paper, Broomhead and King [20] recommend to use a window length $\tau _w$ of approximately one tenth of the period of oscillation about the unstable saddle-foci. In this work, this window length has been chosen as one twentieth of this oscillating period, even though choosing one tenth does not change qualitatively (nor quantitatively) the results presented. Note that the same rationale has been used when applying this technique to the data from the thermosyphon. For our choice of parameters, Fig. 18 provides the temporal evolution of the system within the embedding space and the corresponding reconstruction of the attractor. The theoretical properties of such time-delay embedding of attractors have been recently studied by Kamb et al. [55].

Fig. 17

Projection of the Lorenz attractor on the (x, z) plane (left) and corresponding time-series of x(t) , y(t) and z(t) (right). The parameters of the system are chosen as $\left( \sigma , \rho , \beta \right) = \left( 10, 28, {8}\big /{3} \right)$

Fig. 18

Projection of the Lorenz attractor reconstructed using the Broomhead–King technique on the $(v_1, v_2)$ plane (left) and corresponding time-series of $v_1(t)$, $v_2(t)$ and $v_3(t)$ (right). A window length $\tau _w = 0.032$, corresponding to roughly one twentieth of the oscillating period around the unstable saddle-foci, has been chosen