Greedy Kernel Methods for Center Manifold Approximation

Abstract

For certain dynamical systems it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to a non-hyperbolic equilibrium point, and to obtain meaningful predictions of its behavior by analyzing a reduced dimensional problem. Since the manifold is usually not known, approximation methods are of great interest to obtain qualitative estimates. In this work, we use a data-based greedy kernel method to construct a suitable approximation of the manifold close to the equilibrium. The data are collected by repeated numerical simulation of the full system by means of a high-accuracy solver, which generates sets of discrete trajectories that are then used to construct a surrogate model of the manifold. The method is tested on different examples which show promising performance and good accuracy.

1 Introduction

Center manifold theory plays an important role in the study of the stability of dynamical systems when the equilibrium point is not hyperbolic. It isolates the complicated asymptotic behavior by locating the center manifold which is an invariant manifold tangent to the subspace spanned by the eigenspace of eigenvalues on the imaginary axis. Then, the dynamics of the original system will be essentially determined by the restriction of this dynamics on the center manifold since the local dynamic behavior “transverse” to this invariant manifold is relatively simple as it corresponds to the flows in the local stable (and unstable) manifolds. In practice, one does not compute the center manifold and its dynamics exactly since this requires the resolution of a quasilinear partial differential equation which is not easily solvable. In most cases of interest, an approximation of degree two or three of the solution is sufficient. Then, the reduced dynamics on the center manifold can be determined, its stability can be studied and then conclusions about the stability of the original system can be obtained [1, 3, 4, 6, 8].

In this article, we use greedy kernel methods to construct a data-based approximation of the center manifold. The present work is a preliminary study that is intended to introduce our concept and algorithm, and to test it on some examples.

2 Background

We consider a large dimensional dynamical system

$$\displaystyle \begin{aligned} \dot{x}=f(x),\;\;x\in D, \end{aligned} $$

(1)

where $f : D \rightarrow \mathbb {R}^n$ is a continuously differentiable function over the domain $D \subset \mathbb {R}^n$ such that 0 ∈ D. We are interested in the study of the behavior of the system around an equilibrium point x ∈ D, i.e., f(x) = 0, possibly analyzing a smaller dimensional system.

Without loss of generality, we may assume that the equilibrium is x = 0, and, letting $L=\frac {\partial \,f}{\partial \,x}(x)|{ }_{x=0}$, we can rewrite (1) as

$$\displaystyle \begin{aligned} \dot{x}=f(x)=L x+N(x), \end{aligned} $$

with a suitable nonlinear component N, and denote as $\sigma _{\mathbb {R}}(L)$ the set of real parts of the eigenvalues of L. A classical result relates the stability of the equilibrium with the spectrum of L, and in particular it is known that if L has all its eigenvalues with negative real parts, i.e., $\sigma _{\mathbb {R}}(L)\subset \mathbb {R}_{<0}$, then the origin is asymptotically stable, and if L has some eigenvalues with positive real parts, then the origin is unstable. If instead $\sigma _{\mathbb {R}}(L)\subset \mathbb {R}_{\leq 0}$, the linearization fails to determine the stability properties of the origin, and thus the analysis of this situation requires to employ additional tools.

In this case, we can first use a linear change of coordinates to separate the zero and the negative eigenvalues, i.e., we can rewrite (1) as

$$\displaystyle \begin{aligned} \dot{x}=& L_1 x + N_1(x,y) \\ \dot{y}=& L_2 y + N_2(x,y){} \end{aligned} $$

(2)

where $L_1 \in \mathbb {R}^{d \times d}$ is such that $\sigma _{\mathbb {R}}(L_1)=\{0\}$ and $L_2 \in \mathbb {R}^{m \times m}$ with m:= n − d is such that $\sigma _{\mathbb {R}}(L_2)\subset \mathbb {R}_{<0}$. The nonlinear functions $N_1 : \mathbb {R}^d \times \mathbb {R}^m \rightarrow \mathbb {R}^d$ and $N_2 : \mathbb {R}^d \times \mathbb {R}^m \rightarrow \mathbb {R}^m$ are continuously differentiable. Intuitively, we expect the stability of the equilibrium to only depend on the nonlinear term N ₁(x, y). This intuition turns out to be correct, and indeed it can be properly formalized by means of the center manifold theorem.

We start by recalling a sufficient condition for the existence of a center manifold.

Theorem 1 ([1])

If N ₁ and N ₂ are twice continuously differentiable and are such that

$$\displaystyle \begin{aligned} N_i(0,0)=0,\;\; \frac{\partial N_i}{\partial x}(0,0)=0,\;\; \frac{N_i}{\partial y}(0,0)=0,\;\; i = 1, 2, \end{aligned} $$

and if the eigenvalues of L ₁ have zero real parts, and all the eigenvalues of L ₂ have negative real parts, then there exists a neighbourhood $\varOmega \subset \mathbb {R}^d$ of the origin $0 \in \mathbb {R}^d$ and a center manifold $h:\varOmega \to \mathbb {R}^m$ for (2), i.e., y = h(x) is an invariant manifold for (2), ¹ h is smooth, and

$$\displaystyle \begin{aligned} h(0)=0,\;\; Dh(0)=0. \end{aligned} $$

(3)

Under the assumptions of this theorem, using (2) we deduce that h satisfies the PDE

$$\displaystyle \begin{aligned} L_2 h(x) + N_2(x,h(x))=D h(x)\left(L_1 x +N_1(x,h(x))\right), \end{aligned} $$

(4)

and the following center manifold theorem ensures that there are smooth solutions to this PDE. Moreover, it also allows to deduce the stability of the origin of the full order system (2) from the stability of the origin of a reduced order system called the center dynamics.

Theorem 2 (Center Manifold Theorem [1])

The equilibria x = 0, y = 0 of the original dynamics is locally asymptotically stable (resp. unstable) if and only if the equilibrium x = 0 of the center dynamics (dynamics on the center manifold)

$$\displaystyle \begin{aligned} \dot{x}= L_1 x +N_1(x,h(x)), \end{aligned} $$

(5)

is locally asymptotically stable (resp. unstable).

In particular, this result guarantees that, after solving the PDE (4), the problem of analyzing the stability properties of the system (2) reduces to analyzing the nonlinear stability of the lower dimensional system (5). This second problem is of smaller dimension and thus, provided the knowledge of h, the approach is attractive to obtain information on the system (1) via a reduced model.

Moreover, we remark that an exact knowledge of h is not required for this purpose, i.e., it is sufficient to have an approximate solution of the PDE (4). Indeed, it is frequently sufficient to compute only the low degree terms of the Taylor series expansion of h around x = 0, i.e., if (⋅)^[k] is the degree k part of the Taylor series of h, the approximation

$$\displaystyle \begin{aligned} h(x)\approx h^{[1]} x+h^{[2]}(x)+h^{[3]}(x)+\ldots+h^{[d-1]}(x) \end{aligned} $$

(6)

is sufficient to obtain an approximation of the dynamics of order ε ^d as ∥x∥≤ ε. The approximation (6) can be obtained by coefficient comparison, thus rewriting the PDE (4) as a set of algebraic equations as

$$\displaystyle \begin{aligned} &L_2 h^{[1]}=h^{[1]} L_1\\ &L_2 h^{[2]}(x) + N^{[2]}_2(x,h^{[1]}(x))=\frac{\partial h^{[2]}}{\partial x}(x)\left(L_1 x_1 +N^{[2]}_1(x,h^{[1]}(x))\right)\\ &L_2 h^{[3]}(x) + \left(N_2(x,h^{[2]}(x))\right)^{[2]}=\frac{\partial h^{[2]}}{\partial x}(x) \left(L_1 x +\left(N_1(x,h^{[2]}(x))\right)^{[2]}\right)\\ &\dots. \end{aligned} $$

We remark that this methodology is valid for parameterized dynamical systems and is used to study the stability of dynamical systems with bifurcations.

Nevertheless, even this approximated knowledge of h can be difficult to obtain in practice for a general ODE. To overcome this limitation, and since an approximated knowledge of the manifold is sufficient, our goal in this paper is to find a data-based approximation of the center manifold. This approximation is based solely on the knowledge of the splitting (1) and on the numerical computation of a set of trajectories of the system, and it provides an approximation of h which can be used to study the system stability.

3 Kernel Approximation

We want to build a surrogate model $s_h :\varOmega \rightarrow \mathbb {R}^m$ which approximates the center manifold h on a suitable set $\varOmega \subset \mathbb {R}^d$, in the sense that s _h(x) ≈ h(x) for all x ∈ Ω. This model is constructed in a data-based way, i.e., we assume some knowledge of the map h on a finite set of input parameters, or training data. In practice, such values are computed from high-fidelity numerical approximations, which will be discussed in detail in the following.

The surrogate is based on kernel approximation, which allows the use of scattered data, i.e., we do not require any grid structure on the set of training data. Moreover, since the unknown function h is vector-valued, we employ here matrix-valued kernels. Details on kernel-based approximation can be found e.g. in [9], and the extension to the vectorial case is detailed e.g. in [5, 10]. We recall here only that a positive definite matrix-valued kernel on Ω is a function $K : \varOmega \times \varOmega \rightarrow \mathbb {R}^{m \times m}$ such that K(x, y) = K(y, x)^T for all x, y ∈ Ω and $[K(x_i, x_j)]_{i,j=1}^N\in \mathbb {R}^{mN\times mN}$ is positive semidefinite for any set {x ₁, …, x _N}⊂ Ω of pairwise distinct points, for all $N \in \mathbb {N}$. Associated to a positive definite kernel there is a unique Hilbert space $\ mathscr H$ of functions $\varOmega \to \mathbb {R}^m$, named native space, where the kernel is reproducing, meaning that K(⋅, x)α is the Riesz representer of the directional point evaluation $\delta _x^{\alpha }(f):= \alpha ^T f(x)$, for all $\alpha \in \mathbb {R}^m$, x ∈ Ω.

We consider here a twice continuously differentiable matrix-valued kernel k on Ω, and we use a specific functional formulation for our approximation and a specific cost function, in order to construct a surrogate that is well suited for the particular approximation task.

In detail, the approximant takes the form

$$\displaystyle \begin{aligned} s_h(x) = \sum_{i=1}^{n_1} K(x,x_i^{(1)})\alpha_i + \sum_{j=1}^{n_2} \sum_{i=1}^m \partial_i^{(2)} K(x,x_j^{(2)})\beta_{i,j}, \end{aligned}$$

with centers $x_i^{(1)} \in X^{(1)} = \left \{ x_1^{(1)}, \dots , x_{n_1}^{(1)} \right \}$, $x_j^{(2)} \in X^{(2)} = \left \{ x_1^{(2)}, \dots , x_{n_2}^{(2)} \right \}$ and coefficient vectors $\alpha _i, \beta _{i,j} \in \mathbb {R}^m$. Here the superscript ∂ ⁽²⁾ denotes that the derivative with regards to the second kernel component is taken.

Subsequently, we assume to have a sufficient amount of data $X_{N^{\ast }} = \{x_1, \dots ,x_{N^{\ast }} \}$ and $Y_{N^{\ast }} = \{ y_1, \dots , y_{N^{\ast }} \}$ which, for example, is generated by running a numerical scheme to compute discrete trajectories for different initial values (x ₀, y ₀). For this step, we need to assume that the variable splitting (2) is known in advance. Note that this is not a severe restriction, as for a general ODE (1) the required state transformation can be determined by eigenvalue decomposition of L.

Observe that we do not know if a data pair (x _i, y _i) lies on the center manifold, i.e. if y _i = h(x _i) holds. We only know that the data converges asymptotically to the center manifold as x _i → 0. Thus, an interpolation-based surrogate which merely interpolates the data on a given subset $X \subset X_{N^{\ast }}$ seems ill-suited for our purposes. Instead we consider another set of conditions to define the approximant. First, we still require the conditions in (3) to be satisfied by our approximation. Moreover, for the given subsets X = {x ₁, …, x _N} and Y = {y ₁, …, y _N}, we compute our approximant by minimizing the following functional $J : \ mathscr {H} \rightarrow \mathbb {R}$ under the constraint s(0) = 0, Ds(0) = 0:

$$\displaystyle \begin{aligned} J(s) := \| s \|{}_{\ mathscr{H}}^2 + \sum_{i=1}^N (s(x_i) - y_i)^T \omega_i (s(x_i) - y_i). {} \end{aligned} $$

(7)

Here $\omega _i \in \mathbb {R}^{m \times m}$ is a positive definite weight matrix. It can be shown that (7) has a unique minimizer s _h (see [11]). In particular s _h and its derivative Ds _h have the form

$$\displaystyle \begin{aligned} s_h(x) & = \sum_{i=1}^{N+1} K(x,x_i)\alpha_i + \sum_{i=1}^m \partial_i^{(2)}K(x,0)\beta_{i}, \\ Ds_h(x) & = \sum_{i=1}^{N+1} D^{(1)}K(x,x_i)\alpha_i + \sum_{i=1}^m D^{(1)}\partial_i^{(2)}K(x,0)\beta_{i}, \end{aligned} $$

(8)

where we set x _N+1:= 0. The coefficient vectors α _i, β _i can be computed by solving the system

$$\displaystyle \begin{aligned} \begin{pmatrix} A + W & B \\ B^T & C \end{pmatrix} \begin{pmatrix} \boldsymbol{\alpha} \\ \boldsymbol{\beta} \end{pmatrix} = \begin{pmatrix} \boldsymbol{Y} \\ \boldsymbol{Z} \end{pmatrix}, \end{aligned} $$

(9)

with

$$\displaystyle \begin{aligned} A & := \left( K(x_i,x_j) \right)_{i,j} \in \mathbb{R}^{m(N+1) \times m(N+1)}, \\ W & := \operatorname{\mathrm{diag}}\left( \omega_1^{-1},\dots, \omega_N^{-1},0 \right) \in \mathbb{R}^{m(N+1) \times m(N+1)}, \\ B & := \left( \partial_{j}^{(2)}K(x_i,0) \right)_{i,j} \in \mathbb{R}^{m(N+1) \times m^2}, \\ C & := \left( \partial_{i}^{(1)}\partial_{j}^{(2)}k(0,0) \right)_{i,j} \in \mathbb{R}^{m^2 \times m^2}, \\ \boldsymbol{Y} & := (y_1^T, \dots,y_n^T,0)^T \in \mathbb{R}^{m(N+1)}, \\ \boldsymbol{Z} & := 0 \in \mathbb{R}^{m^2 \times m}. \end{aligned} $$

The weight matrices ω _i can either be chosen manually, or a regularizing function $r : \varOmega \rightarrow \mathbb {R}^{m \times m}$ can be prescribed such that ω _i = r(x _i) is symmetric and positive definite. In our numerical examples in Sect. 4 we chose a constant regularization function, i.e.

$$\displaystyle \begin{aligned} \omega_i = r(x_i) = \lambda I_m \end{aligned}$$

for some λ > 0. However, one might consider a more general approach, where the weight increases as the data tends to the origin, i.e. ω _i ≽ ω _j if ∥x _i∥≤∥x _j∥.

3.1 Greedy Approximation

If the technique of the previous section is used as it is, the surrogate (8) is given by an expansion with N ^∗ terms, where N ^∗ is the number of points in the training set. Therefore, the model evaluation might not be efficient enough if the model is built using a too large dataset. Furthermore, the computation of the coefficients in (8) requires the solution of the linear system (9), whose size again scales with the size of the training set, and which can be severely ill-conditioned for non well-placed points.

To mitigate both problems, we employ an algorithm that aims at selecting small subsets X _N, Y _N of points such that the surrogate computed with these sets is a sufficiently good approximation of the one which uses the full sets. The algorithm selects the points in a greedy way, i.e., one point at a time is selected and added to the current training set. In this way, it is possible to identify a good set without the need to solve a nearly infeasible combinatorial problem.

The selection is performed using the P-greedy method of [2] applied to the kernel K, such that the set of points is selected before the computation of the surrogate. The number of points, and therefore the expansion size and evaluation speed, is depending on a prescribed target accuracy ε _tol > 0. For details on the method implementation and its convergence properties we refer to [7].

4 Numerical Examples

We test now our method on three different examples. In each of them, we specify the setting and the parameters used to build the surrogate and visualize our approximation to the center manifold. Additionally, we compute the pointwise residual

$$\displaystyle \begin{aligned} r(x) = Ds_h(x)\left(L_1x +N_1(x,s_h(x))\right) - \left(L_2s_h(x) + N_2(x,s_h(x))\right), \end{aligned}$$

which measures how well the surrogate s _h satisfies the ODE (4).

In all the three examples, the greedy algorithm is used to select a suitable subset of the points, and in all cases the procedure is stopped with a prescribed ε _tol. In the first two examples we set ε _tol:= 10⁻¹⁵, while ε _tol:= 10⁻¹⁰ is used in the last one.

4.1 Example 1

We consider the 2-dimensional system

$$\displaystyle \begin{aligned} \begin{aligned} \dot{x} & = L_1x + N_1(x,y) = 0 + xy \\ \dot{y} & = L_2y + N_2(x,y) = -y + x^2. \end{aligned} \end{aligned} $$

(10)

We generate the training data by solving (10) with an implicit Euler scheme for initial time t ₀ = 0, final time T = 1000 and with the time step Δt = 0.1. We initiate the numerical procedure with initial values $(x_0,y_0) \in \left \{ \pm 0.8 \right \} \times \left \{ \pm 0.8 \right \}$ and store the resulting data pairs in X and Y  after discarding all data whose x-values are not contained in the neighborhood [−0.1, 0.1] which results in N ^∗ = 38, 248 data pairs.

We run the greedy algorithm for the kernels $k_1(x,y) := \left (1 + xy/2\right )^4$ and $k_2(x,y) = e^{-(x-y)^2/2}$. This results in the sets X ₁ and X ₂ which contain 14 and 6 points, respectively. The corresponding approximations s ₁ and s ₂ for the constant regularization function r ≡ 10⁻¹⁰ are plotted in Fig. 1 over the domain [−0.1, 0.1]. The pointwise residual is depicted in Fig. 2.

Fig. 1

Approximations s ₁ and s ₂ of the center manifold

Fig. 2

Residuals r ₁ and r ₂ of the center manifold

4.2 Example 2

We consider the 2-dimensional system

$$\displaystyle \begin{aligned} \begin{aligned} \dot{x} & = L_1x + N_1(x,y) = 0 -xy \\ \dot{y} & = L_2y + N_2(x,y) = -y + x^2 - 2y^2. \end{aligned} \end{aligned} $$

(11)

The training data is generated the same way as in Example 1. We again use the kernels k ₁ and k ₂. The greedy algorithm gives sets X ₁ and X ₂ of size 12 and 6, respectively. The evaluation of the approximations s ₁ and s ₂ over the neighborhood [−0.1, 0.1] can be seen in Fig. 3, while the respective pointwise residuals are plotted in Fig. 4.

Fig. 3

Approximations s ₁ and s ₂ of the center manifold

Fig. 4

Residuals r ₁ and r ₂ of the center manifold

4.3 Example 3

We consider the (2 + 1)-dimensional system

$$\displaystyle \begin{aligned} \begin{aligned} \dot{x} & = L_1 x + N_1(x,y) = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} + \begin{pmatrix} x_1y \\ x_2 y \end{pmatrix} \\ \dot{y} & = L_2y + N_2(x,y) = -y -x_1^2 - x_2^2 + y^2. \end{aligned} \end{aligned} $$

(12)

We generate the training data in a similar fashion as before. We again use the implicit Euler scheme with start time t = 0, final time T = 1000 and with time step Δt = 0.1. The Euler method is performed for initial data $(x_0,y_0) \in \left \{ \pm 0.8 \right \} ^3$ and the resulting trajectories are stored in X and Y , where only data with x ∈ [−0.1, 0.1]² was considered; this leads to N ^∗ = 78, 796 data pairs. We use the kernels k ₁(x, y) = (1 + x ^T y∕2)⁴ and $k_2(x,y) = e^{- \|x - y\|{ }_2^2/2}$, and the greedy-selected sets have the size 21 (for k ₁) and 25 (for k ₂), respectively. The approximations s ₁, s ₂ and their corresponding residuals r ₁ and r ₂ computed over the domain [−0.1, 0.1]². The results can be seen in Fig. 5.

Fig. 5

Approximations s ₁ and s ₂ of the center manifold and corresponding residuals r ₁ and r ₂

We remark that in all the three experiments both kernels give comparable results in terms of error magnitude, and they both provide a good approximation of the manifold.

5 Conclusions

In this paper we introduced a novel algorithm to approximate the center manifold of a given ODE using a data-based surrogate.

This algorithm computes an approximation of the manifold from a set of numerical trajectories with different initial data. It is based on kernel methods, which allow the use of the scattered data generated by these simulations as training points. Moreover, an application-specific ansatz and cost function have been employed in order to enforce suitable properties on the surrogate.

Several numerical experiments suggested that the present method can reach a significant accuracy, and that it has the potential to be used as an effective model reduction technique. It seems promising to apply this approach to high dimensional systems as the approximation technique straightforwardly can be extended and is less prone to the curse of dimensionality than grid-based approximation techniques. An interesting extension would consist of determining the decomposition (2) in a data-based fashion by suitable processing of the trajectory data.