A Kalman Filter with the Roessler Model

Abstract

In this chapter, we discuss an application of the Kalman filter on simple systems to study its behavior in linear and nonlinear situations. We will use the Roessler model, which we can configure both as a linear and nonlinear system. We start with the linear system, for which the Kalman filter provides an exact and optimal solution, and then study the extended Kalman filter’s performance in the nonlinear system.

In this chapter, we discuss an application of the Kalman filter on simple systems to study its behavior in linear and nonlinear situations. We will use the Roessler model, which we can configure as both a linear and nonlinear system. We start with the linear system, for which the Kalman filter provides an exact and optimal solution, and then study the extended Kalman filter’s performance in the nonlinear system.

1 Roessler Model System

The governing equations defining the Roessler model system for the three variables x, y, and z are

$$\begin{aligned} \frac{\partial x}{\partial t}&= -y - z , \end{aligned}$$

(12.1)

$$\begin{aligned} \frac{\partial y}{\partial t}&= x +a y , \end{aligned}$$

(12.2)

$$\begin{aligned} \frac{\partial z}{\partial t}&= b -cz + \beta x z , \end{aligned}$$

(12.3)

in which $\beta$ is a parameter that determines the system’s nonlinearity. If $\beta =1$, the system has stable, periodic, and chaotic solutions, depending on the values of the three other parameters. Its bifurcation diagrams are similar to that of the logistic map. Choosing $\beta =0$ results in a linear system with either growing, decaying, or periodic solutions.

2 Kalman Filter with the Roessler System

We first look at the behavior of the Kalman filter from Chap. 6 applied to a linear version of the Roessler system resulting by setting $\beta =0$. The system matrix ${\mathbf {M}}$ then becomes

$$\begin{aligned} {\mathbf {M}}= \begin{pmatrix} 0 &{} -1 &{} -1 \\ 1 &{} a &{} 0 \\ 0 &{} 0 &{} -c \end{pmatrix}, \end{aligned}$$

(12.4)

and we have chosen $b=0$ to simplify the analysis.

Fig. 12.1

Evolution of the linear system without data assimilation using an accurate RK4 scheme (left) and a less precise Euler scheme (right) with the same time step

We can find the eigenvalues of this linear system via the characteristic equation $\det ({\mathbf {M}}- \lambda {\mathbf {I}})=0$, leading to

$$\begin{aligned} \lambda = -c \quad \vee \quad \lambda = \frac{a \pm \sqrt{a^2-4}}{2}. \end{aligned}$$

(12.5)

We will always define a $c>0$, so the first eigenvalue corresponds to a decaying mode. Positive values of a lead to ever-growing modes that can be spiraling for $a<2$ and growing exponentially for $a\ge 2$. Negative values for a lead to spiraling decaying modes. In the following, we will use $a=0$, so a purely oscillatory mode.

We generate the reference solution by solving the system using a fourth-order Runge–Kutta (RK4) time-stepping scheme with a time step of $\Delta t = 0.1$. For the data-assimilation experiment, we use the same time step but employ an Euler Scheme. The Euler scheme is less accurate and will give rise to a model error at every time step.

Figure 12.1 gives the evolution for the 3-dimensional system over 200 time steps starting from initial condition (6, 0, 0), for the RK4 and the Euler scheme over 500 time steps. The parameter setting is $(a,b,c) = (0.0,0.5,0.1)$. The true solution converges towards a periodic orbit, while the Euler solution spins out of control. Thus, we need to use data assimilation to keep the system on track!

Fig. 12.2

True solution (left) and Kalman filter solution (right) with all variables observed every 5 time steps

The Kalman filter needs evolution equations for both the mean and the covariance. The evolution equation for the mean is simply the forward model using the Euler scheme, i.e.,

$$\begin{aligned} {\mathbf {x}}_{k+1} = {\mathbf {x}}_k + \Delta t {\mathbf {M}}{\mathbf {x}}_k = \bigl ({\mathbf {I}}+ \Delta t {\mathbf {M}}\bigr ){\mathbf {x}}_k , \end{aligned}$$

(12.6)

which corresponds to the model defined in Eq. (6.34). We can derive the evolution equation for the covariance Eq. (6.35) by first subtracting the true solution at time $k+1$ from Eq. (12.6), leading to

$$\begin{aligned} \begin{aligned} {\mathbf {x}}_{k+1} - {\mathbf {x}}^\mathrm {t}_k&= \bigl ({\mathbf {I}}+ \Delta t {\mathbf {M}}\bigr ){\mathbf {x}}_k - {\mathbf {x}}^\mathrm {t}_{k+1} \\&= \bigl ({\mathbf {I}}+ \Delta t {\mathbf {M}}\bigr )\bigl ({\mathbf {x}}_k - {\mathbf {x}}^\mathrm {t}_k \bigr ) + {\mathbf {q}}_k \end{aligned}, \end{aligned}$$

(12.7)

in which ${\mathbf {q}}_k$ denotes the difference between using the RK4 or the Euler scheme. If we now multiply the equation from the right with $\bigl ({\mathbf {x}}_{k+1} - {\mathbf {x}}^\mathrm {t}_k \bigr )^\mathrm {T}$, expand the right-hand side, and take the expectation, we find

$$\begin{aligned} {\mathbf {C}}_{\textit{xx},k+1}= \bigl ({\mathbf {I}}+ \Delta t {\mathbf {M}}\bigr ){\mathbf {C}}_{\textit{xx},k}\bigl ({\mathbf {I}}+ \Delta t {\mathbf {M}}\bigr )^\mathrm {T}+ {\mathbf {C}}_{\textit{qq},k}, \end{aligned}$$

(12.8)

where we assumed the model errors to be independent of the errors in the model state. Finally, keeping only terms up to $\Delta t$, consistent with the Euler scheme, we arrive at

(12.9)

which is the evolution equation for the covariance we will use in our Kalman filter.

In our first experiment, we use observations of all three variables every 5th time step, with observation error standard deviation $\sigma _{obs}=0.1$, over 1000 timesteps. Furthermore, the model error covariance matrix ${\mathbf {C}}_\textit{qq}$ is diagonal with diagonal elements equal to 0.01. We give the results of the time evolution over 1000 time steps in Fig. 12.2. We see that the Kalman filter solution wiggles around the true solution and is stable. Figure 12.3 shows the evolution of the root-mean-square error in black and the Kalman filter uncertainty of the first variable in blue. The blue line shows the characteristic behavior of Kalman filter estimated errors, with growth between observation errors and a strong reduction at assimilation time. The black line looks rather chaotic, which is not unexpected because it is a random variable. We see that the error estimation is successful as the blue and black lines have similar means. It is also interesting to note that the minimum estimated error is slightly below the observation error, as expected when the state error is larger than the observation error.

Fig. 12.3

Time evolution of the RMSE (black) and estimated error standard deviation of 1st variable (blue) with all variables observed every 5th time step

Fig. 12.4

Kalman filters solutions for only 1st variable observed every 5th time step (left) and all variables observed every 100th time-step (right)

Fig. 12.5

Time evolution of the RMSE (black) and estimated error standard deviation of 2nd variable (blue)

The following experiment has the same settings as before, but we only observe the first variable every five time steps. The solution looks quite similar in Fig. 12.4, which shows that information from the observed variable also affects the other two variables. Two processes can be responsible for this transfer. The first process is the update of unobserved variables via their covariance with the observed variables. Just before the last assimilation step, we found the relative magnitude of that correlation as 0.2 and 0.1 with the 2nd and the 3rd variable, respectively. These correlations are low, and the variance in unobserved and observed variables differs by about a factor of 10. Hence, the second process, i.e., the connection via the model dynamics, is also essential. The periodic orbit is a stable solution of the system, and forcing one of the variables to follow the true evolution draws in the others to do the same. The root-mean-square error confirms the importance of this effect in the 2nd and 3rd variables, which is about a factor $\sqrt{10}$ lower than the estimated error from the covariance matrix.

Figure 12.4 also shows the Kalman filter solution for observations every 100 time steps, approximately once every two revolutions, again observing all variables. While the estimate is not perfect, it remains stable and keeps tracking the reference evolution, although at some distance. We confirm this result by the parallel development of the true and the estimated error in the 2nd variable in Fig. 12.5. Note that the errors grow considerably between observation times but return to the observation error value at assimilation times.

These experiments demonstrate the strength of the Kalman filter in linear systems. Our next set of experiments concerns the extended Kalman filter applied to the nonlinear Roessler system.

Fig. 12.6

Examples of the time evolution of the Roessler system for $c=4$ (left), $c=6$ (middle), and $c=9$ (right), showing, respectively periodic behavior with periods one and two, and chaotic behavior

3 Extended Kalman Filter with the Roessler System

In this section, we study the application of the Kalman filter, or rather the Extended Kalman filter, to the full nonlinear Roessler model, where we now set $\beta =1$. Figure 12.6 shows the evolution of this system using a Runge–Kutta 4 scheme for 2000 time steps with time step $\Delta t =0.01$, and initial condition (6, 0, 0). We have chosen $a=b=0.1$, and c varies from 4 to 6 to 9, corresponding to a one-period solution, a two-period solution, and a chaotic solution. We can recognize the remnants of the periodic orbit of the linear system in the x-y plane and the burst into the z plane resulting from the nonlinearity. Also, note the remnants of the one-period solution in the two-period solution, but with enhanced amplitude. Likewise, we see the remnants of the two-period solution in the chaotic case.

Fig. 12.7

True evolution (left) and mean of the EKF solution (right). Note the difference in scales

We perform the data-assimilation runs using the following parameters settings for the full Roessler system: $(a,b,c) = (0.1,0.1,9)$, which brings us into the chaotic regime depicted in the right plot in Fig. 12.6. The time step is $\Delta t = 0.01$, and we choose a diagonal model error covariance matrix ${\mathbf {C}}_\textit{qq}$ with $10^{-4}$ on the diagonal. The value of this model error depends on the system setup and we find this value by trial and error as there is no general guideline for such nonlinear systems as the Roessler model. We observe the x variable every 100 time steps, but not the other variables. The observation error standard deviation is 0.1.

Fig. 12.8

Evolution of x (top), y (middle), and z (botom). The truth (black), observations (red crosses), and the EKF mean (blue) are displayed. Note the large mismatch in the z variable

Fig. 12.9

Evolution of the RMSE of x (black) and its estimated standard deviation (blue). Note that while the RMSE remains bounded the estimated error grows exponentially

We provide an overview of the true solution and the mean of the EKF in Fig. 12.7. The EKF solution does perform well in the x-y plane but tends to overshoot and be less accurate in general in the z variable. This result is not surprising as the z direction is where the nonlinearity in the system has maximal effect via the xz term in Eq. (12.3). The jumps in the solution at assimilation time are visible and not restricted to the x variable only.

Figure 12.8 provides a more detailed view of the EKF performance. The observed variable x and the unobserved y variable remain very close to the true solution, but the z variable has more significant deviations from the truth, typically slightly overshooting.

One advantage of the EKF is that it does provide an error estimate. We present the root-mean-square error (RMSE) of the second variable y and its estimated standard deviation in Fig. 12.9. The RMSE remains bounded, and the estimated error is of the same order of magnitude, but in general, not accurate at higher values. This result points to a general weakness of the EKF for strongly nonlinear systems. The linearization of the evolution equation for the error covariance can lead to inaccurate and unbounded error growth. This result supports the findings of Evensen  (1992), who experienced unbounded error growth when applying the EKF with a nonlinear QG model. The error estimates for the other variables behave similarly.