Theory and Applications of the Orthogonal Fast Lyapunov Indicator (OFLI and OFLI2) Methods

Abstract

During the last decades the Nonlinear Dynamics field has produced a large number of numerical techniques oriented to the analysis of the behavior of the orbits in different systems. These methods are mainly focused to distinguish chaotic from regular behavior. Among the variational methods, based into the variational equations, we discuss in this paper the so-called Orthogonal Fast Lyapunov Indicator (OFLI and OFLI2) methods that are variants of the FLI method but designed to obtain also some information about the periodic orbits of the systems. We review the OFLI and OFLI2 methods and we show several computational aspects related with avoiding the appearance of spurious structures, with their use in the analysis of regular/chaotic behaviors, but also with the analysis of periodic orbits and regular regions, and with the efficient computation of the solution of the variational equations by means of Taylor series methods. Finally, the methods are shown in several Hamiltonian problems, as well as in several classical dissipative systems, as the Lorenz and Rössler models.

Dedicated to the Memory of Eugenio Barrio (1934–2014)

Appendix: Adaptive ODE Integrator—Taylor Series Method

A basic feature in using any chaos indicator is to obtain numerically solutions of ODE systems and variational equations. In Dynamical Systems there are also more requirements, for example, in the process of determination of periodic orbits we obviously have to integrate the differential system, normally for a short time, with very high precision, especially for highly unstable periodic orbits. Moreover, in the study of the bifurcations and stability of periodic orbits we also have to integrate the first order variational equations using as initial conditions the identity matrix as also occurs in the use of chaos indicators. To reach this goal we may, obviously, use any numerical ODE method like, for example, a Runge–Kutta method. The last few years, in the computational dynamics community [66] one of the preferred methods is the Taylor series method.

The Taylor method is one of the oldest numerical methods for solving ordinary differential equations but it is scarcely used in the numerical analysis community. Its formulation is quite simple [67]. Let us consider the initial value problem $\dot{\mathbf{y}} = \mathbf{f}(t,\,\mathbf{y})$. Now, the value of the solution at t _i (that is, y(t _i )) is approximated by y _i from the nth degree Taylor series of y(t) at t = t _i (the function f has to be a smooth function). So, denoting $h_{i} = t_{i} - t_{i-1}$,

$$\displaystyle{\begin{array}{rcl} \mathbf{y}(t_{0})& =:&\mathbf{y}_{0}, \\ \mathbf{y}(t_{i})& \simeq &\mathbf{y}_{i-1} + \mathbf{f}(t_{i-1},\mathbf{y}_{i-1})\,h_{i} + \frac{1} {2!}\,\frac{d\mathbf{f}(t_{i-1},\mathbf{y}_{i-1})} {dt} \,h_{i}^{2}+\ldots \\ & &\ldots + \frac{1} {n!}\,\frac{d^{n-1}\mathbf{f}(t_{i-1},\mathbf{y}_{i-1})} {dt^{n-1}} \,h_{i}^{n}\,\,\, =:\,\,\, \mathbf{y}_{ i}.\end{array} }$$

Therefore, the problem is reduced to the determination of the Taylor coefficients $\{1/(\,j + 1)!\,d^{j}\mathbf{f}/dt^{j}\}$. This may be done quite efficiently by means of the automatic differentiation (AD) techniques (for more details see [20]). Note that the Taylor method has several good features; one of them is that it gives directly a dense output in the form of a power series being therefore quite useful when an event location criteria may be used (as in the computation of Poincaré sections), it can be formulated as an interval method giving guaranteed integration methods (used, by instance, in the computer assisted proof of chaos [1] and skeletons of periodic orbits [68]), Taylor methods may manage directly high order differential equations just taking into account that the Taylor coefficients for the solution and its derivatives are evidently related, Taylor methods of degree n are also of order n and so Taylor methods of high degree give us numerical methods of high order (therefore, they are very useful for high-precision solution of ODEs, as needed, for example, in some fine studies in dynamical systems [69] and in the computation of unstable periodic orbits [34, 70]).

Just as a short look at the practical implementation of the Taylor series method we remark that in the literature there are efficient variable-stepsize variable-order (VSVO) formulations. For example, in [20, 71, 72] the variable-stepsize formulation is based on the error estimator using the last two coefficients and gives the following stepsize prediction

$$\displaystyle{h_{i+1} = \mathtt{fac}\cdot \min \left \{\,\left ( \frac{\mathtt{Tol}} {\|\{ \frac{1} {(n-1)!}\,\mathbf{f}^{(n-2)}(t_{i})\|_{\infty }}\right )^{ \frac{1} {n-1} },\,\left ( \frac{\mathtt{Tol}} {\| \frac{1} {n!}\,\mathbf{f}^{(n-1)}(t_{i})\|_{\infty }}\right )^{ \frac{1} {n} }\,\right \}}$$

where fac is a safety factor and Tol the user error tolerance. A very simple order selection that only depends on the user error tolerance is given [73] by the formula $n(\mathtt{Tol}) = -\frac{1} {2}\,\ln \mathtt{Tol}$. See [20, 71] for a more extensive analysis and comparison with variable-stepsize variable-order formulations of the Taylor method. In Fig. 3.15 we present some comparisons on the Hénon–Heiles problem with initial conditions (x ₀, y ₀, X ₀, Y ₀) = (0, 0. 52, 0. 371956090598519, 0) and E = 0. 157494996 in the time interval [0, 200] using the Taylor method (software TIDES [72]) and the well established codes dop853 and odex developed by Hairer and Wanner [74]. These codes are based on an explicit Runge–Kutta of order 8(5,3) given by Dormand and Prince with stepsize control and dense output and the extrapolation method, respectively. All the methods are compared only in double and quadruple precision using the Lahey LF 95 compiler (fortran) because the dop853 cannot be directly used in multiple precision. The multiple-precision tests are done using C++ and the GMP and MPFR [75] multiple precision packages. From Fig. 3.15 we note that for low precision the dop853 code is a bit faster but when the precision demands are increased the Taylor method is by far the fastest, being for very high precision the only reliable method. Moreover, we can appreciate the different slope of the variable order method (Taylor method) and the fixed order one (dop853), being clear that for high precision the variable order schemes become the more competitive because they are more versatile.

Fig. 3.15

Comparison of the CPU time in seconds vs. relative error in the numerical integration of a KAM orbit of the Hénon–Heiles problem using two well established codes, dop853 (explicit Runge–Kutta method) and odex (extrapolation method), and the Taylor series method (TIDES code) with variable-order and variable-stepsize in double-precision (DP), quadruple-precision (QP) and multiple-precision (MP)

For the computation of the OFLI and OFLI2 we are interested not only in the differential equations but also in the variational equations. In order to avoid their explicit generation we have devised [21] an alternative that permits us to obtain the solution of the variational equations without computing them explicitly. Therefore, we have to obtain a numerical solution of y(t) and $\mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{y}(t)$, being $\mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{y}(t)$ the Lie derivative of the solution y(t) with respect to the vector δ y(t ₀) (that is, in this case the directional derivative). Note that the partial derivatives of the solution with respect to the initial conditions are given by

$$\displaystyle{\varPi =\big (\,\mathcal{L}_{\mathbf{e}_{1}}\mathbf{y}(t)\,\mid \,\mathcal{L}_{\mathbf{e}_{2}}\mathbf{y}(t)\,\mid \,\ldots \,\mid \,\mathcal{L}_{\mathbf{e}_{n}}\mathbf{y}(t)\,\big)}$$

with (e ₁, e ₂, …, e _n ) the canonical base of $\mathbb{R}^{n}$.

The Taylor series method computes the Taylor series of the solution of the differential equation and the Taylor series of the partial derivatives of the solution

$$\displaystyle{\begin{array}{rcl} \delta \mathbf{y}(t_{i})& =& \frac{\partial \mathbf{y}(t_{i})} {\partial \mathbf{y}(t_{0})} \cdot \delta \mathbf{y}(t_{0}) = \mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{y}(t_{i}) \\ & \simeq &\mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{y}(t_{i-1}) + \mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{f}(t_{i-1})\,h_{i} + \frac{1} {2!}\,\mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{f}^{(2)}(t_{ i-1})\,h_{i}^{2} \\ & & +\ldots + \frac{1} {n!}\,\mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{f}^{(n-1)}(t_{ i-1})\,h_{i}^{n}. \end{array} }$$

We now may compute the coefficients $1/(\,j + 1)!\,\mathcal{L}_{\delta \mathbf{y}(t_{0})}\mathbf{f}^{(\,j)}(t_{i-1})$ by rules of automatic differentiation of the elementary functions (±, ×, ∕, $\ln$, $\sin$, …) obtained in [21]. Automatic differentiation gives a recursive procedure to obtain the numerical value of the reiterated derivatives of the elementary functions at a given point. We present here, as example, the rules for the sum, product by a constant, product, division and real power of functions (see [21] for the complete list of rules of any elementary operation):

Proposition

If $f(t,\,\mathbf{y}(t)),g(t,\,\mathbf{y}(t)): (t,\,\mathbf{y}) \in \mathbb{R}^{s+1}\mapsto \mathbb{R}$ are functions of class $\mathcal{C}^{n}$ and given a vector $\mathbf{v} \in \mathbb{R}^{s}$ , we denote

$$\displaystyle{f^{[\,j,\,0]}:= \frac{1} {j!}\,\frac{d^{j}f(t)} {dt^{j}},\qquad f^{[\,j,\,1]}:= \frac{1} {j!}\,\mathcal{L}_{\mathbf{v}}f^{(\,j)},}$$

that is, the jth Taylor coefficient of the function f(t,  y (t)) and of its Lie derivative with respect to v , respectively. Then, we have

• (i) If h(t) = f(t) ± g(t) then h ^[n, i] = f ^[n, i] ± g ^[n, i] .

• (ii) If h(t) = α f(t) with $\alpha \in \mathbb{R}$ then h ^[n, i] = α f ^[n, i] .

• (iii) If h(t) = f(t) ⋅ g(t) then

  $$\displaystyle{\begin{array}{lcl} h^{[n,\,0]} & =&\sum _{j=0}^{n}f^{[n-j,\,0]} \cdot g^{[\,j,\,0]}, \\ h^{[n,\,1]} & =&\sum _{j=0}^{n}\big(\,f^{[n-j,\,0]} \cdot g^{[\,j,\,1]} + f^{[n-j,\,1]} \cdot g^{[\,j,\,0]}\big).\end{array} }$$

• (iv) If $h(t) = f(t)/g(t)$ then

  $$\displaystyle{\begin{array}{rcl} h^{[n,\,0]} & =& \frac{1} {g^{[0,0]}}\,\bigg(\,f^{[n,\,0]} -\sum _{ j=0}^{n-1}h^{[\,j,\,0]} \cdot f^{[n-j,\,0]}\bigg), \\ h^{[n,\,1]} & =& \frac{1} {g^{[0,0]}}\,\bigg\{f^{[n,\,1]} - h^{[n,\,0]} \cdot f^{[0,\,1]} \\ & & -\sum _{j=0}^{n-1}\big(h^{[\,j,\,0]} \cdot f^{[n-j,\,1]} + h^{[\,j,\,1]} \cdot f^{[n-j,\,0]}\big)\bigg\}.\end{array} }$$

• (v) If h(t) = f(t) ^α with $\alpha \in \mathbb{R}$ and f ^[0,0] ≠0, then

  $$\displaystyle{\begin{array}{lcl} h^{[0,\,0]} & =&(\,f^{[0,0]}(t))^{\alpha }, \\ h^{[n,\,0]} & =& \frac{1} {nf^{[0,0]}}\,\sum _{j=0}^{n-1}\big(n\,\alpha - j(\alpha +1)\big)\,h^{[\,j,\,0]} \cdot f^{[n-j,\,0]}, \\ h^{[0,\,1]} & =& \frac{1} {f^{[0,0]}}\,\alpha \,h^{[0,\,0]} \cdot f^{[0,\,1]}, \\ h^{[n,\,1]} & =& \frac{1} {nf^{[0,0]}}\,\bigg\{ - nh^{[n,\,0]} \cdot f^{[0,\,1]} \\ & & +\sum _{ j=0}^{n-1}\big(n\,\alpha - j(\alpha +1)\big)\,\big(h^{[\,j,\,0]} \cdot f^{[n-j,\,1]} + h^{[\,j,\,1]} \cdot f^{[n-j,\,0]}\big)\bigg\}.\end{array} }$$

The use of high-precision numerical integrators in the determination of periodic orbits is justified, for instance, by the search of highly unstable periodic orbits [34].

In Fig. 3.16 we show some comparisons for the Lorenz model (3.9) in double precision, all obtained with the code TIDES using the traditional way to compute the solution of the variational equations (VAR), that is writing them explicitly, and with the use of the extended Taylor series method (ETS) and using TIDES with this capability (using the extended Automatic Differentiation rules of Proposition 3.5). In the pictures we present computational relative error vs. CPU time diagrams in seconds. The extended Taylor series method is the fastest option with a low difference, but the most important thing is that the difference in the formulation is very high. Everyone knows how cumbersome is to write variational equations of order one, two and higher!! The picture is done for computing the complete order two and just the partial derivative ∂ ² x∕∂ x ₀ ². Note that TIDES can compute also sensitivities with respect to parameters of a system, not only with respect to the initial conditions.

Fig. 3.16

Computational relative error in the computation of sensitivities vs. CPU time diagrams in seconds for the Lorenz model in double-precision using TIDES code using the extended Taylor series method for the solution of the variational equations (ETS) or just the standard Taylor series method with explicit formulation of the variational equations (VAR)

To end this section we remark that the use of the Taylor series method is currently helped by the free available new state-of-the-art numerical library TIDES (Taylor Integrator of Differential EquationS) that has just been developed by Profs. Abad, Barrio, Blesa and Rodríguez [72, 76]. The reader can contact the authors to obtain the software.¹

Nowadays it is quite standard to preserve several geometric properties of the differential systems by means of “geometric integrators”. This kind of methods are specially useful when we want to solve a problem with not very high precision but with a “constant” value of the energy, for instance. The problem for very long numerical integrations is that it doesn’t matter how you perform the integration, finally the rounding errors of the computer will affect the integration, giving an increment of the error in the geometric object [77]. The optimal error in these quantities was studied first by Brouwer [78], who established that the error in energy grows at least as $\mathcal{O}(t^{1/2})$. This error is obtained for long integrations of careful used symplectic integrators [77] or when one is able to suppress the truncation error in any numerical integrator (and also with a careful use, of course). In other circumstances we may observe a typical linear growing $\mathcal{O}(t)$. In the case of the positions, we will have a root-mean-squared (RMS) error $\mathcal{O}(t^{3/2})$ in the best case, and a typical error $\mathcal{O}(t^{2})$ (as in any non-symplectic RK code).

Now, we just show how easy is to eliminate the truncation error in the Taylor series method, and so, in TIDES. The advantage is that using the error estimator of the Taylor method, as they are based in just studying some Taylor coefficients, we may use any tolerance level. A completely different situation occurs when our error estimator is based on the substraction of two similar expressions (like in some formulations of embedded Runge–Kutta pairs where the local error estimator is given by the substraction of the solution of two methods of different order) that makes impossible to use them for tolerances lower than the rounding error due to the “catastrophic digit cancelation”. So, if we fix the tolerance far below the roundoff unit of the computer we, in theory, can control the truncation error. This technique has been used previously by the group of Carles Simó [79] and by others [80–82]. We have to combine this technique with a “compensated sum” formulation [83] of the time increment as we use variable stepsize strategies (in contrast with symplectic integrators that have to use fixed stepsize implementations). So, the truncation-free formulation can be described as:

(use TOL ≪ u, with u = the roundoff unit) + (“compensated sum”)

TIDES uses compensated sum in some stages of the method, so, if we want to preserve some geometric properties of the systems we just have to fix a low enough tolerance level. Obviously, this approach is computationally more expensive than other approaches and it is valid only if you also look for high precision numerical results.

In Fig. 3.17 we present the evolution of the error using TIDES with the truncated-free formulation. It is clear that this approach permits to achieve the optimal Brouwer’s law (see Fig. 3.17), like well-programmed symplectic integrators [77], but it can be used in variable-stepsize formulations being therefore a quite flexible approach.

Fig. 3.17

Evolution of the error in the position and in the Energy of one orbit of the Hénon–Heiles system using a tolerance lower than the unit roundoff of the computer (truncation-free formulation). For the position we show the evolution of the error for the x and X variables of the Hamiltonian (3.6)