Abstract
Quantifying the predictability limits of chaotic systems and their forecast models is an important issue with both theoretical and practical significance. This paper introduces three invariant statistical properties of attractors, namely the attractor radius, global attractor radius (GAR), and the global average distance between two attractors, to define the geometric characteristics and average behavior of a chaotic system and its error growth. The GAR is \(\sqrt 2\) times the attractor radius. These invariant quantities are applied to quantitatively measure the global and local predictability limits (both have practical and potential predictability limits, which correspond to the attractor radius and GAR, respectively) of both global ensemble average forecasts and one single initial state, respectively. Both the attractor radius and GAR are intrinsic properties of a chaotic system and independent of the forecast model and model errors, and thus provide more accurate, objective metrics to assess the global and local predictability limits of forecast models compared with the traditional error saturation or asymptotic value (AV). Both the Lorenz63 model and operational forecast data are used to demonstrate the theoretical aspects of these geometric characteristics and evaluate the feasibility and effectiveness of their application to predictability analysis.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Many nonlinear physical systems in the real world (e.g., the atmosphere and the ocean) display complex chaotic behavior that makes prediction of their variations challenging. Because of their high sensitivity to errors in the initial state, there is a predictability problem in these systems, as well as in their forecast models, which are subject to model errors (Lorenz 1963; Wolf et al. 1985; Fraedrich 1987; Li and Ding 2011, 2015; Tang et al. 2013; Duan and Huo 2016; Liu et al. 2016; Lucarini et al. 2016; Mu et al. 2017). For example, the atmosphere, as a nonlinear dynamical system, has a chaotic attractor (Lorenz 1963; Ruelle and Takens 1971; Ott 1981; Lions et al. 1997; Li and Chou 1996, 1997a, b; Li and Wang 2008), and two initially close trajectories can diverge in phase space. Numerical weather prediction (NWP) and climate prediction inevitably tend to be less accurate with simulation time and become useless beyond a certain time limit, which is defined as the predictability limit (Mu et al. 2003, 2010; Bauer et al. 2015; Vannitsem and Lucarini 2016). Since pioneering works (Lorenz 1965; Charney et al. 1966; Smagorinsky 1969) recognized that the predictability limit for the synoptic scale is about 2 weeks, the atmospheric predictability limit has been used as a fundamental assessment indicator in the development of numerical models and forecasting techniques (Mu et al. 2017; Duan and Zhao 2015).
Early studies used the doubling time of the root-mean-square error (RMSE) of forecasts to measure the initial error growth and the predictability (Charney et al. 1966; Smagorinsky 1969; Lorenz 1982). Dalcher and Kalnay (1987) suggested that the doubling time of small errors is not a good measure of error growth, and the saturation or asymptotic value (AV) of RMSE provides a better measure of the predictability limit. In most studies, the predictability limit has been defined as the time when the forecast error exceeds 95% of the AV (Dalcher and Kalnay 1987; Simmons and Hollingsworth 2002; Buizza 2010). A threshold of 71% of the forecast error AV has also been used, and corresponds to climatic variability (Savijärvi 1995; Buizza 2010). Notwithstanding the widespread use of AV, it turns out to be a limited approach to measure the predictability limit. The error growth in deterministic forecasts is largely determined by model deficiencies for medium- and extended-range forecasts (Orrell et al. 2001). This means that the AV of forecast error depends on the numerical model used, and thus is not an objective threshold to quantify and compare the predictability limits between predictions from different models. Particularly, when a forecast model has significant drift-related error compared to the attractor of verifying analyses, the AV of forecast error may be an inaccurate threshold for quantifying the predictability limit and the 71% threshold may not be close to the variability of the actual system. Additionally, the forecast-error growth and predictability of individual cases are flow-dependent (Corazza et al. 2003). Therefore, it may be difficult to identify the error AV to evaluate the local predictability limit (i.e., the predictability limit of individual cases). Another approach, the nonlinear local Lyapunov exponent theory (Ding and Li 2007, 2008; Ding et al. 2008, 2010, 2015; Li and Ding 2011, 2013, 2015), was proposed to quantify the global and local predictability limits of chaotic systems. When the exact equations of a dynamical system are known or observational data covering a long period of time are available, this is an effective method. However, similar to the AV threshold approach mentioned above, this approach may be susceptible to uncertainties in measuring global and local predictability limits using forecast models that have associated model errors.
This study proposes a unified framework for quantifying global and local predictability limits of both chaotic dynamical systems and their forecast models. Three invariant statistical properties of attractors are introduced, namely the attractor radius, global attractor radius (GAR), and the global average distance (GAD) between two attractors, as metrics to quantify global and local predictability limits. Moreover, some basic properties of these geometric characteristics of attractors are investigated.
The remainder of the paper is arranged as follows. Section 2 introduces the definitions of the attractor radius, GAR, and GAD, their relevant theoretical properties, and criteria for quantifying global and local predictability limits. The models and data used in this work are described in Sect. 3. Section 4 presents the results of applying the attractor radius and GAR to quantifying global and local predictability limits of an original dynamical system and its forecast model, along with a comparison with the traditional error AV method. Finally, conclusions are drawn in Sect. 5.
2 Methodology
2.1 Attractor radius, GAR and GAD definitions
Let \({\fancyscript{A}}\) be a compact attractor of the semigroup S, and \(\mathbf{x}\) be a state column vector on \({\fancyscript{A}}\). \({\fancyscript{A}}\) is a closed, bounded and invariant set, i.e., \(S(t){\fancyscript{A}}={\fancyscript{A}}\). Probability theory can be used to study the statistical characteristics of the state of attractor \({\fancyscript{A}}\). Let \({\fancyscript{A}}\) be the sample space, \(\mathcal{F}\) be a set of subsets of \({\fancyscript{A}}\) and a σ-field, the probability measure P:\(\mathcal{F}\)→[0,1]. Then, the triplet (\({\fancyscript{A}}\), \(\mathcal{F}\), P) is a probability space. According to the ergodic theory of attractors (Eckmann and Ruelle 1985), there are invariant (ergodic) measures and Eckmann and Ruelle (1985) and Farmer et al. (1983) introduced some invariant probability measures such as entropy, dimensions and characteristic exponents (also called Lyapunov exponents) to study the properties of dynamical systems. To quantitatively estimate predictability limit of strange attractors, some other invariant probability measures of attractors are introduced. First, let \(F({\mathbf{x}})\) be the probability density function (PDF) of \({\mathbf{x}}\) on \({\fancyscript{A}}\), and \(f\left( {{x^{\left( I \right)}}} \right)\) be the marginal density function in the direction of \({x^{(I)}}\), then for a compact attractor \({\fancyscript{A}}\), its PDF and marginal density function are invariant. In the following the norm \(|| \cdot ||\) is the \({L^2}\)-norm. To reduce space, the derivation process of some formulas and theorems is given in “Appendix”, and main concepts are summarized in Table 1.
Definition 1
Let \({{\mathbf{x}}_i}\) be a specific state on a compact attractor \({\fancyscript{A}}\), then the local attractor radius (LAR, R L ) with respect to the point is defined as follows:
where E represents the expectation, and the corresponding marginal LAR in the direction of \({x^{(I)}}\) is
Particularly, let \({\mathbf{x}_E}\) be the mean state of \({\fancyscript{A}}\), i.e., \({{\mathbf{x}}_E}=E({\mathbf{x}})\), \({\mathbf{x}} \in {\fancyscript{A}}\). The point \({{\mathbf{x}}_E}\) is referred to as the center of the attractor (not necessary on the attractor), and the attractor radius (R E ) and marginal attractor radius (R E (I)) are expressed as follows:
The geometric meaning of LAR is the expectation of the root-mean-square distance between one specific state and all other states on an attractor, which is an extension of the definition of the radius of an n-sphere. In fact, they are equivalent if \({\fancyscript{A}}\) is a periodic attractor or an n-sphere. Physically, the LAR is an indicator of the average behavior of the attractor subset relative to a specific state as the subset expands and fills up the whole attractor. The attractor radius has the same form as the standard deviation (SD) in statistics, which represents the variability of a variable.
Definition 2
The global attractor radius (GAR, R G ) of a compact attractor \({\fancyscript{A}}\) is defined as the average or expected value of the LAR of \({\fancyscript{A}}\), i.e.,
The corresponding marginal GAR in the direction of \({x^{(I)}}\) is
This indicates that the GAR is an expectation of the root-mean-square distance between any two randomly chosen points on an attractor.
Theorem 1
The (marginal) LAR and (marginal) attractor radius of a compact attractor \({\fancyscript{A}}\) satisfy
Moreover, one has
where d iE and d jE denote the root-mean-square distances of x i and x j on \({\fancyscript{A}}\) from the mean state of \({\fancyscript{A}}\), respectively, and \(d_{{iE}}^{{(I)}}\) and \(d_{{jE}}^{{(I)}}\) are the corresponding distances in the direction of \({x^{(I)}}\). Furthermore, in the phase space of \({\fancyscript{A}}\) the points \({\mathbf{x}_i}\) with the same LAR are on the intersection of the attractor set and a sphere of center \({{\mathbf{x}}_E}\) and radius \({d_{iE}}\).
The minimum value of the LAR of an attractor is the attractor radius.
Theorem 2
A constant proportion relationship between the (marginal) GAR and (marginal) attractor radius of a compact attractor \({\fancyscript{A}}\) exists as:
This conclusion is consistent with the error covariance analysis of numerical predictions from Leith (1974) and Kalnay (2002). In addition, Theorem 2 provides a simple, practical approach to calculate the GAR by calculating the attractor radius, which can greatly reduce the computational burden. In order to discuss the predictability of a model that is an approximation of a chaotic system (e.g., atmospheric models, which are approximations of the real atmosphere), some average distances between two attractors must be introduced.
Definition 3
Let \({\fancyscript{A}}\) and \({\fancyscript{B}}\) be two compact attractors, and \({{\mathbf{x}}_i}\) be a point on \({\fancyscript{A}}\). The local average distance (LAD) from \({\fancyscript{A}}\) to \({\fancyscript{B}}\) with respect to \({{\mathbf{x}}_i}\) is:
and the corresponding marginal LAD in the direction \((I)\) is \(R_{L}^{{(I)}}({{\mathbf{x}}_i},{\fancyscript{B}})=\sqrt {E\left( {{{\left\| {x_{i}^{{(I)}} - {y^{(I)}}} \right\|}^2}} \right)}\).
Definition 4
The global average distance (GAD) between two compact attractors \({\fancyscript{A}}\) and \({\fancyscript{B}}\) is the expected value of the LAD between the two attractors, i.e.
Theorem 3
For two compact attractors \({\fancyscript{A}}\) and \({\fancyscript{B}}\), one has
Theorem 3 provides an effective method to calculate the GAD between two attractors, which can save considerable computational time.
Theorem 4
If the GAD between two compact attractors \({\fancyscript{A}}\) and \({\fancyscript{B}}\) equals their GARs, then the two attractors have the same radius and the same center as well. Additionally, with respect to any point on \({\fancyscript{A}}\) or \({\fancyscript{B}}\) the LAD equals the LAR.
In practice, the long-term historical records of a chaotic system and a large amount of forecast data from a model of the system can be used to estimate its attractor radius and GAR, as well as the GAD between the attractors of the system and its forecast model, even if its exact equations are unknown. We have the following corollary.
Corollary 1
The attractor radius and GAR of a compact attractor and the GAD between two compact attractors are invariant quantities.
2.2 Quantifying global and local predictability limits
The attractor radius and GAR are in essence statistics of the average behavior of chaos. For NWP, if the forecast RMSE averaged over a great number of samples exceeds the GAR calculated from the attractor of verifying analyses, the forecasts cannot be more accurate than a randomly chosen state from the attractor on average, and thus become useless. In other words, the GAR as an invariant property of an attractor provides an objective metric that can be used to determine whether forecasts are still skillful, i.e., predictability. For an n-dimensional nonlinear dynamical system,
where \(\mathbf{x}\) and \(\mathbf{F}\) are n-dimensional column vectors, the system has a compact attractor \({\fancyscript{A}}\). Considering the evolution of the state \(\mathbf{x}\) on \({\fancyscript{A}}\), let \(\mathbf{x}(t)\) be a fiducial orbit with the initial value \({{\mathbf{x}}_0}={\mathbf{x}}(0)\), \({\mathbf{\hat {x}}}(t)\) be its perturbed orbit with the initial value \({{\mathbf{\hat {x}}}_0}={\mathbf{\hat {x}}}(0)\), and \({\delta _0}=\delta (0)\) be an initial perturbation vector. Then,
The RMSE between the two orbits is defined as
For global predictability, the global ensemble average of error growth must be investigated, and is defined as
where \({\left\langle {{e^{\text{2}}}({{\mathbf{x}}_0},{\delta _{\text{0}}},t)} \right\rangle _N}=\int_{{\fancyscript{A}}} {{e^{\text{2}}}\left( {{\mathbf{x}}({t_0}),\delta ({t_{\text{0}}}),t} \right)d} {\mathbf{x}}\), \(<{\text{ }}{>_N}\) denotes the ensemble average of samples of sufficiently large number N (N → ∞). For the local predictability of one initial state, we define a local ensemble average of error growth,
where \({\Omega _0}\) is a sphere of center \({{\mathbf{x}}_0}\) and radius \({\delta _0}=\left\| {{\delta _0}} \right\|\), \({\left\langle {{e^{\text{2}}}({{\mathbf{x}}_0},{\delta _{\text{0}}},t)} \right\rangle _{N \to \infty }}=\int_{{\Omega (t)}} {{e^{\text{2}}}({\mathbf{x}}({t_0}),\delta ({t_{\text{0}}}),t)d} {\mathbf{x}}\), \(\Omega (t)=S(t){\Omega _0}\).
Ding and Li (2007) and Li and Ding (2011, 2015) indicated that for chaotic systems, the ensemble mean relative growth of initial error has a property of saturation—i.e., it tends to a saturation value. Similarly, the global ensemble RMSE tends to the GAR (which equals the AV) with time. The saturation value of error growth (i.e., the GAR) represents the average distance between two randomly chosen points on an attractor (Li and Ding 2015). Almost all of the initial information is lost when the global ensemble RMSE reaches the GAR, and further prediction therefore becomes meaningless. Note that the attractor radius in Li and Ding (2015) is actually the GAR described here. Because \(71\% \approx {1 \mathord{\left/ {\vphantom {1 {\sqrt 2 }}} \right. \kern-0pt} {\sqrt 2 }}\), 71% of the forecast error AV used as the threshold in Savijärvi (1995) and Buizza (2010) is equivalent to using the attractor radius \({R_E}\) as the threshold. There are two types of predictability limit corresponding to the attractor radius and GAR, the practical predictability limit and the potential predictability limit (also known as the maximum predictability limit). Figure 1a illustrates typical mean error growth in a nonlinear chaotic system as a function of time, along with the practical predictability limit and the potential predictability limit. As shown, the global practical predictability limit and the global potential predictability limit are determined as the times when the global ensemble average RMSE reaches the attractor radius and GAR, respectively, i.e.,
Where \({T_{pr}}\) and \({T_{po}}\) are the global practical and potential predictability limits, respectively. The error growth will enter a strong nonlinear phase with a steadily decreasing growth rate when the magnitude of the mean error exceeds the attractor radius (Fig. 1a). For a “perfect” model without model errors, we may choose 95% of the GAR as the criterion to measure \({T_{po}}\), following previous studies (Dalcher and Kalnay 1987; Simmons and Hollingsworth 2002; Buizza 2010).
Different initial states have different predictability. The local practical and potential predictability limits of any initial state can be measured by the times when the local ensemble average RMSE reaches the attractor radius and GAR for the first time, respectively—i.e.,
Next, the predictability of model forecasts is discussed. Typically, model errors are present. For example, an approximate model of the dynamical system of Eq. (14) can be written as follows:
where \(\tilde {\mathbf{F}}\) is an approximation of \(\mathbf{F}\) in Eq. (14). Let \({\fancyscript{A}}\) and \({{\fancyscript{A}}_M}\) be the attractor of the chaotic system of Eq. (14) and the one of its approximate model, Eq. (23), respectively. Typically, they are different and thus their predictabilities are also different. Considering the two orbits \(\mathbf{x}(t)\) on \({\fancyscript{A}}\) and \({\mathbf{y}}(t)\) on \({{\fancyscript{A}}_M}\). Here, \(\mathbf{x}(t)\) is a fiducial orbit and the initial value \({{\mathbf{y}}_0}={{\mathbf{x}}_0}+{\delta _0}\), where \({\delta _0}\) is a small perturbation. The error of the model forecast is
The global ensemble average of error growth, \({\bar {e}_M}\), will tend to a saturation value with time, which is the GAD between the two attractors \({\fancyscript{A}}\) and \({{\fancyscript{A}}_M}\). However, the GAD cannot directly be used as a threshold to determine the predictability limit, because the forecast model has associated model errors. Following Theorem 3, let \({R_e}=\hbox{min} \left( {R_{E}^{{}}\left( {{{\fancyscript{A}}_M}} \right),\;R_{E}^{{}}\left({\fancyscript{A}} \right)} \right)\), and \({R_g}=\hbox{min} \left( {R_{G}^{{}}\left( {{{\fancyscript{A}}_M}} \right),\;R_{G}^{{}}\left({\fancyscript{A}} \right)} \right)\), then the global and local predictability limits of the model are determined by
where \(T_{{pr}}^{M}\) and \(T_{{po}}^{M}\) are the global practical and potential predictability limits of the model (Fig. 1b), respectively, and \(T_{{pr,{{\mathbf{x}}_0}}}^{M}\) and \(T_{{po,{{\mathbf{x}}_0}}}^{M}\) are the local practical and potential predictability limits of the model. All of the concepts of predictability limits mentioned above are summarized in Table 1.
The GAR and the traditional error AV are similar in that both are used to determine the predictability limit by identifying the time during which forecasts are chaotic in contrast to analyses. However, because a forecast model usually has associated model errors and the model attractor \({{\fancyscript{A}}_M}\) is different from the real attractor \({\fancyscript{A}}\), the saturation level of the global ensemble average RMSE \({\bar {e}_M}({\delta _{\text{0}}},t)\), which corresponds to the traditional AV and the GAD between the two attractors \({\fancyscript{A}}\) and \({{\fancyscript{A}}_M}\), is not the same as that of \(\bar {e}({\delta _{\text{0}}},t)\) (i.e. the GAR). Typically, because of model errors, the AV overestimates the error saturation value, resulting in an overestimation of the predictability limit. Furthermore, the difference between the GAR of the real attractor and the GAD is a better indicator of the degree of model drift or systematic error. Additionally, in light of Theorem 3 the 71% of AV threshold cannot serve as an accurate approximation of the variability of the real system and has therefore lost both physical and geometric meaning. In contrast, the GAR and attractor radius only depend on the real attractor and are not affected by model drift errors. Thus, the GAR and attractor radius are more suitable metrics to use to determine the predictability limit of a forecast model (Fig. 1b).
3 Model and data
The model used in this study is the simple Lorenz63 model (Lorenz 1963):
where \(\sigma =10\), \(r=28\), \(b={8 \mathord{\left/ {\vphantom {8 3}} \right. \kern-0pt} 3}\), and for which the well-known ‘butterfly’ attractor exists. To discuss the predictability of model forecasts, an approximation of the Lorenz63 model with associated model errors is chosen, in which the parameter \(b={{10} \mathord{\left/ {\vphantom {{10} 3}} \right. \kern-0pt} 3}\).This approximate model is referred to as the imperfect Lorenz63 model. A fourth-order Runge–Kutta time integration scheme is selected with a time step \(h=0.01\)time units (tu) and double precision is used to calculate numerical solutions.
The Lorenz63 model is continuously integrated for 1 × 106 tu with a 0.01 tu output interval and used to represent the ‘true’ system after eliminating the first 104 tu trajectory as spin-up. Results are used to calculate the attractor radius, GAR, and LAR, and are used as the reference for verifying forecasts. In addition to a large data set simulated from this simple model, the 45-day retrospective forecasts of the coupled Climate Forecast System (CFSv2) from the National Centers for Environmental Prediction (NCEP) from Jan 1999 to Dec 2010 every 6 h are used to evaluate the utility of the attractor characteristic parameters presented in this work. The 00Z initial conditions, which are approximately equivalent to the observed atmospheric state, are used to verify the forecasts. Note that the annual cycle of the series of 00Z initial conditions has been removed before the observations are applied to the computation of the GAR and LAR of the atmosphere. The predictability experiments are summarized in Table 2.
4 Attractor radius, GAR and LAR in the Lorenz63 model
4.1 GAR and attractor radius
Figure 2 shows the variations of the marginal GARs for the variables x, y, z and the GAR for the vector (x, y, z) in the Lorenz63 model as a function of time-series length. The figure shows that for any variable or vector, the marginal GAR or GAR initially varies slightly, and then gradually approaches a constant value with time. The fact that the evolution trajectory tends to spread over the whole attractor implies that the GAR is an invariant quantity. The marginal GARs for the variables x, y, z and the GAR for the vector (x, y, z) are 11.21, 12.74, 12.19, and 20.90, and the marginal attractor radii and the attractor radius of the system are 7.92, 9.01, 8.62 and 14.78, respectively. A proportional relationship of \(\sqrt 2\) between the attractor radius and the GAR exists, as proved in Sect. 2.
4.2 LAR
As described in Sect. 2, the LAR varies with the specific location of a point in phase space. Figure 3 illustrates the distributions (solid line) of the marginal LAR and the PDF for the variables x, y, and z in the Lorenz63 model with respect to the reference state. The PDF patterns of the variables x, y, and z noticeably differ, but in general the probability near the mean is larger than at the extremities. In contrast, the LAR patterns are similar among the variables, reaching minima at the mean state (dashed line) and continuously increasing toward the edge of the attractor. This is consistent with Theorem 2, as described in Sect. 2, and follows from the fact that predictions of extreme events generally have larger forecast errors compared with events closer to the mean, a trend that is independent of the probability that specific events occur, and is a natural property of an attractor. The application of the LAR can be extended to higher-dimensional systems. Figure 4 shows the distribution of the LAR on the x–y plane in phase space for the Lorenz63 model. The figure shows that points with the same LAR are on the intersection of the attractor set and a sphere centered at the mean state with a radius equal to the distance between the points and the mean state, in agreement with Theorem 1, as described in Sect. 2.
For the real atmosphere, the LAR can be estimated from observational data collected over a long period of time. Figure 5 shows the distribution of LAR as a function of states on the attractor at 500 hPa geopotential height (GHT; with the annual cycle removed) at five selected grid points. The results are very similar to those in Fig. 3. The anomalies of GHT approximately follow a Gaussian distribution at individual grid point. The states farther away from the mean state have relatively larger LARs compared with the ones close to the mean state, which may help explain the more remarkable forecast errors for extreme events from atmospheric models.
5 Global predictability limits
5.1 A ‘perfect’ model scenario
Forecast RMSE averaged over large samples will gradually increase with time, and approach a certain saturation value because of the presence of nonlinear chaos. Traditional approaches generally use a threshold value of the error AV to define the predictability limit. In this section, the AV and GAR are compared in a ‘perfect’ model scenario. Figure 6 shows the evolution of the global ensemble mean RMSE of the variables x, y, and z and vector (x, y, z) with the ‘perfect’ Lorenz63 forecast model (the same model described in Sect. 3 for the ‘true’ system). A set of initial errors with magnitudes 10−2, 10−3, 10−4, 10−5, 10−6, 10−7, are generated with random numbers and superimposed on the ‘true’ states to produce the initial states (5 × 107 in total). Forecasts are then carried out for 50 tu. For such simple system, forecasts with smaller initial errors have higher accuracy and longer predictability limits, which is consistent with the previous study (Ding and Li 2007, 2008). However, for the multi-scale system (Lorenz 1969) and coupled system (Ding and Li 2012) this relationship between the predictability limits and initial errors no longer applies. For the two classes of systems we will further investigate this relationship using the approach presented here in the future. All forecast errors for a particular point or vector tend to the same saturation value. The saturation values corresponding to the variables x, y, and z and vector (x, y, z) are 11.21, 12.74, 12.19, and 20.90, respectively, and are consistent with the GARs shown in Fig. 2. This indicates that the GAR and the AV are equivalent in the perfect model scenario, and that both are able to distinguish the different predictability limits that result from variations in the initial error size in these experiments. Figure 7 shows the global practical and potential predictability limits (\({T_{pr}}\) and \({T_{po}}\), respectively) of the Lorenz 63 model as a function of initial error. Negative linear relationships of \({T_{pr}}\) and \({T_{po}}\) with the magnitude of the initial error are observed—i.e., both the global practical and potential predictability limits increase as the magnitude of the initial error increases. This result is in agreement with Ding and Li (2007) who used the nonlinear local Lyapunov exponent.
5.2 An imperfect model scenario
Deficiencies in numerical models are an important source of forecast errors. A simple case, the Lorenz63 model, is used as the real system with an imperfect forecast model, as described in Sect. 3. The difference between the two models results from a small deviation in the parameter b in the imperfect model (see Sect. 3). Figure 8 shows the evolution of the global ensemble mean RMSE of the variables x, y, and z and vector (x, y, z) of the imperfect Lorenz63 forecast model. The error growth of the imperfect Lorenz63 forecast model is similar to that of the Lorenz63 model (Fig. 6), but because of the system bias, the GAR of the former is significantly larger than that of the latter. Thus, the error growth of the imperfect model alone cannot be used to determine its predictability of the perfect system, and the global ensemble mean RMSE between the Lorenz63 model and the imperfect Lorenz63 model must be used. As shown in Fig. 9, the global ensemble mean RMSE between the Lorenz63 model and the imperfect Lorenz63 model converges to the GAD between the two models. Moreover, error growth curves with initial errors of different magnitude coincide—i.e., they are independent of the magnitude of the initial errors. This implies that, in this case, model errors, and not small initial errors, play the dominant role in forecast error growth. The global practical and potential predictability limits of the imperfect Lorenz63 model are 0.8 and 1.9 for the vector (x, y, z), 0.8 and 1.9 for x, 0.8 and 1.8 for y, and 0.8 and 2.6 for z, respectively. These predictability limits are much shorter than those of the original Lorenz63 model (Fig. 7). This reflects the predictability deficiencies of the imperfect model when used to forecast the evolution of the ‘perfect’ Lorenz system. However, a practical problem is that the existing models are inevitably of model errors. Applying a forecast model to investigate predictability is also an important tool and approach. It is very important to understand how to use a forecast model to estimate the attractor radius and global attractor radius and the predictability limits. To this end, we need to learn how large is the GAD between the real attractor (it could be estimated by using observational or long analysis data series if the exact equations of the system are unknown) and the model attractor, and take it into account. Usually we may choose a forecast model with smaller GAD between its attractor and the real attractor to study predictability and estimate the predictability limits.
5.3 Global predictability limits of the CFSv2
Figure 10 illustrates the mean RMSE of all the CFSv2 operational forecast cases (from Jan 1999 to Dec 2010) of GHT at 500 hPa averaged globally (90°S–90°N) and over the extratropical Northern Hemisphere (NH 20°–90°N), the extratropical Southern Hemisphere (SH 90°–20°S) and the tropics (TRO 20°S–20°N). Because the CFSv2 only makes 45-day retrospective forecasts for each set of initial conditions, the average of the last 25 forecasts is chosen as an estimation of the AV. It can be seen in Fig. 10 that there are smaller differences between the AV and GAR for the global, extratropical NH and extratropical SH, than for the tropics. This indicates that the forecast model has significant model drift (or systematic error) in the tropics. Using the GAR, the global practical (potential) predictability limits for the global, extratropical NH, extratropical SH, and tropics are 7.7 (17.1), 8.0 (18.8), 7.4 (16.0), and 5.6 (8.8) days, respectively. The CFSv2 has much lower predictability for the tropics compared with the extratropical regions, consistent with Li and Ding (2015). The mean RMSE reaches the AV for the global, extratropical NH, extratropical SH, and tropics at 29.4, 29.3, 39.1, and 29.4 days, respectively, times which are significantly longer than the predictability limits found using the GAR, which results from that fact that the AV ignores the effects of model-drift-related errors on degrading forecast accuracy, and thus overestimates the predictability limit.
6 Local predictability limits
This section considers one case for the Lorenz63 model with initial state (− 4.0, 5.0, 20.0) to investigate the quantification of the local predictability limit for a specific state. Projections of the LAR on the x–y plane in the phase space of the Lorenz63 model are presented in Fig. 11. Although the LAR changes with time, the LAR itself is an attractor. As the figure shows, the projections of the LAR on the x–y plane have a pattern similar to a single butterfly wing. Figure 12 shows the evolution of the LAR and the local ensemble average RMSE for the initial state specified above with two initial perturbation magnitudes, \({10^{ - 2}}\) and \({10^{ - 7}}\). In this case study, 1 × 106 initial perturbations of the initial state are generated with the same perturbation size but different directions. These initial errors are added to the specified initial state to produce a set of individual initial ‘analyses’ and the forecasts are run for 50 tu. In fact, this many initial perturbations are not needed, and hundreds of initial perturbations (or less) are sufficient. Results indicate that unlike the global ensemble mean forecast error (for which saturation and convergence to the GAR are evident; see Fig. 6), the RMSE in the case of a specific initial state varies with time because of a time-dependent basic flow or ‘flow-dependence’ (Corazza et al. 2003). The RMSE are initially smaller than the attractor radius, which indicates better forecasts. Then, the errors gradually increase and exceed both the attractor radius and the GAR. The practical and potential predictability limits of this initial condition are 10.1 (23.6) and 11.7 (23.6) for initial perturbation size 10−2 (10−7). Beyond the potential predictability limit, forecast errors begin to undergo fluctuations similar to those of the LAR, and finally converge on the trajectory of the LAR, at which point forecasts become meaningless.
7 Conclusions and discussion
This paper introduces several geometric properties of attractors, the LAR, attractor radius, and GAR of an attractor, as well as the LAD and GAD between two attractors, to characterize features and the average behavior of chaotic systems and evaluate error growth in nonlinear dynamical systems. The attractor radius, GAR, and GAD are invariant quantities, which are applied to quantitatively determine the global and local predictability limits of both global ensemble average forecasts and a single state in phase space, respectively. Both global and local predictability limits have practical and potential values, which correspond to the times at which the ensemble RMSE reaches the attractor radius and GAR, respectively. Both the Lorenz63 model and operational forecast data demonstrate the theoretical aspects of these geometric characteristics, and the feasibility and effectiveness of their application to predictability limit analysis.
The GAR is \(\sqrt 2\) times the attractor radius. Therefore, when a nonlinear dynamical model has no associated model error, the global potential and practical predictability limits quantified by the GAR and attractor radius are equivalent to the thresholds of the traditional AV and 71% of AV, respectively. The theory presented here provides a general, unified framework for the two traditional criteria used to measure predictability limits. However, for a forecast model with associated model errors, the two traditional thresholds do not work, because the attractor of the forecast model is different from that of the original dynamical system, and the AV thresholds overestimate forecast skill and the predictability limit. Additionally, for the local predictability limit, because of the ‘flow-dependence’ of individual cases, the local ensemble average RMSE varies with time and does not reach saturation or an AV. Thus, the traditional AV threshold is not applicable. Both the attractor radius and GAR of a nonlinear system are intrinsic properties of the system and independent of the forecast model used, and thus provide more accurate and objective thresholds to assess the global and local predictability limits of forecast models compared with the traditional AV threshold. In addition, because of the invariant properties of the GAR and attractor radius, the theoretical work presented here can be used to reduce computational burden and storage and avoid repetitive computation, facilitating faster, easier measurements of predictability limits.
In practice, the predictability limits on different spatial–temporal scales can be estimated by the attractor radius and GAR by using observational data; and then, we may analyze the spatial–temporal structures of predictability limits and their variations. A noticeable question is how much the observational errors influence on the estimation of predictability limits. During the recent decades, the performance of NWP models is significantly improved, and the observational network become much more accurate and extensively covered, resulting in largely reduced analyses errors from the truth and reducing the influence of the observational errors on the estimation of predictability limits. However, this is worthy of future work. Besides, the Lorenze63 model has converging domains, which occur within a torus in phase space and also satisfy the fluctuation theorem (Smith et al. 1999; Schalge et al. 2012). The retrospective forecasts of the CFSv2 are used to evaluate the utility of the attractor invariant quantities presented in this paper, however, we still need to employ some high-dimensional complex dynamical systems or coupled systems to further validate the predictability theory and methods in the future so as to better apply them to practice.
References
Bauer P, Thorpe A, Brunet G (2015) The quiet revolution of numerical weather prediction. Nature 525:47–55. https://doi.org/10.1038/nature14956
Buizza R (2010) Horizontal resolution impact on short- and long-range forecast error. Q J R Meteorol Soc 136:1020–1035
Charney JG, Fleagle RG, Riehl H, Lally VE, Wark DQ (1966) The feasibility of a global observation and analysis experiment. Bull Am Meteorol Soc 47:200–220
Corazza M et al (2003) Use of the breeding technique to estimate the structure of the analysis “errors of the day”. Nonlinear Process Geophys 10:233–243
Dalcher A, Kalnay E (1987) Error growth and predictability in operational ECMWF forecasts. Tellus 39A:474–491
Ding RQ, Li JP (2007) Nonlinear finite-time Lyapunov exponent and predictability. Phys Lett A 364:396–400
Ding RQ, Li JP (2008) Study on the regularity of predictability limit of chaotic systems with different initial errors. Acta Phys Sin 57(12):7494–7499
Ding RQ, Li JP (2012) Relationships between the limit of predictability and initial error in the uncoupled and coupled Lorenz models. Adv Atmos Sci 29:1078–1088
Ding RQ, Li JP, Ha K (2008) Trends and interdecadal changes of weather predictability during 1950s–1990s. J Geophys Res 113:D24112. https://doi.org/10.1029/2008JD010404
Ding RQ, Li JP, Seo K (2010) Predictability of the Madden–Julian oscillation estimated using observational data. Mon Weather Rev 138:1004–1013. https://doi.org/10.1175/2009MWR3082.1
Ding RQ, Li JP, Zheng F, Feng J, Liu DQ (2015) Estimating the limit of decadal-scale climate predictability using observational data. Clim Dyn 46(5):1563–1580. https://doi.org/10.1007/s00382-015-2662-6
Duan WS, Huo ZH (2016) An approach to generating mutually independent initial perturbations for ensemble forecasts: orthogonal conditional nonlinear optimal perturbations. J Atmos Sci 73:997–1014. https://doi.org/10.1175/JAS-D-15-0138.1
Duan WS, Zhao P (2015) Revealing the most disturbing tendency error of Zebiak–Cane model associated with El Niño predictions by nonlinear forcing singular vector approach. Clim Dyn 44:2351–2367. https://doi.org/10.1007/s00382-014-2369-0
Eckmann JP, Ruelle D (1985) Ergodic theory of chaos and strange attractors. Rev Mod Phys 57:617–656
Farmer JD, Ott E, Yorke JA (1983) The dimension of chaotic attractors. Phys D 7:153–180
Fraedrich K (1987) Estimating weather and climate predictability on attractors. J Atmos Sci 44:722–728
Kalnay E (2002) Atmospheric modeling, data assimilation and predictability. Cambridge University Press, Cambridge, New York, p 230
Leith CE (1974) Theoretical skill of Monte Carlo forecasts. Mon Weather Rev 102(6):409–418
Li JP, Chou JF (1996) The property of solutions of large-scale atmosphere with the non-stationary external forcing. China Sci Bull 41:587–590
Li JP, Chou J (1997a) The existence of the atmosphere attractor. Sci China Ser D 40:215–224
Li JP, Chou J (1997b) Further study on the properties of operators of atmospheric equations and the existence of attractor. Acta Meteorol Sin 11:216–223
Li JP, Ding RQ (2011) Temporal-spatial distribution of atmospheric predictability limit by local dynamical analogues. Mon Weather Rev 139:3265–3283
Li JP, Ding RQ (2013) Temporal–spatial distribution of the predictability limit of monthly sea surface temperature in the global oceans. Int J Climatol 33:1936–1947. https://doi.org/10.1002/joc.3562
Li JP, Ding RQ (2015) Seasonal and interannual weather prediction. In: North G, Pyle J, Zhang F (eds) Encyclopedia of atmospheric sciences, vol 6, 2nd edn. Academic Press and Elsevier, Amsterdam, Boston, pp 303–312
Li JP, Wang S (2008) Some mathematical and numerical issues in geophysical fluid dynamics and climate dynamics. Commun Comput Phys 3(4):759–793
Lions JL, Manley OP, Temam R, Wang S (1997) Physical interpretation of the attractor dimension for the primitive equations of atmospheric circulation. J Atmos Sci 54:1137–1143
Liu H, Tang Y, Chen D, Lian T (2016) Predictability of the Indian Ocean Dipole in the coupled models. Clim Dyn 48:2005–2024. https://doi.org/10.1007/s00382-016-3187-3
Lorenz EN (1963) Deterministic nonperiodic flow. J Atmos Sci 20:130–141
Lorenz EN (1965) A study of the predictability of a 28-variable atmospheric model. Tellus 17:321–333
Lorenz EN (1969) The predictability of a flow which possesses many scales of motion. Tellus 21:289–307
Lorenz EN (1982) Atmospheric predictability experiments with a large numerical model. Tellus 34:505–513
Lucarini V, Lunkeit F, Ragone F (2016) Predicting climate change using response theory: Global averages and spatial patterns. J Stat Phys. https://doi.org/10.1007/s10955-016-1506-z
Mu M, Duan WS, Wang B (2003) Conditional nonlinear optimal perturbation and its applications. Nonlinear Process Geophys 10:493–501
Mu M, Duan WS, Wang Q, Zhang R (2010) An extension of conditional nonlinear optimal perturbation and its applications. Nonlinear Process Geophys 17:211–220
Mu M, Duan W, Tang Y (2017) The predictability of atmospheric and oceanic motions: further understanding, prospects and exploration. Sci China Earth Sci 60:2001–2012. https://doi.org/10.1007/s11430-016-9101-x
Orrell D, Smith L, Barkmeijer J, Palmer TN (2001) Model error in weather forecasting. Nonlinear Process Geophys 8:357–371
Ott E (1981) Strange attractors and chaotic motions of dynamical systems. Rev Mod Phys 53:655–671
Ruelle D, Takens F (1971) One the nature of turbulence. Commun Math Phys 20:167–192
Savijärvi H (1995) Error growth in a large numerical forecast system. Mon Weather Rev 123:212–221
Schalge B, Blender R, Wouters J, Fraedrich K, Lunkeit F (2012) Towards a fluctuation theorem in an atmospheric circulation model. Physics 11:1945–1953
Simmons AJ, Hollingsworth A (2002) Some aspects of the improvement in skill of numerical weather prediction. Q J R Meteorol Soc 128:647–677
Smagorinsky J (1969) Problems and promises of deterministic extended range forecasting. Bull Am Meteorol Soc 50:286–311
Smith LA, Ziehmann C, Fraedrich K (1999) Uncertainty dynamics and predictability in chaotic systems. Q J R Meteorol Soc 125:2855–2886
Tang Y, Chen D, Yang D, Lian T (2013) Methods of estimating uncertainty of climate prediction and climate change projection. In: Singh BR (ed) Climate change—realities, impacts over ice cap, sea level and risks. InTech. https://doi.org/10.5772/54810
Vannitsem S, Lucarini V (2016) Statistical and dynamical properties of covariant Lyapunov vectors in a coupled atmosphere–ocean model—multiscale effects, geometric degeneracy, and error dynamics. J Phys A 49:224001
Wolf A, Swift JB, Swinney HL, Vastano JA (1985) Determining Lyapunov exponents from a time series. Phys D 16:285–317
Acknowledgements
The authors acknowledge funding from the National Natural Science Foundation of China (NSFC) (Grant Nos. 41375110 and 41522502).
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
This Appendix shows some derivation processes of some formulas and theorems in the text. First, it follows that
The proof of Theorem 1. For a compact attractor \({\fancyscript{A}}\) and \(\mathbf{x} \in {\fancyscript{A}}\), and \({\mathbf{x}_i}\) is a specific point on \({\fancyscript{A}}\), since
where \({d_{iE}}=d({\mathbf{x}_i},{\mathbf{x}_E})=\sqrt {{{\left\| {{\mathbf{x}_i} - {\mathbf{x}_E}} \right\|}^2}}\), and \({\mathbf{x}_E}=E(\mathbf{x})\). Thus follows Theorem 1.
The proof of Theorem 3. For \({\mathbf{x}_i}\), \({\mathbf{x}} \in {\fancyscript{A}}\), \(\mathbf{y} \in {\fancyscript{B}}\) here \({\fancyscript{A}}\) and \({\fancyscript{B}}\) are two compact attractors, we have
where \({d^2}({\mathbf{x}_i},{\mathbf{y}_E})={\left\| {{\mathbf{x}_i} - E(\mathbf{y})} \right\|^2}\), \({\mathbf{x}_E}=E(\mathbf{x})\), and \({\mathbf{y}_E}=E(\mathbf{y})\). Then one has
Thus follows Theorem 3.
The proof of Theorem 4. For two compact attractors \({\fancyscript{A}}\) and \({\fancyscript{B}}\), if
thanks to Theorems 2 and 3, we have
Thus, \({d^2}({\mathbf{x}_E},{\mathbf{y}_E})=0\), \(R_{E}^{2}(~{\fancyscript{A}})=R_{E}^{2}({\fancyscript{B}})\) and \({\mathbf{x}_E}={\mathbf{y}_E}\). Furthermore,
Thus follows Theorem 4.
The marginal global practical predictability limit \(T_{{pr}}^{{(I)}}\)and the marginal global potential predictability limit \(T_{{po}}^{{(I)}}\) are determined as the times when the marginal global ensemble average RMSE reaches the marginal attractor radius and marginal GAR, respectively, i.e.,
The marginal local practical predictability limit \(T_{{pr,{{\mathbf{x}}_0}}}^{{(I)}}\)and the marginal local potential predictability limit \(T_{{po,{{\mathbf{x}}_0}}}^{{(I)}}\) are determined as the times when the marginal global ensemble average RMSE reaches the marginal attractor radius and marginal GAR, respectively, i.e.,
The marginal global practical predictability limit \(T_{{pr}}^{{M(I)}}\)and marginal global potential predictability limit \(T_{{po}}^{{M(I)}}\) of a forecast model are defined as
where \(R_{e}^{{(I)}}=\hbox{min} \left( {R_{E}^{{(I)}}({{\fancyscript{A}}_M}),\;R_{E}^{{(I)}}({\fancyscript{A}})} \right)\), \(R_{g}^{{(I)}}=\hbox{min} \left( {R_{G}^{{(I)}}({{\fancyscript{A}}_M}),\;R_{G}^{{(I)}}({\fancyscript{A}})} \right)\), and the marginal local practical limit \(T_{{pr,{{\mathbf{x}}_0}}}^{{M(I)}}\) and marginal local potential predictability limit \(T_{{po,{{\mathbf{x}}_0}}}^{{M(I)}}\) of a forecast model are defined as
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Li, J., Feng, J. & Ding, R. Attractor radius and global attractor radius and their application to the quantification of predictability limits. Clim Dyn 51, 2359–2374 (2018). https://doi.org/10.1007/s00382-017-4017-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-017-4017-y