1 Introduction

Many nonlinear physical systems in the real world (e.g., the atmosphere and the ocean) display complex chaotic behavior that makes prediction of their variations challenging. Because of their high sensitivity to errors in the initial state, there is a predictability problem in these systems, as well as in their forecast models, which are subject to model errors (Lorenz 1963; Wolf et al. 1985; Fraedrich 1987; Li and Ding 2011, 2015; Tang et al. 2013; Duan and Huo 2016; Liu et al. 2016; Lucarini et al. 2016; Mu et al. 2017). For example, the atmosphere, as a nonlinear dynamical system, has a chaotic attractor (Lorenz 1963; Ruelle and Takens 1971; Ott 1981; Lions et al. 1997; Li and Chou 1996, 1997a, b; Li and Wang 2008), and two initially close trajectories can diverge in phase space. Numerical weather prediction (NWP) and climate prediction inevitably tend to be less accurate with simulation time and become useless beyond a certain time limit, which is defined as the predictability limit (Mu et al. 2003, 2010; Bauer et al. 2015; Vannitsem and Lucarini 2016). Since pioneering works (Lorenz 1965; Charney et al. 1966; Smagorinsky 1969) recognized that the predictability limit for the synoptic scale is about 2 weeks, the atmospheric predictability limit has been used as a fundamental assessment indicator in the development of numerical models and forecasting techniques (Mu et al. 2017; Duan and Zhao 2015).

Early studies used the doubling time of the root-mean-square error (RMSE) of forecasts to measure the initial error growth and the predictability (Charney et al. 1966; Smagorinsky 1969; Lorenz 1982). Dalcher and Kalnay (1987) suggested that the doubling time of small errors is not a good measure of error growth, and the saturation or asymptotic value (AV) of RMSE provides a better measure of the predictability limit. In most studies, the predictability limit has been defined as the time when the forecast error exceeds 95% of the AV (Dalcher and Kalnay 1987; Simmons and Hollingsworth 2002; Buizza 2010). A threshold of 71% of the forecast error AV has also been used, and corresponds to climatic variability (Savijärvi 1995; Buizza 2010). Notwithstanding the widespread use of AV, it turns out to be a limited approach to measure the predictability limit. The error growth in deterministic forecasts is largely determined by model deficiencies for medium- and extended-range forecasts (Orrell et al. 2001). This means that the AV of forecast error depends on the numerical model used, and thus is not an objective threshold to quantify and compare the predictability limits between predictions from different models. Particularly, when a forecast model has significant drift-related error compared to the attractor of verifying analyses, the AV of forecast error may be an inaccurate threshold for quantifying the predictability limit and the 71% threshold may not be close to the variability of the actual system. Additionally, the forecast-error growth and predictability of individual cases are flow-dependent (Corazza et al. 2003). Therefore, it may be difficult to identify the error AV to evaluate the local predictability limit (i.e., the predictability limit of individual cases). Another approach, the nonlinear local Lyapunov exponent theory (Ding and Li 2007, 2008; Ding et al. 2008, 2010, 2015; Li and Ding 2011, 2013, 2015), was proposed to quantify the global and local predictability limits of chaotic systems. When the exact equations of a dynamical system are known or observational data covering a long period of time are available, this is an effective method. However, similar to the AV threshold approach mentioned above, this approach may be susceptible to uncertainties in measuring global and local predictability limits using forecast models that have associated model errors.

This study proposes a unified framework for quantifying global and local predictability limits of both chaotic dynamical systems and their forecast models. Three invariant statistical properties of attractors are introduced, namely the attractor radius, global attractor radius (GAR), and the global average distance (GAD) between two attractors, as metrics to quantify global and local predictability limits. Moreover, some basic properties of these geometric characteristics of attractors are investigated.

The remainder of the paper is arranged as follows. Section 2 introduces the definitions of the attractor radius, GAR, and GAD, their relevant theoretical properties, and criteria for quantifying global and local predictability limits. The models and data used in this work are described in Sect. 3. Section 4 presents the results of applying the attractor radius and GAR to quantifying global and local predictability limits of an original dynamical system and its forecast model, along with a comparison with the traditional error AV method. Finally, conclusions are drawn in Sect. 5.

2 Methodology

2.1 Attractor radius, GAR and GAD definitions

Let \({\fancyscript{A}}\) be a compact attractor of the semigroup S, and \(\mathbf{x}\) be a state column vector on \({\fancyscript{A}}\). \({\fancyscript{A}}\) is a closed, bounded and invariant set, i.e., \(S(t){\fancyscript{A}}={\fancyscript{A}}\). Probability theory can be used to study the statistical characteristics of the state of attractor \({\fancyscript{A}}\). Let \({\fancyscript{A}}\) be the sample space, \(\mathcal{F}\) be a set of subsets of \({\fancyscript{A}}\) and a σ-field, the probability measure P:\(\mathcal{F}\)→[0,1]. Then, the triplet (\({\fancyscript{A}}\), \(\mathcal{F}\), P) is a probability space. According to the ergodic theory of attractors (Eckmann and Ruelle 1985), there are invariant (ergodic) measures and Eckmann and Ruelle (1985) and Farmer et al. (1983) introduced some invariant probability measures such as entropy, dimensions and characteristic exponents (also called Lyapunov exponents) to study the properties of dynamical systems. To quantitatively estimate predictability limit of strange attractors, some other invariant probability measures of attractors are introduced. First, let \(F({\mathbf{x}})\) be the probability density function (PDF) of \({\mathbf{x}}\) on \({\fancyscript{A}}\), and \(f\left( {{x^{\left( I \right)}}} \right)\) be the marginal density function in the direction of \({x^{(I)}}\), then for a compact attractor \({\fancyscript{A}}\), its PDF and marginal density function are invariant. In the following the norm \(|| \cdot ||\) is the \({L^2}\)-norm. To reduce space, the derivation process of some formulas and theorems is given in “Appendix”, and main concepts are summarized in Table 1.

Table 1 Main concepts and their definitions
Full size table

Definition 1

Let \({{\mathbf{x}}_i}\) be a specific state on a compact attractor \({\fancyscript{A}}\), then the local attractor radius (LAR, R L ) with respect to the point is defined as follows:

$${R_{Li}}={R_L}({{\mathbf{x}}_i})=\sqrt {E\left( {{{\left\| {{{\mathbf{x}}_i} - {\mathbf{x}}} \right\|}^2}} \right)} ,\quad {{\mathbf{x}}_i},{\mathbf{x}} \in {\fancyscript{A}},$$
(1)

where E represents the expectation, and the corresponding marginal LAR in the direction of \({x^{(I)}}\) is

$$R_{{Li}}^{{(I)}}=R_{L}^{{(I)}}\left( {x_{i}^{{(I)}}} \right)=\sqrt {E\left( {{{\left\| {x_{i}^{{(I)}} - {x^{(I)}}} \right\|}^2}} \right)} ,$$
(2)

Particularly, let \({\mathbf{x}_E}\) be the mean state of \({\fancyscript{A}}\), i.e., \({{\mathbf{x}}_E}=E({\mathbf{x}})\), \({\mathbf{x}} \in {\fancyscript{A}}\). The point \({{\mathbf{x}}_E}\) is referred to as the center of the attractor (not necessary on the attractor), and the attractor radius (R E ) and marginal attractor radius (R E (I)) are expressed as follows:

$${R_E}=\sqrt {E\left( {{{\left\| {{\mathbf{x}} - {{\mathbf{x}}_E}} \right\|}^2}} \right)} ,\quad {\mathbf{x}} \in {\fancyscript{A}},$$
(3)
$$R_{E}^{{(I)}}=\sqrt {E\left( {{{\left\| {{x^{(I)}} - x_{E}^{{(I)}}} \right\|}^2}} \right)} .$$
(4)

The geometric meaning of LAR is the expectation of the root-mean-square distance between one specific state and all other states on an attractor, which is an extension of the definition of the radius of an n-sphere. In fact, they are equivalent if \({\fancyscript{A}}\) is a periodic attractor or an n-sphere. Physically, the LAR is an indicator of the average behavior of the attractor subset relative to a specific state as the subset expands and fills up the whole attractor. The attractor radius has the same form as the standard deviation (SD) in statistics, which represents the variability of a variable.

Definition 2

The global attractor radius (GAR, R G ) of a compact attractor \({\fancyscript{A}}\) is defined as the average or expected value of the LAR of \({\fancyscript{A}}\), i.e.,

$${R_G}=\sqrt {E(R_{L}^{2})} =\sqrt {E\left( {{{\left\| {{\mathbf{x}} - {\mathbf{y}}} \right\|}^2}} \right)} ,\quad {\mathbf{x}},\,{\mathbf{y}} \in {\fancyscript{A}}.$$
(5)

The corresponding marginal GAR in the direction of \({x^{(I)}}\) is

$$R_{G}^{{(I)}}=\sqrt {E\left( {{{\left( {R_{L}^{{(I)}}} \right)}^2}} \right)} .$$
(6)

This indicates that the GAR is an expectation of the root-mean-square distance between any two randomly chosen points on an attractor.

Theorem 1

The (marginal) LAR and (marginal) attractor radius of a compact attractor \({\fancyscript{A}}\) satisfy

$${R_{Li}} \geqslant {R_E},\;R_{{Li}}^{{(I)}} \geqslant R_{E}^{{(I)}}.$$
(7)

Moreover, one has

$${R_{Li}}>{R_{Lj}},\;{\text{if}}\;{d_{iE}}>{d_{jE}},$$
(8)
$$R_{{Li}}^{{(I)}}>R_{{Lj}}^{{(I)}},\;{\text{if}}\;d_{{iE}}^{{(I)}}>d_{{jE}}^{{(I)}},$$
(9)

where d iE and d jE denote the root-mean-square distances of x i and x j on \({\fancyscript{A}}\) from the mean state of \({\fancyscript{A}}\), respectively, and \(d_{{iE}}^{{(I)}}\) and \(d_{{jE}}^{{(I)}}\) are the corresponding distances in the direction of \({x^{(I)}}\). Furthermore, in the phase space of \({\fancyscript{A}}\) the points \({\mathbf{x}_i}\) with the same LAR are on the intersection of the attractor set and a sphere of center \({{\mathbf{x}}_E}\) and radius \({d_{iE}}\).

The minimum value of the LAR of an attractor is the attractor radius.

Theorem 2

A constant proportion relationship between the (marginal) GAR and (marginal) attractor radius of a compact attractor \({\fancyscript{A}}\) exists as:

$${R_G}=\sqrt 2 {R_E},\quad R_{G}^{{(I)}}=\sqrt 2 R_{E}^{{(I)}}.$$
(10)

This conclusion is consistent with the error covariance analysis of numerical predictions from Leith (1974) and Kalnay (2002). In addition, Theorem 2 provides a simple, practical approach to calculate the GAR by calculating the attractor radius, which can greatly reduce the computational burden. In order to discuss the predictability of a model that is an approximation of a chaotic system (e.g., atmospheric models, which are approximations of the real atmosphere), some average distances between two attractors must be introduced.

Definition 3

Let \({\fancyscript{A}}\) and \({\fancyscript{B}}\) be two compact attractors, and \({{\mathbf{x}}_i}\) be a point on \({\fancyscript{A}}\). The local average distance (LAD) from \({\fancyscript{A}}\) to \({\fancyscript{B}}\) with respect to \({{\mathbf{x}}_i}\) is:

$${R_L}({{\mathbf{x}}_i},{\fancyscript{B}})=\sqrt {E\left( {{{\left\| {{{\mathbf{x}}_i} - {\mathbf{y}}} \right\|}^2}} \right)} ,\quad {{\mathbf{x}}_i} \in {\fancyscript{A}},\;{\mathbf{y}} \in {\fancyscript{B}},$$
(11)

and the corresponding marginal LAD in the direction \((I)\) is \(R_{L}^{{(I)}}({{\mathbf{x}}_i},{\fancyscript{B}})=\sqrt {E\left( {{{\left\| {x_{i}^{{(I)}} - {y^{(I)}}} \right\|}^2}} \right)}\).

Definition 4

The global average distance (GAD) between two compact attractors \({\fancyscript{A}}\) and \({\fancyscript{B}}\) is the expected value of the LAD between the two attractors, i.e.

$${R_G}({\fancyscript{A}},{\fancyscript{B}})=\sqrt {E\left( {R_{L}^{2}({\fancyscript{A}},{\fancyscript{B}})} \right)} =\sqrt {E\left( {{{\left\| {{\mathbf{x}} - {\mathbf{y}}} \right\|}^2}} \right)} ,\quad {\mathbf{x}} \in {\fancyscript{A}},\;{\mathbf{y}} \in {\fancyscript{B}}.$$
(12)

Theorem 3

For two compact attractors \({\fancyscript{A}}\) and \({\fancyscript{B}}\), one has

$$R_{G}^{{}}(~{\fancyscript{A}},{\fancyscript{B}})=\sqrt {R_{E}^{2}({\fancyscript{A}})+R_{E}^{2}({\fancyscript{B}})+d_{{}}^{2}({\mathbf{x}_E},{\mathbf{y}_E})} .$$
(13)

Theorem 3 provides an effective method to calculate the GAD between two attractors, which can save considerable computational time.

Theorem 4

If the GAD between two compact attractors \({\fancyscript{A}}\) and \({\fancyscript{B}}\) equals their GARs, then the two attractors have the same radius and the same center as well. Additionally, with respect to any point on \({\fancyscript{A}}\) or \({\fancyscript{B}}\) the LAD equals the LAR.

In practice, the long-term historical records of a chaotic system and a large amount of forecast data from a model of the system can be used to estimate its attractor radius and GAR, as well as the GAD between the attractors of the system and its forecast model, even if its exact equations are unknown. We have the following corollary.

Corollary 1

The attractor radius and GAR of a compact attractor and the GAD between two compact attractors are invariant quantities.

2.2 Quantifying global and local predictability limits

The attractor radius and GAR are in essence statistics of the average behavior of chaos. For NWP, if the forecast RMSE averaged over a great number of samples exceeds the GAR calculated from the attractor of verifying analyses, the forecasts cannot be more accurate than a randomly chosen state from the attractor on average, and thus become useless. In other words, the GAR as an invariant property of an attractor provides an objective metric that can be used to determine whether forecasts are still skillful, i.e., predictability. For an n-dimensional nonlinear dynamical system,

$$\frac{{d\mathbf{x}(t)}}{{dt}}=\mathbf{F}\left( {\mathbf{x}\left( t \right)} \right),$$
(14)

where \(\mathbf{x}\) and \(\mathbf{F}\) are n-dimensional column vectors, the system has a compact attractor \({\fancyscript{A}}\). Considering the evolution of the state \(\mathbf{x}\) on \({\fancyscript{A}}\), let \(\mathbf{x}(t)\) be a fiducial orbit with the initial value \({{\mathbf{x}}_0}={\mathbf{x}}(0)\), \({\mathbf{\hat {x}}}(t)\) be its perturbed orbit with the initial value \({{\mathbf{\hat {x}}}_0}={\mathbf{\hat {x}}}(0)\), and \({\delta _0}=\delta (0)\) be an initial perturbation vector. Then,

$${{\mathbf{\hat {x}}}_0}={{\mathbf{x}}_0}+{\delta _0}.$$
(15)

The RMSE between the two orbits is defined as

$$e({{\mathbf{x}}_0},{\delta _{\text{0}}},t)=\left\| {\delta (t)} \right\|=\left\| {{\mathbf{\hat {x}}}(t) - {\mathbf{x}}(t)} \right\|.$$
(16)

For global predictability, the global ensemble average of error growth must be investigated, and is defined as

$$\bar {e}({\delta _{\text{0}}},t)=\sqrt {{{\left\langle {{e^{\text{2}}}({{\mathbf{x}}_0},{\delta _{\text{0}}},t)} \right\rangle }_N}} ,\quad {\mathbf{x}_0} \in {\fancyscript{A}},$$
(17)

where \({\left\langle {{e^{\text{2}}}({{\mathbf{x}}_0},{\delta _{\text{0}}},t)} \right\rangle _N}=\int_{{\fancyscript{A}}} {{e^{\text{2}}}\left( {{\mathbf{x}}({t_0}),\delta ({t_{\text{0}}}),t} \right)d} {\mathbf{x}}\), \(<{\text{ }}{>_N}\) denotes the ensemble average of samples of sufficiently large number N (N → ∞). For the local predictability of one initial state, we define a local ensemble average of error growth,

$${\bar {e}_{{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t)=\sqrt {{{\left\langle {{e^{\text{2}}}({{{\mathbf{\hat {x}}}}_0},{\delta _{\text{0}}},t)} \right\rangle }_N}} ,\quad {\hat {\mathbf{x}}_0} \in {\Omega _0},$$
(18)

where \({\Omega _0}\) is a sphere of center \({{\mathbf{x}}_0}\) and radius \({\delta _0}=\left\| {{\delta _0}} \right\|\), \({\left\langle {{e^{\text{2}}}({{\mathbf{x}}_0},{\delta _{\text{0}}},t)} \right\rangle _{N \to \infty }}=\int_{{\Omega (t)}} {{e^{\text{2}}}({\mathbf{x}}({t_0}),\delta ({t_{\text{0}}}),t)d} {\mathbf{x}}\), \(\Omega (t)=S(t){\Omega _0}\).

Ding and Li (2007) and Li and Ding (2011, 2015) indicated that for chaotic systems, the ensemble mean relative growth of initial error has a property of saturation—i.e., it tends to a saturation value. Similarly, the global ensemble RMSE tends to the GAR (which equals the AV) with time. The saturation value of error growth (i.e., the GAR) represents the average distance between two randomly chosen points on an attractor (Li and Ding 2015). Almost all of the initial information is lost when the global ensemble RMSE reaches the GAR, and further prediction therefore becomes meaningless. Note that the attractor radius in Li and Ding (2015) is actually the GAR described here. Because \(71\% \approx {1 \mathord{\left/ {\vphantom {1 {\sqrt 2 }}} \right. \kern-0pt} {\sqrt 2 }}\), 71% of the forecast error AV used as the threshold in Savijärvi (1995) and Buizza (2010) is equivalent to using the attractor radius \({R_E}\) as the threshold. There are two types of predictability limit corresponding to the attractor radius and GAR, the practical predictability limit and the potential predictability limit (also known as the maximum predictability limit). Figure 1a illustrates typical mean error growth in a nonlinear chaotic system as a function of time, along with the practical predictability limit and the potential predictability limit. As shown, the global practical predictability limit and the global potential predictability limit are determined as the times when the global ensemble average RMSE reaches the attractor radius and GAR, respectively, i.e.,

Fig. 1
figure 1

a Schematic illustration of the typical mean error growth (measured as the global ensemble average RMSE, red) of a nonlinear chaotic system as a function of time, and the relationships of the practical predictability limit (T pr ) and potential predictability limit (T po ) to the attractor radius (R E ; lower dashed line) and GAR (R G ; upper dashed line). b Same as in (a) but for the error growth of a model forecast (blue) and the determination of the practical predictability limit (\(T_{pr}^{M}\)) and potential predictability limit (\(T_{po}^{M}\)) of the model. The upper, middle and lower horizontal dashed lines represent the GAD (GAD M,O) between the real attractor and model attractor, the minimum R g  the GARs of real attractor and model attractor, and the minimum R e of the attractor radii of real attractor and model attractor, respectively

Full size image
$$\bar {e}({\delta _{\text{0}}},t){|_{t<{T_{pr}}}}<{R_E}\;{\text{and}}\;\bar {e}({\delta _{\text{0}}},t){|_{t={T_{pr}}}}={R_E},$$
(19)
$$\bar {e}({\delta _{\text{0}}},t){|_{t<{T_{po}}}}<{R_G}\;{\text{and}}\;\bar {e}({\delta _{\text{0}}},t){|_{t={T_{po}}}}={R_G},$$
(20)

Where \({T_{pr}}\) and \({T_{po}}\) are the global practical and potential predictability limits, respectively. The error growth will enter a strong nonlinear phase with a steadily decreasing growth rate when the magnitude of the mean error exceeds the attractor radius (Fig. 1a). For a “perfect” model without model errors, we may choose 95% of the GAR as the criterion to measure \({T_{po}}\), following previous studies (Dalcher and Kalnay 1987; Simmons and Hollingsworth 2002; Buizza 2010).

Different initial states have different predictability. The local practical and potential predictability limits of any initial state can be measured by the times when the local ensemble average RMSE reaches the attractor radius and GAR for the first time, respectively—i.e.,

$${\bar {e}_{{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t={T_{pr,{{\mathbf{x}}_0}}}}}={R_E}\quad {\text{and}}\quad {\bar {e}_{{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t<{T_{pr,{{\mathbf{x}}_0}}}}}<{R_E},$$
(21)
$${\bar {e}_{{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t={T_{po,{{\mathbf{x}}_0}}}}}={R_G}\quad {\text{and}}\quad {\bar {e}_{{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t<{T_{po,{{\mathbf{x}}_0}}}}}<{R_G},$$
(22)

Next, the predictability of model forecasts is discussed. Typically, model errors are present. For example, an approximate model of the dynamical system of Eq. (14) can be written as follows:

$$\frac{{d\mathbf{x}(t)}}{{dt}}=\tilde {\mathbf{F}}\left( {\mathbf{x}(t)} \right),$$
(23)

where \(\tilde {\mathbf{F}}\) is an approximation of \(\mathbf{F}\) in Eq. (14). Let \({\fancyscript{A}}\) and \({{\fancyscript{A}}_M}\) be the attractor of the chaotic system of Eq. (14) and the one of its approximate model, Eq. (23), respectively. Typically, they are different and thus their predictabilities are also different. Considering the two orbits \(\mathbf{x}(t)\) on \({\fancyscript{A}}\) and \({\mathbf{y}}(t)\) on \({{\fancyscript{A}}_M}\). Here, \(\mathbf{x}(t)\) is a fiducial orbit and the initial value \({{\mathbf{y}}_0}={{\mathbf{x}}_0}+{\delta _0}\), where \({\delta _0}\) is a small perturbation. The error of the model forecast is

$${e_M}({{\mathbf{x}}_0},{\delta _{\text{0}}},t)=\left\| {{\delta _M}(t)} \right\|=\left\| {{\mathbf{y}}(t) - {\mathbf{x}}(t)} \right\|,\quad \mathbf{x} \in {\fancyscript{A}},\;\mathbf{y} \in {{\fancyscript{A}}_M}.$$
(24)

The global ensemble average of error growth, \({\bar {e}_M}\), will tend to a saturation value with time, which is the GAD between the two attractors \({\fancyscript{A}}\) and \({{\fancyscript{A}}_M}\). However, the GAD cannot directly be used as a threshold to determine the predictability limit, because the forecast model has associated model errors. Following Theorem 3, let \({R_e}=\hbox{min} \left( {R_{E}^{{}}\left( {{{\fancyscript{A}}_M}} \right),\;R_{E}^{{}}\left({\fancyscript{A}} \right)} \right)\), and \({R_g}=\hbox{min} \left( {R_{G}^{{}}\left( {{{\fancyscript{A}}_M}} \right),\;R_{G}^{{}}\left({\fancyscript{A}} \right)} \right)\), then the global and local predictability limits of the model are determined by

$${\bar {e}_M}({\delta _{\text{0}}},t){|_{t=T_{{pr}}^{M}}}={R_e}\quad {\text{and}}\quad {\bar {e}_M}({\delta _{\text{0}}},t){|_{t<T_{{pr}}^{M}}}<{R_e},$$
(25)
$${\bar {e}_{M,{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t=T_{{pr,{{\mathbf{x}}_0}}}^{M}}}={R_e}\quad {\text{and}}\quad {\bar {e}_{M,{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t<T_{{pr,{{\mathbf{x}}_0}}}^{M}}}<{R_e},$$
(26)
$${\bar {e}_M}({\delta _{\text{0}}},t){|_{t=T_{{po}}^{M}}}={R_g}\quad {\text{and}}\quad {\bar {e}_M}({\delta _{\text{0}}},t){|_{t<T_{{po}}^{M}}}<{R_g},$$
(27)
$${\bar {e}_{M,{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t=T_{{po,{{\mathbf{x}}_0}}}^{M}}}={R_g}\quad {\text{and}}\quad {\bar {e}_{M,{{\mathbf{x}}_0}}}({\delta _{\text{0}}},t){|_{t<T_{{po,{{\mathbf{x}}_0}}}^{M}}}<{R_g},$$
(28)

where \(T_{{pr}}^{M}\) and \(T_{{po}}^{M}\) are the global practical and potential predictability limits of the model (Fig. 1b), respectively, and \(T_{{pr,{{\mathbf{x}}_0}}}^{M}\) and \(T_{{po,{{\mathbf{x}}_0}}}^{M}\) are the local practical and potential predictability limits of the model. All of the concepts of predictability limits mentioned above are summarized in Table 1.

The GAR and the traditional error AV are similar in that both are used to determine the predictability limit by identifying the time during which forecasts are chaotic in contrast to analyses. However, because a forecast model usually has associated model errors and the model attractor \({{\fancyscript{A}}_M}\) is different from the real attractor \({\fancyscript{A}}\), the saturation level of the global ensemble average RMSE \({\bar {e}_M}({\delta _{\text{0}}},t)\), which corresponds to the traditional AV and the GAD between the two attractors \({\fancyscript{A}}\) and \({{\fancyscript{A}}_M}\), is not the same as that of \(\bar {e}({\delta _{\text{0}}},t)\) (i.e. the GAR). Typically, because of model errors, the AV overestimates the error saturation value, resulting in an overestimation of the predictability limit. Furthermore, the difference between the GAR of the real attractor and the GAD is a better indicator of the degree of model drift or systematic error. Additionally, in light of Theorem 3 the 71% of AV threshold cannot serve as an accurate approximation of the variability of the real system and has therefore lost both physical and geometric meaning. In contrast, the GAR and attractor radius only depend on the real attractor and are not affected by model drift errors. Thus, the GAR and attractor radius are more suitable metrics to use to determine the predictability limit of a forecast model (Fig. 1b).

3 Model and data

The model used in this study is the simple Lorenz63 model (Lorenz 1963):

$$\left\{ \begin{aligned} \dot {x} & = - \sigma x+\sigma y \\ \dot {y} & =rx - y - xz \\ \dot {z} & =xy - bz \\ \end{aligned} \right.,$$
(29)

where \(\sigma =10\), \(r=28\), \(b={8 \mathord{\left/ {\vphantom {8 3}} \right. \kern-0pt} 3}\), and for which the well-known ‘butterfly’ attractor exists. To discuss the predictability of model forecasts, an approximation of the Lorenz63 model with associated model errors is chosen, in which the parameter \(b={{10} \mathord{\left/ {\vphantom {{10} 3}} \right. \kern-0pt} 3}\).This approximate model is referred to as the imperfect Lorenz63 model. A fourth-order Runge–Kutta time integration scheme is selected with a time step \(h=0.01\)time units (tu) and double precision is used to calculate numerical solutions.

The Lorenz63 model is continuously integrated for 1 × 106 tu with a 0.01 tu output interval and used to represent the ‘true’ system after eliminating the first 104 tu trajectory as spin-up. Results are used to calculate the attractor radius, GAR, and LAR, and are used as the reference for verifying forecasts. In addition to a large data set simulated from this simple model, the 45-day retrospective forecasts of the coupled Climate Forecast System (CFSv2) from the National Centers for Environmental Prediction (NCEP) from Jan 1999 to Dec 2010 every 6 h are used to evaluate the utility of the attractor characteristic parameters presented in this work. The 00Z initial conditions, which are approximately equivalent to the observed atmospheric state, are used to verify the forecasts. Note that the annual cycle of the series of 00Z initial conditions has been removed before the observations are applied to the computation of the GAR and LAR of the atmosphere. The predictability experiments are summarized in Table 2.

Table 2 Predictability experiments and the corresponding descriptions
Full size table

4 Attractor radius, GAR and LAR in the Lorenz63 model

4.1 GAR and attractor radius

Figure 2 shows the variations of the marginal GARs for the variables x, y, z and the GAR for the vector (x, y, z) in the Lorenz63 model as a function of time-series length. The figure shows that for any variable or vector, the marginal GAR or GAR initially varies slightly, and then gradually approaches a constant value with time. The fact that the evolution trajectory tends to spread over the whole attractor implies that the GAR is an invariant quantity. The marginal GARs for the variables x, y, z and the GAR for the vector (x, y, z) are 11.21, 12.74, 12.19, and 20.90, and the marginal attractor radii and the attractor radius of the system are 7.92, 9.01, 8.62 and 14.78, respectively. A proportional relationship of \(\sqrt 2\) between the attractor radius and the GAR exists, as proved in Sect. 2.

Fig. 2
figure 2

Evolution of the marginal GAR and GAR for the variables (a) x, (b) y, and (c) z and (d) vector (x, y, z) in the Lorenz 63 model as with respect to time-series length (× 102 tu)

Full size image

4.2 LAR

As described in Sect. 2, the LAR varies with the specific location of a point in phase space. Figure 3 illustrates the distributions (solid line) of the marginal LAR and the PDF for the variables x, y, and z in the Lorenz63 model with respect to the reference state. The PDF patterns of the variables x, y, and z noticeably differ, but in general the probability near the mean is larger than at the extremities. In contrast, the LAR patterns are similar among the variables, reaching minima at the mean state (dashed line) and continuously increasing toward the edge of the attractor. This is consistent with Theorem 2, as described in Sect. 2, and follows from the fact that predictions of extreme events generally have larger forecast errors compared with events closer to the mean, a trend that is independent of the probability that specific events occur, and is a natural property of an attractor. The application of the LAR can be extended to higher-dimensional systems. Figure 4 shows the distribution of the LAR on the x–y plane in phase space for the Lorenz63 model. The figure shows that points with the same LAR are on the intersection of the attractor set and a sphere centered at the mean state with a radius equal to the distance between the points and the mean state, in agreement with Theorem 1, as described in Sect. 2.

Fig. 3
figure 3

Distribution (solid line) of the marginal LAR and probability density function (PDF dotted line) for the variables (a) x, (b) y, and (c) z in the Lorenz 63 model. The dashed line represents the mean state

Full size image
Fig. 4
figure 4

Distribution of LAR on the x–y plane in the phase space of the Lorenz63 model

Full size image

For the real atmosphere, the LAR can be estimated from observational data collected over a long period of time. Figure 5 shows the distribution of LAR as a function of states on the attractor at 500 hPa geopotential height (GHT; with the annual cycle removed) at five selected grid points. The results are very similar to those in Fig. 3. The anomalies of GHT approximately follow a Gaussian distribution at individual grid point. The states farther away from the mean state have relatively larger LARs compared with the ones close to the mean state, which may help explain the more remarkable forecast errors for extreme events from atmospheric models.

Fig. 5
figure 5

Distribution (solid line) of LAR and the probability density function (PDF; dotted line) for geopotential height (GHT) at 500 hPa in the coupled Climate Forecast System (CFSv2) of the National Centers for Environmental Prediction (NCEP) at five grid points: a (0°N, 120°E), b (30°N, 120°E), c (30°S, 120°E), d (60°N, 120°E), and e (60°N, 120°E)

Full size image

5 Global predictability limits

5.1 A ‘perfect’ model scenario

Forecast RMSE averaged over large samples will gradually increase with time, and approach a certain saturation value because of the presence of nonlinear chaos. Traditional approaches generally use a threshold value of the error AV to define the predictability limit. In this section, the AV and GAR are compared in a ‘perfect’ model scenario. Figure 6 shows the evolution of the global ensemble mean RMSE of the variables x, y, and z and vector (x, y, z) with the ‘perfect’ Lorenz63 forecast model (the same model described in Sect. 3 for the ‘true’ system). A set of initial errors with magnitudes 10−2, 10−3, 10−4, 10−5, 10−6, 10−7, are generated with random numbers and superimposed on the ‘true’ states to produce the initial states (5 × 107 in total). Forecasts are then carried out for 50 tu. For such simple system, forecasts with smaller initial errors have higher accuracy and longer predictability limits, which is consistent with the previous study (Ding and Li 2007, 2008). However, for the multi-scale system (Lorenz 1969) and coupled system (Ding and Li 2012) this relationship between the predictability limits and initial errors no longer applies. For the two classes of systems we will further investigate this relationship using the approach presented here in the future. All forecast errors for a particular point or vector tend to the same saturation value. The saturation values corresponding to the variables x, y, and z and vector (x, y, z) are 11.21, 12.74, 12.19, and 20.90, respectively, and are consistent with the GARs shown in Fig. 2. This indicates that the GAR and the AV are equivalent in the perfect model scenario, and that both are able to distinguish the different predictability limits that result from variations in the initial error size in these experiments. Figure 7 shows the global practical and potential predictability limits (\({T_{pr}}\) and \({T_{po}}\), respectively) of the Lorenz 63 model as a function of initial error. Negative linear relationships of \({T_{pr}}\) and \({T_{po}}\) with the magnitude of the initial error are observed—i.e., both the global practical and potential predictability limits increase as the magnitude of the initial error increases. This result is in agreement with Ding and Li (2007) who used the nonlinear local Lyapunov exponent.

Fig. 6
figure 6

Evolution of global ensemble mean RMSE of the variables (a) x, (b) y, and (c) z, and (d) vector (x, y, z) in the Lorenz 63 model with respect to lead time (unit: tu) and the magnitude of initial error. The magnitudes of initial perturbation are 10−2, 10−3, 10−4, 10−5, 10−6, and 10−7, from left to right. The straight line represents the corresponding GAR of each variable

Full size image
Fig. 7
figure 7

a Global practical predictability limit \({T_{pr}}\) and b potential predictability limit \({T_{po}}\) of the Lorenz 63 model with respect to initial error. The open circles, crosses, gray dots, and black dots correspond to the variables x, y, and z, and the vector (x, y, z)

Full size image

5.2 An imperfect model scenario

Deficiencies in numerical models are an important source of forecast errors. A simple case, the Lorenz63 model, is used as the real system with an imperfect forecast model, as described in Sect. 3. The difference between the two models results from a small deviation in the parameter b in the imperfect model (see Sect. 3). Figure 8 shows the evolution of the global ensemble mean RMSE of the variables x, y, and z and vector (x, y, z) of the imperfect Lorenz63 forecast model. The error growth of the imperfect Lorenz63 forecast model is similar to that of the Lorenz63 model (Fig. 6), but because of the system bias, the GAR of the former is significantly larger than that of the latter. Thus, the error growth of the imperfect model alone cannot be used to determine its predictability of the perfect system, and the global ensemble mean RMSE between the Lorenz63 model and the imperfect Lorenz63 model must be used. As shown in Fig. 9, the global ensemble mean RMSE between the Lorenz63 model and the imperfect Lorenz63 model converges to the GAD between the two models. Moreover, error growth curves with initial errors of different magnitude coincide—i.e., they are independent of the magnitude of the initial errors. This implies that, in this case, model errors, and not small initial errors, play the dominant role in forecast error growth. The global practical and potential predictability limits of the imperfect Lorenz63 model are 0.8 and 1.9 for the vector (x, y, z), 0.8 and 1.9 for x, 0.8 and 1.8 for y, and 0.8 and 2.6 for z, respectively. These predictability limits are much shorter than those of the original Lorenz63 model (Fig. 7). This reflects the predictability deficiencies of the imperfect model when used to forecast the evolution of the ‘perfect’ Lorenz system. However, a practical problem is that the existing models are inevitably of model errors. Applying a forecast model to investigate predictability is also an important tool and approach. It is very important to understand how to use a forecast model to estimate the attractor radius and global attractor radius and the predictability limits. To this end, we need to learn how large is the GAD between the real attractor (it could be estimated by using observational or long analysis data series if the exact equations of the system are unknown) and the model attractor, and take it into account. Usually we may choose a forecast model with smaller GAD between its attractor and the real attractor to study predictability and estimate the predictability limits.

Fig. 8
figure 8

Same as in Fig. 6, but for the imperfect Lorenz63 model. The dashed lines represent the corresponding GAR of the Lorenz63 model

Full size image
Fig. 9
figure 9

Same as in Fig. 6, but for the global ensemble mean RMSE between the Lorenz63 model and imperfect Lorenz63 model. The solid straight line denotes the GAD of each variable between the two models. The upper and lower dashed lines represent the corresponding GAR and attractor radius of the Lorenz63 model, respectively. Note that the six error growth curves overlap

Full size image

5.3 Global predictability limits of the CFSv2

Figure 10 illustrates the mean RMSE of all the CFSv2 operational forecast cases (from Jan 1999 to Dec 2010) of GHT at 500 hPa averaged globally (90°S–90°N) and over the extratropical Northern Hemisphere (NH 20°–90°N), the extratropical Southern Hemisphere (SH 90°–20°S) and the tropics (TRO 20°S–20°N). Because the CFSv2 only makes 45-day retrospective forecasts for each set of initial conditions, the average of the last 25 forecasts is chosen as an estimation of the AV. It can be seen in Fig. 10 that there are smaller differences between the AV and GAR for the global, extratropical NH and extratropical SH, than for the tropics. This indicates that the forecast model has significant model drift (or systematic error) in the tropics. Using the GAR, the global practical (potential) predictability limits for the global, extratropical NH, extratropical SH, and tropics are 7.7 (17.1), 8.0 (18.8), 7.4 (16.0), and 5.6 (8.8) days, respectively. The CFSv2 has much lower predictability for the tropics compared with the extratropical regions, consistent with Li and Ding (2015). The mean RMSE reaches the AV for the global, extratropical NH, extratropical SH, and tropics at 29.4, 29.3, 39.1, and 29.4 days, respectively, times which are significantly longer than the predictability limits found using the GAR, which results from that fact that the AV ignores the effects of model-drift-related errors on degrading forecast accuracy, and thus overestimates the predictability limit.

Fig. 10
figure 10

Mean RMSE from Jan 1999 to Dec 2010 of GHT at 500 hPa from CFSv2 operational forecast data averaged (a) globally (90°S–90°N), and over (b) the extratropical Northern Hemisphere (NH 20°N–90°N), c the extratropical Southern Hemisphere (SH 90°S–20°S), and d the tropics (TRO 20°S–20°N). Black, red, and blue dashed lines are the error AV, the attractor radius (AR), and the GAR of the time series with 00Z initial conditions, respectively. The numbers (in days) in the legends correspond to the time at which the mean RMSE reaches the AR, GAR, and AV

Full size image

6 Local predictability limits

This section considers one case for the Lorenz63 model with initial state (− 4.0, 5.0, 20.0) to investigate the quantification of the local predictability limit for a specific state. Projections of the LAR on the x–y plane in the phase space of the Lorenz63 model are presented in Fig. 11. Although the LAR changes with time, the LAR itself is an attractor. As the figure shows, the projections of the LAR on the x–y plane have a pattern similar to a single butterfly wing. Figure 12 shows the evolution of the LAR and the local ensemble average RMSE for the initial state specified above with two initial perturbation magnitudes, \({10^{ - 2}}\) and \({10^{ - 7}}\). In this case study, 1 × 106 initial perturbations of the initial state are generated with the same perturbation size but different directions. These initial errors are added to the specified initial state to produce a set of individual initial ‘analyses’ and the forecasts are run for 50 tu. In fact, this many initial perturbations are not needed, and hundreds of initial perturbations (or less) are sufficient. Results indicate that unlike the global ensemble mean forecast error (for which saturation and convergence to the GAR are evident; see Fig. 6), the RMSE in the case of a specific initial state varies with time because of a time-dependent basic flow or ‘flow-dependence’ (Corazza et al. 2003). The RMSE are initially smaller than the attractor radius, which indicates better forecasts. Then, the errors gradually increase and exceed both the attractor radius and the GAR. The practical and potential predictability limits of this initial condition are 10.1 (23.6) and 11.7 (23.6) for initial perturbation size 10−2 (10−7). Beyond the potential predictability limit, forecast errors begin to undergo fluctuations similar to those of the LAR, and finally converge on the trajectory of the LAR, at which point forecasts become meaningless.

Fig. 11
figure 11

Projections of the LAR on the x–y plane in the phase space of the Lorenz63 model. 1 × 105 points are plotted

Full size image
Fig. 12
figure 12

Evolution of the LAR (orange) and local ensemble average RMSE (green and blue curves for initial perturbation sizes of \({10^{ - 2}}\) and \({10^{ - 7}}\), respectively) of the vector (x, y, z) for the Lorenz63 model. The initial value is (− 4.0, 5.0, 20.0). The upper and lower black lines are the GAR and attractor radius, respectively

Full size image

7 Conclusions and discussion

This paper introduces several geometric properties of attractors, the LAR, attractor radius, and GAR of an attractor, as well as the LAD and GAD between two attractors, to characterize features and the average behavior of chaotic systems and evaluate error growth in nonlinear dynamical systems. The attractor radius, GAR, and GAD are invariant quantities, which are applied to quantitatively determine the global and local predictability limits of both global ensemble average forecasts and a single state in phase space, respectively. Both global and local predictability limits have practical and potential values, which correspond to the times at which the ensemble RMSE reaches the attractor radius and GAR, respectively. Both the Lorenz63 model and operational forecast data demonstrate the theoretical aspects of these geometric characteristics, and the feasibility and effectiveness of their application to predictability limit analysis.

The GAR is \(\sqrt 2\) times the attractor radius. Therefore, when a nonlinear dynamical model has no associated model error, the global potential and practical predictability limits quantified by the GAR and attractor radius are equivalent to the thresholds of the traditional AV and 71% of AV, respectively. The theory presented here provides a general, unified framework for the two traditional criteria used to measure predictability limits. However, for a forecast model with associated model errors, the two traditional thresholds do not work, because the attractor of the forecast model is different from that of the original dynamical system, and the AV thresholds overestimate forecast skill and the predictability limit. Additionally, for the local predictability limit, because of the ‘flow-dependence’ of individual cases, the local ensemble average RMSE varies with time and does not reach saturation or an AV. Thus, the traditional AV threshold is not applicable. Both the attractor radius and GAR of a nonlinear system are intrinsic properties of the system and independent of the forecast model used, and thus provide more accurate and objective thresholds to assess the global and local predictability limits of forecast models compared with the traditional AV threshold. In addition, because of the invariant properties of the GAR and attractor radius, the theoretical work presented here can be used to reduce computational burden and storage and avoid repetitive computation, facilitating faster, easier measurements of predictability limits.

In practice, the predictability limits on different spatial–temporal scales can be estimated by the attractor radius and GAR by using observational data; and then, we may analyze the spatial–temporal structures of predictability limits and their variations. A noticeable question is how much the observational errors influence on the estimation of predictability limits. During the recent decades, the performance of NWP models is significantly improved, and the observational network become much more accurate and extensively covered, resulting in largely reduced analyses errors from the truth and reducing the influence of the observational errors on the estimation of predictability limits. However, this is worthy of future work. Besides, the Lorenze63 model has converging domains, which occur within a torus in phase space and also satisfy the fluctuation theorem (Smith et al. 1999; Schalge et al. 2012). The retrospective forecasts of the CFSv2 are used to evaluate the utility of the attractor invariant quantities presented in this paper, however, we still need to employ some high-dimensional complex dynamical systems or coupled systems to further validate the predictability theory and methods in the future so as to better apply them to practice.