Regional Arctic sea–ice prediction: potential versus operational seasonal forecast skill

Abstract

Seasonal predictions of Arctic sea ice on regional spatial scales are a pressing need for a broad group of stakeholders, however, most assessments of predictability and forecast skill to date have focused on pan-Arctic sea–ice extent (SIE). In this work, we present the first direct comparison of perfect model (PM) and operational (OP) seasonal prediction skill for regional Arctic SIE within a common dynamical prediction system. This assessment is based on two complementary suites of seasonal prediction ensemble experiments performed with a global coupled climate model. First, we present a suite of PM predictability experiments with start dates spanning the calendar year, which are used to quantify the potential regional SIE prediction skill of this system. Second, we assess the system’s OP prediction skill for detrended regional SIE using a suite of retrospective initialized seasonal forecasts spanning 1981–2016. In nearly all Arctic regions and for all target months, we find a substantial skill gap between PM and OP predictions of regional SIE. The PM experiments reveal that regional winter SIE is potentially predictable at lead times beyond 12 months, substantially longer than the skill of their OP counterparts. Both the OP and PM predictions display a spring prediction skill barrier for regional summer SIE forecasts, indicating a fundamental predictability limit for summer regional predictions. We find that a similar barrier exists for pan-Arctic sea–ice volume predictions, but is not present for predictions of pan-Arctic SIE. The skill gap identified in this work indicates a promising potential for future improvements in regional SIE predictions.

Appendix

Reliability condition for ensemble forecasts

Claim The PM MSE given by Eq. (8) satisfies the necessary condition for forecast reliability:

$$\begin{aligned} MSE(\tau )=\frac{N}{N-1}\sigma _e^2(\tau ). \end{aligned}$$

(18)

Proof The mean intra-ensemble variance, $\sigma _e^2$, is given by

$$\begin{aligned} \displaystyle \sigma _e^2(\tau )= \frac{1}{M}\sum _{j=1}^M \frac{1}{N-1} \sum _{i=1}^N \Big (\langle \mathbf {x}_{j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2, \end{aligned}$$

(19)

where $\langle \mathbf {x}_{j}(\tau ) \rangle$ is the ensemble mean of the jth ensemble. The MSE is given by

$$\begin{aligned} \displaystyle MSE(\tau )= \frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{MN}. \end{aligned}$$

(20)

First, we note a relation between the ensemble mean $\langle \mathbf {x}_{j}(\tau ) \rangle$ and the ensemble mean with the ith member removed $\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle$. These ensemble means are defined respectively as

$$\begin{aligned} \langle \mathbf {x}_{j}(\tau ) \rangle =\frac{1}{N} \sum _{k=1}^N x_{kj}(\tau ), \end{aligned}$$

(21)

and

$$\begin{aligned} \langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle = \frac{1}{N-1} \sum _{k \ne i}^N x_{kj}(\tau ), \end{aligned}$$

(22)

and are related by:

$$\begin{aligned} \langle \mathbf {x}_{j}(\tau ) \rangle =\frac{1}{N} \sum _{k=1}^N x_{kj}(\tau )=\frac{x_{ij}(\tau )}{N}+\frac{1}{N}\sum _{k \ne i}^N x_{kj}(\tau )=\frac{x_{ij}(\tau )}{N}+\frac{N-1}{N}\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle . \end{aligned}$$

(23)

Therefore,

$$\begin{aligned} \sigma _e^2(\tau )&= \frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$

(24)

$$\begin{aligned}&=\frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\frac{1}{N}x_{ij}(\tau )+\frac{N-1}{N}\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$

(25)

$$\begin{aligned}&=\frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\frac{N-1}{N}\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - \frac{N-1}{N}x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$

(26)

$$\begin{aligned}&=\bigg ( \frac{N-1}{N} \bigg )^2 \frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$

(27)

$$\begin{aligned}&=\frac{N-1}{N}\frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{MN}\end{aligned}$$

(28)

$$\begin{aligned}&=\frac{N-1}{N}MSE(\tau ). \end{aligned}$$

(29)

Relation of perfect model skill metrics to other metrics

PPP

A commonly used PM skill metric is the potential prognostic predictability (PPP, Pohlmann et al. 2004), which compares the ensemble variance, $\sigma _e^2(\tau )$, to the climatological variance, $\sigma _c^2$. The PPP is defined as

$$\begin{aligned} PPP(\tau )=1-\frac{\sigma _e^2(\tau )}{\sigma _c^2}, \end{aligned}$$

(30)

which has a similar form to the MSSS defined in Eq. (11). Since $MSE=\frac{N}{N-1}\sigma _e^2$, for any finite N, $MSSS < PPP$ and $MSSS \rightarrow PPP$ as $N \rightarrow \infty$. For most typical values of N, the PPP and MSSS will be quite similar and share the same qualitative interpretations. However, we believe that the MSSS metric provides a more natural comparison with the MSSS metric used in OP predictions. In the PPP formulation, the ensemble mean $\langle x_j \rangle$ is used to predict a given truth member $x_{ij}$. This implies that the prediction has knowledge of the observed value, since the $x_{ij}$ truth member is included in the ensemble mean computation. This is an undesirable property for a skill metric, and will tend to bias skill scores high. The MSSS does not suffer from this issue, as only non-truth members are used to predict a given truth member.

RMSE

In the PM MSE formula given in Eq. (8), we have used the $(N-1)$-member ensemble mean to predict a given truth member. In general, we could use an E-member ensemble mean to make this prediction, where $1 \le E \le N-1$. It can be shown that an MSE based on an E-member ensemble mean satisfies $MSE=\frac{E+1}{E}\sigma _e^2$, where the proof uses the Central Limit Theorem and follows the same approach as that of Jolliffe and Stephenson (2012). The formula in 6.1 is the special case when $E=N-1$. The PM RMSE definition of Collins (2002), uses 1-member ensembles to predict a given truth member, and therefore satisfies $MSE=2\sigma _e^2$. At long lead times, the PM RMSE of Collins (2002) converges to $\sqrt{2} \sigma _c$ (note that this is strictly true only if the normalization of $MN(N-1)-1$ used in Collins (2002) is replaced with $MN(N-1)$).

This factor of $\sqrt{2}$ is a potential source of confusion, since in the PM literature a “no skill” forecast has $RMSE=\sqrt{2}\sigma _c$, whereas in the OP literature a “no skill” (climatological) forecast has an RMSE of $\sigma _c$. This can lead to confusion when quoting PM RMSE in physical units, or when comparing PM and OP RMSE values (e.g. as done in Blanchard-Wrigglesworth et al. 2015; Tietsche et al. 2014). In particular, the RMSE values obtained via the formula of Collins (2002) are too large, since they do not benefit from ensemble averaging. If ensemble means are used for the PM prediction, this issue is greatly ameliorated, since the PM RMSE values converge to $\sqrt{\frac{N}{N-1}}\sigma _c$, allowing for cleaner comparison with OP predictions.