Regional Arctic sea–ice prediction: potential versus operational seasonal forecast skill

Abstract

Seasonal predictions of Arctic sea ice on regional spatial scales are a pressing need for a broad group of stakeholders, however, most assessments of predictability and forecast skill to date have focused on pan-Arctic sea–ice extent (SIE). In this work, we present the first direct comparison of perfect model (PM) and operational (OP) seasonal prediction skill for regional Arctic SIE within a common dynamical prediction system. This assessment is based on two complementary suites of seasonal prediction ensemble experiments performed with a global coupled climate model. First, we present a suite of PM predictability experiments with start dates spanning the calendar year, which are used to quantify the potential regional SIE prediction skill of this system. Second, we assess the system’s OP prediction skill for detrended regional SIE using a suite of retrospective initialized seasonal forecasts spanning 1981–2016. In nearly all Arctic regions and for all target months, we find a substantial skill gap between PM and OP predictions of regional SIE. The PM experiments reveal that regional winter SIE is potentially predictable at lead times beyond 12 months, substantially longer than the skill of their OP counterparts. Both the OP and PM predictions display a spring prediction skill barrier for regional summer SIE forecasts, indicating a fundamental predictability limit for summer regional predictions. We find that a similar barrier exists for pan-Arctic sea–ice volume predictions, but is not present for predictions of pan-Arctic SIE. The skill gap identified in this work indicates a promising potential for future improvements in regional SIE predictions.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. Anderson JL (2001) An ensemble adjustment Kalman filter for data assimilation. Mon Weather Rev 129(12):2884–2903

    Article  Google Scholar 

  2. Bitz C, Holland M, Weaver A, Eby M (2001) Simulating the ice-thickness distribution in a coupled climate model. J Geophys Res Oceans 106(C2):2441–2463

    Article  Google Scholar 

  3. Bitz C, Roe G (2004) A mechanism for the high rate of sea ice thinning in the Arctic Ocean. J Clim 17(18):3623–3632

    Article  Google Scholar 

  4. Blanchard-Wrigglesworth E, Armour KC, Bitz CM, DeWeaver E (2011) Persistence and inherent predictability of Arctic sea ice in a GCM ensemble and observations. J Clim 24:231–250

    Article  Google Scholar 

  5. Blanchard-Wrigglesworth E, Barthélemy A, Chevallier M, Cullather R, Fučkar N, Massonnet F, Posey P, Wang W, Zhang J, Ardilouze C et al (2017) Multi-model seasonal forecast of Arctic sea–ice: forecast uncertainty at pan-Arctic and regional scales. Clim Dyn 49(4):1399–1410

    Article  Google Scholar 

  6. Blanchard-Wrigglesworth E, Bitz C, Holland M (2011) Influence of initial conditions and climate forcing on predicting Arctic sea ice. Geophys Res Lett 38(18)

  7. Blanchard-Wrigglesworth E, Cullather R, Wang W, Zhang J, Bitz C (2015) Model forecast skill and sensitivity to initial conditions in the seasonal Sea Ice Outlook. Geophys Res Lett 42(19):8042–8048

    Article  Google Scholar 

  8. Bretherton CS, Widmann M, Dymnikov VP, Wallace JM, Bladé I (1999) The effective number of spatial degrees of freedom of a time-varying field. J Clim 12(7):1990–2009

    Article  Google Scholar 

  9. Bushuk M, Giannakis D (2015) Sea-ice reemergence in a model hierarchy. Geophys Res Lett 42:5337–5345

    Article  Google Scholar 

  10. Bushuk M, Giannakis D (2017) The seasonality and interannual variability of Arctic sea–ice reemergence. J Clim 30:4657–4676

    Article  Google Scholar 

  11. Bushuk M, Giannakis D, Majda AJ (2015) Arctic sea–ice reemergence: the role of large-scale oceanic and atmospheric variability. J Clim 28:5477–5509

    Article  Google Scholar 

  12. Bushuk M, Msadek R, Winton M, Vecchi G, Gudgel R, Rosati A, Yang X (2017) Skillful regional prediction of Arctic sea ice on seasonal timescales. Geophys Res Lett 44

  13. Bushuk M, Msadek R, Winton M, Vecchi G, Gudgel R, Rosati A, Yang X (2017) Summer enhancement of Arctic sea–ice volume anomalies in the September-ice zone. J Clim 30:2341–2362

    Article  Google Scholar 

  14. Cavalieri DJ, Parkinson CL, Gloersen P, Zwally HJ (1996) Sea ice concentrations from Nimbus-7 SMMR and DMSP SSM/I-SSMIS Passive Microwave Data, Version 1. NASA DAAC at the Natl. Snow and Ice Data Cent. https://doi.org/10.5067/8GQ8LZQVL0VL

  15. Chen Z, Liu J, Song M, Yang Q, Xu S (2017) Impacts of assimilating satellite sea ice concentration and thickness on Arctic sea ice prediction in the NCEP Climate Forecast System. J Clim 30(21):8429–8446

    Article  Google Scholar 

  16. Cheng W, Blanchard-Wrigglesworth E, Bitz CM, Ladd C, Stabeno PJ (2016) Diagnostic sea ice predictability in the pan-Arctic and US Arctic regional seas. Geophys Res Lett 43(22)

  17. Chevallier M, Salas y Mélia D (2012) The role of sea ice thickness distribution in the Arctic sea ice potential predictability: a diagnostic approach with a coupled GCM. J Clim 25(8):3025–3038

    Article  Google Scholar 

  18. Chevallier M, Salas y Mélia D, Voldoire A, Déqué M, Garric G (2013) Seasonal forecasts of the pan-Arctic sea ice extent using a GCM-based seasonal prediction system. J Clim 26(16):6092–6104

    Article  Google Scholar 

  19. Collins M (2002) Climate predictability on interannual to decadal time scales: the initial value problem. Clim Dyn 19:671–692

    Article  Google Scholar 

  20. Collow TW, Wang W, Kumar A, Zhang J (2015) Improving Arctic sea ice prediction using PIOMAS initial sea ice thickness in a coupled ocean–atmosphere model. Mon Weather Rev 143(11):4618–4630

    Article  Google Scholar 

  21. Day J, Tietsche S, Hawkins E (2014) Pan-Arctic and regional sea ice predictability: initialization month dependence. J Clim 27(12):4371–4390

    Article  Google Scholar 

  22. Day JJ, Goessling HF, Hurlin WJ, Keeley SP (2016) The Arctic predictability and prediction on seasonal-to-interannual timescales (APPOSITE) data set version 1. Geosci Model Dev 9(6):2255

    Article  Google Scholar 

  23. Delworth TL, Broccoli AJ, Rosati A, Stouffer RJ, Balaji V, Beesley JA, Cooke WF, Dixon KW, Dunne J, Dunne K et al (2006) GFDL’s CM2 global coupled climate models. Part I: Formulation and simulation characteristics. J Clim 19(5):643–674

    Article  Google Scholar 

  24. Delworth TL, Rosati A, Anderson W, Adcroft AJ, Balaji V, Benson R, Dixon K, Griffies SM, Lee HC, Pacanowski RC et al (2012) Simulated climate and climate change in the GFDL CM2. 5 high-resolution coupled climate model. J Clim 25(8):2755–2781

    Article  Google Scholar 

  25. Deser C, Magnusdottir G, Saravanan R, Phillips A (2004) The effects of North Atlantic SST and sea ice anomalies on the winter circulation in CCM3. Part II: Direct and indirect components of the response. J Clim 17(5):877–889

    Article  Google Scholar 

  26. Deser C, Walsh JE, Timlin MS (2000) Arctic sea ice variability in the context of recent atmospheric circulation trends. J Clim 13:617–633

    Article  Google Scholar 

  27. Dirkson A, Merryfield WJ, Monahan A (2017) Impacts of sea ice thickness initialization on seasonal Arctic sea ice predictions. J Clim 30(3):1001–1017

    Article  Google Scholar 

  28. Drobot SD (2007) Using remote sensing data to develop seasonal outlooks for Arctic regional sea–ice minimum extent. Remote Sens Environ 111(2–3):136–147

    Article  Google Scholar 

  29. Drobot SD, Maslanik JA, Fowler C (2006) A long-range forecast of Arctic summer sea–ice minimum extent. Geophys Res Lett 33(10)

  30. Germe A, Chevallier M, y Mélia DS, Sanchez-Gomez E, Cassou C (2014) Interannual predictability of Arctic sea ice in a global climate model: regional contrasts and temporal evolution. Clim Dyn 43(9-10):2519–2538

  31. Griffies S (2012) Elements of the modular ocean model (MOM), GFDL Ocean Group Technical Report. Tech. Rep. No. 7, NOAA/Geophysical Fluid Dynamics Laboratory

  32. Griffies SM, Winton M, Donner LJ, Horowitz LW, Downes SM, Farneti R, Gnanadesikan A, Hurlin WJ, Lee HC, Liang Z et al (2011) The GFDL CM3 coupled climate model: characteristics of the ocean and sea ice simulations. J Clim 24(13):3520–3544

    Article  Google Scholar 

  33. Guemas V, Chevallier M, Dqu M, Bellprat O, Doblas-Reyes F (2016) Impact of sea ice initialisation on sea ice and atmosphere prediction skill on seasonal timescales. Geophys Res Lett 43(8):3889–3896

    Article  Google Scholar 

  34. Hawkins E, Tietsche S, Day JJ, Melia N, Haines K, Keeley S (2016) Aspects of designing and evaluating seasonal-to-interannual Arctic sea–ice prediction systems. Q J R Meteorol Soc 142(695):672–683

    Article  Google Scholar 

  35. Holland MM, Bailey DA, Vavrus S (2011) Inherent sea ice predictability in the rapidly changing Arctic environment of the Community Climate System Model, version 3. Clim Dyn 36(7–8):1239–1253

    Article  Google Scholar 

  36. Holland, M.M., Stroeve, J.: Changing seasonal sea ice predictor relationships in a changing arctic climate. Geophys Res Lett 38(18)

  37. Hunke E, Dukowicz J (1997) An elastic-viscous-plastic model for sea ice dynamics. J Phys Oceanogr 27(9):1849–1867

    Article  Google Scholar 

  38. Jia L, Yang X, Vecchi G, Gudgel R, Delworth T, Fueglistaler S, Lin P, Scaife AA, Underwood S, Lin SJ (2017) Seasonal prediction skill of northern extratropical surface temperature driven by the stratosphere. J Clim 30(1):4463–4475

    Article  Google Scholar 

  39. Jia L, Yang X, Vecchi GA, Gudgel RG, Delworth TL, Rosati A, Stern WF, Wittenberg AT, Krishnamurthy L, Zhang S et al (2015) Improved seasonal prediction of temperature and precipitation over land in a high-resolution GFDL climate model. J Clim 28(5):2044–2062

    Article  Google Scholar 

  40. Johnson C, Bowler N (2009) On the reliability and calibration of ensemble forecasts. Mon Weather Rev 137(5):1717–1720

    Article  Google Scholar 

  41. Jolliffe IT, Stephenson DB (2012) Forecast verification: a practitioner’s guide in atmospheric science, 2nd edn. Wiley

  42. Jung T, Gordon ND, Bauer P, Bromwich DH, Chevallier M, Day JJ, Dawson J, Doblas-Reyes F, Fairall C, Goessling HF et al (2016) Advancing polar prediction capabilities on daily to seasonal time scales. Bull Am Meteorol Soc. https://doi.org/10.1175/BAMS-D-14-00246.1

  43. Kapsch ML, Graversen RG, Economou T, Tjernström M (2014) The importance of spring atmospheric conditions for predictions of the Arctic summer sea ice extent. Geophys Res Lett 41(14):5288–5296

    Article  Google Scholar 

  44. Kauker F, Kaminski T, Karcher M, Giering R, Gerdes R, Voßbeck M (2009) Adjoint analysis of the 2007 all time Arctic sea–ice minimum. Geophys Res Lett 36(3)

  45. Koenigk T, Mikolajewicz U (2009) Seasonal to interannual climate predictability in mid and high northern latitudes in a global coupled model. Clim Dyn 32(6):783–798

    Article  Google Scholar 

  46. Krikken F, Schmeits M, Vlot W, Guemas V, Hazeleger W (2016) Skill improvement of dynamical seasonal Arctic sea ice forecasts. Geophys Res Lett

  47. Kumar A, Peng P, Chen M (2014) Is there a relationship between potential and actual skill? Mon Weather Rev 142(6):2220–2227

    Article  Google Scholar 

  48. Leutbecher M, Palmer TN (2008) Ensemble forecasting. J Comput Phys 227(7):3515–3539

    Article  Google Scholar 

  49. Lin SJ (2004) A vertically Lagrangian finite-volume dynamical core for global models. Mon Weather Rev 132(10):2293–2307

    Article  Google Scholar 

  50. Lindsay R, Zhang J, Schweiger A, Steele M (2008) Seasonal predictions of ice extent in the Arctic Ocean. J Geophys Res Oceans 113(C2)

  51. Martinson DG (1990) Evolution of the Southern Ocean winter mixed layer and sea ice: open ocean deepwater formation and ventilation. J Geophys Res Oceans 95(C7):11641–11654

    Article  Google Scholar 

  52. Merryfield W, Lee WS, Wang W, Chen M, Kumar A (2013) Multi-system seasonal predictions of Arctic sea ice. Geophys Res Lett 40(8):1551–1556

    Article  Google Scholar 

  53. Milly PC, Malyshev SL, Shevliakova E, Dunne KA, Findell KL, Gleeson T, Liang Z, Phillipps P, Stouffer RJ, Swenson S (2014) An enhanced model of land water and energy for global hydrologic and earth-system studies. J Hydrometeorol 15(5):1739–1761

    Article  Google Scholar 

  54. Msadek R, Vecchi G, Winton M, Gudgel R (2014) Importance of initial conditions in seasonal predictions of Arctic sea ice extent. Geophys Res Lett 41(14):5208–5215

    Article  Google Scholar 

  55. Murakami H, Vecchi GA, Delworth TL, Wittenberg AT, Underwood S, Gudgel R, Yang X, Jia L, Zeng F, Paffendorf K et al (2017) Dominant role of subtropical pacific warming in extreme Eastern Pacific hurricane seasons: 2015 and the future. J Clim 30(1):243–264

    Article  Google Scholar 

  56. Murphy AH (1988) Skill scores based on the mean square error and their relationships to the correlation coefficient. Mon Weather Rev 116(12):2417–2424

    Article  Google Scholar 

  57. Owens WB, Lemke P (1990) Sensitivity studies with a sea ice-mixed layer-pycnocline model in the Weddell sea. J Geophys Res Oceans (1978–2012) 95(C6):9527–9538

  58. Palmer T, Buizza R, Hagedorn R, Lawrence A, Leutbecher M, Smith L (2006) Ensemble prediction: a pedagogical perspective. ECMWF Newslett 106:10–17

    Google Scholar 

  59. Peterson KA, Arribas A, Hewitt H, Keen A, Lea D, McLaren A (2015) Assessing the forecast skill of Arctic sea ice extent in the GloSea4 seasonal prediction system. Clim Dyn 44(1–2):147–162

    Article  Google Scholar 

  60. Petty AA, Schröder D, Stroeve J, Markus T, Miller J, Kurtz N, Feltham D, Flocco D (2017) Skillful spring forecasts of September Arctic sea ice extent using passive microwave sea ice observations. Earth’s Future 5(2):254–263

    Article  Google Scholar 

  61. Pohlmann H, Botzet M, Latif M, Roesch A, Wild M, Tschuck P (2004) Estimating the decadal predictability of a coupled AOGCM. J Clim 17(22):4463–4472

    Article  Google Scholar 

  62. Putman WM, Lin SJ (2007) Finite-volume transport on various cubed-sphere grids. J Comput Phys 227(1):55–78

    Article  Google Scholar 

  63. Schröder D, Feltham DL, Flocco D, Tsamados M (2014) September Arctic sea-ice minimum predicted by spring melt-pond fraction. Nat Clim Change

  64. Schweiger A, Lindsay R, Zhang J, Steele M, Stern H, Kwok R (2011) Uncertainty in modeled Arctic sea ice volume. J Geophys Res Oceans 116(C8)

  65. Sigmond M, Fyfe J, Flato G, Kharin V, Merryfield W (2013) Seasonal forecast skill of Arctic sea ice area in a dynamical forecast system. Geophys Res Lett 40(3):529–534

    Article  Google Scholar 

  66. Sigmond M, Reader M, Flato G, Merryfield W, Tivy A (2016) Skillful seasonal forecasts of Arctic sea ice retreat and advance dates in a dynamical forecast system. Geophys Res Lett 43

  67. Stock CA, Pegion K, Vecchi GA, Alexander MA, Tommasi D, Bond NA, Fratantoni PS, Gudgel RG, Kristiansen T, OBrien TD et al (2015) Seasonal sea surface temperature anomaly prediction for coastal ecosystems. Prog Oceanogr 137:219–236

    Article  Google Scholar 

  68. Stroeve J, Hamilton LC, Bitz CM, Blanchard-Wrigglesworth E (2014) Predicting September sea ice: ensemble skill of the SEARCH sea ice outlook 2008–2013. Geophys Res Lett 41(7):2411–2418

    Article  Google Scholar 

  69. Sun L, Deser C, Tomas RA (2015) Mechanisms of stratospheric and tropospheric circulation response to projected Arctic sea ice loss. J Clim 28(19):7824–7845

    Article  Google Scholar 

  70. Tietsche S, Day J, Guemas V, Hurlin W, Keeley S, Matei D, Msadek R, Collins M, Hawkins E (2014) Seasonal to interannual Arctic sea ice predictability in current global climate models. Geophys Res Lett 41(3):1035–1043

    Article  Google Scholar 

  71. Tivy A, Howell SE, Alt B, Yackel JJ, Carrieres T (2011) Origins and levels of seasonal forecast skill for sea ice in Hudson Bay using Canonical Correlation Analysis. J Clim 24(5):1378–1395

    Article  Google Scholar 

  72. Vecchi GA, Delworth T, Gudgel R, Kapnick S, Rosati A, Wittenberg AT, Zeng F, Anderson W, Balaji V, Dixon K et al (2014) On the seasonal forecasting of regional tropical cyclone activity. J Clim 27(21):7994–8016

    Article  Google Scholar 

  73. Wang L, Ting M, Kushner P (2017) A robust empirical seasonal prediction of winter NAO and surface climate. Sci Rep 7(1):279

    Article  Google Scholar 

  74. Wang L, Yuan X, Ting M, Li C (2016) Predicting summer Arctic sea ice concentration intraseasonal variability using a vector autoregressive model*. J Clim 29(4):1529–1543

    Article  Google Scholar 

  75. Wang W, Chen M, Kumar A (2013) Seasonal prediction of Arctic sea ice extent from a coupled dynamical forecast system. Mon Weather Rev 141(4):1375–1394

    Article  Google Scholar 

  76. Weigel AP, Liniger MA, Appenzeller C (2009) Seasonal ensemble forecasts: are recalibrated single models better than multimodels? Mon Weather Rev 137(4):1460–1479

    Article  Google Scholar 

  77. Williams J, Tremblay B, Newton R, Allard R (2016) Dynamic preconditioning of the minimum September sea–ice extent. J Clim 29(16):5879–5891

    Article  Google Scholar 

  78. Winton M (2000) A reformulated three-layer sea ice model. J Atmos Oceanic Technol 17(4):525–531

    Article  Google Scholar 

  79. Yang X, Vecchi GA, Gudgel RG, Delworth TL, Zhang S, Rosati A, Jia L, Stern WF, Wittenberg AT, Kapnick S et al (2015) Seasonal predictability of extratropical storm tracks in GFDLs high-resolution climate prediction model. J Clim 28(9):3592–3611

    Article  Google Scholar 

  80. Yeager SG, Karspeck AR, Danabasoglu G (2015) Predicted slowdown in the rate of Atlantic sea ice loss. Geophys Res Lett 42(24)

  81. Yuan X, Chen D, Li C, Wang L, Wang W (2016) Arctic sea ice seasonal prediction by a linear markov model. J Clim 29(22):8151–8173

    Article  Google Scholar 

  82. Zhang J, Rothrock D (2003) Modeling global sea ice with a thickness and enthalpy distribution model in generalized curvilinear coordinates. Mon Weather Rev 131(5):845–861

    Article  Google Scholar 

  83. Zhang S, Harrison M, Rosati A, Wittenberg A (2007) System design and evaluation of coupled ensemble data assimilation for global oceanic climate studies. Mon Weather Rev 135(10):3541–3564

    Article  Google Scholar 

  84. Zhang S, Rosati A (2010) An inflated ensemble filter for ocean data assimilation with a biased coupled GCM. Mon Weather Rev 138(10):3905–3931

    Article  Google Scholar 

Download references

Acknowledgements

This paper is dedicated to Walter Bushuk. We thank two anonymous reviewers for constructive comments which improved the manuscript. We also acknowledge Olga Sergienko and Hiroyuki Murakami for comments on a preliminary version of the manuscript. We thank Seth Underwood, Bill Hurlin, and Chris Blanton for assistance in setting up the model experiments. M. Bushuk was supported by NOAA’s Climate Program Office, Climate Variability and Predictability Program (Award GC15-504).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mitchell Bushuk.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 747 KB)

Appendix

Appendix

Reliability condition for ensemble forecasts

Claim The PM MSE given by Eq. (8) satisfies the necessary condition for forecast reliability:

$$\begin{aligned} MSE(\tau )=\frac{N}{N-1}\sigma _e^2(\tau ). \end{aligned}$$
(18)

Proof The mean intra-ensemble variance, \(\sigma _e^2\), is given by

$$\begin{aligned} \displaystyle \sigma _e^2(\tau )= \frac{1}{M}\sum _{j=1}^M \frac{1}{N-1} \sum _{i=1}^N \Big (\langle \mathbf {x}_{j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2, \end{aligned}$$
(19)

where \(\langle \mathbf {x}_{j}(\tau ) \rangle\) is the ensemble mean of the jth ensemble. The MSE is given by

$$\begin{aligned} \displaystyle MSE(\tau )= \frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{MN}. \end{aligned}$$
(20)

First, we note a relation between the ensemble mean \(\langle \mathbf {x}_{j}(\tau ) \rangle\) and the ensemble mean with the ith member removed \(\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle\). These ensemble means are defined respectively as

$$\begin{aligned} \langle \mathbf {x}_{j}(\tau ) \rangle =\frac{1}{N} \sum _{k=1}^N x_{kj}(\tau ), \end{aligned}$$
(21)

and

$$\begin{aligned} \langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle = \frac{1}{N-1} \sum _{k \ne i}^N x_{kj}(\tau ), \end{aligned}$$
(22)

and are related by:

$$\begin{aligned} \langle \mathbf {x}_{j}(\tau ) \rangle =\frac{1}{N} \sum _{k=1}^N x_{kj}(\tau )=\frac{x_{ij}(\tau )}{N}+\frac{1}{N}\sum _{k \ne i}^N x_{kj}(\tau )=\frac{x_{ij}(\tau )}{N}+\frac{N-1}{N}\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle . \end{aligned}$$
(23)

Therefore,

$$\begin{aligned} \sigma _e^2(\tau )&= \frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$
(24)
$$\begin{aligned}&=\frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\frac{1}{N}x_{ij}(\tau )+\frac{N-1}{N}\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$
(25)
$$\begin{aligned}&=\frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\frac{N-1}{N}\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - \frac{N-1}{N}x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$
(26)
$$\begin{aligned}&=\bigg ( \frac{N-1}{N} \bigg )^2 \frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{M(N-1)}\end{aligned}$$
(27)
$$\begin{aligned}&=\frac{N-1}{N}\frac{\sum _{j=1}^M \sum _{i=1}^N \Big (\langle \mathbf {x}_{\hat{i}j}(\tau ) \rangle - x_{ij}(\tau ) \Big )^2}{MN}\end{aligned}$$
(28)
$$\begin{aligned}&=\frac{N-1}{N}MSE(\tau ). \end{aligned}$$
(29)

Relation of perfect model skill metrics to other metrics

PPP

A commonly used PM skill metric is the potential prognostic predictability (PPP, Pohlmann et al. 2004), which compares the ensemble variance, \(\sigma _e^2(\tau )\), to the climatological variance, \(\sigma _c^2\). The PPP is defined as

$$\begin{aligned} PPP(\tau )=1-\frac{\sigma _e^2(\tau )}{\sigma _c^2}, \end{aligned}$$
(30)

which has a similar form to the MSSS defined in Eq. (11). Since \(MSE=\frac{N}{N-1}\sigma _e^2\), for any finite N, \(MSSS < PPP\) and \(MSSS \rightarrow PPP\) as \(N \rightarrow \infty\). For most typical values of N, the PPP and MSSS will be quite similar and share the same qualitative interpretations. However, we believe that the MSSS metric provides a more natural comparison with the MSSS metric used in OP predictions. In the PPP formulation, the ensemble mean \(\langle x_j \rangle\) is used to predict a given truth member \(x_{ij}\). This implies that the prediction has knowledge of the observed value, since the \(x_{ij}\) truth member is included in the ensemble mean computation. This is an undesirable property for a skill metric, and will tend to bias skill scores high. The MSSS does not suffer from this issue, as only non-truth members are used to predict a given truth member.

RMSE

In the PM MSE formula given in Eq. (8), we have used the \((N-1)\)-member ensemble mean to predict a given truth member. In general, we could use an E-member ensemble mean to make this prediction, where \(1 \le E \le N-1\). It can be shown that an MSE based on an E-member ensemble mean satisfies \(MSE=\frac{E+1}{E}\sigma _e^2\), where the proof uses the Central Limit Theorem and follows the same approach as that of Jolliffe and Stephenson (2012). The formula in 6.1 is the special case when \(E=N-1\). The PM RMSE definition of Collins (2002), uses 1-member ensembles to predict a given truth member, and therefore satisfies \(MSE=2\sigma _e^2\). At long lead times, the PM RMSE of Collins (2002) converges to \(\sqrt{2} \sigma _c\) (note that this is strictly true only if the normalization of \(MN(N-1)-1\) used in Collins (2002) is replaced with \(MN(N-1)\)).

This factor of \(\sqrt{2}\) is a potential source of confusion, since in the PM literature a “no skill” forecast has \(RMSE=\sqrt{2}\sigma _c\), whereas in the OP literature a “no skill” (climatological) forecast has an RMSE of \(\sigma _c\). This can lead to confusion when quoting PM RMSE in physical units, or when comparing PM and OP RMSE values (e.g. as done in Blanchard-Wrigglesworth et al. 2015; Tietsche et al. 2014). In particular, the RMSE values obtained via the formula of Collins (2002) are too large, since they do not benefit from ensemble averaging. If ensemble means are used for the PM prediction, this issue is greatly ameliorated, since the PM RMSE values converge to \(\sqrt{\frac{N}{N-1}}\sigma _c\), allowing for cleaner comparison with OP predictions.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bushuk, M., Msadek, R., Winton, M. et al. Regional Arctic sea–ice prediction: potential versus operational seasonal forecast skill. Clim Dyn 52, 2721–2743 (2019). https://doi.org/10.1007/s00382-018-4288-y

Download citation

Keywords

  • Sea ice
  • Seasonal predictability
  • Arctic