ECMWF seasonal forecast system 3 and its prediction of sea surface temperature

Abstract

The latest operational version of the ECMWF seasonal forecasting system is described. It shows noticeably improved skill for sea surface temperature (SST) prediction compared with previous versions, particularly with respect to El Nino related variability. Substantial skill is shown for lead times up to 1 year, although at this range the spread in the ensemble forecast implies a loss of predictability large enough to account for most of the forecast error variance, suggesting only moderate scope for improving long range El Nino forecasts. At shorter ranges, particularly 3–6 months, skill is still substantially below the model-estimated predictability limit. SST forecast skill is higher for more recent periods than earlier ones. Analysis shows that although various factors can affect scores in particular periods, the improvement from 1994 onwards seems to be robust, and is most plausibly due to improvements in the observing system made at that time. The improvement in forecast skill is most evident for 3-month forecasts starting in February, where predictions of NINO3.4 SST from 1994 to present have been almost without fault. It is argued that in situations where the impact of model error is small, the value of improved observational data can be seen most clearly. Significant skill is also shown in the equatorial Indian Ocean, although predictive skill in parts of the tropical Atlantic are relatively poor. SST forecast errors can be especially high in the Southern Ocean.

Appendix

If a specific season and lead time have been selected because they give the best results, it is necessary to ensure that what is seen is not just a sampling effect. As a simple combinatorial test, define a “poor” forecast as one in which the mean absolute error for the first 3 months exceeds 0.15° C. If we consider all the S3 forecasts from 1961 to 2009 present (49 years), a total of 23 of them are poor, all in the years prior to 1994. If it is assumed, as a null hypothesis, that the poor forecasts are equally likely to occur in any year, the chances of the last 16 years including zero occurrences of the 23 poor forecasts which exist in the sample is (33/49)*(32/48)* …(11/27) or (33!/49!)*(26!/10!) = 0.0000016. Using this method of looking for a change in skill (defining thresholds, counting poor forecasts, and applying combinatorial tests to get a p value), the following selections have been made: the best of 12 start months, the best of 7 possible forecast ranges, and the best of (say) 5 plausible thresholds for the definition of a poor forecast. This multiplies to 420 possible tests for a change in skill, of which this is the highest scoring. Under the null hypothesis, the chances of such a high score are only 0.0007.