Seasonal prediction skill of ECMWF System 4 and NCEP CFSv2 retrospective forecast for the Northern Hemisphere Winter
The seasonal prediction skill for the Northern Hemisphere winter is assessed using retrospective predictions (1982–2010) from the ECMWF System 4 (Sys4) and National Center for Environmental Prediction (NCEP) CFS version 2 (CFSv2) coupled atmosphere–ocean seasonal climate prediction systems. Sys4 shows a cold bias in the equatorial Pacific but a warm bias is found in the North Pacific and part of the North Atlantic. The CFSv2 has strong warm bias from the cold tongue region of the eastern Pacific to the equatorial central Pacific and cold bias in broad areas over the North Pacific and the North Atlantic. A cold bias in the Southern Hemisphere is common in both reforecasts. In addition, excessive precipitation is found in the equatorial Pacific, the equatorial Indian Ocean and the western Pacific in Sys4, and in the South Pacific, the southern Indian Ocean and the western Pacific in CFSv2. A dry bias is found for both modeling systems over South America and northern Australia. The mean prediction skill of 2 meter temperature (2mT) and precipitation anomalies are greater over the tropics than the extra-tropics and also greater over ocean than land. The prediction skill of tropical 2mT and precipitation is greater in strong El Nino Southern Oscillation (ENSO) winters than in weak ENSO winters. Both models predict the year-to-year ENSO variation quite accurately, although sea surface temperature trend bias in CFSv2 over the tropical Pacific results in lower prediction skill for the CFSv2 relative to the Sys4. Both models capture the main ENSO teleconnection pattern of strong anomalies over the tropics, the North Pacific and the North America. However, both models have difficulty in forecasting the year-to-year winter temperature variability over the US and northern Europe.
Despite the chaotic internal dynamics of the atmosphere, the time average of atmospheric variables is predictable to some degree due to those components that have slow variations on time scales from months to seasons. The socioeconomic importance of accurate seasonal climate prediction has motivated development of better seasonal prediction systems. Recently, the development of coupled ocean–atmosphere dynamical model prediction systems has provided important advances in seasonal predictability (Kumar et al. 2005; Wang et al. 2005a; Kug et al. 2008). Several international projects have been undertaken to compare coupled climate predictions, including the Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER) (Palmer et al. 2004) and Asia–Pacific economic cooperation climate center (APCC)/climate prediction and its application to society (CliPAS) projects (Wang et al. 2009). Seasonal prediction skill and the model performance have been examined based on retrospective predictions of DEMETER and APCC/CliPAS (Jin et al. 2008; Kim et al. 2008; Kug et al. 2008; Wang et al. 2009; Lee et al. 2010).
Operational coupled seasonal forecast systems include Climate Forecast System from the National Center for Environmental Prediction (NCEP CFS) (Saha et al. 2006), the Australian POAMA (Wang et al. 2001), European Centre for Medium-Range Weather Forecasts (ECMWF), UK Meteorological Office and Meteo-France (Palmer et al. 2004). Operational climate forecast centers are now updating their seasonal prediction systems with improved physics and increased resolution. This study focuses on the ECMWF and NCEP CFS seasonal forecasting systems. ECMWF has been operating a seasonal forecast system since 1997 and the operational system, known as System 3, was introduced in March 2007. System 3 shows greater prediction skill for the sea surface temperature (SST) in the eastern Pacific and equatorial Indian Ocean than previous ECMWF operational systems (systems 1 and 2) (Stockdale et al. 2011). The ECMWF has now upgraded its operational seasonal forecasts from System 3 to System 4 with the later version being operational since late 2011. In the upgrade, it utilizes the use of the most recent atmospheric model version, higher resolution forecasts with a higher top of the atmosphere, more ensemble members and a larger reforecast data set (Molteni et al. 2011).
The NCEP CFS has been making coupled ocean–atmosphere forecasts since 2004. Skill of the CFS model has been examined in simulating and predicting El Nino-Southern Oscillation (ENSO) variability (Wang et al. 2005b), Asian-Australian/Indian monsoon (Yang et al. 2008; Wang et al. 2008; Pattanaik and Kumar 2010) and climatic variation in the US (Yang et al. 2009). The NCEP CFS version 2 (CFSv2, http://cfs.ncep.noaa.gov/cfsv2.info/) represents a substantial change to all aspects of the forecast system including model components, data assimilation system and ensemble configuration. The MJO simulation shows improvement in CFSv2 owing to a positive response to upgrades in the initial state compared to CFSv1 (Weaver et al. 2011).
The seasonal predictions of individual coupled seasonal forecast systems has been analyzed separately for various target of seasons, different time periods and regions with wide range of variables using regression and correlation analysis, composite analysis and principal component analysis (Wang et al. 2005b; Saha et al. 2006; Yang et al. 2008, 2009; Lee et al. 2010; Tompkins and Feudale 2010; Wang et al. 2010; Stockdale et al. 2011). However, the ECMWF and NCEP CFS seasonal forecast systems have not been compared with the same validation matrix. The choice of one model over the other, or the use of both models in a multi-model ensemble requires information that compares the predictions of both models and the determination of the bias of each model. We compare the simulation ability and seasonal prediction skill of the two systems using the same validation matrix. The results of this comparison may be useful for the community as a benchmark for future generations of seasonal prediction systems, and may provide valuable information for forecast providers and decision makers that use seasonal forecast products.
In this paper, we focus on the Northern Hemisphere (NH) winter when the magnitude of ENSO anomalies and teleconnections to the extratropics can be particularly high (Peng et al. 2000). A companion paper for the NH summer has also been prepared. In particular, this study addresses how well the ECMWF System 4 and NCEP CFSv2 simulate the spatio-temporal climate variability for the NH winter. Section 2 introduces details of reforecast and observational data used in the present study. Section 3 examines the simulated climates and the seasonal prediction skill of surface temperature and precipitation. Section 4 examines the prediction of ENSO whilst Sect. 5 focuses on the prediction of the winter teleconnection patterns. A summary of the results and a general discussion are provided in Sect. 6.
2 Retrospective forecasts and observation data
The ECMWF System 4 (hereafter Sys4) and the NCEP CFSv2 (hereafter CFSv2) are fully coupled general circulation models (GCMs) that provide operational seasonal predictions. Both systems provide reforecast simulations for the purpose of evaluating and calibrating the model simulations. The ECMWF System 4 seasonal reforecasts, commencing in 1981, include 15 member ensembles consist of 7 month simulations initialized on the 1st day of every month. The atmospheric initial conditions come from ERA Interim reanalysis for the period 1981–2010. A new ocean model (NEMO) and ocean data assimilation system (NEMOVAR) is implemented, improving the mean state and SST forecast skill in the East Pacific and Tropical Atlantic oceans. Details for the ECMWF System 4 can be found in Molteni et al. (2011) and http://www.ecmwf.int/products/forecasts/seasonal/documentation/system4. The NCEP CFSv2 is an upgraded version of CFSv1 (Saha et al. 2006). CFSv2 produces a set of 9-month reforecast initiated from every 5th day with four ensemble members for the period 1982–2010. Initial conditions for the atmosphere and ocean come from NCEP Climate Forecast System Reanalysis (CFSR, Saha et al. 2010).
As prediction skill depends strongly on the ensemble size (Kumar and Hoerling 2000), we match the ensemble size, as well as lead-time for the comparison of the Sys4 and CFSv2 forecasts. The Sys4 reforecast consists of 15 ensembles initialized on November 1st and for CFSv2 16 member ensembles initialized from October 23rd to November 7th from the target variables and those from December to February (DJF), which we define as the NH winter. For example, 1997 winter is an average of December 1997 and January and February of 1998. A total of 28 boreal winters from 1982/1983 to 2009/2010 are examined in this study.
For the forecast evaluation, SST data is obtained from monthly NOAA Optimum Interpolation (OI) SST V2 (Reynolds et al. 2002). The air temperature at 2 meter (2mT), mean sea level pressure (SLP), and geopotential height at 500 hPa data are obtained from the CFS reanalysis and ERA-Interim reanalysis products (Berrisford et al. 2009) from 1981. The CFSR is a major improvement over the first generation NCEP reanalyses (NCEP R1 and R2) as it is the product of a coupled ocean–atmosphere–land system at higher spatial resolution (Higgins et al. 2010; Saha et al. 2010). ERA-Interim (hereafter ERA) is the latest global atmospheric reanalysis produced by the ECMWF and shows improvements on ERA-40 (Uppala et al. 2005) due to the use of four-dimensional data assimilation (4D-Var), higher horizontal resolution, and bias correction of satellite radiance data (Dee and Uppala 2009; Dee et al. 2011). Global Precipitation Climatology Project (GPCP) version 2.1 combined precipitation dataset (Adler et al. 2003) is used as the validation dataset. It has to be noted that there are substantial differences in trends across different reanalyses (Ebisuzaki and Zhang 2011; Zhang et al. 2012).
3 Seasonal prediction skill
Here, we examine the capability of the systems in simulating the spatial patterns of seasonal climatology and their predictive skill of seasonal anomalies. The prediction skill is calculated as an anomaly correlation based on the ensemble mean of each seasonal prediction and the target observations.
Figure 1c, d shows the bias for winter mean precipitation (PRCP). The spatial pattern of the precipitation climatology in both Sys4 and CFSv2 are similar to the observation but include systematic biases. In Sys4, excessive precipitation is found along the Inter-Tropical Convergence Zone (ITCZ), equatorial Indian Ocean and western Pacific. In CFSv2, a strong wet bias is found along the South Pacific Convergence Zone (SPCZ) and the southern Indian Ocean as well as the western Pacific and dry biases are shown over the South America and the northern Australia consistent with Weaver et al. (2011). Wet bias in East Asia and the equatorial Atlantic is common in both systems.
4 ENSO prediction
As described above, the amplitude of ENSO dominates the winter seasonal prediction skill. Jin et al. (2008) examined the current status of ENSO prediction using retrospective forecasts made with ten different coupled GCMs from DEMETER and CliPAS/APCC model sets and found that the ENSO prediction skill in the state-of-the-art dynamical predictions depends on the ENSO phase and amplitude. Generally, dynamical models tend to have better prediction skill when initialized at NH winter than spring due to the ‘spring predictability barrier’ (Webster and Yang 1992; Webster 1995; Torrence and Webster 1998; Jin et al. 2008; Kim et al. 2009; Hendon et al. 2009). This study focuses on the boreal winter prediction when the initial condition already contains a strong ENSO signal. The ECMWF forecast model has been found to be better than statistical models at forecasting ENSO events (Van Oldenborgh et al. 2005) and NCEP CFS is shown to be competitive with other statistical models in predicting tropical SST variability (Saha et al. 2006). Here we compare ECMWF System 4 and CFSv2 in terms of winter ENSO prediction.
5 Teleconnection patterns in the extratropics
5.1 ENSO teleconnection
The composite patterns in CFSR are similar to the ERA analyses (not shown). The conventional El Nino pattern is apparent, with warm/wet anomaly across the equatorial central to eastern Pacific produced by the shifting pattern of the Walker circulation (Figs. 14, 15). A boomerang pattern of cold and dry anomaly appears to the north and south of the equatorial western Pacific. Although the La Nina pattern is not exactly the mirror image of El Nino, it is almost the opposite from El Nino in the extratropics. Both prediction systems simulate well the general pattern of ENSO response over the tropics, although the boomerang pattern in the western Pacific is not well simulated by either system. The magnitude of the SST anomaly in both prediction systems is larger than the observed anomaly. The warm anomaly over the South Indian Ocean during El Nino and the warm/cold anomaly over the northern part of Australia in El Nino/La Nina are well captured in Sys4 (Fig. 14b, e).
The ENSO forcing of the Polar Jet over the North Pacific and North America is known to be responsible for ENSO teleconnections such as Pacific North America (PNA) (Wallace and Gutzler 1981). The southern part of North America experiences a cold and wet winter during El Nino and a warm and dry winter during La Nina (Figs. 14, 15). The northwestern part of North America experiences milder winter during the El Nino and colder winter during the La Nina phase. Both modeling systems capture the gross global patterns in strong ENSO winters. The 500 hPa high pressure area over the North America in El Nino winter is well captured in Sys4 but with weaker magnitude, and it is shifted to the west in CFSv2. The strong low pressure area in the North Pacific is well captured in both models, but slightly shifted to the south in CFSv2 (Figs. 14, 15). The other low pressure area in the southern part of US and the Atlantic Ocean is not well simulated in Sys4. In La Nina winters, the models have a tendency that is similar but slightly asymmetric to El Nino winters (Figs. 14, 15).
5.2 PNA and NAO
We have shown that the ENSO teleconnection pattern over the North Pacific and the North America is generally well predicted for strong ENSO winters. However, the year-to-year winter climate variability in extratropics is influenced not only by tropical forcing but by oscillations of atmospheric mass between mid- and high-latitudes, such as PNA or North Atlantic Oscillation (NAO; Wallace and Gutzler 1981; Barnston and Livezey 1987). The NAO and PNA patterns are the two most important modes of variability in the NH mid- and high-latitudes, thus the prediction skill of the NH extratropics is related to the skill of predicting these patterns. In this section, we examine how well the models predict the dominant winter climate oscillations.
6 Summary and discussion
This study has examined the seasonal prediction skill for NH winter using retrospective predictions (reforecasts) by the ECMWF System 4 and NCEP CFSv2. The temperature, precipitation and geopotential height from the reforecast for the period 1982–2010 were compared with two reanalysis products: the ERA interim and the CFSR. The simulation ability of long-term mean climatology and the year-to-year variation were assessed. Both Sys4 and CFSv2 reproduce realistically the observed climatology pattern. However, systematic biases are found in both simulations. For the Sys4, a cold bias is found across the equatorial Pacific although a warm bias is found in the North Pacific and part of the North Atlantic. The CFSv2 has strong warm bias from the cold tongue region of the Pacific to the equatorial central Pacific and cold bias in broad areas of the North Pacific and the North Atlantic. A cold bias over large regions of the Southern Hemisphere is a common property of both reforecasts. With respect to precipitation, the Sys4 produced excesses along the ITCZ, the equatorial Indian Ocean and the western Pacific in Sys4. In the CFSv2, a strong wet bias is found along the SPCZ and the southern Indian Ocean as well as in the western Pacific. A dry bias is found for both modeling systems over South America and northern Australia and wet bias in East Asia and the equatorial Atlantic.
For both the Sys4 and CFSv2 systems, the mean prediction skill of 2mT and precipitation is higher over the tropics than the extra-tropics and higher over ocean than land. The 2mT over the South Indian Ocean, the North Pacific and equatorial North Atlantic shows high predictive skill in both reforecasts. The actual prediction skill of the 2mT depends on the reanalysis data set which is used as verification field. The discrepancy in two reanalysis (ERA interim and CFSR) is clear over the Indian Ocean, the equatorial western Pacific, the South America, over part of the equatorial Atlantic Ocean and over the Arctic. Therefore, the analyses are conducted using both reanalysis datasets. The 2mT and precipitation show the greatest skill in the tropical belt, especially in ENSO region when it is verified with both ERA interim and CFSR. In both modeling systems, the prediction skill of both tropical 2mT and precipitation is higher during strong ENSO winters than during weak ENSO winters.
This study has examined the prediction skill of the NH winter from the most recently upgraded seasonal forecast systems from ECMWF and NCEP. However, to provide physical insights to differences in prediction skill regarding to the set up of forecast systems, it would be useful to compare the skill between CFSv1 and CFSv2 and between ECMWF System 3 and system 4. This will be the subject of future research.
We would like to thank the reviewers for thoughtful and helpful comments. The ECMWF System 4 reforecasts were obtained by the authors through a commercial agreement with ECMWF. We would like to acknowledge the continued support of ECMWF especially, with regard to the paper, the comments of Drs. Franco Molteni and Tim Stockdale. The Climate Dynamics Division of the National Science Foundation under grant NSF-AGS 0965610 provided funding support for this research.
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.