Skip to main content

Assessing the performance of the CFSR by an ensemble of analyses


The Climate Forecast System Reanalysis (CFSR, Saha et al. in Bull Am Meteor Soc 91:1015–1057, 2010) is the latest global reanalysis from the National Centers of Environmental Prediction (NCEP). In this study, we compare the CFSR tropospheric analyses to two ensembles of analyses. The first ensemble consists of 12 h analyses from various operational analyses for the year 2007. This ensemble shows how well the CFSR analyses can capture the daily variability. The second ensemble consists of monthly means from the available reanalyses from the years 1979 to 2009 which is used to examine the trends. With the 2007 ensemble, we find that the CFSR captures the daily variability in 2007 better than the older reanalyses and is comparable to the operational analyses. With the ensemble of monthly means, the CFSR is often the outlier. The CFSR shows a strong warming trend in the tropics which is not seen in the observations or other reanalyses.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10


  • Bosilovich MG (2008) NASA’s modern era retrospective-analysis for research and applications: integrating earth observations. Accessed 2 June 2010

  • Chelliah M, Ebisuzaki W, Weaver S, Kumar A (2011) Evaluating the tropospheric analyses from NCEP’s climate forecast system reanalysis (submitted)

  • Daley R (1991) Atmospheric data analysis. Cambridge University Press, Cambridge, pp 98–101

    Google Scholar 

  • Ebisuzaki W, Worley S, Shih CF (2007) The NCEP-NCAR 2007 annual analyses DVD. Available from the National Center for Atmospheric Research (NCAR), Boulder CO

  • Kalnay E, Kanamitsu M, Kistler R, Collins W, Deaven D, Gandin L, Iredell M, Saha S, White G, Woollen J, Zhu Y, Chelliah M, Ebisuzaki W, Higgins W, Janowiak J, Mo KC, Ropelewski C, Wang J, Leetmaa A, Reynolds R, Jenne R, Joseph D (1996) The NMC/NCAR 40-year reanalysis project. Bull Am Meteor Soc 77:437–471

    Article  Google Scholar 

  • Kanamitsu M, Ebisuzaki W, Woollen J, Yang S-K, Hnilo JJ, Fiorino M, Potter GL (2002) NCEP-DOE AMIP-II reanalysis (R-2). Bull Am Meteor Soc 83:1631–1643

    Article  Google Scholar 

  • Onogi K, Tsutsui J, Koide H, Sakamoto M, Kobayashi S, Hatsushika H, Matsumoto T, Yamazaki N, Kamahori H, Takahashi K, Kadokura S, Wada K, Kato K, Oyama R, Ose T, Mannoji N, Taira R (2007) The JRA-25 reanalysis. J Met Soc Jap 85(3):369–432

    Article  Google Scholar 

  • Rienecker MM, Suarez MJ, Todling R, Bacmeister J, Takacs L, Liu H-C, Gu W, Sienkiewicz M, Koster RD, Gelaro R, Stajner I (2007) The GEOS-5 Data Assimilation System—Documentation of Versions 5.0.1 and 5.1.0. NASA GSFC Technical Report Series on Global Modeling and Data Assimilation, NASA/TM-2007-104606, vol 27, pp 92

  • Saha S, Moorthi S, Pan HL, Wu X, Wang J, Nadiga S, Tripp P, Kistler R, Woolen J, Behringer D, Liu H, Stokes D, Grumbine R, Gayno G, Wang J, Hou YT, Chuang H, Juang HMJ, Sela J, Irdell M, Treadon R, Klesits D, Felst PV, Keyser D, Derber J, Ek M, Meng J, Wei H, Yang R, Lord S, van den Dool H, Kumar A, Wang W, Long C, Chelliah M, Xue Y, Huang B, Schemm J, Ebisuzaki W, Lin R, Xie PP, Chen M, Zhou S, Higgins W, Zou CZ, Liu Q, Chen Y, Han Y, Cucurull L, Reynolds RW, Rutledge G, Goldberg M (2010) The NCEP climate forecast system reanalysis. Bull Am Meteor Soc 91:1015–1057

    Article  Google Scholar 

  • Uppala SM, Kållberg PW, Simmons AJ, Andrae U, da Costa Bechtold V, Fiorino M, Gibson JK, Haseler J, Hernandez A, Kelly GA, Li X, Onogi K, Saarinen S, Sokka N, Allan RP, Andersson E, Arpe K, Balmaseda MA, Beljaars ACM, van de Berg L, Bidlot J, Bormann N, Caires S, Chevallier F, Dethof A, Dragosavac M, Fisher M, Fuentes M, Hagemann S, Hólm E, Hoskins BJ, Isaksen L, Janssen PAEM, Jenne R, McNally AP, Mahfouf J-F, Morcrette J-J, Rayner NA, Saunders RW, Simon P, Sterl A, Trenberth KE, Untch A, Vasiljevic D, Viterbo P, Woollen J (2005) The ERA-40 re-analysis. Quart J R Meteorol Soc 131:2961–3012

    Article  Google Scholar 

  • Xue Y, Huang B, Hu Z-Z, Kumar A, Wen C, Behringer D, Nadiga S (2010) An assessment of oceanic variability in the NCEP climate system reanalysis. Clim Dyn. doi: 10.1007/s00382-010-0954-4

  • Yang F (2010) 2009 Review of GFS forecast skill. Accessed 20 Oct 2010

Download references


Craig Long supplied the sonde data for Singapore and Accension Island. Suru Saha and the rest of the CFSR team for creating and supplying the data for the CFSR. Emily Becker downloaded much of the JRA-25 and MERRA data. Bhaskar Jha for supplying the CFSR-AMIP simulations and the reviewers for their comments which improved this paper.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Wesley Ebisuzaki.



Part A: expected values of the correlations in Table 3

Suppose the ith ensemble member can be written as Ai + T where T is the truth. Then the ensemble mean can be written as M = (1/n) ∑Ai + T where n is the number of ensemble members. Assuming that the Ai are statistically independent, unbiased and have the same variance (σ2), then

$$ \begin{aligned} & {\text{E}}\left( {{\text{A}}_{\text{i}} } \right) = 0 \\ & {\text{E}}\left( {{\text{A}}_{\text{i}} {\text{A}}_{\text{i}} } \right) = \sigma^{2} \\ & {\text{E}}\left( {{\text{A}}_{\text{i}} {\text{A}}_{\text{j}} } \right) = 0\,{\text{for}}\,{\text{i}} \ne j \\ \end{aligned} $$

The temporal correlation of the perturbations from the ensemble mean can be written as

$$ {{{{\uprho}}_{\text{ij}} = \sum \left[ {\left( {{\text{A}}_{\text{i}} - {\text{M}}} \right)\left( {{\text{A}}_{\text{j}} - {\text{M}}} \right)} \right]} \mathord{\left/ {\vphantom {{{{\uprho}}_{\text{ij}} = \sum \left[ {\left( {{\text{A}}_{\text{i}} - {\text{M}}} \right)\left( {{\text{A}}_{\text{j}} - {\text{M}}} \right)} \right]} {\left( {\sum \left( {{\text{A}}_{\text{i}} - {\text{M}}} \right)^{2} \sum \left( {{\text{A}}_{\text{j}} - {\text{M}}} \right)^{2} } \right)^{1/2} }}} \right. \kern-\nulldelimiterspace} {\left( {\sum \left( {{\text{A}}_{\text{i}} - {\text{M}}} \right)^{2} \sum \left( {{\text{A}}_{\text{j}} - {\text{M}}} \right)^{2} } \right)^{1/2} }} $$

where the values are summed over the samples. Now the expected value of ∑(Ai−M)2 is

$$ {\text{E}}\left( {\sum \left( {{\text{A}}_{\text{i}} - {\text{M}}} \right)^{2} } \right) = {{\text{E}}\left( {\sum \left( {{\text{A}}_{\text{i}}^{2} } \right)} \right)} - {\text{E}}\left( {2\sum {\text{A}}_{\text{i}} {\text{M}}} \right) + {\text{E}}\left( {\sum {\text{M}}^{2} } \right) = {\text{m}}\left( {1 - 1/{\text{n}}} \right)\sigma^{2} $$

where m is the number of samples

The expected value of ∑ [(Ai−M)(Aj−M)] when i ≠ j is

$$ {\text{E}}\left( {\sum \left[ {\left( {{\text{A}}_{\text{i}} - {\text{M}}} \right)\left( {{\text{A}}_{\text{j}} - {\text{M}}} \right)} \right]} \right) = {\text{E}}\left( {\sum \left( {{\text{A}}_{\text{i}} {\text{A}}_{\text{j}} } \right)} \right) - {\text{E}}\left( {{\text{A}}_{\text{i}} {\text{M}}} \right) - {\text{E}}\left( {{\text{A}}_{\text{j}} {\text{M}}} \right) + {\text{E}}\left( {{\text{M}}^{2} } \right) = - {\text{m}}/{\text{n }}\sigma^{2} $$

So the expected value of the temporal correlation (when the time series is long) is −1/(n−1). For the high-frequency ensemble, n = 5 and the expected correlation of −0.25 which is in the middle of the range of calculated values in Table 3.

Part B: the expected values of the correlations in Table 5

Let AEC represent the EC analyses.

Let Ai i = 1,…,4 represent the CFSR, CMC, FNO, UK analyses


$$ \begin{aligned} & {\text{E}}\left( {{\text{A}}_{\text{i}} {\text{A}}_{\text{i}} } \right) = \sigma_{\text{i}}^{2} \\ & {\text{E}}\left( {{\text{A}}_{\text{i}} {\text{A}}_{\text{j}} } \right) = 0\,{\text{for}}\,{\text{i}} \ne {\text{j}} \\ & {\text{E}}\left( {{\text{A}}_{\text{i}} {\text{A}}_{\text{EC}} } \right) = 0 \\ & {\text{E}}\left( {{\text{A}}_{\text{EC}} {\text{A}}_{\text{EC}} } \right) = \sigma_{\text{EC}}^{2} \\ \end{aligned} $$

The correlation between the terms in Table E is given by

$$ {{{{\uprho}}_{\text{ij}} = \sum \left[ {\left( {{\text{A}}_{\text{i}} - {\text{A}}_{\text{EC}} } \right)\left( {{\text{A}}_{\text{j}} - {\text{A}}_{\text{EC}} } \right)} \right]} \mathord{\left/ {\vphantom {{{{\uprho}}_{\text{ij}} = \sum \left[ {\left( {{\text{A}}_{\text{i}} - {\text{A}}_{\text{EC}} } \right)\left( {{\text{A}}_{\text{i}} - {\text{A}}_{\text{EC}} } \right)} \right]} {\left( {\sum \left( {{\text{A}}_{\text{i}} - {\text{A}}_{\text{EC}} } \right)^{2} \sum \left( {{\text{A}}_{\text{j}} - {\text{A}}_{\text{EC}} } \right)^{2} } \right)^{1/2} }}} \right. \kern-\nulldelimiterspace} {\left( {\sum \left( {{\text{A}}_{\text{i}} - {\text{A}}_{\text{EC}} } \right)^{2} \sum \left( {{\text{A}}_{\text{j}} - {\text{A}}_{\text{EC}} } \right)^{2} } \right)^{1/2} }} $$

Now for i ≠ j, the expected value of ∑ [(Ai−AEC)(Aj−AEC)] is m·σ 2EC and the expected valued of ∑(Ai−AEC)2 is m(σ 2i  + σ 2EC ). So the expected value of ρij when the m is large is

$$ {\text{E}}({{\uprho}}_{\text{ij}} ) = \sigma_{\text{EC}}^{2} \left[ {\left( {\sigma_{\text{i}}^{2} + \sigma_{\text{EC}}^{2} } \right)\left( {\sigma_{\text{j}}^{2} + \sigma_{\text{EC}}^{2} } \right)} \right]^{ - 1/2} $$

Using σ 2i estimated by Column A of Table 4, the expected values of the correlation when deviations from “truth” are random can be calculated are shown in parenthesis in Table 5.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ebisuzaki, W., Zhang, L. Assessing the performance of the CFSR by an ensemble of analyses. Clim Dyn 37, 2541–2550 (2011).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • CFSR
  • Reanalysis
  • Ensemble