Acta Meteorologica Sinica

, Volume 26, Issue 1, pp 26–40 | Cite as

Preliminary comparison of the CMA, ECMWF, NCEP, and JMA ensemble prediction systems

  • Mingkeng Duan (段明铿)Email author
  • Juhui Ma (麻巨慧)
  • Panxing Wang (王盘兴)


Based on The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Global Ensemble (TIGGE) dataset, using various verification methods, the performances of four typical ensemble prediction systems (EPSs) from the China Meteorological Administration (CMA), the European Centre for Medium-Range Weather Forecasts (ECMWF), the US National Centers for Environmental Prediction (NCEP), and the Japan Meteorological Agency (JMA) are compared preliminarily. The verification focuses on the 500-hPa geopotential height forecast fields in the mid- and high-latitude Eurasian region during July 2007 and January 2008. The results show that for the forecast of 500-hPa geopotential height, in both summer and winter, the ECMWF EPS exhibits the highest forecast skill, followed by that of NCEP, then by JMA, and the CMA EPS gets in the last. The better system behaviors benefit from the better combination of the following: data assimilation system, numerical models, initial perturbations, and stochastic model perturbations. For the medium-range forecast, the ensemble forecasting can effectively filter out the forecast errors associated with the initial uncertainty, and the reliability and resolution (the two basic attributions of the forecast system) of these EPSs are better in winter than in summer. Specifically, the CMA EPS has certain advantage on the reliability of ensemble probabilistic forecasts. The forecasts are easy to be underestimated by the JMA EPS. The deficiency of ensemble spread, which is the universal problem of EPS, also turns up in this study. Although the systems of ECMWF, NCEP, and JMA have more ensemble members, this problem cannot be ignored. This preliminary comparison helps to further recognize the prediction capability of the four EPSs over the Eurasian region, provides important references for wide applications of the TIGGE dataset, and supplies useful information for improving the CMA EPS.

Key words

TIGGE ensemble prediction system comparison verification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Atger, F., 1999: The skill of ensemble prediction systems. Mon. Wea. Rev., 127, 1941–1953.CrossRefGoogle Scholar
  2. Bougeault, P., Z. Toth, C. Bishop, et al., 2010: The THORPEX interactive grand global ensemble. Bull. Amer. Meteor. Soc., 91, 1059–1072.CrossRefGoogle Scholar
  3. Bourke, W., R. Buizza, and M. Naughton, 2004: Performance of the ECMWF and the BoM ensemble prediction systems in the Southern Hemisphere. Mon. Wea. Rev., 132, 2338–2357.CrossRefGoogle Scholar
  4. Buizza, R., and T. N. Palmer, 1995: The singular-vector structure of the atmospheric global circulation. J. Atmos. Sci., 52, 1434–1456.CrossRefGoogle Scholar
  5. —, M. Miller, and T. N. Palmer, 1999: Stochastic representation of model uncertainties in the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 125, 2887–2908.CrossRefGoogle Scholar
  6. —, P. L. Houtekamer, Z. Toth, et al., 2005: A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Mon. Wea. Rev., 133, 1076–1097.CrossRefGoogle Scholar
  7. Chen Jing, Xue Jishan, and Yan Hong, 2005: A new initial perturbation method of ensemble mesoscale heavy rain prediction. J. Appl. Meteor. Sci., 29, 717–726. (in Chinese)Google Scholar
  8. Duan Mingkeng, 2006: Ensemble forecasting performance studies on the important medium-range circulation processes affecting China. Ph. D. dissertation, Nanjing University of Information Science & Technology, China, 144 pp. (in Chinese)Google Scholar
  9. — and Wang Panxing, 2006: A new weighted average method on ensemble mean forecasting. J. Appl. Meteor. Sci., 17, 488–493. (in Chinese)Google Scholar
  10. —, —, Wu Hongbao, et al., 2009: The ensemble forecasting verification on the summer Eurasian middlehigh latitude circulation. J. Appl. Meteor. Sci., 20, 56–61. (in Chinese)Google Scholar
  11. Froude, L. S. R., 2010: TIGGE: Comparison of the prediction of Northern Hemisphere extratropical cyclones by different ensemble prediction systems. Wea. Forecasting, 25, 819–836.CrossRefGoogle Scholar
  12. —, L. Bengtsson, and K. I. Hodges, 2007: The prediction of extratropical storm tracks by the ECMWF and NCEP ensemble prediction systems. Mon. Wea. Rev., 135, 2545–2567.CrossRefGoogle Scholar
  13. Gong, J. D., W. J. Li, and J. F. Chou, 1999: Forming proper ensemble forecast initial members with four-dimensional variational data assimilation method. Chinese Sci. Bull., 44, 1527–1531.CrossRefGoogle Scholar
  14. Hamill, T. M., 1997: Reliability diagrams for multicategory probabilistic forecasts. Wea. Forecasting, 12, 736–741.CrossRefGoogle Scholar
  15. —, 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550–560.CrossRefGoogle Scholar
  16. Houtekamer, P. L., L. Lefaivre, J. Derome, et al., 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124, 1225–1242.CrossRefGoogle Scholar
  17. Huangfu Xueguan, 2002: Verification of the ensemble prediction system of the National Meteorological Center. J. Appl. Meteor. Sci., 13, 29–36. (in Chinese)Google Scholar
  18. Jin Ronghua, Li Weijing, and Sun Zhaobo, 2002: Study on the interpretation method of ensemble prediction products. J. Appl. Meteor. Sci., 25, 442–448. (in Chinese)Google Scholar
  19. Kalnay, E., M. Kanamitsu, R. Kistler, et al., 1996: The NCEP/NCAR 40-year reanalysis project. Bull. Amer. Meteor. Soc., 77, 437–472.CrossRefGoogle Scholar
  20. Kistler, R., E. Kalnay, W. Collins, et al., 2001: The NCEP/NCAR 50-year reanalysis: Monthly means CD-ROM and documentation. Bull. Amer. Meteor. Soc., 82, 247–268.CrossRefGoogle Scholar
  21. Leith, C. E., 1974: Theoretical skill of Monte Carlo forecasts. Mon. Wea. Rev., 102, 409–418.CrossRefGoogle Scholar
  22. Liu Huanzhu and Zhang Shaoqing, 1992: Statistical veri-fication on the medium-range numerical forecasting. Meteor. Mon., 18, 50–54. (in Chinese)Google Scholar
  23. Lorenz, E. N., 1963: Deterministic nonperiodic flow. J. Atmos. Sci., 20, 130–141.CrossRefGoogle Scholar
  24. Ma Xulin, Xue Jishan, and Lu Weisong, 2009: Study on ETKF-based initial perturbation scheme for GRAPES global ensemble prediction. Acta Meteor. Sinica, 23, 562–574.Google Scholar
  25. Molteni, F., R. Buizza, T. N. Palmer, et al., 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122, 73–120.CrossRefGoogle Scholar
  26. Mu, M., and Z. N. Jiang, 2008: A new approach to the generation of initial perturbations for ensemble prediction: Conditional nonlinear optimal perturbation. Chinese Sci. Bull., 53, 2062–2068.CrossRefGoogle Scholar
  27. Mullen, S. L., and R. Buizza, 2001: Quantitative precipitation forecasts over the United States by the ECMWF ensemble prediction system. Mon. Wea. Rev., 129, 638–663.CrossRefGoogle Scholar
  28. Park, Y. Y., R. Buizza, and M. Leutbecher, 2008: TIGGE: Preliminary reliminary results on comparing and combining ensembles. Quart. J. Roy. Meteor. Soc., 134, 2029–2050.CrossRefGoogle Scholar
  29. Richardson, D., 2000: Skill and relative economic value of the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 126, 649–667.CrossRefGoogle Scholar
  30. — —, R. Buizza, and R. Hagedorn, 2005: Final report of the 1st Workshop on the THORPEX Interactive Grand Global Ensemble (TIGGE), WMO/TDNo. 1273, WWRP/THORPEX, No.5, 39 pp.Google Scholar
  31. Shapiro, M., and A. Thorpe, 2004: THORPEX International Science Plan. WMO/TD-No. 1246, WWRP/THORPEX, No. 2, 51 pp.Google Scholar
  32. Talagrand, O., R. Vautard, and B. Strauss, 1997: Evaluation of probabilistic prediction systems. Proceedings of the ECMWF Workshop on Predictability, Reading, United Kingdom, ECMWF, 1–25.Google Scholar
  33. Titley, H., N. Savage, R. Swinbank, et al., 2008: Comparison between Met Office and ECMWF medium-range ensemble forecast systems. Meteorology Research and Development Technical Report No. 512, Met Office, United Kingdom, 41 pp.Google Scholar
  34. Tracton, M. S., and E. Kalnay, 1993: Operational ensemble prediction at the National Meteorological Center: Practical aspects. Wea. Forecasting, 8, 379–398.CrossRefGoogle Scholar
  35. Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330.CrossRefGoogle Scholar
  36. —, O. Talagrand, and Y. Zhu, 2006: The attributes of forecast systems: A general framework for the evaluation and calibration of weather forecasts. Predictability of Weather and Climate, T. N. Palmer and R. Hagedorn, Eds., Cambridge University Press, 584–595.Google Scholar
  37. —, Y. Zhu, and T. Marchok, 2001: The use of ensembles to identify forecasts with small and large uncertainty. Wea. Forecasting, 16, 463–477.CrossRefGoogle Scholar
  38. Wang Chenxi, Yao Jianqun, and Liang Xudong, 2007: The establishment and verification of the operational ensemble forecast system for Shanghai regional precipitation. J. Appl. Meteor. Sci., 18, 173–180. (in Chinese)Google Scholar
  39. Wei, M., and Z. Toth, 2003: A new measure of ensemble performance: Perturbation versus error correlation analysis (PECA). Mon. Wea. Rev., 131, 1549–1565.CrossRefGoogle Scholar
  40. Zhou Bing, Zhao Cuiguang, and Zhao Shengrong, 2006: Multi-model ensemble forecast technology with analysis and verification of the results. J. Appl. Meteor. Sci., 17, 104–109. (in Chinese)Google Scholar
  41. Zhu, Y., Z. Toth, R. Wobus, et al., 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83, 73–83.CrossRefGoogle Scholar

Copyright information

© The Chinese Meteorological Society and Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Mingkeng Duan (段明铿)
    • 1
    Email author
  • Juhui Ma (麻巨慧)
    • 1
  • Panxing Wang (王盘兴)
    • 1
  1. 1.Key Laboratory of Meteorological Disaster of Ministry of EducationNanjing University of Information Science & TechnologyNanjingChina

Personalised recommendations