Acta Meteorologica Sinica

, Volume 25, Issue 5, pp 568–580 | Cite as

Evaluation of the NMC regional ensemble prediction system during the Beijing 2008 Olympic Games

  • Xiaoli Li (李晓莉)Email author
  • Hua Tian (田 华)
  • Guo Deng (邓 国)


Based on the B08RDP (Beijing 2008 Olympic Games Mesoscale Ensemble Prediction Research and Development Project) that was launched by the World Weather Research Programme (WWRP) in 2004, a regional ensemble prediction system (REPS) at a 15-km horizontal resolution was developed at the National Meteorological Center (NMC) of the China Meteorological Administration (CMA). Supplementing to the forecasters’ subjective affirmation on the promising performance of the REPS during the 2008 Beijing Olympic Games (BOG), this paper focuses on the objective verification of the REPS for precipitation forecasts during the BOG period. By use of a set of advanced probabilistic verification scores, the value of the REPS compared to the quasi-operational global ensemble prediction system (GEPS) is assessed for a 36-day period (21 July–24 August 2008). The evaluation here involves different aspects of the REPS and GEPS, including their general forecast skills, specific attributes (reliability and resolution), and related economic values. The results indicate that the REPS generally performs significantly better for the short-range precipitation forecasts than the GEPS, and for light to heavy rainfall events, the REPS provides more skillful forecasts for accumulated 6- and 24-h precipitation. By further identifying the performance of the REPS through the attribute-focused measures, it is found that the advantages of the REPS over the GEPS come from better reliability (smaller biases and better dispersion) and increased resolution. Also, evaluation of a decision-making score reveals that a much larger group of users benefits from using the REPS forecasts than using the single model (the control run) forecasts, especially for the heavy rainfall events.

Key words

regional ensemble prediction ensemble verification probabilistic scores 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Brier, G. W., 1950: Verification of forecasts expressed in terms of probability. Mon. Wea. Rev., 78, 1–3.CrossRefGoogle Scholar
  2. Bowler, N. E., A. Arribas, K. R. Mylne, K. B. Robertson, and S. E. Beare, 2008: The MOGEPS short-range ensemble prediction system. Quart. J. Roy. Meteor. Soc., 134, 703–722.CrossRefGoogle Scholar
  3. Buizza, R., P. L. Houtekamer, Z. Toth, G. Pellerin, M. Wei, and Y. Zhu, 2005: A comparison of the ECMWF, MSC and NCEP global ensemble. Mon. Wea. Rev., 133, 1076–1097.CrossRefGoogle Scholar
  4. Candille, G., J. Cote, P. L. Houtekamer, and G. Pellerin, 2007: Verification of an ensemble prediction system against observations. Mon. Wea. Rev., 135, 2688–2699.CrossRefGoogle Scholar
  5. Chessa, P. A., G. Ficca, M. Marrocu, and R. Buizza, 2004: Application of a limited-area short-range ensemble forecast system to a case of heavy rainfall in the Mediterranean region. Wea. Forecasting, 19, 566–581.CrossRefGoogle Scholar
  6. Deng Guo, Gong Jiandong, Deng Liantang, et al., 2010: The development of mesoscale ensemble prediction system at National Meteorological Center. J. Appl. Meteor. Sci., 21(5), 513–523. (in Chinese)Google Scholar
  7. Du, J., and M. S. Tracton, 2001: Implementation of a real-time short-range ensemble forecasting system at NCEP: An update. Preprint, Ninth Conf. on Mesoscale Processes, Ft. Lauderdale, Florida, Amer. Meteor. Soc., 355–356.Google Scholar
  8. —, G. DiMego, M. S. Tracton, and B. Zhou, 2003: NCEP short-range ensemble forecasting (SREF) system: Multi-IC, multi-model and multi-physics approach. Research Activities in Atmospheric and Oceanic Modelling (edited by J. Cote), Report 33, CAS/JSC Working Group Numerical Experimentation (WGNE), WMO/TD-No. 1161, 5.09–5.10.Google Scholar
  9. Duan, Y., J. Gong, J. Du, M. Charron, J. Chen, G. Deng, G.. DiMego, M. Hara, M. Kunni, X. Li, Y. Li, K. Saito, H. Seko, Y. Wang, and C. Wittmann, 2011: An overview of the Beijing 2008 Olympics Research and Development Project (B08RDP). Bull. Amer. Meteor. Soc., doi: 10.1175/BAMS-D-11-00115.1.Google Scholar
  10. Efron, B., and R. Tibshirani, 1993: An Introduction to the Bootstrap. Chapman and Hall, 436 pp.Google Scholar
  11. Heizenreder, D., S. Trepte, and M. Denhard, 2006: SRNWP-PEPS: A regional multi-model ensemble in Europe. The European Forecaster, Newsletter of the WGCEF, 11, 29–35.Google Scholar
  12. Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Forecasting, 15, 559–570.CrossRefGoogle Scholar
  13. Janjié, Z. I., J. P. Gerrity Jr., and S. Nickovic, 2001: An alternative approach to nonhydrostatic modeling. Mon. Wea. Rev., 129, 1164–1178.CrossRefGoogle Scholar
  14. Jolliffe, I. T., and D. B. Stephenson, 2003: Forecast Verification: A Practitioner’s Guide in Atmospheric Science. Wiley and Sons, 240 pp.Google Scholar
  15. Li Zechun and Chen Dehui, 2002: The development and application of the operational ensemble prediction system at National Meteorological Center. J. Appl. Meteor. Sci., 13(1), 1–15. (in Chinese)Google Scholar
  16. Marsigli, C., A. Montani, F. Nerozzi, and T. Paccangnella, 2005: The COSMO-LEPS mesoscale ensemble system: Validation of the methodology and verification. Nonlinear Processes Geophys., 12, 527–536.CrossRefGoogle Scholar
  17. Mason, I., 1982: A model for assessment of weather forecasts. Aus. Meteor. Mag., 30, 291–303.Google Scholar
  18. Montani, A., and Coauthors, 2003: Operational limitedarea ensemble forecasts based on the Lokal Modell. ECMWF Newsletter, No. 98, ECMWF, Reading, United Kingdom, 2–7.Google Scholar
  19. Murphy, A. H., 1973: A new vector partition of the probability score. J. Appl. Meteor., 12, 595–600.CrossRefGoogle Scholar
  20. Park, Y. -Y., R. Buizza, and M. Leutbecher, 2008: TIGGE: Preliminary results on comparing and combining ensembles. Quart. J. Roy. Meteor. Soc., 134, 2029–2050.CrossRefGoogle Scholar
  21. Richardson, D. S., 2000: Skill and economic value of the ECMWF ensemble prediction system. Quart. J. Roy. Meteor. Soc., 126, 649–668.CrossRefGoogle Scholar
  22. Stanski, H. R., L. J. Wilson, and W. R. Burrows, 1989: Survey of common verification methods in meteorology. World Meteorological Organization, World Weather Watch Rep. 8, Tech. Doc., 358, 114 pp.Google Scholar
  23. Stensrud, D. J., H. E. Brooke, J. Du, M. S. Tracton, and E. Rogers, 1999: Using ensembles for short-range forecasting. Mon. Wea. Rev., 127, 433–446.CrossRefGoogle Scholar
  24. —, J. Bao, and T. T. Warner, 2000: Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 2077–2107.CrossRefGoogle Scholar
  25. Talagrand, O., R. Vautard, and B. Strauss, 1999: Evaluation of probabilistic prediction systems. Proc. Workshop on Predictability, Reading, United Kingdom, ECMWF, 1–25.Google Scholar
  26. Toth, Z., and E. Kalnay, 1993: Ensemble forecasting at NMC: The generation of perturbations. Bull. Amer. Meteor. Soc., 74, 2317–2330.CrossRefGoogle Scholar
  27. —, and —, 1997: Ensemble forecasting at NCEP: The breeding method. Mon. Wea. Rev., 125, 3297–3319.CrossRefGoogle Scholar
  28. Wang, Y., M. Bellus, C. Wittmann, M. Steinheimer, F. Weidle, S. Ivatek-Sahdan, A. Kann, W. Tian, X. Ma, S. Tascu, and E. Bazile, 2011: The Central European limited area ensemble forecasting system: ALADIN-LAEF. Quart. J. Roy. Meteor. Soc., 137, 483–502, doi: 10.1002/qj.751.CrossRefGoogle Scholar
  29. Zhu, Y., Z. Toth, R. Wobus, D. Richarson, and K. Mylne, 2002: The economic value of ensemble-based weather forecasts. Bull. Amer. Meteor. Soc., 83, 74–83.CrossRefGoogle Scholar

Copyright information

© The Chinese Meteorological Society and Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Xiaoli Li (李晓莉)
    • 1
    Email author
  • Hua Tian (田 华)
    • 1
  • Guo Deng (邓 国)
    • 1
  1. 1.The Center of Numerical Weather PredictionChina Meteorological AdministrationBeijingChina

Personalised recommendations