Climate Dynamics

, Volume 38, Issue 7–8, pp 1433–1457 | Cite as

Evaluation of regional climate model simulations versus gridded observed and regional reanalysis products using a combined weighting scheme

  • Hyung-Il EumEmail author
  • Philippe Gachon
  • René Laprise
  • Taha Ouarda


This study presents a combined weighting scheme which contains five attributes that reflect accuracy of climate data, i.e. short-term (daily), mid-term (annual), and long-term (decadal) timescales, as well as spatial pattern, and extreme values, as simulated from Regional Climate Models (RCMs) with respect to observed and regional reanalysis products. Southern areas of Quebec and Ontario provinces in Canada are used for the study area. Three series of simulation from two different versions of the Canadian RCM (CRCM4.1.1, and CRCM4.2.3) are employed over 23 years from 1979 to 2001, driven by both NCEP and ERA40 global reanalysis products. One series of regional reanalysis dataset (i.e. NARR) over North America is also used as reference for comparison and validation purpose, as well as gridded historical observed daily data of precipitation and temperatures, both series have been beforehand interpolated on the CRCM 45-km grid resolution. Monthly weighting factors are calculated and then combined into four seasons to reflect seasonal variability of climate data accuracy. In addition, this study generates weight averaged references (WARs) with different weighting factors and ensemble size as new reference climate data set. The simulation results indicate that the NARR is in general superior to the CRCM simulated precipitation values, but the CRCM4.1.1 provides the highest weighting factors during the winter season. For minimum and maximum temperature, both the CRCM4.1.1 and the NARR products provide the highest weighting factors, respectively. The NARR provides more accurate short- and mid-term climate data, but the two versions of the CRCM provide more precise long-term data, spatial pattern and extreme events. Or study confirms also that the global reanalysis data (i.e. NCEP vs. ERA40) used as boundary conditions in the CRCM runs has non-negligible effects on the accuracy of CRCM simulated precipitation and temperature values. In addition, this study demonstrates that the proposed weighting factors reflect well all five attributes and the performances of weighted averaged references are better than that of the best single model. This study also found that the improvement of WARs’ performance is due to the reliability (accuracy) of RCMs rather than the ensemble size.


Ensemble simulations Regional climate model Weighting scheme Criteria of evaluation 



This research was made possible by a financial support from Québec’s Ministère du Développement Économique, de l’Innovation et de l’Exportation (MDEIE) and National Sciences and Engineering Research Council (NSERC) of Canada. The authors would like to acknowledge the Data Access Integration (DAI, see Team for providing the data and technical support, in particular the help of Milka Radojevic in preparing the data. The DAI data download gateway is made possible through collaboration among the FQRNT-funded Global Environmental and Climate Change Centre (GEC3), the Adaptation and Impacts Research Section (AIRS) of Environment Canada, and the Drought Research Initiative (DRI). The Ouranos Consortium also provides IT support to the DAI team. The CRCM time series data has been generated and supplied by Ouranos’ Climate Simulations Team. We would like also to acknowledge the National Centers for Environmental Prediction (NCEP) for the access of the North American Regional Reanalysis (NARR) datasets (see Finally, the authors would like also to express their gratitude to the two anonymous reviewers for their constructive comments.


  1. Blenkinsop S, Fowler JH (2007) Changes in drought frequency and severity over the British Isles projected by the PRUDENCE regional climate models. J Hydrol 342:50–71CrossRefGoogle Scholar
  2. Brochu R, Laprise R (2007) Surface water and energy budgets over the Mississippi and Columbia river basins as simulated by two generations of the Canadian regional climate model. Atmos Ocean 45(1):19–35Google Scholar
  3. Caya D, Laprise R (1999) A semi-implicit semi-Lagrangian regional climate model: the Canadian RCM. Mon Weather Rev 127:341–362CrossRefGoogle Scholar
  4. Christensen JH, Carter TR, Rummukainen M, Amanatidis G (2007) Evaluating the performance and utility of regional climate models: the PRUDENCE project. Clim Change 81(Suppl 1):1–6Google Scholar
  5. Christensen JH, Kjellstrom E, Giorgi F, Lenderink G, Rummukainen M (2010) Weight assignment in regional climate models. Clim Res 44:179–194CrossRefGoogle Scholar
  6. Collins WD, Bitz CM, Blackmon M, Bonan GB, Bretherton CS, Carton JA, Chang P, Doney S, Hack JJ, Henderson TB, Kiehl JT, Large WG, Mckenna DS, Santer BD, Smith RD (2006) The community climate system model: CCSM3. J Clim 19:2122–2143CrossRefGoogle Scholar
  7. Coppola E, Giorgi F, Rauscher SA, Piani C (2010) Model weighting based on mesoscale structures in precipitation and temperature in an ensemble of regional climate models. Clim Res 44:121–134CrossRefGoogle Scholar
  8. De Elía R, Caya D, Côté H, Frigon A, Biner S, Giguère M, Paquin D, Harvey R, Plummer D (2008) Evaluation of uncertainties in the CRCM-simulated North American climate. Clim Dyn 30:113–132CrossRefGoogle Scholar
  9. Deque M, Rowell DP, Luthi D, Giorgi F, Christensen JH, Rockel B, Jacob D, Kjellstrom E, Castro M, van den Hurk B (2007) An intercomparison of regional climate simulations for Europe: assessing uncertainties in model projections. Clim Change 81:53–70CrossRefGoogle Scholar
  10. Dessai S, Lu X, Hulme M (2005) Limited sensitivity analysis of regional climate change probabilities for the 21st century. J Geophys Res 110:D19108CrossRefGoogle Scholar
  11. Doblas-Reyes FJ, Hagendorn R, Palmer TN (2005) The rationale behind the success of multi-model ensembles in seasonal forecasting—II. Calibration and combination. Tellus A57:234–252Google Scholar
  12. Fiorino M (1997) AMIP II sea surface temperature and sea ice concentration observations.
  13. Fowler HJ, Ekstrom M (2009) Multi-model ensemble estimates of climate change impacts on UK seasonal precipitation extremes. Int J Climatol 29:385–416CrossRefGoogle Scholar
  14. Gachon P, Dibike Y (2007) Temperature change signals in northern Canada: convergence of statistical downscaling results using two driving GCMs. Int J Climatol 27:1623–1641CrossRefGoogle Scholar
  15. Gent PR, McWilliams JC (1990) Isopycnal mxing in ocean circulation models. J Phys Oceanogr 20:150–155CrossRefGoogle Scholar
  16. Giorgi F, Mearns LO (2002) Calculation of average, uncertainty range, and reliability of regional climate changes from AOGCM simulations via the reliability ensemble averaging (REA) method. J Clim 15:1141–1158CrossRefGoogle Scholar
  17. Hall J (2007) Probabilistic climate scenarios may misrepresent uncertainty and lead to bad adaptation decisions. Hydrol Process 21:1127–1129CrossRefGoogle Scholar
  18. Houghton JT, Ding Y, Griggs DJ, Noguer M, van der Linden PJ, Dai X, Maskell K, Johnson CA (2001) Climate change 2001: the scientific basis. In: Contribution of Working Group I to the 3rd Assessment Report of the Intergovernmental Panel on Climate Change (IPCC). Cambridge University Press, CambridgeGoogle Scholar
  19. Hutchinson M, Mckenney DW, Lawrence K, Pedlar JH (2009) Development and testing of Canada-wide interpolated spatial models of daily minimum–maximum temperature and precipitation for 1961–2003. J Appl Meteorol Climatol 48:725–741CrossRefGoogle Scholar
  20. IPCC (2007) Climate change 2007: the physical science basis. In: Contribution of the Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, CambridgeGoogle Scholar
  21. IPCC (2010) Meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. In: Stocker T, Dahe Q, Plattner G-K, Tignor M, Midgley P (eds) IPCC Working Group I Technical Support Unit. University of Bern, Bern, p 117Google Scholar
  22. Jenkins G, Lowe J (2003) Handling uncertainties in the UKCIP02 scenarios of climate change. Hadley Cent. Tech. Note 44, ExeterGoogle Scholar
  23. Jun M, Knutti R, Nychka DW (2008) Spatial analysis to quantify numerical model bias and dependence: how many climate models are there? J Am Stat Assoc 103:934–947CrossRefGoogle Scholar
  24. Jungclaus JH, Keenlyside N, Botzet M, Haak H, Luo J-J, Latif M, Marotzke J, Mikolajewicz U, Roeckner E (2006) Ocean circulation and tropical variability in the coupled model ECHAC5/MPI-OM. J Clim 19:3952–3972CrossRefGoogle Scholar
  25. Kendon EJ, Rowell DP, Jones RG, Buonomo E (2008) Robustness of future changes in local precipitation extremes. J Clim 21:4280–4297CrossRefGoogle Scholar
  26. Kevin W, Brandon D, Clark M, Gangopadhyay S (2004) Climate index weighting schemes for NWS ESP-based seasonal volume forecasts. J Hydrometeorol 5:1076–1090CrossRefGoogle Scholar
  27. Kim Y-O, Lee J-K (2010) Addressing heterogeneities in climate change studies for water resources in Korea. Curr Sci 98(8):1077–1083Google Scholar
  28. Knutti R, Furrer R, Tebaldi C, Cermak J (2010) Challenges in combining projections from multiple climate models. J Clim 23:2739–2758CrossRefGoogle Scholar
  29. Lapen DR, Hayhoe HN (2003) Spatial analysis of seasonal and annual temperature and precipitation normals in Southern Ontario, Canada. J Great Lakes Res 29(4):529–544CrossRefGoogle Scholar
  30. Laprise R, Caya D, Giguere M, Bergeron G, Cote H, Blanchet J-P, Boer GJ, McFarlane NA (1998) Climate and climate change in western Canada as simulated by the Canadian Regional Climate Model. Atmos Ocean 36(2):119–167CrossRefGoogle Scholar
  31. Laprise R, Caya D, Frigon A, Paquin D (2003) Current and perturbed climate as simulated by the second-generation Canadian Regional Climate Model (CRCM-II) over northwestern North America. Clim Dyn 21:405–421CrossRefGoogle Scholar
  32. Maidment D (1993) Handbook of hydrology. McGraw-Hill, New YorkGoogle Scholar
  33. Manning LJ, Hall JW, Fowler JJ, Kilsby CG, Tebaldi C (2009) Using probabilistic climate change information from a multimodel ensemble for water resources assessment. Water Resour Res 45:W11411CrossRefGoogle Scholar
  34. Mesinger F, DiMego G, Kalnay E, Mitchell K, Shafran PC, Ebisuzaki W, Jović D, Woollen J, Rogers E, Berbery EH, Ek MB, Fan Y, Grumbine R, Higgins W, Li H, Lin Y, Manikin G, Parrish D, Shi W (2006) North American regional reanalysis. Bull Am Meteorol Soc 87(3):343–360Google Scholar
  35. Mizuta R, Oouchi K, Yoshimura H, Noda A, Katayama K, Yukimoto S, Hosaka M, Kusunoki S, Kawai H, Nakagawa M (2006) 20-km-mesh global climate simulations using JMA-GSM model—mean climate states. J Meteorol Soc Jpn 84:165–185CrossRefGoogle Scholar
  36. Murphy JM et al (2004) Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature 430:768–772CrossRefGoogle Scholar
  37. Music B, Caya D (2007) Evaluation of the hydrological cycle over the Mississippi River basin as simulated by the Canadian Regional Climate Model (CRCM). J Hydrometeorol 8:969–988Google Scholar
  38. Nakicenovic N et al (2000) IPCC special report on emission scenarios. Cambridge University Press, CambridgeGoogle Scholar
  39. New M, Hulme M (2000) Representing uncertainty in climate change scenarios: a Monte-Carlo approach. Integr Assess 1:203–214CrossRefGoogle Scholar
  40. Palmer TN, Alessandri A, Andersen U, Cantelaube P, Davey M, Delecluse P, Deque M, Diez E, Doblas-Reyes FJ, Grahan R, Gualdi S, Gueremy J-F, Hagedorn R, Hoshen M, Keenlyside N, Latif M, Lazar A, Maisonnave E, Marletto V, Morse AP, Orfila B, Rogel P, Terres J-M, Thomson MC (2004) Development of a European multimodel ensemble system for seasonal-to-interannual prediction (Demeter). Bull Am Meteorol Soc 85(6):853–872CrossRefGoogle Scholar
  41. Pope VD, Gallani ML, Rowntree PR, Stratton A (2000) The impact of new physical parametrizations in the Hadley Centre Climate Model–HadAM3. Clim Dyn 16:123–146CrossRefGoogle Scholar
  42. Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using Bayesian model averaging to calibrate forecast ensembles. Mon Weather Rev 133:1155–1174CrossRefGoogle Scholar
  43. Raisanen J, Palmer TN (2001) A probability and decision-model analysis of a multimodel ensemble of climate change simulation. J Clim 14:3212–3226CrossRefGoogle Scholar
  44. Rajagopalan B, Lall U, Zebiak SE (2002) Categorical climate forecasts through regularization and optimal combination of multiple GCM ensembles. Mon Weather Rev 130:1792–1811CrossRefGoogle Scholar
  45. Rinke A, Marbaix P, Dethloff K (2004) Internal variability in arctic regional climate simulations: case study for the SHEBA year. Clim Res 27:197–209CrossRefGoogle Scholar
  46. Rummukainen M, Raissanen J, Bringfelt B, Ullerstig A, Omstedt A, Willen U, Hansson U, Jones C (2001) A regional climate model for northern Europe: model description and results from the downscaling of two GCM control simulations. Clim Dyn 17:339–359CrossRefGoogle Scholar
  47. Sanchez E, Romera R, Gaertner MA, Gallardo C, Castro M (2009) A weighting proposal for an ensemble of regional climate models over Europe driven by 1961–2000 ERA40 based on monthly precipitation probability density functions. Atmos Sci Lett 10:241–248Google Scholar
  48. Sharma A, O’Neill R (2002) A nonparametric approach for representing interannual dependence in monthly streamflow sequences. Water Resour Res 38(7):5.1–5.10. doi: 10.1029/2001WR000953 CrossRefGoogle Scholar
  49. Sharma A, Tarboton DG, Lall U (1997) Streamflow simulation: a nonparametric approach. Water Resour Res 33:291–308CrossRefGoogle Scholar
  50. Sloughter JM, Raftery AE, Gneiting T, Fraley C (2007) Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Mon Weather Rev 135:3209–3220CrossRefGoogle Scholar
  51. Straus DM, Shukla J (2000) Distinguishing between the SST-forced variability and internal variability in mid-latitudes: analysis of observations and GCM simulations. Q J R Meteorol Soc 126:2323–2350CrossRefGoogle Scholar
  52. Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Phil Trans R Soc A 365:2053–2075CrossRefGoogle Scholar
  53. Uppala SM (2001) ECMWF reanalysis, 1957– 2001, ERA-40, paper presented at the Workshop on Reanalysis, ECMWF, Reading, UKGoogle Scholar
  54. Uppala S, Kallberg P, Simmons A, Andrae U, da Costa Bechtold V, Fiorino M, Gibson J, Haseler J, Hernandez A, Kelly G, Li X, Onogi K, Saarinen S, Sokka N, Allan R, Andersson E, Arpe K, Balmaseda M, Beljaars A, Van de Berg L, Bidlot J, Bormann N, Caires S, Chevallier F, Dethof A, Dragosavac M, Fisher M, Fuentes M, Hagemann S, Hlm E, Hoskins B, Isaksen L, Janssen P, Jenne R, McNally A, Mahfouf J-F, Morcrette J-J, Rayner N, Saunders R, Simon P, Sterl A, Trenberth K, Untch A, Vasiljevic D, Viterbo P, Woollen J (2005) The ERA-40 re-analysis. Q J R Meteorol Soc 131:2961–3012. doi: 10.1256/qj.04.176
  55. van der Linden P, Mitchell JFB (2009) ENSEMBLES: climate change and its impacts: summary of research and results from the ENSEMBLES project. Met Office Hadley Centre, ExeterGoogle Scholar
  56. Visser H, Folkert RJM, Hoekstra J, deWolff JJ (2000) Identifying key sources of uncertainty in climate change projections. Clim Change 45:421–457CrossRefGoogle Scholar
  57. Weigel AP, Knutti R, Liniger MA, Appenzeller C (2010) Risks of model weighting in multimodel climate projections. J Clim 23:4175–4191CrossRefGoogle Scholar
  58. Wilby RL, Harris I (2006) A framework for assessing uncertainties in climate change impacts: low-flow scenarios for the River Thames, UK. Water Resour Res 42:W02419CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Hyung-Il Eum
    • 1
    Email author
  • Philippe Gachon
    • 1
    • 2
  • René Laprise
    • 1
  • Taha Ouarda
    • 3
  1. 1.ESCER (Étude et Simulation du Climat à l’Échelle Régionale)University of Québec at MontrealMontrealCanada
  2. 2.Adaptation and Impacts Research Section, Climate Research DivisionEnvironment CanadaMontrealCanada
  3. 3.INRS-ETE (Institut National de la Recherche Scientifique, Centre Eau-Terre-Environnement)University of QuébecQuebecCanada

Personalised recommendations