Climate Dynamics

, Volume 47, Issue 5–6, pp 1881–1898 | Cite as

Evaluation of precipitation and temperature simulation performance of the CMIP3 and CMIP5 historical experiments

  • A. G. Koutroulis
  • M. G. Grillakis
  • I. K. Tsanis
  • L. Papadimitriou


The fifth phase of the Coupled Model Intercomparison Project (CMIP5) is the most recent coordinated experiment of global climate modeling. Compared to its predecessor CMIP3, the fifth phase of the homonymous experiment—CMIP5 involves a greater number of GCMs, run at higher resolutions with more complex components. Here we use daily GCM data from both projects to test their efficiency in representing precipitation and temperature parameters with the use of a state of the art high resolution gridded global dataset for land areas and for the period 1960–2005. Two simple metrics, a comprehensive histogram similarity metric based on the match of simulated and observed empirical pdfs and a metric for the representation of the annual cycle were employed as performance indicators. The metrics were used to assess the skill of each GCM at the entire spectrum of precipitation and temperature pdfs but also for the upper and lower tails of it. Results are presented globally and regionally for 26 land regions that represent different climatic regimes, covering the total earth’s land surface except for Antarctica. Compared to CMIP3, CMIP5 models perform better in simulating precipitation including relatively intense events and the fraction of wet days. For temperature the improvement is not as clear except for the upper and lower hot and cold events of the distribution. The agreement of model simulations is also considerably increased in CMIP5. Substantial improvement in intense precipitation is observed over North Europe, Central and Eastern North America and North East Europe. Nevertheless, in both ensembles some models clearly perform better than others from a histogram similarity point of view. The derived skill score metrics provide essential information for impact studies based on global or regional land area multi-model ensembles.


Precipitation Temperature CMIP3 CMIP5 Intercomparison Skill score 



A. Koutroulis and L. Papadimitriou were partly supported by the High-End cLimate Impacts and eXtremes (HELIX) project which has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under Grant Agreement No. 603864. We acknowledge the World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP, and we thank the climate modeling groups (listed in Table 1 of this paper) for producing and making available their model output. We also acknowledge the modeling groups, the Program for Climate Model Diagnosis and Intercomparison (PCMDI) and the WCRP’s Working Group on Coupled Modelling (WGCM) for their roles in making available the WCRP CMIP3 multi-model dataset. For CMIP the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led development of software infrastructure in partnership with the Global Organization for Earth System Science Portals.

Supplementary material

382_2015_2938_MOESM1_ESM.docx (9.7 mb)
Supplementary material 1 (DOCX 9926 kb)


  1. Abdullah K (2006) Use of water and land for food security and environmental sustainability. Irrig Drain 55:219–222. doi: 10.1002/ird.254 CrossRefGoogle Scholar
  2. Adler RF, Huffman GJ, Chang A et al (2003) The Version-2 Global Precipitation Climatology Project (GPCP) monthly precipitation analysis (1979–Present). J Hydrometeorol 4:1147–1167. doi: 10.1175/1525-7541(2003)004<1147:TVGPCP>2.0.CO;2 CrossRefGoogle Scholar
  3. Adler RF, Gu G, Huffman GJ (2012) Estimating climatological bias errors for the Global Precipitation Climatology Project (GPCP). J Appl Meteorol Climatol 51:84–99. doi: 10.1175/JAMC-D-11-052.1 CrossRefGoogle Scholar
  4. Bao Q, Lin P, Zhou T et al (2013) The Flexible Global Ocean-Atmosphere-Land system model, Spectral Version 2: FGOALS-s2. Adv Atmos Sci 30:561–576. doi: 10.1007/s00376-012-2113-9 CrossRefGoogle Scholar
  5. Bhend J, Whetton P (2013) Consistency of simulated and observed regional changes in temperature, sea level pressure and precipitation. Clim Change 118:799–810. doi: 10.1007/s10584-012-0691-2 CrossRefGoogle Scholar
  6. Christensen JH, Kjellström E, Giorgi F et al (2010) Weight assignment in regional climate models. Clim Res 44:179–194CrossRefGoogle Scholar
  7. Collins M, Tett SFB, Cooper C (2001) The internal climate variability of HadCM3, a version of the Hadley Centre coupled model without flux adjustments. Clim Dyn 17:61–81. doi: 10.1007/s003820000094 CrossRefGoogle Scholar
  8. Dee DP, Uppala SM, Simmons AJ et al (2011) The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q J R Meteorol Soc 137:553–597. doi: 10.1002/qj.828 CrossRefGoogle Scholar
  9. Deliverable E (2008) D3. 2.2. RCM-specific Weight based their Abil. to simulate Present Clim. calibrated ERA40-based simulations.
  10. Delworth TL, Broccoli AJ, Rosati A et al (2006) GFDL’s CM2 global coupled climate models. Part I: formulation and simulation characteristics. J Clim 19:643–674. doi: 10.1175/JCLI3629.1 CrossRefGoogle Scholar
  11. K-1 model Developers (2004) K-1 coupled model (MIROC) description. Ed by H Hasumi Emori, S Cent Clim Syst Res Univ Tokyo K-1 technics, pp 1–34Google Scholar
  12. Dufresne J-L, Foujols M, Denvil S et al (2013) Climate change projections using the IPSL-CM5 Earth System Model: from CMIP3 to CMIP5. Clim Dyn 40:2123–2165. doi: 10.1007/s00382-012-1636-1 CrossRefGoogle Scholar
  13. Flato G, Marotzke J, Abiodun B, Braconnot P, Chou SC, Collins W, Cox P, Driouech F, Emori S, Eyring V, Forest C, Gleckler P, Guilyardi E, Jakob C, Kattsov V, Reason C, Rummukainen M (2013) Evaluation of climate models. In: Stocker TF et al (eds) Climate change 2013: the physical science basis. Cambridge University Press, Cambridge, pp 741–866Google Scholar
  14. Franklin CN, Sun Z, Bi D et al (2013) Evaluation of clouds in ACCESS using the satellite simulator package COSP: regime-sorted tropical cloud properties. J Geophys Res Atmos 118:6663–6679. doi: 10.1002/jgrd.50496 CrossRefGoogle Scholar
  15. Furevik T, Bentsen M, Drange H et al (2003) Description and evaluation of the bergen climate model: ARPEGE coupled with MICOM. Clim Dyn 21:27–51. doi: 10.1007/s00382-003-0317-5 CrossRefGoogle Scholar
  16. Giorgi F, Bi X (2005) Updated regional precipitation and temperature changes for the 21st century from ensembles of recent AOGCM simulations. Geophys Res Lett 32:L21715. doi: 10.1029/2005GL024288 CrossRefGoogle Scholar
  17. Gordon HB, Rotstayn LD, McGregor JL et al (2002) The CSIRO Mk3 Climate System Model. CSIRO Atmos Res Tech Pap no 60:130Google Scholar
  18. Gordon H, O’Farrell S, Collier M et al (2010) The CSIRO Mk3. 5 climate model. CAWCRGoogle Scholar
  19. Grillakis MG, Koutroulis AG, Tsanis IK (2013) Multisegment statistical bias correction of daily GCM precipitation output. J Geophys Res Atmos 118:3150–3162. doi: 10.1002/jgrd.50323 CrossRefGoogle Scholar
  20. Guemas V, Codron F (2011) Differing impacts of resolution changes in latitude and longitude on the midlatitudes in the LMDZ atmospheric GCM. J Clim 24:5831–5849. doi: 10.1175/2011JCLI4093.1 CrossRefGoogle Scholar
  21. Harding R, Best M, Blyth E et al (2011) WATCH: current knowledge of the terrestrial global water cycle. J Hydrometeorol 12:1149–1156. doi: 10.1175/JHM-D-11-024.1 CrossRefGoogle Scholar
  22. Hirota N, Takayabu YN (2013) Reproducibility of precipitation distribution over the tropical oceans in CMIP5 multi-climate models compared to CMIP3. Clim Dyn 41:2909–2920CrossRefGoogle Scholar
  23. Hourdin F, Foujols M-A, Codron F et al (2012) Impact of the LMDZ atmospheric grid configuration on the climate and sensitivity of the IPSL-CM5A coupled model. Clim Dyn 40:2167–2192. doi: 10.1007/s00382-012-1411-3 CrossRefGoogle Scholar
  24. Huang W, Wang B, Li L et al (2013) Variability of atlantic meridional overturning circulation in FGOALS-g2. Adv Atmos Sci 31:95–109. doi: 10.1007/s00376-013-2155-7 CrossRefGoogle Scholar
  25. Iversen T, Bentsen M, Bethke I et al (2013) The Norwegian Earth System Model, NorESM1-M—Part 2: climate response and scenario projections. Geosci Model Dev 6:389–415. doi: 10.5194/gmd-6-389-2013 CrossRefGoogle Scholar
  26. Johns TC, Durman CF, Banks HT et al (2006) The New Hadley Centre Climate Model (HadGEM1): evaluation of coupled simulations. J Clim 19:1327–1353. doi: 10.1175/JCLI3712.1 CrossRefGoogle Scholar
  27. Jones C, Carvalho LMV (2013) Climate change in the South American Monsoon System: present climate and CMIP5 projections. J Clim 26:6660–6678. doi: 10.1175/JCLI-D-12-00412.1 CrossRefGoogle Scholar
  28. Jones CD, Hughes JK, Bellouin N et al (2011) The HadGEM2-ES implementation of CMIP5 centennial simulations. Geosci Model Dev 4:543–570. doi: 10.5194/gmd-4-543-2011 CrossRefGoogle Scholar
  29. Jungclaus JH, Keenlyside N, Botzet M et al (2006) Ocean circulation and tropical variability in the coupled model ECHAM5/MPI-OM. J Clim 19Google Scholar
  30. Kalnay E, Kanamitsu M, Kistler R et al (1996) The NCEP/NCAR 40-year reanalysis project. Bull Am Meteorol Soc 77:437–471. doi: 10.1175/1520-0477(1996)077<0437:TNYRP>2.0.CO;2 CrossRefGoogle Scholar
  31. Kanamitsu M, Ebisuzaki W, Woollen J et al (2002) NCEP–DOE AMIP-II reanalysis (R-2). Bull Am Meteorol Soc 83:1631–1643. doi: 10.1175/BAMS-83-11-1631 CrossRefGoogle Scholar
  32. Kjellström E, Boberg F, Castro M et al (2010) Daily and monthly temperature and precipitation statistics as performance indicators for regional climate models. Clim Res 44:135–150. doi: 10.3354/cr00932 CrossRefGoogle Scholar
  33. Knutti R, Sedláček J (2013) Robustness and uncertainties in the new CMIP5 climate model projections. Nat Clim Change 3:369–373. doi: 10.1038/nclimate1716 CrossRefGoogle Scholar
  34. Knutti R, Masson D, Gettelman A (2013) Climate model genealogy: generation CMIP5 and how we got there. Geophys Res Lett 40:1194–1199. doi: 10.1002/grl.50256 CrossRefGoogle Scholar
  35. Kopparla P, Fischer EM, Hannay C, Knutti R (2013) Improved simulation of extreme precipitation in a high-resolution atmosphere model. Geophys Res Lett 40:5803–5808CrossRefGoogle Scholar
  36. Kottek M, Grieser J, Beck C et al (2006) World map of the Köppen–Geiger climate classification updated. Meteorol Zeitschrift 15:259–263CrossRefGoogle Scholar
  37. Kumar S, Merwade V, Kinter JL, Niyogi D (2013) Evaluation of temperature and precipitation trends and long-term persistence in CMIP5 twentieth-century climate simulations. J Clim. doi: 10.1175/JCLI-D-12-00259.1 Google Scholar
  38. Kumar D, Kodra E, Ganguly AR (2014) Regional and seasonal intercomparison of CMIP3 and CMIP5 climate model ensembles for temperature and precipitation. Clim Dyn 43:1–28. doi: 10.1007/s00382-014-2070-3 CrossRefGoogle Scholar
  39. Marti O, Braconnot P, Bellier J et al (2005) The new IPSL climate system model: IPSL-CM4. Note du Pôle Modélisation, IPSL 26:1–86Google Scholar
  40. Meehl GA, Washington WM, Santer BD et al (2006) Climate change projections for the twenty-first century and climate change commitment in the CCSM3. J Clim 19Google Scholar
  41. Meehl GA, Washington WM, Arblaster JM et al (2012) Climate system response to external forcings and climate change projections in CCSM4. J Clim 25:3661–3683. doi: 10.1175/JCLI-D-11-00240.1 CrossRefGoogle Scholar
  42. Perkins SE, Pitman AJ, Holbrook NJ, McAneney J (2007) Evaluation of the AR4 climate models’ simulated daily maximum temperature, minimum temperature, and precipitation over australia using probability density functions. J Clim 20:4356–4376. doi: 10.1175/JCLI4253.1 CrossRefGoogle Scholar
  43. Ramesh KV, Goswami P (2014) Assessing reliability of regional climate projections: the case of Indian monsoon. Sci Rep 4:4071. doi: 10.1038/srep04071 CrossRefGoogle Scholar
  44. Ramirez-Villegas J, Challinor AJ, Thornton PK, Jarvis A (2013) Implications of regional improvement in global climate models for agricultural impact research. Environ Res Lett 8:024018. doi: 10.1088/1748-9326/8/2/024018 CrossRefGoogle Scholar
  45. Roehrig R, Bouniol D, Guichard F et al (2013) The present and future of the West African monsoon: a process-oriented assessment of CMIP5 simulations along the AMMA transect. J Clim 26:6471–6475. doi: 10.1175/JCLI-D-12-00505.1 CrossRefGoogle Scholar
  46. Rotstayn LD, Jeffrey SJ, Collier MA et al (2012) Aerosol- and greenhouse gas-induced changes in summer rainfall and circulation in the Australasian region: a study using single-forcing climate simulations. Atmos Chem Phys 12:6377–6404. doi: 10.5194/acp-12-6377-2012 CrossRefGoogle Scholar
  47. Russell GL, Miller JR, Rind D (1995) A coupled atmosphere-ocean model for transient climate change studies. Atmos Ocean 33:683–730CrossRefGoogle Scholar
  48. Sakamoto T, Komuro Y, Nishimura T et al (2012) MIROC4 h—a new high-resolution atmosphere-ocean coupled general circulation model. J Meteorol Soc Japan Ser II 90:325–359. doi: 10.2151/jmsj.2012-301 CrossRefGoogle Scholar
  49. Salas-Mélia D, Chauvin F, Déqué M et al (2005) Description and validation of the CNRM-CM3 global coupled model. CNRM Work note 103 36Google Scholar
  50. Saurral RI, Camilloni IA, Ambrizzi T (2014) Links between topography, moisture fluxes pathways and precipitation over South America. Clim Dyn 45:777–789. doi: 10.1007/s00382-014-2309-z CrossRefGoogle Scholar
  51. Schmidt GA, Ruedy R, Hansen JE et al (2006) Present-day atmospheric simulations using GISS ModelE: comparison to in situ, satellite, and reanalysis data. J Clim 19:153–192. doi: 10.1175/JCLI3612.1 CrossRefGoogle Scholar
  52. Scinocca JF, McFarlane NA, Lazare M et al (2008) The CCCma third generation AGCM and its extension into the middle atmosphere. Atmos Chem Phys Discuss 8:7883–7930. doi: 10.5194/acpd-8-7883-2008 CrossRefGoogle Scholar
  53. Scoccimarro E, Gualdi S, Bellucci A et al (2011) Effects of tropical cyclones on ocean heat transport in a high-resolution coupled general circulation model. J Clim 24:4368–4384. doi: 10.1175/2011JCLI4104.1 CrossRefGoogle Scholar
  54. Sillmann J, Kharin VV, Zhang X et al (2013) Climate extremes indices in the CMIP5 multimodel ensemble: Part 1. Model evaluation in the present climate. J Geophys Res Atmos 118:1716–1733. doi: 10.1002/jgrd.50203 CrossRefGoogle Scholar
  55. Smith DM, Cusack S, Colman AW et al (2007) Improved surface temperature prediction for the coming decade from a global climate model. Science 317:796–799. doi: 10.1126/science.1139540 CrossRefGoogle Scholar
  56. Sperber KR, Annamalai H, Kang I-S et al (2012) The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century. Clim Dyn. doi: 10.1007/s00382-012-1607-6 Google Scholar
  57. Stephens GL, L’Ecuyer T, Forbes R et al (2010) Dreary state of precipitation in global models. J Geophys Res 115:D24211. doi: 10.1029/2010JD014532 CrossRefGoogle Scholar
  58. Stevens B, Giorgetta M, Esch M et al (2013) Atmospheric component of the MPI-M Earth System Model: ECHAM6. J Adv Model Earth Syst 5:146–172. doi: 10.1002/jame.20015 CrossRefGoogle Scholar
  59. Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res 106:7183. doi: 10.1029/2000JD900719 CrossRefGoogle Scholar
  60. Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93:485–498. doi: 10.1175/BAMS-D-11-00094.1 CrossRefGoogle Scholar
  61. Terando A, Easterling WE, Keller K, Easterling DR (2012) Observed and modeled twentieth-century spatial and temporal patterns of selected agro-climate indices in North America. J Clim 25:473–490. doi: 10.1175/2011JCLI4168.1 CrossRefGoogle Scholar
  62. Uppala SM, Kållberg PW, Simmons AJ et al (2005) The ERA-40 re-analysis. Q J R Meteorol Soc 131:2961–3012. doi: 10.1256/qj.04.176 CrossRefGoogle Scholar
  63. Voldoire A, Sanchez-Gomez E, Salas y Mélia D et al (2012) The CNRM-CM5.1 global climate model: description and basic evaluation. Clim Dyn 40:2091–2121. doi: 10.1007/s00382-011-1259-y CrossRefGoogle Scholar
  64. Volodin EM, Dianskii NA, Gusev AV (2010) Simulating present-day climate with the INMCM4.0 coupled model of the atmospheric and oceanic general circulations. Izv Atmos Ocean Phys 46:414–431. doi: 10.1134/S000143381004002X CrossRefGoogle Scholar
  65. Von Salzen K, Scinocca JF, McFarlane NA et al (2013) The Canadian Fourth Generation Atmospheric Global Climate Model (CanAM4). Part I: representation of physical processes. Atmos Ocean 51:104–125. doi: 10.1080/07055900.2012.755610 CrossRefGoogle Scholar
  66. Wada Y, Gleeson T, Esnault L (2014) Wedge approach to water stress. Nat Geosci 7:615–617. doi: 10.1038/ngeo2241 CrossRefGoogle Scholar
  67. Warszawski L, Frieler K, Huber V et al (2014) The Inter-Sectoral Impact Model Intercomparison Project (ISI-MIP): project framework. Proc Natl Acad Sci USA 111:3228–3232. doi: 10.1073/pnas.1312330110 CrossRefGoogle Scholar
  68. Washington WM, Weatherly JW, Meehl GA et al (2000) Parallel climate model (PCM) control and transient simulations. Clim Dyn 16:755–774CrossRefGoogle Scholar
  69. Watanabe M, Suzuki T, O’ishi R et al (2010) Improved climate simulation by MIROC5: mean states, variability, and climate sensitivity. J Clim 23:6312–6335. doi: 10.1175/2010JCLI3679.1 CrossRefGoogle Scholar
  70. Watanabe S, Hajima T, Sudo K et al (2011) MIROC-ESM 2010: model description and basic results of CMIP5-20c3m experiments. Geosci Model Dev 4:845–872. doi: 10.5194/gmd-4-845-2011 CrossRefGoogle Scholar
  71. Weedon GP, Gomes S, Viterbo P et al (2010) The WATCH forcing data 1958–2001: a meteorological forcing dataset for land surface and hydrological models. Watch. Ed. Watch Tech. Rep. 22:Google Scholar
  72. Weedon GP, Balsamo G, Bellouin N et al (2014) The WFDEI meteorological forcing data set: WATCH Forcing Data methodology applied to ERA-Interim reanalysis data. Water Resour Res. doi: 10.1002/2014WR015638 Google Scholar
  73. Wei T, Yang S, Moore JC et al (2012) Developed and developing world responsibilities for historical climate change and CO2 mitigation. Proc Natl Acad Sci USA 109:12911–12915. doi: 10.1073/pnas.1203282109 CrossRefGoogle Scholar
  74. Wu T, Li W, Ji J et al (2013) Global carbon budgets simulated by the Beijing Climate Center Climate System Model for the last century. J Geophys Res Atmos 118:4326–4347. doi: 10.1002/jgrd.50320 CrossRefGoogle Scholar
  75. Yin L, Fu R, Shevliakova E, Dickinson RE (2012) How well can CMIP5 simulate precipitation and its controlling processes over tropical South America? Clim Dyn 41:3127–3143. doi: 10.1007/s00382-012-1582-y CrossRefGoogle Scholar
  76. Yongqiang Y, Xuehong Z, Yufu G (2004) Global coupled ocean-atmosphere general circulation models in LASG/IAP. Adv Atmos Sci 21:444–455CrossRefGoogle Scholar
  77. Yukimoto S, Noda A, Kitoh A et al (2006) Present-day climate and climate sensitivity in the Meteorological Research Institute coupled GCM version 2.3 (MRI-CGCM2.3). J Meteorol Soc Jpn Ser 2 84:333CrossRefGoogle Scholar
  78. Yukimoto S, Adachi Y, Hosaka M et al (2012) A New Global Climate Model of the Meteorological Research Institute: MRI-CGCM3—model description and basic performance. J Meteorol Soc Jpn Ser II 90A:23–64. doi: 10.2151/jmsj.2012-A02 CrossRefGoogle Scholar
  79. Zhang J, Li L, Zhou T, Xin X (2013) Evaluation of spring persistent rainfall over East Asia in CMIP3/CMIP5 AGCM simulations. Adv Atmos Sci 30:1587–1600. doi: 10.1007/s00376-013-2139-7 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.School of Environmental EngineeringTechnical University of CreteChaniaGreece
  2. 2.Department of Civil EngineeringMcMaster UniversityHamiltonCanada

Personalised recommendations