Skip to main content

Advertisement

SpringerLink
Log in

Evaluation of precipitation and temperature simulation performance of the CMIP3 and CMIP5 historical experiments

  • Published:
Climate Dynamics Aims and scope Submit manuscript

Abstract

The fifth phase of the Coupled Model Intercomparison Project (CMIP5) is the most recent coordinated experiment of global climate modeling. Compared to its predecessor CMIP3, the fifth phase of the homonymous experiment—CMIP5 involves a greater number of GCMs, run at higher resolutions with more complex components. Here we use daily GCM data from both projects to test their efficiency in representing precipitation and temperature parameters with the use of a state of the art high resolution gridded global dataset for land areas and for the period 1960–2005. Two simple metrics, a comprehensive histogram similarity metric based on the match of simulated and observed empirical pdfs and a metric for the representation of the annual cycle were employed as performance indicators. The metrics were used to assess the skill of each GCM at the entire spectrum of precipitation and temperature pdfs but also for the upper and lower tails of it. Results are presented globally and regionally for 26 land regions that represent different climatic regimes, covering the total earth’s land surface except for Antarctica. Compared to CMIP3, CMIP5 models perform better in simulating precipitation including relatively intense events and the fraction of wet days. For temperature the improvement is not as clear except for the upper and lower hot and cold events of the distribution. The agreement of model simulations is also considerably increased in CMIP5. Substantial improvement in intense precipitation is observed over North Europe, Central and Eastern North America and North East Europe. Nevertheless, in both ensembles some models clearly perform better than others from a histogram similarity point of view. The derived skill score metrics provide essential information for impact studies based on global or regional land area multi-model ensembles.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Abdullah K (2006) Use of water and land for food security and environmental sustainability. Irrig Drain 55:219–222. doi:10.1002/ird.254

    Article  Google Scholar 

  • Adler RF, Huffman GJ, Chang A et al (2003) The Version-2 Global Precipitation Climatology Project (GPCP) monthly precipitation analysis (1979–Present). J Hydrometeorol 4:1147–1167. doi:10.1175/1525-7541(2003)004<1147:TVGPCP>2.0.CO;2

    Article  Google Scholar 

  • Adler RF, Gu G, Huffman GJ (2012) Estimating climatological bias errors for the Global Precipitation Climatology Project (GPCP). J Appl Meteorol Climatol 51:84–99. doi:10.1175/JAMC-D-11-052.1

    Article  Google Scholar 

  • Bao Q, Lin P, Zhou T et al (2013) The Flexible Global Ocean-Atmosphere-Land system model, Spectral Version 2: FGOALS-s2. Adv Atmos Sci 30:561–576. doi:10.1007/s00376-012-2113-9

    Article  Google Scholar 

  • Bhend J, Whetton P (2013) Consistency of simulated and observed regional changes in temperature, sea level pressure and precipitation. Clim Change 118:799–810. doi:10.1007/s10584-012-0691-2

    Article  Google Scholar 

  • Christensen JH, Kjellström E, Giorgi F et al (2010) Weight assignment in regional climate models. Clim Res 44:179–194

    Article  Google Scholar 

  • Collins M, Tett SFB, Cooper C (2001) The internal climate variability of HadCM3, a version of the Hadley Centre coupled model without flux adjustments. Clim Dyn 17:61–81. doi:10.1007/s003820000094

    Article  Google Scholar 

  • Dee DP, Uppala SM, Simmons AJ et al (2011) The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q J R Meteorol Soc 137:553–597. doi:10.1002/qj.828

    Article  Google Scholar 

  • Deliverable E (2008) D3. 2.2. RCM-specific Weight based their Abil. to simulate Present Clim. calibrated ERA40-based simulations. www.ensembleseu.org

  • Delworth TL, Broccoli AJ, Rosati A et al (2006) GFDL’s CM2 global coupled climate models. Part I: formulation and simulation characteristics. J Clim 19:643–674. doi:10.1175/JCLI3629.1

    Article  Google Scholar 

  • K-1 model Developers (2004) K-1 coupled model (MIROC) description. Ed by H Hasumi Emori, S Cent Clim Syst Res Univ Tokyo K-1 technics, pp 1–34

  • Dufresne J-L, Foujols M, Denvil S et al (2013) Climate change projections using the IPSL-CM5 Earth System Model: from CMIP3 to CMIP5. Clim Dyn 40:2123–2165. doi:10.1007/s00382-012-1636-1

    Article  Google Scholar 

  • Flato G, Marotzke J, Abiodun B, Braconnot P, Chou SC, Collins W, Cox P, Driouech F, Emori S, Eyring V, Forest C, Gleckler P, Guilyardi E, Jakob C, Kattsov V, Reason C, Rummukainen M (2013) Evaluation of climate models. In: Stocker TF et al (eds) Climate change 2013: the physical science basis. Cambridge University Press, Cambridge, pp 741–866

  • Franklin CN, Sun Z, Bi D et al (2013) Evaluation of clouds in ACCESS using the satellite simulator package COSP: regime-sorted tropical cloud properties. J Geophys Res Atmos 118:6663–6679. doi:10.1002/jgrd.50496

    Article  Google Scholar 

  • Furevik T, Bentsen M, Drange H et al (2003) Description and evaluation of the bergen climate model: ARPEGE coupled with MICOM. Clim Dyn 21:27–51. doi:10.1007/s00382-003-0317-5

    Article  Google Scholar 

  • Giorgi F, Bi X (2005) Updated regional precipitation and temperature changes for the 21st century from ensembles of recent AOGCM simulations. Geophys Res Lett 32:L21715. doi:10.1029/2005GL024288

    Article  Google Scholar 

  • Gordon HB, Rotstayn LD, McGregor JL et al (2002) The CSIRO Mk3 Climate System Model. CSIRO Atmos Res Tech Pap no 60:130

    Google Scholar 

  • Gordon H, O’Farrell S, Collier M et al (2010) The CSIRO Mk3. 5 climate model. CAWCR

  • Grillakis MG, Koutroulis AG, Tsanis IK (2013) Multisegment statistical bias correction of daily GCM precipitation output. J Geophys Res Atmos 118:3150–3162. doi:10.1002/jgrd.50323

    Article  Google Scholar 

  • Guemas V, Codron F (2011) Differing impacts of resolution changes in latitude and longitude on the midlatitudes in the LMDZ atmospheric GCM. J Clim 24:5831–5849. doi:10.1175/2011JCLI4093.1

    Article  Google Scholar 

  • Harding R, Best M, Blyth E et al (2011) WATCH: current knowledge of the terrestrial global water cycle. J Hydrometeorol 12:1149–1156. doi:10.1175/JHM-D-11-024.1

    Article  Google Scholar 

  • Hirota N, Takayabu YN (2013) Reproducibility of precipitation distribution over the tropical oceans in CMIP5 multi-climate models compared to CMIP3. Clim Dyn 41:2909–2920

    Article  Google Scholar 

  • Hourdin F, Foujols M-A, Codron F et al (2012) Impact of the LMDZ atmospheric grid configuration on the climate and sensitivity of the IPSL-CM5A coupled model. Clim Dyn 40:2167–2192. doi:10.1007/s00382-012-1411-3

    Article  Google Scholar 

  • Huang W, Wang B, Li L et al (2013) Variability of atlantic meridional overturning circulation in FGOALS-g2. Adv Atmos Sci 31:95–109. doi:10.1007/s00376-013-2155-7

    Article  Google Scholar 

  • Iversen T, Bentsen M, Bethke I et al (2013) The Norwegian Earth System Model, NorESM1-M—Part 2: climate response and scenario projections. Geosci Model Dev 6:389–415. doi:10.5194/gmd-6-389-2013

    Article  Google Scholar 

  • Johns TC, Durman CF, Banks HT et al (2006) The New Hadley Centre Climate Model (HadGEM1): evaluation of coupled simulations. J Clim 19:1327–1353. doi:10.1175/JCLI3712.1

    Article  Google Scholar 

  • Jones C, Carvalho LMV (2013) Climate change in the South American Monsoon System: present climate and CMIP5 projections. J Clim 26:6660–6678. doi:10.1175/JCLI-D-12-00412.1

    Article  Google Scholar 

  • Jones CD, Hughes JK, Bellouin N et al (2011) The HadGEM2-ES implementation of CMIP5 centennial simulations. Geosci Model Dev 4:543–570. doi:10.5194/gmd-4-543-2011

    Article  Google Scholar 

  • Jungclaus JH, Keenlyside N, Botzet M et al (2006) Ocean circulation and tropical variability in the coupled model ECHAM5/MPI-OM. J Clim 19

  • Kalnay E, Kanamitsu M, Kistler R et al (1996) The NCEP/NCAR 40-year reanalysis project. Bull Am Meteorol Soc 77:437–471. doi:10.1175/1520-0477(1996)077<0437:TNYRP>2.0.CO;2

    Article  Google Scholar 

  • Kanamitsu M, Ebisuzaki W, Woollen J et al (2002) NCEP–DOE AMIP-II reanalysis (R-2). Bull Am Meteorol Soc 83:1631–1643. doi:10.1175/BAMS-83-11-1631

    Article  Google Scholar 

  • Kjellström E, Boberg F, Castro M et al (2010) Daily and monthly temperature and precipitation statistics as performance indicators for regional climate models. Clim Res 44:135–150. doi:10.3354/cr00932

    Article  Google Scholar 

  • Knutti R, Sedláček J (2013) Robustness and uncertainties in the new CMIP5 climate model projections. Nat Clim Change 3:369–373. doi:10.1038/nclimate1716

    Article  Google Scholar 

  • Knutti R, Masson D, Gettelman A (2013) Climate model genealogy: generation CMIP5 and how we got there. Geophys Res Lett 40:1194–1199. doi:10.1002/grl.50256

    Article  Google Scholar 

  • Kopparla P, Fischer EM, Hannay C, Knutti R (2013) Improved simulation of extreme precipitation in a high-resolution atmosphere model. Geophys Res Lett 40:5803–5808

    Article  Google Scholar 

  • Kottek M, Grieser J, Beck C et al (2006) World map of the Köppen–Geiger climate classification updated. Meteorol Zeitschrift 15:259–263

    Article  Google Scholar 

  • Kumar S, Merwade V, Kinter JL, Niyogi D (2013) Evaluation of temperature and precipitation trends and long-term persistence in CMIP5 twentieth-century climate simulations. J Clim. doi:10.1175/JCLI-D-12-00259.1

    Google Scholar 

  • Kumar D, Kodra E, Ganguly AR (2014) Regional and seasonal intercomparison of CMIP3 and CMIP5 climate model ensembles for temperature and precipitation. Clim Dyn 43:1–28. doi:10.1007/s00382-014-2070-3

    Article  Google Scholar 

  • Marti O, Braconnot P, Bellier J et al (2005) The new IPSL climate system model: IPSL-CM4. Note du Pôle Modélisation, IPSL 26:1–86

    Google Scholar 

  • Meehl GA, Washington WM, Santer BD et al (2006) Climate change projections for the twenty-first century and climate change commitment in the CCSM3. J Clim 19

  • Meehl GA, Washington WM, Arblaster JM et al (2012) Climate system response to external forcings and climate change projections in CCSM4. J Clim 25:3661–3683. doi:10.1175/JCLI-D-11-00240.1

    Article  Google Scholar 

  • Perkins SE, Pitman AJ, Holbrook NJ, McAneney J (2007) Evaluation of the AR4 climate models’ simulated daily maximum temperature, minimum temperature, and precipitation over australia using probability density functions. J Clim 20:4356–4376. doi:10.1175/JCLI4253.1

    Article  Google Scholar 

  • Ramesh KV, Goswami P (2014) Assessing reliability of regional climate projections: the case of Indian monsoon. Sci Rep 4:4071. doi:10.1038/srep04071

    Article  Google Scholar 

  • Ramirez-Villegas J, Challinor AJ, Thornton PK, Jarvis A (2013) Implications of regional improvement in global climate models for agricultural impact research. Environ Res Lett 8:024018. doi:10.1088/1748-9326/8/2/024018

    Article  Google Scholar 

  • Roehrig R, Bouniol D, Guichard F et al (2013) The present and future of the West African monsoon: a process-oriented assessment of CMIP5 simulations along the AMMA transect. J Clim 26:6471–6475. doi:10.1175/JCLI-D-12-00505.1

    Article  Google Scholar 

  • Rotstayn LD, Jeffrey SJ, Collier MA et al (2012) Aerosol- and greenhouse gas-induced changes in summer rainfall and circulation in the Australasian region: a study using single-forcing climate simulations. Atmos Chem Phys 12:6377–6404. doi:10.5194/acp-12-6377-2012

    Article  Google Scholar 

  • Russell GL, Miller JR, Rind D (1995) A coupled atmosphere-ocean model for transient climate change studies. Atmos Ocean 33:683–730

    Article  Google Scholar 

  • Sakamoto T, Komuro Y, Nishimura T et al (2012) MIROC4 h—a new high-resolution atmosphere-ocean coupled general circulation model. J Meteorol Soc Japan Ser II 90:325–359. doi:10.2151/jmsj.2012-301

    Article  Google Scholar 

  • Salas-Mélia D, Chauvin F, Déqué M et al (2005) Description and validation of the CNRM-CM3 global coupled model. CNRM Work note 103 36

  • Saurral RI, Camilloni IA, Ambrizzi T (2014) Links between topography, moisture fluxes pathways and precipitation over South America. Clim Dyn 45:777–789. doi:10.1007/s00382-014-2309-z

    Article  Google Scholar 

  • Schmidt GA, Ruedy R, Hansen JE et al (2006) Present-day atmospheric simulations using GISS ModelE: comparison to in situ, satellite, and reanalysis data. J Clim 19:153–192. doi:10.1175/JCLI3612.1

    Article  Google Scholar 

  • Scinocca JF, McFarlane NA, Lazare M et al (2008) The CCCma third generation AGCM and its extension into the middle atmosphere. Atmos Chem Phys Discuss 8:7883–7930. doi:10.5194/acpd-8-7883-2008

    Article  Google Scholar 

  • Scoccimarro E, Gualdi S, Bellucci A et al (2011) Effects of tropical cyclones on ocean heat transport in a high-resolution coupled general circulation model. J Clim 24:4368–4384. doi:10.1175/2011JCLI4104.1

    Article  Google Scholar 

  • Sillmann J, Kharin VV, Zhang X et al (2013) Climate extremes indices in the CMIP5 multimodel ensemble: Part 1. Model evaluation in the present climate. J Geophys Res Atmos 118:1716–1733. doi:10.1002/jgrd.50203

    Article  Google Scholar 

  • Smith DM, Cusack S, Colman AW et al (2007) Improved surface temperature prediction for the coming decade from a global climate model. Science 317:796–799. doi:10.1126/science.1139540

    Article  Google Scholar 

  • Sperber KR, Annamalai H, Kang I-S et al (2012) The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century. Clim Dyn. doi:10.1007/s00382-012-1607-6

    Google Scholar 

  • Stephens GL, L’Ecuyer T, Forbes R et al (2010) Dreary state of precipitation in global models. J Geophys Res 115:D24211. doi:10.1029/2010JD014532

    Article  Google Scholar 

  • Stevens B, Giorgetta M, Esch M et al (2013) Atmospheric component of the MPI-M Earth System Model: ECHAM6. J Adv Model Earth Syst 5:146–172. doi:10.1002/jame.20015

    Article  Google Scholar 

  • Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res 106:7183. doi:10.1029/2000JD900719

    Article  Google Scholar 

  • Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93:485–498. doi:10.1175/BAMS-D-11-00094.1

    Article  Google Scholar 

  • Terando A, Easterling WE, Keller K, Easterling DR (2012) Observed and modeled twentieth-century spatial and temporal patterns of selected agro-climate indices in North America. J Clim 25:473–490. doi:10.1175/2011JCLI4168.1

    Article  Google Scholar 

  • Uppala SM, Kållberg PW, Simmons AJ et al (2005) The ERA-40 re-analysis. Q J R Meteorol Soc 131:2961–3012. doi:10.1256/qj.04.176

    Article  Google Scholar 

  • Voldoire A, Sanchez-Gomez E, Salas y Mélia D et al (2012) The CNRM-CM5.1 global climate model: description and basic evaluation. Clim Dyn 40:2091–2121. doi:10.1007/s00382-011-1259-y

    Article  Google Scholar 

  • Volodin EM, Dianskii NA, Gusev AV (2010) Simulating present-day climate with the INMCM4.0 coupled model of the atmospheric and oceanic general circulations. Izv Atmos Ocean Phys 46:414–431. doi:10.1134/S000143381004002X

    Article  Google Scholar 

  • Von Salzen K, Scinocca JF, McFarlane NA et al (2013) The Canadian Fourth Generation Atmospheric Global Climate Model (CanAM4). Part I: representation of physical processes. Atmos Ocean 51:104–125. doi:10.1080/07055900.2012.755610

    Article  Google Scholar 

  • Wada Y, Gleeson T, Esnault L (2014) Wedge approach to water stress. Nat Geosci 7:615–617. doi:10.1038/ngeo2241

    Article  Google Scholar 

  • Warszawski L, Frieler K, Huber V et al (2014) The Inter-Sectoral Impact Model Intercomparison Project (ISI-MIP): project framework. Proc Natl Acad Sci USA 111:3228–3232. doi:10.1073/pnas.1312330110

    Article  Google Scholar 

  • Washington WM, Weatherly JW, Meehl GA et al (2000) Parallel climate model (PCM) control and transient simulations. Clim Dyn 16:755–774

    Article  Google Scholar 

  • Watanabe M, Suzuki T, O’ishi R et al (2010) Improved climate simulation by MIROC5: mean states, variability, and climate sensitivity. J Clim 23:6312–6335. doi:10.1175/2010JCLI3679.1

    Article  Google Scholar 

  • Watanabe S, Hajima T, Sudo K et al (2011) MIROC-ESM 2010: model description and basic results of CMIP5-20c3m experiments. Geosci Model Dev 4:845–872. doi:10.5194/gmd-4-845-2011

    Article  Google Scholar 

  • Weedon GP, Gomes S, Viterbo P et al (2010) The WATCH forcing data 1958–2001: a meteorological forcing dataset for land surface and hydrological models. Watch. Ed. Watch Tech. Rep. 22:

  • Weedon GP, Balsamo G, Bellouin N et al (2014) The WFDEI meteorological forcing data set: WATCH Forcing Data methodology applied to ERA-Interim reanalysis data. Water Resour Res. doi:10.1002/2014WR015638

    Google Scholar 

  • Wei T, Yang S, Moore JC et al (2012) Developed and developing world responsibilities for historical climate change and CO2 mitigation. Proc Natl Acad Sci USA 109:12911–12915. doi:10.1073/pnas.1203282109

    Article  Google Scholar 

  • Wu T, Li W, Ji J et al (2013) Global carbon budgets simulated by the Beijing Climate Center Climate System Model for the last century. J Geophys Res Atmos 118:4326–4347. doi:10.1002/jgrd.50320

    Article  Google Scholar 

  • Yin L, Fu R, Shevliakova E, Dickinson RE (2012) How well can CMIP5 simulate precipitation and its controlling processes over tropical South America? Clim Dyn 41:3127–3143. doi:10.1007/s00382-012-1582-y

    Article  Google Scholar 

  • Yongqiang Y, Xuehong Z, Yufu G (2004) Global coupled ocean-atmosphere general circulation models in LASG/IAP. Adv Atmos Sci 21:444–455

    Article  Google Scholar 

  • Yukimoto S, Noda A, Kitoh A et al (2006) Present-day climate and climate sensitivity in the Meteorological Research Institute coupled GCM version 2.3 (MRI-CGCM2.3). J Meteorol Soc Jpn Ser 2 84:333

    Article  Google Scholar 

  • Yukimoto S, Adachi Y, Hosaka M et al (2012) A New Global Climate Model of the Meteorological Research Institute: MRI-CGCM3—model description and basic performance. J Meteorol Soc Jpn Ser II 90A:23–64. doi:10.2151/jmsj.2012-A02

    Article  Google Scholar 

  • Zhang J, Li L, Zhou T, Xin X (2013) Evaluation of spring persistent rainfall over East Asia in CMIP3/CMIP5 AGCM simulations. Adv Atmos Sci 30:1587–1600. doi:10.1007/s00376-013-2139-7

    Article  Google Scholar 

Download references

Acknowledgements

A. Koutroulis and L. Papadimitriou were partly supported by the High-End cLimate Impacts and eXtremes (HELIX) project which has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under Grant Agreement No. 603864. We acknowledge the World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP, and we thank the climate modeling groups (listed in Table 1 of this paper) for producing and making available their model output. We also acknowledge the modeling groups, the Program for Climate Model Diagnosis and Intercomparison (PCMDI) and the WCRP’s Working Group on Coupled Modelling (WGCM) for their roles in making available the WCRP CMIP3 multi-model dataset. For CMIP the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led development of software infrastructure in partnership with the Global Organization for Earth System Science Portals.

Author information

Authors and Affiliations

  1. School of Environmental Engineering, Technical University of Crete, Chania, Greece

    A. G. Koutroulis, M. G. Grillakis, I. K. Tsanis & L. Papadimitriou

  2. Department of Civil Engineering, McMaster University, Hamilton, Canada

    I. K. Tsanis

Authors
  1. A. G. Koutroulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. G. Grillakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I. K. Tsanis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. L. Papadimitriou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to I. K. Tsanis.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 9926 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koutroulis, A.G., Grillakis, M.G., Tsanis, I.K. et al. Evaluation of precipitation and temperature simulation performance of the CMIP3 and CMIP5 historical experiments. Clim Dyn 47, 1881–1898 (2016). https://doi.org/10.1007/s00382-015-2938-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00382-015-2938-x

Keywords

Search

Navigation