Evaluation of CMIP5 ability to reproduce twentieth century regional trends in surface air temperature and precipitation over CONUS

  • Jinny LeeEmail author
  • Duane Waliser
  • Huikyo Lee
  • Paul Loikith
  • Kenneth E. Kunkel


The ability of the 5th phase of the Coupled Model Intercomparison Project (CMIP5) to reproduce twentieth-century climate trends over the seven CONUS regions of the National Climate Assessment is evaluated. This evaluation is carried out for summer and winter for three time periods, 1895–1939, 1940–1979, and 1980–2005. The evaluation includes all 206 CMIP5 historical simulations from 48 unique models and their multi-model ensemble (MME), as well as a gridded in situ dataset of surface air temperature and precipitation. Analysis is performed on both individual members and the MME, and considers reproducing the correct sign of the trends by the members as well as reproducing the trend values. While the MME exhibits some trend bias in most cases, it reproduces historical temperature trends with reasonable fidelity for summer for all time periods and all regions, including at the CONUS scale, except the Northern Great Plains from 1895 to 1939 and Southeast during 1980–2005. Likewise, for DJF, the MME reproduces historical temperature trends across all time periods over all regions, including at the CONUS scale, except the Southeast from 1895 to 1939 and the Midwest during 1940–1979. Model skill was highest across all of the seven regions during JJA and DJF for the 1980–2005 period. The quantitatively best result is seen during DJF in the Southwest region with at least 74% of the ensemble members correctly reproducing the observed trend across all of the time periods. No clear trends in MME precipitation were identified at these scales due to high model precipitation variability.


CMIP5 Model evaluation Surface air temperature Multi-model ensemble 



We would like to acknowledge the World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP, and we thank the climate modeling groups (listed in “Appendix B” of this paper) for producing and making available their model output. For CMIP the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led development of software infrastructure in partnership with the Global Organization for Earth System Science Portals. The primary author would also like to acknowledge California State University, Los Angeles NASA DIRECT-STEM program and director, Dr. Hengchun Ye for funding and support. This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. Support for this project was provided by NASA National Climate Assessment 11-NCA 11-0028. Kenneth Kunkel was supported by NOAA through the Cooperative Institute for Climate and Satellites—North Carolina under Cooperative Agreement NA14NES432003.

Supplementary material

382_2019_4875_MOESM1_ESM.docx (886 kb)
Supplementary material 1 (DOCX 886 kb)


  1. Baek H-J, Lee J, Lee H-S, Hyun Y-K, Cho C, Kwon W-T, Marzin C, Gan SY, Kim MJ, Choi DH, Lee J, Lee J, Boo K-O, Kang H-S, Byun Y-H (2013) Climate change in the 21st century simulated by HadGEM2-AO under representative concentration pathways. Asia Pac J Atmos Sci 49(5):603–618. CrossRefGoogle Scholar
  2. Barnett TP, Pierce DW, Hidalgo HG, Bonfils C, Santer BD, Das T, Bala G, Wood AW, Nozawa T, Mirin AA, Cayan DR, Dettinger MD (2008) Human-induced changes in the hydrology of the western United States. Science 319(5866):1080–1083. Retrieved from
  3. Bentsen M, Bethke I, Debernard JB, Iversen T, Kirkevåg A, Seland Ø, Drange H, Roelandt C, Seierstad IA, Hoose C, Kristjánsson JE (2013) The Norwegian earth system model, NorESM1-M—part 1: description and basic evaluation of the physical climate. Geosci Model Dev 6(3):687–720. CrossRefGoogle Scholar
  4. Bukovsky MS (2012) Temperature trends in the NARCCAP regional climate models. J Clim 25(11):3985–3991. CrossRefGoogle Scholar
  5. Cayan DR, Tyree M, Kunkel KE, Castro C, Gershunov A, Barsugli J, Ray AJ, Overpeck J, Anderson A, Russell J, Rajagopalan B, Rangwala I, Duffy P, Barlow M (2013) Future climate: projected average. In: Garfin G, Jardine A, Merideth R, Black M, LeRoy S (eds) Assessment of climate change in the southwest United States: a report prepared for the national climate assessment. Island Press, Washington, DC, pp 101–125. CrossRefGoogle Scholar
  6. Chylek P, Li J, Dubey MK, Wang M, Lesins G (2011) Observed and model simulated 20th century Arctic temperature variability: Canadian earth system model CanESM2. Atmos Chem Phys Discuss 11(8):22893–22907. CrossRefGoogle Scholar
  7. Collier M, Uhe P (2012) CMIP5 datasets from the ACCESS1.0 and ACCESS1.3 coupled climate models. CAWCR technical report no. 059. ISBN: 20978-1-922173-29-4Google Scholar
  8. Collins WD, Rasch PJ, Boville BA, Hack JJ, Williamson DL, Kiehl JT, Briegleb B, Bitz C, Lin SJ, Zhang M, Dai Y (2004) Description of the NCAR community atmosphere model (CAM 3.0). Ncar/Tn-464+ Str (June), 214Google Scholar
  9. Collins WJ, Bellouin N, Doutriaux-Boucher M, Gedney N, Halloran P, Hinton T, Hughes J, Jones CD, Joshi M, Liddicoat S, Martin G, O’Connor F, Rae J, Senior C, Sitch S, Totterdell I, Wiltshire A, Woodward S (2011) Development and evaluation of an earth-system model—HadGEM2. Geosci Model Dev 4(4):1051–1075. CrossRefGoogle Scholar
  10. Crowley TJ (2000) Causes of climate change over the past 1000 years. Science 289(5477):270–277. Retrieved from
  11. Delworth TL, Broccoli AJ, Rosati A, Stouffer RJ, Balaji V, Beesley JA, Cooke WF, Dixon KW, Dunne J, Dunne KA, Durachta JW, Findell KL, Ginoux P, Gnanadesikan A, Gordon CT, Griffies SM, Gudgel R, Harrison MJ, Held IM, Hemler RS, Horowitz LW, Klein SA, Knutson TR, Kushner PJ, Langenhorst AR, Lee H-C, Lin S-J, Lu J, Malyshev SL, Milly PCD, Ramaswamy V, Russell J, Schwarzkopf MD, Shevliakova E, Sirutis JJ, Spelman MJ, Stern WF, Winton M, Wittenberg AT, Wyman B, Zeng F, Zhang R (2006) GFDL’s CM2 global coupled climate models. Part I: formulation and simulation characteristics. J Clim 19(5):643–674. CrossRefGoogle Scholar
  12. Donner LJ, Wyman BL, Hemler RS, Horowitz LW, Ming Y, Zhao M, Golaz J-C, Ginoux P, Lin S-J, Schwarzkopf MD, Austin J, Alaka G, Cooke WF, Delworth TL, Freidenreich SM, Gordon CT, Griffies SM, Held IM, Hurlin WJ, Klein SA, Knutson TR, Langenhorst AR, Lee H-C, Lin Y, Magi BI, Malyshev SL, Milly PCD, Naik V, Nath MJ, Pincus R, Ploshay JJ, Ramaswamy V, Seman CJ, Shevliakova E, Sirutis JJ, Stern WF, Stouffer RJ, Wilson RJ, Winton M, Wittenberg AT, Zeng F (2011) The dynamical core, physical parameterizations, and basic simulation characteristics of the atmospheric component AM3 of the GFDL global coupled model CM3. J Clim 24(13):3484–3519. CrossRefGoogle Scholar
  13. Dufresne J-L, Foujols M-A, Denvil S, Caubel A, Marti O, Aumont O, Balkanski Y, Bekki S, Bellenger H, Benshila R, Bony S, Bopp L, Braconnot P, Brockmann P, Cadule P, Cheruy F, Codron F, Cozic A, Cugnet D, de Noblet N, Duvel J-P, Ethé C, Fairhead L, Fichefet T, Flavoni S, Friedlingstein P, Grandpeix J-Y, Guez L, Guilyardi E, Hauglustaine D, Hourdin F, Idelkadi A, Ghattas J, Joussaume S, Kageyama M, Krinner G, Labetoulle S, Lahellec A, Lefebvre M-P, Lefevre F, Levy C, Li ZX, Lloyd J, Lott F, Madec G, Mancip M, Marchand M, Masson S, Meurdesoif Y, Mignot J, Musat I, Parouty S, Polcher J, Rio C, Schulz M, Swingedouw D, Szopa S, Talandier C, Terray P, Viovy N, Vuichard N (2013) Climate change projections using the IPSL-CM5 earth system model: from CMIP3 to CMIP5. Clim Dyn 40(9):2123–2165. CrossRefGoogle Scholar
  14. Dunne JP, John JG, Shevliakova E, Stouffer RJ, Krasting JP, Malyshev SL, Milly PC, Sentman LT, Adcroft AJ, Cooke W, Dunne KA, Griffies SM, Hallberg RW, Harrison MJ, Levy H, Wittenberg AT, Phillips PJ, Zadeh N (2013) GFDL’s ESM2 global coupled climate–carbon earth system models. Part II: carbon system formulation and baseline simulation characteristics. J Clim 26(7):2247–2267. CrossRefGoogle Scholar
  15. Easterling DR, Kunkel KE, Arnold JR, Knutson T, LeGrande AN, Leung LR, Vose RS, Waliser DE, Wehner MF (2017) Precipitation change in the United States. In: Wuebbles DJ, Fahey DW, Hibbard KA, Dokken DJ, Stewart BC, Maycock TK (eds) Climate science special report: fourth national climate assessment, vol I. U.S. Global Change Research Program, pp 207–230.
  16. Fogli PG, Iovino D (2014) CMCC–CESM–NEMO: toward the new CMCC earth system model. In: CMCC research paper, no 248.
  17. Gordon H, Farrell SO, Collier M, Dix M, Rotstayn L, Kowalczyk E, Hirst T, Watterson I (2010) The CSIRO Mk3.5 Climate Model. In: CAWCR Technical Report, no 21. CAWCR, Melbourne, pp 1–74Google Scholar
  18. Hazeleger W, Wang X, Severijns C, Ştefănescu S, Bintanja R, Sterl A, Wyser K, Semmler T, Yang S, Van den Hurk B (2012) EC-Earth V2. 2: description and validation of a new seamless earth system prediction model. Clim Dyn 39(11):2611–2629CrossRefGoogle Scholar
  19. Hogg RV, Tanis EA (2009) Probability and statistical inference. Pearson Educational International, LondonGoogle Scholar
  20. IPCC (2013) Climate change 2013: the physical science basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, CambridgeGoogle Scholar
  21. Janssen E, Wuebbles D, Kunkel K (2014) Observational and model based trends and projections of extreme precipitation over the contiguous United States. Earth’s Future. Google Scholar
  22. Janssen E, Sriver RL, Wuebbles DJ, Kunkel KE (2016) Seasonal and regional variations in extreme precipitation event frequency using CMIP5. Geophys Res Lett 43(10):5385–5393. CrossRefGoogle Scholar
  23. Ji D, Wang L, Feng J, Wu Q, Cheng H, Zhang Q, Yang J, Dong W, Dai Y, Gong D, Zhang RH, Wang X, Liu J, Moore JC, Chen D, Zhou M (2014) Description and basic evaluation of Beijing Normal University earth system model (BNU-ESM) version 1. Geosci Model Dev 7(5):2039–2064. CrossRefGoogle Scholar
  24. Karl TR, Knight RW, Easterling DR, Quayle RG (1996) Indices of climate change for the United States. Bull Am Meteorol Soc 77(2):279–292.;2 CrossRefGoogle Scholar
  25. Knutson TR, Delworth TL, Dixon KW, Held IM, Lu J, Ramaswamy V, Schwarzkopf MD, Stenchikov G, Stouffer RJ (2006) Assessment of twentieth-century regional surface temperature trends using the GFDL CM2 coupled models. J Clim 19(9):1624–1651. CrossRefGoogle Scholar
  26. Knutson TR, Zeng F, Wittenberg AT (2013) Multimodel assessment of regional surface temperature trends: CMIP3 and CMIP5 twentieth-century simulations. J Clim 26(22):8709–8743. CrossRefGoogle Scholar
  27. Kumar S III, Kinter J, Dirmeyer PA, Pan Z, Adams J (2013a) Multidecadal climate variability and the “warming hole” in North America: results from CMIP5 twentieth- and twenty-first-century climate simulations. J Clim 26(11):3511–3527. CrossRefGoogle Scholar
  28. Kumar S, Merwade V, Kinter JL, Niyogi D (2013b) Evaluation of temperature and precipitation trends and long-term persistence in CMIP5 twentieth-century climate simulations. J Clim 26(12):4168–4185. CrossRefGoogle Scholar
  29. Kunkel KE, Liang XZ (2005) GCM simulations of the climate in the central United States. J Clim 18(7):1016–1031. CrossRefGoogle Scholar
  30. Kunkel KE, Easterling DR, Redmond K, Hubbard K (2003) Temporal variations of extreme precipitation events in the United States: 1895–2000. Geophys Res Lett. Google Scholar
  31. Kunkel KE, Liang X-Z, Zhu J, Lin Y (2006) Can CGCMs simulate the twentieth-century “warming hole” in the central United States? J Clim 19(17):4137–4153. CrossRefGoogle Scholar
  32. Kunkel K, Stevens LE, Stevens SE, Sun L, Janssen E, Wuebbles D, Dobson JG (2013) Regional climate trends and scenarios for the U. S. National Climate Assessment part 9. Climate of the contiguous United States (January), 77. Retrieved from
  33. Kunkel KE, Vose RS, Stevens LE, Knight RW (2015) Is the monthly temperature climate of the United States becoming more extreme? Geophys Res Lett 42(2):629–636. CrossRefGoogle Scholar
  34. Li L, Lin P, Yu Y, Wang B, Zhou T, Liu L, Liu J, Bao Q, Xu S, Huang W, Xia K, Pu Y, Dong L, Shen S, Liu Y, Hu N, Liu M, Sun W, Shi X, Zheng W, Wu B, Song M, Liu H, Zhang X, Wu G, Xue W, Huang X, Yang G, Song Z, Qiao F (2013) The flexible global ocean-atmosphere-land system model, grid-point version 2: FGOALS-g2. Adv Atmos Sci 30(3):543–560. CrossRefGoogle Scholar
  35. Marsh DR, Mills MJ, Kinnison DE, Lamarque J-F, Calvo N, Polvani LM (2013) Climate change from 1850 to 2005 simulated in CESM1(WACCM). J Clim 26(19):7372–7391. CrossRefGoogle Scholar
  36. Meehl GA, Tebaldi C (2004) More intense, more frequent, and longer lasting heat waves in the 21st century. Science 305(5686):994–997. Retrieved from
  37. Meehl GA, Arblaster JM, Branstator G (2012) Mechanisms contributing to the warming hole and the consequent U.S. east–west differential of heat extremes. J Clim 25(18):6394–6408. CrossRefGoogle Scholar
  38. Melillo JM, Richmond TC, Yohe GW, US National Climate Assessment (2014) Climate change impacts in the United States: the third national climate assessment. US Global Change Research Program, vol 841.
  39. Menne MJ, Durre I, Vose RS, Gleason BE, Houston TG (2012) An overview of the global historical climatology network-daily database. J Atmos Ocean Technol 29(7):897–910. CrossRefGoogle Scholar
  40. Pope VD, Gallani ML, Rowntree PR, Stratton RA (2000) The impact of new physical parametrizations in the Hadley Centre climate model: HadAM3. Clim Dyn 16(2):123–146. CrossRefGoogle Scholar
  41. Qiao F, Song Z, Bao Y, Song Y, Shu Q, Huang C, Zhao W (2013) Development and evaluation of an earth system model with surface gravity waves. J Geophys Res Oceans 118(9):4514–4524CrossRefGoogle Scholar
  42. Sakamoto TT, Komuro Y, Nishimura T, Ishii M, Tatebe H, Shiogama H, Hasegawa A, Toyoda T, Mori M, Suzuki T, Imada Y, Nozawa T, Takata K, Mochizuki T, Ogochi K, Emori S, Hasumi H, Kimoto M (2012) MIROC4h—a new high-resolution atmosphere-ocean coupled general circulation model. J Meteorol Soc Jpn Ser II 90(3):325–359. CrossRefGoogle Scholar
  43. Schmidt GA, Kelley M, Nazarenko L, Ruedy R, Russell GL, Aleinov I, Bauer M, Bauer SE, Bhat MK, Bleck R, Canuto V, Chen Y-H, Cheng Y, Clune TL, Del Genio A, de Fainchtein R, Faluvegi G, Hansen JE, Healy RJ, Kiang NY, Koch D, Lacis AA, LeGrande AN, Lerner J, Lo KK, Matthews EE, Menon S, Miller RL, Oinas V, Oloso AO, Perlwitz JP, Puma MJ, Putman WM, Rind D, Romanou A, Sato M, Shindell DT, Sun S, Syed RA, Tausnev N, Tsigaridis K, Unger N, Voulgarakis A, Yao M-S, Zhang J (2014) Configuration and assessment of the GISS ModelE2 contributions to the CMIP5 archive. J Adv Model Earth Syst 6(1):141–184. CrossRefGoogle Scholar
  44. Scoccimarro E, Gualdi S, Bellucci A, Sanna A, Fogli PG, Manzini E, Vichi M, Oddo P, Navarra A (2011) Effects of tropical cyclones on ocean heat transport in a high-resolution coupled general circulation model. J Clim 24(16):4368–4384. CrossRefGoogle Scholar
  45. Stevens B, Giorgetta M, Esch M, Mauritsen T, Crueger T, Rast S, Salzmann M, Schmidt H, Bader J, Block K, Brokopf R, Fast I, Kinne S, Kornblueh L, Lohmann U, Pincus R, Reichler T, Roeckner E (2013) Atmospheric component of the MPI-M earth system model: ECHAM6. J Adv Model Earth Syst 5(2):146–172. CrossRefGoogle Scholar
  46. Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93(4):485–498. CrossRefGoogle Scholar
  47. Voldoire A, Sanchez-Gomez E, y Mélia D, Decharme B, Cassou C, Sénési S, Valcke S, Beau I, Alias A, Chevallier M, Déqué M, Deshayes J, Douville H, Fernandez E, Madec G, Maisonnave E, Moine M-P, Planton S, Saint-Martin D, Szopa S, Tyteca S, Alkama R, Belamari S, Braun A, Coquart L, Chauvin F (2013) The CNRM-CM5.1 global climate model: description and basic evaluation. Clim Dyn 40(9):2091–2121. CrossRefGoogle Scholar
  48. Volodin EM, Dianskii NA, Gusev AV (2010) Simulating present-day climate with the INMCM4.0 coupled model of the atmospheric and oceanic general circulations. Izv Atmos Ocean Phys 46(4):414–431. CrossRefGoogle Scholar
  49. Vose RS, Applequist S, Squires M, Durre I, Menne CJ, Williams CN, Fenimore C, Gleason K, Arndt D (2014) Improved historical temperature and precipitation time series for U.S. climate divisions. J Appl Meteorol Climatol 53(5):1232–1251. CrossRefGoogle Scholar
  50. Watanabe M, Suzuki T, O’ishi R, Komuro Y, Watanabe S, Emori S, Takemura T, Chikira M, Ogura T, Sekiguchi M, Takata K, Yamazaki D, Yokohata T, Nozawa T, Hasumi H, Tatebe H, Kimoto M (2010) Improved climate simulation by MIROC5: mean states, variability, and climate sensitivity. J Clim 23(23):6312–6335. CrossRefGoogle Scholar
  51. Watanabe S, Hajima T, Sudo K, Nagashima T, Takemura T, Okajima H, Nozawa T, Kawase H, Abe M, Yokohata T, Ise T, Sato H, Kato E, Takata K, Emori S, Kawamiya M (2011) MIROC-ESM 2010: model description and basic results of CMIP5-20c3m experiments. Geosci Model Dev 4(4):845–872. CrossRefGoogle Scholar
  52. Wu T, Song L, Li W, Wang Z, Zhang H, Xin X, Zhang Zhang L, Li J, Wu F, Liu Y, Zhang F, Shi X, Chu M, Zhang J, Fang Y, Wang F, Lu Y, Liu X, Wei M, Liu Q, Zhou W, Dong M, Zhao Q, Ji J, Li L, Zhou M (2014) An overview of BCC climate system model development and application for climate change studies. J Meteorol Res 28(1):34–56. Google Scholar
  53. Wuebbles DJ, Kunkel K, Wehner M, Zobel Z (2014) Severe weather in United States under a changing climate. Eos Trans Am Geophys Union 95(18):149–150. CrossRefGoogle Scholar
  54. Wuebbles DJ, Fahey DW, Hibbard KA, Dokken DJ, Stewart BC, Maycock TK (eds) (2017) US Global Change Research Program, Washington, DC, USA.
  55. Yukimoto S (2011) Meteorological research institute earth system model version 1 (MRI-ESM1): model descriptionGoogle Scholar
  56. Yukimoto S, Adachi Y, Hosaka M, Sakami T, Yoshimura H, Hirabara M, Tanaka TY, Shindo E, Tsujino H, Deushi M, Mizuta R, Yabu S, Obata A, Nakano H, Koshiro T, Ose T, Kitoh A (2012) A new global climate model of the meteorological research institute: MRI-CGCM3—model description and basic performance. J Meteorol Soc Jpn Ser II 90A:23–64. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Center for Hydrometeorology and Remote Sensing, Department of Civil and Environmental EngineeringUniversity of California IrvineIrvineUSA
  2. 2.Jet Propulsion LaboratoryCalifornia Institute of TechnologyPasadenaUSA
  3. 3.Department of GeographyPortland State UniversityPortlandUSA
  4. 4.Cooperative Institute for Climate and Satellites - North CarolinaNorth Carolina State UniversityAshevilleUSA

Personalised recommendations