Climate Dynamics

, Volume 48, Issue 1–2, pp 89–112 | Cite as

Regime-based evaluation of cloudiness in CMIP5 models

  • Daeho JinEmail author
  • Lazaros Oreopoulos
  • Dongmin Lee


The concept of cloud regimes (CRs) is used to develop a framework for evaluating the cloudiness of 12 fifth Coupled Model Intercomparison Project (CMIP5) models. Reference CRs come from existing global International Satellite Cloud Climatology Project (ISCCP) weather states. The evaluation is made possible by the implementation in several CMIP5 models of the ISCCP simulator generating in each grid cell daily joint histograms of cloud optical thickness and cloud top pressure. Model performance is assessed with several metrics such as CR global cloud fraction (CF), CR relative frequency of occurrence (RFO), their product [long-term average total cloud amount (TCA)], cross-correlations of CR RFO maps, and a metric of resemblance between model and ISCCP CRs. In terms of CR global RFO, arguably the most fundamental metric, the models perform unsatisfactorily overall, except for CRs representing thick storm clouds. Because model CR CF is internally constrained by our method, RFO discrepancies yield also substantial TCA errors. Our results support previous findings that CMIP5 models underestimate cloudiness. The multi-model mean performs well in matching observed RFO maps for many CRs, but is still not the best for this or other metrics. When overall performance across all CRs is assessed, some models, despite shortcomings, apparently outperform Moderate Resolution Imaging Spectroradiometer cloud observations evaluated against ISCCP like another model output. Lastly, contrasting cloud simulation performance against each model’s equilibrium climate sensitivity in order to gain insight on whether good cloud simulation pairs with particular values of this parameter, yields no clear conclusions.


CMIP5 evaluation CFMIP COSP ISCCP simulator Cloud assessment Cloud climatology Cloud regime 



We acknowledge the World Climate Research Programme’s Working Group on Coupled Modeling, which is responsible for CMIP, and we thank the climate modeling groups (listed in Table 1 of this paper) for producing and making available their model output. For CMIP, the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led development of software infrastructure in partnership with the Global Organization for Earth System Science Portals. Lastly, funding by NASA’s Modeling Analysis and Prediction (MAP) program is gratefully acknowledged.

Supplementary material

382_2016_3064_MOESM1_ESM.pdf (212 kb)
Supplementary material 1 (PDF 211 kb)
382_2016_3064_MOESM2_ESM.pdf (2.3 mb)
Supplementary material 2 (PDF 2329 kb)


  1. Anderberg MR (1973) Cluster analysis for applications. Elsevier, New YorkGoogle Scholar
  2. Andrews T, Gregory JM, Webb MJ, Taylor KE (2012) Forcing, feedbacks and climate sensitivity in CMIP5 coupled atmosphere–ocean climate models. Geophys Res Lett 39:L09712. doi: 10.1029/2012GL051607 Google Scholar
  3. Bodas-Salcedo A, Webb MJ, Bony S et al (2011) COSP: satellite simulation software for model assessment. Bull Am Meteorol Soc 92:1023–1043. doi: 10.1175/2011BAMS2856.1 CrossRefGoogle Scholar
  4. Bony S, Webb M, Bretherton CS et al (2011) CFMIP: towards a better evaluation and understanding of clouds and cloud feedbacks in CMIP5 models. CLIVAR Exch 56(16):20–24Google Scholar
  5. Boucher O, Randall D, Artaxo P et al (2013) Clouds and aerosols. In: Stocker TF, Qin D, Plattner G-K et al (eds) Climate change 2013: the physical science basis. Contribution of working group I to the fifth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge; New York, pp 571–658Google Scholar
  6. Brenguier J-L, Pawlowska H, Schüller L et al (2000) Radiative properties of boundary layer clouds: droplet effective radius versus number concentration. J Atmos Sci 57:803–821. doi: 10.1175/1520-0469(2000)057<0803:RPOBLC>2.0.CO;2 CrossRefGoogle Scholar
  7. Cho H-M, Zhang Z, Meyer K et al (2015) Frequency and causes of failed MODIS cloud property retrievals for liquid phase clouds over global oceans. J Geophys Res Atmos 120:4132–4154. doi: 10.1002/2015JD023161 CrossRefGoogle Scholar
  8. Dolinar EK, Dong X, Xi B et al (2015) Evaluation of CMIP5 simulated clouds and TOA radiation budgets using NASA satellite observations. Clim Dyn 44:2229–2247. doi: 10.1007/s00382-014-2158-9 CrossRefGoogle Scholar
  9. Forster PM, Andrews T, Good P et al (2013) Evaluating adjusted forcing and model spread for historical and future scenarios in the CMIP5 generation of climate models. J Geophys Res Atmos 118:1139–1150. doi: 10.1002/jgrd.50174 CrossRefGoogle Scholar
  10. Jakob C, Tselioudis G (2003) Objective identification of cloud regimes in the Tropical Western Pacific. Geophys Res Lett 30:2082. doi: 10.1029/2003GL018367 CrossRefGoogle Scholar
  11. Jiang JH, Su H, Zhai C et al (2012) Evaluation of cloud and water vapor simulations in CMIP5 climate models using NASA “A-Train” satellite observations. J Geophys Res 117:D14105. doi: 10.1029/2011JD017237 Google Scholar
  12. Jin D, Oreopoulos L, Lee D (2016) Simplified ISCCP cloud regimes for evaluating cloudiness in CMIP5 models. Clim Dyn (accepted) Google Scholar
  13. King MD, Menzel WP, Kaufman YJ et al (2003) Cloud and aerosol properties, precipitable water, and profiles of temperature and water vapor from MODIS. IEEE Trans Geosci Remote Sens 41:442–458. doi: 10.1109/TGRS.2002.808226 CrossRefGoogle Scholar
  14. Klein SA, Jakob C (1999) Validation and sensitivities of frontal clouds simulated by the ECMWF model. Mon Weather Rev 127:2514–2531. doi: 10.1175/1520-0493(1999)127<2514:VASOFC>2.0.CO;2 CrossRefGoogle Scholar
  15. Klein SA, Zhang Y, Zelinka MD et al (2013) Are climate model simulations of clouds improving? An evaluation using the ISCCP simulator. J Geophys Res Atmos 118:1329–1342. doi: 10.1002/jgrd.50141 CrossRefGoogle Scholar
  16. L’Ecuyer TS, Stephens GL (2007) The tropical atmospheric energy budget from the TRMM perspective. Part II: evaluating GCM representations of the sensitivity of regional energy and water cycles to the 1998–99 ENSO cycle. J Clim 20:4548–4571. doi: 10.1175/JCLI4207.1 CrossRefGoogle Scholar
  17. Lauer A, Hamilton K (2013) Simulating clouds with global climate models: a comparison of CMIP5 results with CMIP3 and satellite data. J Clim 26:3823–3845. doi: 10.1175/JCLI-D-12-00451.1 CrossRefGoogle Scholar
  18. Li J-LF, Waliser DE, Stephens G et al (2013) Characterizing and understanding radiation budget biases in CMIP3/CMIP5 GCMs, contemporary GCM, and reanalysis. J Geophys Res Atmos 118:8166–8184. doi: 10.1002/jgrd.50378 CrossRefGoogle Scholar
  19. MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Oakland, CA, USA, pp 281–297Google Scholar
  20. Mason S, Fletcher JK, Haynes JM et al (2015) A hybrid cloud regime methodology used to evaluate southern ocean cloud and shortwave radiation errors in ACCESS. J Clim 28:6001–6018. doi: 10.1175/JCLI-D-14-00846.1 CrossRefGoogle Scholar
  21. Masunaga H, Matsui T, Tao W-K et al (2010) Satellite data simulator unit: a multisensor, multispectral satellite simulator package. Bull Am Meteorol Soc 91:1625–1632. doi: 10.1175/2010BAMS2809.1 CrossRefGoogle Scholar
  22. Matsui T, Santanello J, Shi JJ et al (2014) Introducing multisensor satellite radiance-based evaluation for regional Earth System modeling. J Geophys Res Atmos 119:8450–8475. doi: 10.1002/2013JD021424 CrossRefGoogle Scholar
  23. Nam C, Bony S, Dufresne J-L, Chepfer H (2012) The “too few, too bright” tropical low-cloud problem in CMIP5 models. Geophys Res Lett 39:L21801. doi: 10.1029/2012GL053421 CrossRefGoogle Scholar
  24. O’Dell CW, Wentz FJ, Bennartz R (2008) Cloud liquid water path from satellite-based passive microwave observations: a new climatology over the global oceans. J Clim 21:1721–1739. doi: 10.1175/2007JCLI1958.1 CrossRefGoogle Scholar
  25. Oreopoulos L, Rossow WB (2011) The cloud radiative effects of International Satellite Cloud Climatology Project weather states. J Geophys Res 116:D12202. doi: 10.1029/2010JD015472 CrossRefGoogle Scholar
  26. Oreopoulos L, Cho N, Lee D et al (2014) An examination of the nature of global MODIS cloud regimes. J Geophys Res Atmos 119:8362–8383. doi: 10.1002/2013JD021409 CrossRefGoogle Scholar
  27. Oreopoulos L, Cho N, Lee D, Kato S (2016) Radiative effects of global MODIS cloud regimes. J Geophys Res Atmos 121. doi: 10.1002/2015JD024502
  28. Pincus R, Batstone CP, Hofmann RJP et al (2008) Evaluating the present-day simulation of clouds, precipitation, and radiation in climate models. J Geophys Res 113:D14209. doi: 10.1029/2007JD009334 CrossRefGoogle Scholar
  29. Pincus R, Platnick S, Ackerman SA et al (2012) Reconciling simulated and observed views of clouds: MODIS, ISCCP, and the limits of instrument simulators. J Clim 25:4699–4720. doi: 10.1175/JCLI-D-11-00267.1 CrossRefGoogle Scholar
  30. Rossow WB, Schiffer RA (1991) ISCCP cloud data products. Bull Am Meteorol Soc 72:2–20. doi: 10.1175/1520-0477(1991)072<0002:ICDP>2.0.CO;2 CrossRefGoogle Scholar
  31. Rossow WB, Schiffer RA (1999) Advances in understanding clouds from ISCCP. Bull Am Meteorol Soc 80:2261–2287. doi: 10.1175/1520-0477(1999)080<2261:AIUCFI>2.0.CO;2 CrossRefGoogle Scholar
  32. Rossow WB, Tselioudis G, Polak A, Jakob C (2005) Tropical climate described as a distribution of weather states indicated by distinct mesoscale cloud property mixtures. Geophys Res Lett 32:L21812. doi: 10.1029/2005GL024584 CrossRefGoogle Scholar
  33. Stephens GL, Vane DG, Boain RJ et al (2002) The CloudSat mission and the A-Train: a new dimension of space-based observations of clouds and precipitation. Bull Am Meteorol Soc 83:1771–1790. doi: 10.1175/BAMS-83-12-1771 CrossRefGoogle Scholar
  34. Stephens GL, Vane DG, Tanelli S et al (2008) CloudSat mission: performance and early science after the first year of operation. J Geophys Res 113:2156–2202. doi: 10.1029/2008JD009982 CrossRefGoogle Scholar
  35. Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93:485–498. doi: 10.1175/BAMS-D-11-00094.1 CrossRefGoogle Scholar
  36. Tselioudis G, Rossow W, Zhang Y, Konsta D (2013) Global weather states and their properties from passive and active satellite cloud retrievals. J Clim 26:7734–7746. doi: 10.1175/JCLI-D-13-00024.1 CrossRefGoogle Scholar
  37. Tsushima Y, Ringer MA, Webb MJ, Williams KD (2013) Quantitative evaluation of the seasonal variations in climate model cloud regimes. Clim Dyn 41:2679–2696. doi: 10.1007/s00382-012-1609-4 CrossRefGoogle Scholar
  38. Wang H, Su W (2015) The ENSO effects on tropical clouds and top-of-atmosphere cloud radiative effects in CMIP5 models. J Geophys Res Atmos 120:4443–4465. doi: 10.1002/2014JD022337 CrossRefGoogle Scholar
  39. Webb M, Senior C, Bony S, Morcrette J-J (2001) Combining ERBE and ISCCP data to assess clouds in the Hadley Centre, ECMWF and LMD atmospheric climate models. Clim Dyn 17:905–922CrossRefGoogle Scholar
  40. Williams KD, Tselioudis G (2007) GCM intercomparison of global cloud regimes: present-day evaluation and climate change response. Clim Dyn 29:231–250. doi: 10.1007/s00382-007-0232-2 CrossRefGoogle Scholar
  41. Williams KD, Webb MJ (2009) A quantitative performance assessment of cloud regimes in climate models. Clim Dyn 33:141–157. doi: 10.1007/s00382-008-0443-1 CrossRefGoogle Scholar
  42. Williams KD, Ringer MA, Senior CA et al (2006) Evaluation of a component of the cloud response to climate change in an intercomparison of climate models. Clim Dyn 26:145–165. doi: 10.1007/s00382-005-0067-7 CrossRefGoogle Scholar
  43. Winker DM, Vaughan MA, Omar A et al (2009) Overview of the CALIPSO mission and CALIOP data processing algorithms. J Atmos Ocean Technol 26:2310–2323CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.University Space Research AssociationColumbiaUSA
  2. 2.NASA Goddard Space Flight Center Code 613GreenbeltUSA
  3. 3.Morgan State UniversityBaltimoreUSA

Personalised recommendations