Skip to main content

Quantitative evaluation of the seasonal variations in climate model cloud regimes


An extended cloud-clustering method to assess the seasonal variation of clouds is applied to five CMIP5 models. The seasonal variation of the total cloud radiative effect (CRE) is dominated by variations in the relative frequency of occurrence of the different cloud regimes. Seasonal variations of the CRE within the individual regimes contribute much less. This is the case for both observations, models and model errors. The error in the seasonal variation of cloud regimes, and its breakdown into mean amplitude and time varying components, are quantified with a new metric. The seasonal variation of the CRE of the cloud regimes is relatively well simulated by the models in the tropics, but less well in the extra-tropics. The stratocumulus regime has the largest seasonal variation of shortwave CRE in the tropics, despite having a small magnitude in the climatological mean. Most of the models capture the temporal variation of the CRE reasonably well, with the main differences between models coming from the variation in amplitude. In the extra-tropics, most models fail to correctly represent both the amplitude and time variation of the CRE of congestus, frontal and stratocumulus regimes. The annual mean climatology of the CRE and its amplitude in the seasonal variation are both underestimated for the anvil regime in the tropics, the cirrus regime and the congestus regime in the extra-tropics. The models in this study that best capture the seasonal variation of the cloud regimes tend to have higher climate sensitivities.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6


  • Andrews T, Gregory JM, Webb MJ, Taylor KE (2012) Forcing, feedbacks and climate sensitivity in CMIP5 coupled atmosphere-ocean climate models. Geophys Res Lett. doi:10.1029/2012GL051607

  • Bodas-Salcedo A, Webb MJ, Bony S, Chepfer H, Dufresne JL, Klein S, Zhang Y, Marchand R, Haynes JM, Pincus R, John VO (2011) COSP: satellite simulation software for model assessment. Bull Am Meteorol Soc 92(8):1023–1043. doi:10.1175/2011BAMS2856.1

    Google Scholar 

  • Bodas-Salcedo A, Williams KD, Field PR, Lock AP (2012) The surface downwelling solar radiation surplus over the southern ocean in the Met Office model: the role of midlatitude cyclone clouds. J Clim. doi:10.1175/JCLI-D-11-00702.1

  • Bony S, Dufresne JL, Le Treut H, Morcrette JJ, Senior CA (2004) On dynamic and thermodynamic components of cloud changes. Clim Dyn 22:71–86. doi:10.1007/s00382-003-0369-6

    Article  Google Scholar 

  • Bony S, Webb MJ, Bretherton CS, Klein SA, Siebesma AP, Tselioudis G, Zhang M (2011) CFMIP: towards a better evaluation and understanding of clouds and cloud feedbacks in CMIP5 models. CLIVAR Exch 16(2):20–24

    Google Scholar 

  • Cess RD, Potter GL (1987) Exploratory studies of cloud radiative forcing with a general circulation model. Tellus Ser A 39:460–473

    Article  Google Scholar 

  • Cess RD, Zhang MH, Potter GL, Alekseev V, Barker HW, Bony S, Colman RA, Dazlich DA, Del Genio AD, Déqué M, Dix MR, Dymnikov V, Esch M, Fowler LD, Fraser JR, Galin V, Gates WL, Hack JJ, Ingram WJ, Kiehl JT, Kim Y, Le Treut H, Liang XZ, McAvaney BJ, Meleshko VP, Morcrette JJ, Randall DA, Roeckner E, Schlesinger ME, Sporyshev PV, Taylor KE, Timbal B, Volodin EM, Wang W, Wang WC, Wetherald RT (1997) Comparison of the seasonal change in cloud-radiative forcing from atmospheric general circulation models and satellite observations. J Geophys Res 102:16593–16603

    Article  Google Scholar 

  • Charlock T, Ramanathan V (1985) The Albedo field and cloud radiative forcing in a general circulation model with internally generated cloud optics. J Atmos Sci 42:1408–1429

    Article  Google Scholar 

  • Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113:D06104. doi:10.1029/2007JD008972

  • Harrison EF, Minnis P, Barkstrom BR, Ramanathan V, Cess RD, Gibson GG (1990) Seasonal variation of cloud radiative forcing derived from the earth radiation budget experiment. J Geophys Res 95:18687–18703

    Article  Google Scholar 

  • Jiang JH, et al. (2012) Evaluation of cloud and water vapor simulations in CMIP5 climate models using NASA “A-Train” satellite observations. J Geophys Res. doi:10.1029/2011JD017237 (in press)

  • Klein SA, Jakob C (1999) Validation and sensitivities of frontal clouds simulated by the ECMWF model. Mon Weather Rev 127(10):2514–2531

    Google Scholar 

  • Meehl GA, Stocker TF, Collins WD, Friedlingstein P, Gaye AT, Gregory JM, Kitoh A, Knutti R, Murphy JM, Noda A, Raper SCB, Watterson IG, Weaver AJ, Zhao ZC (2007) Global climate projections,. Cambridge University Press, Cambridge. Climate Change 2007: the physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change

  • Norris JR, Weaver CP (2001) Improved techniques for evaluating GCM cloudiness applied to the NCAR CCM3. J Clim 14:2540–2550

    Article  Google Scholar 

  • Pincus R, Batstone CP, Hofmann RJP, Taylor KE, PJ Glecker (2008) Evaluating the present-day simulation of clouds, precipitation, and radiation in climate models. J Geophys Res 113. doi:10.1029/2007jd009334

  • Pincus R, Platnick S, Ackerman SA, Hemler RS, Patrick Hofmann RJ (2012) Reconciling simulated and observed views of clouds: MODIS, ISCCP, and the limits of instrument simulators. J Climate 25:4699–4720. doi:10.1175/JCLI-D-11-00267.1

    Article  Google Scholar 

  • Randall D, Wood R, Bony S, Colman R, Fichefet T, Fyfe J, Kattsov V, Pitman A, Shukla J, Srinivasan J, Stouffer R, Sumi A, Taylor K (2007) Climate models and their evaluation, Cambridge University Press, Cambridge. Climate Change 2007: the physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change

  • Rossow WB, Schiffer RA (1999) Advances in understanding clouds from ISCCP. Bull Amer Meteor Soc 80:2261–2288

    Article  Google Scholar 

  • Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res 106(D7):7183–7192. doi:10.1029/2000JD900719

    Google Scholar 

  • Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Amer Meteor Soc 93. doi:10.1175/BAMS-D-11-00094.1

  • Trenberth KE, Fasullo JT (2010) Simulation of present-day and twenty-first-century energy budgets of the Southern Oceans. J Clim 23:440–454. doi:10.1175/2009JCLI3152.1

    Google Scholar 

  • Tselioudis G, Zhang Y, Rossow WB (2000) Cloud and radiation variations associated with northern midlatitude low and high sea level pressure regimes. J Clim 13(2):312–327

    Article  Google Scholar 

  • Tsushima Y, Abe-Ouchi A, Manabe S (2005) Radiative damping of annual variation in global mean surface temperature: comparison between observed and simulated feedback. Clim Dyn 24:591–597

    Article  Google Scholar 

  • Webb M, Senior C, Bony S, Morcrette JJ (2001) Combining ERBE and ISCCP data to assess clouds in the Hadley Centre, ECMWF and LMD atmospheric climate models. Clim Dyn 17:905–922

    Google Scholar 

  • Williams KD, Tselioudis G (2007) GCM intercomparison of global cloud regimes: present-day evaluation and climate change response. Clim Dyn 29:231–250. doi:10.1007/s00382-007-0232-2

    Article  Google Scholar 

  • Williams KD, Webb MJ (2009) A quantitative performance assessment of cloud regimes in climate models. Clim Dyn 33(1):141–157

    Article  Google Scholar 

  • Williams KD, Ringer MA, Senior CA, Webb MJ, McAvaney BJ, Andronova N, Bony S, Dufresne JL, Emori S, Gudgel R, Knutson T, Li B, Lo K, Musat I, Wegner J, Slingo A, Mitchell JFB (2006) Evaluation of a component of the cloud response to climate change in an intercomparison of climate models. Clim Dyn 26:145–165. doi:10.1007/s00382-005-0067-7

    Article  Google Scholar 

  • Williams KD, Bodas-Salcedo A, Dquee M, Fermepin S, Medeiros B, Watanabe M, Jakob C, Klein SA, Senior CA, Williamson DL (2012) The transpose-AMIP II experiment and its application to the understanding of Southern Ocean cloud biases in climate models. J Clim. doi:10.1175/JCLI-D-12-00429.1

  • Zelinka, MD, Hartmann DL (2010) Why is longwave cloud feedback positive? J Geophys Res 115:D16117. doi:10.1029/2010JD013817

  • Zelinka MD, Klein SA, Hartmann DL (2012) Computing and partitioning cloud feedbacks using cloud property histograms. Part I: cloud radiative kernels. J Clim 25:3715–3735. doi:10.1175/JCLI-D-11-00248.1.1

    Article  Google Scholar 

  • Zhang MH, Lin WY, Klein SA, Bacmeister JT, Bony S, Cederwall RT, Del Genio AD, Hack JJ, Loeb NG, Lohmann U, Minnis P, Musat I, Pincus R, Stier P, Suarez MJ, Webb MJ, Wu JB, Xie SC, Yao M-S, Zhang JH (2005) Comparing clouds and their seasonal variations in 10 atmospheric general circulation models with satellite measurements. J Geophys Res 110. doi:10.1029/2004JD005021

  • Zhang Y, Rossow WB, Lacis AA (1995) Calculation of surface and top of atmosphere radiative fluxes from physical quantities based on ISCCP datasets 1. Method and sensitivity to input data uncertainties. J Geophys Res 100:1149–1165

    Article  Google Scholar 

Download references


The authors are very grateful to Dr. Alejandro Bodas-Salcedo, who helped constructing the data for the analysis and gave useful comments. The authors are also very grateful to Dr. Hidetoshi Shimodaira, who gave helpful comments on the metrics. They thank the international climate modeling groups, the PCMDI and WCRP’s Working Group on Coupled Modeling for making available the multi-model data set obtained from the phase 5 of the Coupled Model Intercomparison Project (CMIP5). The data set has been indispensable for the study conducted here. The construction of the dataset was supported by the Office of Science, U.S. Department of Energy. The authors are indebted to the Technical Support Unit of IPCC Working Group I for technical support. The research leading to these results has received funding from the European Union, Seventh Framework Programme (FP7/2007-2013) under grant agreement no 244067 via the EU Cloud Intercomparison and Process Study Evaluation Project (EUCLIPSE) and the Joint DECC/Defra Met Office Hadley Centre Climate Programme (GA01101).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Yoko Tsushima.


Appendix 1: Cloud regime error metrics for the present day annual mean climatology

According to WW09, cloud regime error metric of the present-day annual mean climatology

$$ CREMpd = \sqrt {\frac{{\sum\nolimits_{r = 1}^{nregimes} {CREMpd_{r}^{2} } }}{nregimes}} $$
$$ CREMpd_{r} = aw\sqrt {\left( {RFO_{r}^{\prime} NCRE_{obsr} } \right)^{2} + \left( {NCRE_{r}^{\prime} RFO_{obsr} } \right)^{2} } $$

where \( NCRE_{r}^{\prime} \) and \( RFO_{r}^{\prime} \) are regime mean differences between the model and ISCCP observations, RFO obsr and NCRE obsr are the ISCCP observed regime mean values which are used to weight the regimes, and aw is the respective area weight for the region.

Appendix 2: Amplitude of seasonal variation

x n is defined to be a climatological monthly mean of variable x of a regime in a region for month n, and \( \overline{x} \) is the climatological annual mean. The standard deviations (σ) of \( x_{n} \left( {n = 1, \ldots ,N} \right) \) is as follows.

$$ \sigma^{2} = \frac{1}{N}\sum\limits_{n = 1}^{N} {\left( {x_{n} - \overline{x} } \right)}^{2} $$
$$ \overline{x} = \frac{{\sum\nolimits_{n = 1}^{N} {x_{n} } }}{N} $$

For the seasonal variation of NSCRE and longwave cloud radiative effect (LCRE) of each cloud regime, the amplitude is obtained by using monthly climatological means of NSCRE and LCRE of the regime as \( x_{n} \left( {n = 1, \ldots ,N} \right) \) and N is the total number of climatological month n (N =12).

Appendix 3: RMS error in the seasonal variation

Superscript o and m indicates the value from the observation and a model.

The standard deviations of observational and model variable x; σ x,o , σ x,m are

$$ \sigma_{x,o}^{2} = \frac{1}{N}\sum\limits_{n = 1}^{N} {\left( {x_{n}^{o} - \overline{{x^{o} }} } \right)}^{2} $$
$$ \sigma_{x,m}^{2} = \frac{1}{N}\sum\limits_{n = 1}^{N} {\left( {x_{n}^{m} - \overline{{x^{m} }} } \right)}^{2} $$

In centred RMS error, climatological annual mean is subtracted from the climatological monthly mean for both the observation and the model, and the square of the difference of the anomaly of the model from the anomaly of the observation is averaged over the 12 months as follows.

$$ e_{x}^{\prime2} = \frac{1}{N}\sum\limits_{n = 1}^{N} {\left\{ {\left( {x_{n}^{m} - \overline{{x^{m} }} } \right) - \left( {x_{n}^{o} - \overline{{x^{o} }} } \right)} \right\}}^{2} $$

A pattern correlation coefficient (R) between \( \left( {x_{n}^{m} - \overline{{x^{m} }} } \right) \) and \( \left( {x_{n}^{o} - \overline{{x^{o} }} } \right) \) is defined as follows.

$$ R = \frac{{\sum\nolimits_{n = 1}^{N} {\left( {x_{n}^{m} - \overline{{x^{m} }} } \right)\left( {x_{n}^{o} - \overline{{x^{o} }} } \right)} }}{{\sqrt {\left\{ {\sum\nolimits_{n = 1}^{N} {\left( {x_{n}^{m} - \overline{{x^{m} }} } \right)^{2} } } \right\}\left\{ {\sum\nolimits_{n = 1}^{N} {\left( {x_{n}^{o} - \overline{{x^{o} }} } \right)}^{2} } \right\}} }} $$

The second term of Eq. (3) can be rewritten as follows.

$$ 2\sigma_{x,m} \sigma_{x,o} (1 - R) = \frac{{\sigma_{x,m} }}{{\sigma_{x,o} }}\frac{1}{N}\sum\limits_{n = 1}^{N} {\left\{ {\frac{{\sigma_{x,o} }}{{\sigma_{x,m} }}\left( {x_{n}^{m} - \overline{{x^{m} }} } \right) - \left( {x_{n}^{o} - \overline{{x^{o} }} } \right)} \right\}^{2} } $$

It consists of \( \frac{{\sigma_{x,m} }}{{\sigma_{x,o} }} \) and \( \frac{1}{N}\sum\nolimits_{n = 1}^{N} {\left\{ {\frac{{\sigma_{x,o} }}{{\sigma_{x,m} }}\left( {x_{n}^{m} - \overline{{x^{m} }} } \right) - \left( {x_{n}^{o} - \overline{{x^{o} }} } \right)} \right\}^{2} } \), the former is the ratio of the model standard deviation to that observed, while the latter corresponds to the RMS error of a hypothetical model whose standard deviation is normalized to that observed.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Tsushima, Y., Ringer, M.A., Webb, M.J. et al. Quantitative evaluation of the seasonal variations in climate model cloud regimes. Clim Dyn 41, 2679–2696 (2013).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Cloud regime
  • Cloud metric
  • Seasonal variation
  • Cloud radiative effect
  • Cloud feedback
  • Climate model