Climate Dynamics

, Volume 29, Issue 7–8, pp 853–868 | Cite as

Hierarchical evaluation of IPCC AR4 coupled climate models with systematic consideration of model uncertainties



The capability of reproducing observed surface air temperature (SAT) changes for the twentieth century is assessed using 22 multi-models which contribute to the Intergovernmental Panel on Climate Change Fourth Assessment Report. A Bayesian method is utilized for model evaluation by which model uncertainties are considered systematically. We provide a hierarchical analysis for global to sub-continental regions with two settings. First, regions of different size are evaluated separately at global, hemispheric, continental, and sub-continental scales. Second, the global SAT trend patterns are evaluated with gradual refinement of horizontal scales (higher dimensional analysis). Results show that models with natural plus anthropogenic forcing (MME_ALL) generally exhibit better skill than models with anthropogenic only forcing (MME_ANTH) at all spatial scales for different trend periods (entire twentieth century and its first and second halves). This confirms previous studies that suggest the important role of natural forcing. For the second half of the century, we found that MME_ANTH performs well compared to MME_ALL except for a few models with overestimated warming. This indicates not only major contributions of anthropogenic forcing over that period but also the applicability of both MMEs to observationally-constrained future predictions of climate changes. In addition, the skill-weighted averages with the Bayes factors [Bayesian model averaging (BMA)] show a general superiority over other error-based weighted averaging methods, suggesting a potential advantage of BMA for climate change predictions.


Skill Score Bayesian Model Average Couple Climate Model Eastern Hemisphere Weighted Average Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This study was supported by the German Research Foundation (DFG) with grant He1916/8. We thank two anonymous reviewers for their constructive comments and Walter Skinner for his help on English editing. We also acknowledge the international modeling groups for providing their data for analysis, the Program for Climate Model Diagnosis and Intercomparison (PCMDI) for collecting and archiving the model data, the JSC/CLIVAR Working Group on Coupled Modelling (WGCM) and their Coupled Model Intercomparison Project (CMIP) and Climate Simulation Panel for organizing the model data analysis activity, and the IPCC WG1 TSU for technical support. The IPCC Data Archive at Lawrence Livermore National Laboratory is supported by the Office of Science, US Department of Energy.


  1. Allen MR, Stott PA, Mitchell JFB, Schnur R, Thomas LD (2000) Quantifying the uncertainty in forecasts of anthropogenic climate change. Nature 407:617–620CrossRefGoogle Scholar
  2. Boer GJ, Lambert SJ (2001) Second order space-time climate difference statistics. Clim Dyn 17:213–218CrossRefGoogle Scholar
  3. Cubasch U, Meehl GA, Boer GJ, Stouffer RJ, Dix M, Noda A, Senior CA, Raper S, Yap KS (2001) Projections of future climate change. In: Houghton JT et al (eds) Climate change 2001: the scientific basis. Cambridge University Press, UK, pp 525–582Google Scholar
  4. Gillett NP, Zwiers FW, Weaver AJ, Hegerl GC, Allen MR, Stott PA (2002) Detecting anthropogenic influence with a multi-model ensemble. Geophys Res Lett 29:1970.  doi:10.1029/2002GL015836 CrossRefGoogle Scholar
  5. Giorgi F, Bi X (2005) Updated regional precipitation and temperature changes for the 21st century from ensembles of recent AOGCM simulations. Geophys Res Lett 32:L21715.  doi:10.1029/2005GL024288 CrossRefGoogle Scholar
  6. Giorgi F, Mearns LO (2002) Calculation of average, uncertainty range, and reliability of regional climate changes from AOGCM simulations via the “Reliability Ensemble Averaging” (REA) method. J Clim 15:1141–1158CrossRefGoogle Scholar
  7. Greene AM, Goddard L, Lall U (2006) Probabilistic multimodel regional temperature change projections. J Clim 19:4326–4343CrossRefGoogle Scholar
  8. Hoeting JA, Madigan D, Raftery AE, Volinsky CT (1999) Bayesian model averaging: a tutorial (with discussion). Stat Sci 14:382–401 (correction 15:193–195)CrossRefGoogle Scholar
  9. International Ad Hoc Detection and Attribution Group (IDAG) (2005) Detecting and attributing external influences on the climate system: a review of recent advances. J Clim 18:1291–1314Google Scholar
  10. Jeffreys H (1935) Some tests of significance, treated by the theory of probability. Proc Camb Philol Soc 31:203–222CrossRefGoogle Scholar
  11. Jeffreys H (1961) Theory of probability, 3rd edn. Oxford University Press, New York, p 470Google Scholar
  12. Jones PD, Moberg A (2003) Hemispheric and large-scale surface air temperature variations: an extensive revision and an update to 2001. J Clim 16:206–223CrossRefGoogle Scholar
  13. Karoly DJ, Braganza K (2005) Attribution of recent temperature changes in the Australian region. J Clim 18:457–464CrossRefGoogle Scholar
  14. Karoly DJ, Braganza K, Stott PA, Arblaster JM, Meehl GA, Broccoli AJ, Dixon KW (2003) Detection of a human influence on North American climate. Science 302:1200–1203CrossRefGoogle Scholar
  15. Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90:773–795CrossRefGoogle Scholar
  16. Kharin VV, Zwiers FW (2002) Climate predictions with multimodel ensembles. J Clim 15:793–799CrossRefGoogle Scholar
  17. Knutson TR, Delworth TL, Dixon KW, Held IM, Lu J, Ramaswamy V, Schwarzkopf MD, Stenchikov G, Stouffer RJ (2006) Assessment of twentieth-century regional surface temperature trends using the GFDL CM2 coupled models. J Clim 19:1624–1651CrossRefGoogle Scholar
  18. Krishnamurti TN, Kishtawal CM, Zhang Z, LaRow T, Bachiochi D, Williford E, Gadgil S, Surendran S (2000) Multimodel ensemble forecasts for weather and seasonal climate. J Clim 13:4196–4216CrossRefGoogle Scholar
  19. Lambert SJ, Boer GJ (2001) CMIP1 evaluation and intercomparison of coupled climate models. Clim Dyn 17:83–106CrossRefGoogle Scholar
  20. Leroy SS (1998) Detecting climate signals: some Bayesian aspects. J Clim 11:640–651CrossRefGoogle Scholar
  21. McAvaney BJ, Covey C, Joussaume S, Kattsov V, Kitoh A, Ogana W, Pitman AJ, Weaver AJ, Wood RA, Zhao ZC (2001) Model evaluation, In: Houghton JT et al (eds) Climate change 2001: the scientific basis. Cambridge University Press, UK, pp 471–524Google Scholar
  22. Min S-K, Hense A (2006a) A Bayesian approach to climate model evaluation and multi-model averaging with an application to global mean surface temperatures from IPCC AR4 coupled climate models. Geophys Res Lett 33: L08708.  doi:10.1029/2006GL025779 CrossRefGoogle Scholar
  23. Min S-K, Hense A (2006b) A Bayesian assessment of climate change using multimodel ensembles. Part I: Global mean surface temperature.J Clim 19:3237–3256CrossRefGoogle Scholar
  24. Min S-K, Hense A (2007) A Bayesian assessment of climate change using multimodel ensembles. Part II: Regional and seasonal mean surface temperatures. J Clim (in press)Google Scholar
  25. Min S-K, Hense A, Kwon W-T (2005) Regional-scale climate change detection using a Bayesian decision method. Geophys Res Lett 32:L03706.  doi:10.1029/2004GL021028 CrossRefGoogle Scholar
  26. Min S-K, Hense A, Paeth H, Kwon W-T (2004a) A Bayesian decision method for climate change signal analysis. Meteorol Z 13:421–436CrossRefGoogle Scholar
  27. Min S-K, Park E-H, Kwon W-T (2004b) Future projections of East Asian climate change from multi-AOGCM ensembles of IPCC SRES scenario simulations. J Meteor Soc Jpn 82:1187–1211CrossRefGoogle Scholar
  28. Mitchell JFB, Karoly DJ, Hegerl GC, Zwiers FW, Allen MR, Marengo J (2001) Detection of climate change and attribution of causes. In: Houghton JT et al (eds) Climate change 2001: the scientific basis. Cambridge University Press, UK, pp 695–738Google Scholar
  29. Piani C, Frame DJ, Stainforth DA, Allen MR (2005) Constraints on climate change from a multi-thousand member ensemble of simulations. Geophys Res Lett 32:L23825.  doi:10.1029/2005GL024452 CrossRefGoogle Scholar
  30. Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using Bayesian model averaging to calibrate forecast ensembles. Mon Wea Rev 133:1155–1174CrossRefGoogle Scholar
  31. Smith RL, Wigley TML, Santer BD (2003) A Bivariate time series approach to anthropogenic trend detection in hemispheric mean temperatures. J Clim 16:228–1240CrossRefGoogle Scholar
  32. Stone DA, Allen MR, Stott PA (2007) A multimodel update on the detection and attribution of global surface warming. J Clim 20:517–530CrossRefGoogle Scholar
  33. Stott PA (2003) Attribution of regional-scale temperature changes to anthropogenic and natural causes. Geophys Res Lett 30: 1728.  doi:10.1029/2003GL017324 CrossRefGoogle Scholar
  34. Stott PA, Kettleborough JA (2002) Origins and estimates of uncertainty in predictions of twenty-first century temperature rise. Nature 416:723–726CrossRefGoogle Scholar
  35. Stott PA, Kettleborough JA, Allen MR (2006) Uncertainty in continental-scale temperature predictions. Geophys Res Lett 33:L02708.  doi:10.1029/2005GL024423 CrossRefGoogle Scholar
  36. Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res 106(D7):7183–7192CrossRefGoogle Scholar
  37. Tebaldi C, Mearns LO, Nychka D, Smith RL (2004) Regional probabilities of precipitation change: a Bayesian analysis of multimodel simulations. Geophys Res Lett 31:L24213.  doi:10.1029/2004GL021276 CrossRefGoogle Scholar
  38. Tebaldi C, Smith RL, Nychka D, Mearns LO (2005) Quantifying uncertainty in projections of regional climate change: a Bayesian approach to the analysis of multimodel ensembles. J Clim 18:1524–1540CrossRefGoogle Scholar
  39. Watterson IG (1996) Non-dimensional measures of climate model performance. Int J Climatol 16:379–391CrossRefGoogle Scholar
  40. Zhang X, Zwiers FW, Stott PA (2006) Multimodel multisignal climate change detection at regional scale. J Clim 19:4294–4307CrossRefGoogle Scholar
  41. Zwiers FW, Zhang X (2003) Toward regional scale climate change detection. J Clim 16:793–797CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  1. 1.Meteorologisches InstitutUniversität BonnBonnGermany
  2. 2.Climate Research DivisionEnvironment CanadaDownsviewCanada

Personalised recommendations