Abstract
The natural sea surface temperature (SST) variability in the global oceans is evaluated in simulations of the Climate Model Intercomparison Project Phase 3 (CMIP3) and CMIP5 models. In this evaluation, we examine how well the spatial structure of the SST variability matches between the observations and simulations on the basis of their leading empirical orthogonal functions-modes. Here we focus on the high-pass filter monthly mean time scales and the longer 5 years running mean time scales. We will compare the models and observations against simple null hypotheses, such as isotropic diffusion (red noise) or a slab ocean model, to illustrate the models skill in simulating realistic patterns of variability. Some models show good skill in simulating the observed spatial structure of the SST variability in the tropical domains and less so in the extra-tropical domains. However, most models show substantial deviations from the observations and from each other in most domains and particularly in the North Atlantic and Southern Ocean on the longer (5 years running mean) time scale. In many cases the simple spatial red noise null hypothesis is closer to the observed structure than most models, despite the fact that the observed SST variability shows significant deviations from this simple spatial red noise null hypothesis. The CMIP models tend to largely overestimate the effective spatial number degrees of freedom and simulate too strongly localized patterns of SST variability at the wrong locations with structures that are different from the observed. However, the CMIP5 ensemble shows some improvement over the CMIP3 ensemble, mostly in the tropical domains. Further, the spatial structure of the SST modes of the CMIP3 and CMIP5 super ensemble is more realistic than any single model, if the relative explained variances of these modes are scaled by the observed eigenvalues.
Similar content being viewed by others
References
Bayr T, Dommenget D (2014) Comparing the spatial structure of variability in two datasets against each other on the basis of EOF-modes. Clim Dyn 42(5–6):1631–1648
Boer GJ, Lambert SJ (2001) Second order space–time climate difference statistics. Clim Dyn 17:213–218
Brayshaw DJ, Hoskins B, Blackburn M (2008) The storm-track response to idealized SST perturbations in an aquaplanet GCM. J Atmos Sci 65:2842–2860
Bretherton CS, Widmann M, Dymnikov VP, Wallace JM, Bladé I (1999) The effective number of spatial degrees of freedom of a time-varying field. J Clim 12:1990–2009
Cahalan RF, Wharton LE, Wu W-L (1996) Empirical orthogonal functions of monthly precipitation and temperature over the United States and homogeneous stochastic models. J Geophys Res 101:26309–26318
Cai W, Sullivan A, Cowan T, Ribbe J, Shi G (2011) Simulation of the Indian Ocean Dipole: a relevant criterion for selecting models for climate projections. Geophys Res Lett 38:L03704. doi:10.1029/2010GL046242
Davies T, Cullen MJP, Malcolm AJ, Mawson MH, Staniforth A, White AA, Wood N (2005) A new dynamical core for the Met Office’s global and regional modelling of the atmosphere. Q J R Meteorol Soc 131:1759–1782
Delecluse P, Davey MK, Kitamura Y, Philander SGH, Suarez M, Bengtsson L (1998) Coupled general circulation modeling of the tropical Pacific. J Geophys Res 103(C7):14357–14373
Deremble B, Lapeyre G, Ghil M (2012) Atmospheric dynamics triggered by an oceanic SST front in a moist quasigeostrophic model. J Atmos Sci 69:1617–1632
Dommenget D (2007) Evaluating EOF modes against a stochastic null hypothesis. Clim Dyn 28(5):517–531
Dommenget D (2010) The slab ocean El Niño. Geophys Res Lett 37:L20701. doi:10.1029/2010GL044888
Dommenget D (2011) An objective analysis of the observed spatial structure of the tropical Indian Ocean SST variability. Clim Dyn 36:2129–2145
Dommenget D (2012) Analysis of the model climate sensitivity spread forced by mean sea surface temperature biases. J Clim 25:7147–7162
Dommenget D, Latif M (2002) Analysis of observed and simulated SST spectra in the midlatitudes. Clim Dyn 19:277–288
Downes SM, Hogg AM (2013) Southern Ocean circulation and eddy compensation in CMIP5 models. J Clim 26:7198–7220
Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113:D06104. doi:10.1029/2007JD008972
Grenier H, Le Treut H, Fichefet T (2000) Ocean-atmosphere interactions and climate drift in a coupled general circulation model. Clim Dyn 16:701–717
Guilyardi E (2006) El Niño–mean state–seasonal cycle interactions in a multi-model ensemble. Clim Dyn 26:329–348
Gupta AS, Jourdain NC, Brown JN, Monselesan D (2013) Climate drift in the CMIP5 models. J Clim 26:8597–8615
Hirota N, Takayabu YN (2013) Reproducibility of precipitation distribution over the tropical oceans in CMIP5 multi-climate models compared to CMIP3. Clim Dyn 41(11–12):2909–2920
Huang B, Hu Z–Z, Jha B (2007) Evolution of model systematic errors in the tropical Atlantic basin from coupled climate hindcasts. Clim Dyn 28(7–8):661–682
Jamison N, Kravtsov S (2010) Decadal variations of north Atlantic sea surface temperature in observations and CMIP3 simulations. J Clim 23:4619–4636
Jolliffe I (2002) Principal component analysis, 2nd edn. Springer, New York
Kao H, Yu J (2009) Contrasting Eastern-Pacific and Central-Pacific types of ENSO. J Clim 22(3):615–632
Kirtman BP, Bitz C, Bryan F, Collins W, Dennis J, Hearn N, Kinter JL III, Loft R, Rousset C, Siqueira L, Stan C, Tomas R, Vertenstein M (2012) Impact of ocean model resolution on CCSM climate simulations. Clim Dyn 39(6):1303–1328
Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010) Challenges in combining projections from multiple models. J Clim 23:2739–2758
Krzanowski WJ (1979) Between-groups comparison of principal components. J Am Stat Assoc 74:703–707
Li JLF, Waliser DE, Stephens G, Lee S, Ecuyer TL, Kato S, Loeb N, Ma Y (2013) Characterizing and understanding radiation budget biases in CMIP3/CMIP5 GCMs, contemporary GCM, and reanalysis. J Geophys Res Atmos 118:8166–8184
Mantua NJ, Hare SR, Zhang Y, Wallace JM, Francis RC (1997) A Pacific decadal climate oscillation with impacts on salmon. Bull Am Meteorol Soc 78:1069–1079
Martin GM, Milton SF, Senior CA, Brooks ME, Ineson S, Reichler T, Kim J (2010) Analysis and reduction of systematic errors through a seamless approach to modelling weather and climate. J Clim 23:5933–5957
Martin GM, Bellouin N, Collins WJ, Culverwell ID, Halloran P, Hardiman S, Hinton TJ, Jones CD, McLaren A, O’Connor F, Rodriguez J, Woodward S et al (2011) The HadGEM2 family of met office unified model climate configurations. Geosci Model Dev Discuss 4:723–757
Meehl GA, Covey C, Delworth T, Latif M, McAvaney B, Mitchell JFB, Stouffer RJ, Taylor KE (2007) The WCRP CMIP3 multi-model dataset: a new era in climate change research. Bull Am Meteorol Soc 88:1383–1394
Murphy JM, Sexton DMH, Barnett DN, Jones GS, Webb MJ, Collins M, Stainforth DA (2004) Quantification of modeling uncertainties in a large ensemble of climate change simulations. Nature 430:768–772
North GR, Bell TL, Cahalan RF, Moeng FJ (1982) Sampling errors in the estimation of empirical orthogonal functions. Mon Weather Rev 110:699–706
Pierce DW, Barnett TP, Schneider N, Saravanan R, Dommenget D, Mojib L (2001) The role of ocean dynamics in producing decadal climate variability in the North Pacific. Clim Dyn 18(1–2):51–70
Rayner NA, Parker DE, Horton EB, Folland CK, Alexander LV, Rowell DP, Kent EC, Kaplan A (2003) Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century. J Geophys Res 108(D14):4407. doi:10.1029/2002JD002670
Reifen C, Toumi R (2009) Climate projections: past performance no guarantee of future skill? Geophys Res Lett 36:L13704. doi:10.1029/2009GL038082
Santer BD, Taylor KE, Gleckler PJ, Bonfils C, Barnett TP, Pierce DW, Wigley TML, Mears C, Wentz FJ, Brüggemann W, Gillett NP, Klein SA, Solomon S, Stott PA, Wehner MF (2009) Incorporating model quality information in climate change detection and attribution studies. Proc Natl Acad Sci USA 106:14778–14783
Smith TM, Reynolds RW, Peterson TC, Lawrimor J (2008) Improvements to NOAA’s historical merged land–ocean surface temperature analysis (1880–2006). J Clim 21:2283–2296
Stockdale TN (1997) Coupled ocean–atmosphere forecasts in the presence of climate drift. Mon Weather Rev 125:809–818
Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res 106(D7):7183–7192. doi:10.1029/2000JD900719
Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93:485–498
Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philos Trans R Soc Lond A 365:2053–2075
Washington WM, Meehl GA (1984) Seasonal cycle experiment on the climate sensitivity due to a doubling of CO2 with an atmospheric general circulation model coupled to a simple mixed-layer ocean model. J Geophys Res 89(D6):9475–9503. doi:10.1029/JD089iD06p09475
Xavier PK, Duvel J-P, Braconnot P, Doblas-Reyes FJ (2010) An evaluation metric for intraseasonal variability and its application to CMIP3 twentieth-century simulations. J Clim 23:3497–3508
Zhou T, Wu B, Wang B (2009) How well do atmospheric general circulation models capture the leading modes of the interannual variability of the Asian–Australian monsoon? J Clim 22:1159–1173
Acknowledgments
We like to thank Tobias Bayr, Johanna Baehr, Katja Lorbacher and Timofej Woyzichowzki for fruitful discussions and comments. The comments of two anonymous referees have helped to improve the presentation of this study substantially. The ARC Centre of Excellence in Climate System Science (CE110001028) and the Deutsche Forschung Gemeinschaft (DO1038/5-1) supported this study. The slab ocean model simulations were computed on the National Computational Infrastructure in Canberra.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
382_2014_2154_MOESM1_ESM.pdf
First three leading EOF patterns, as in Fig. 10, but of high-pass SSTA in the Tropical Indian Ocean and Pacific for a selection of model simulations and observations. In addition to the observations, the CMIP3 and CMIP5 super models, the models with the largest (GISS-AOM) and smallest (ECHAM5/MPI-OM) RMSEEOF value are shown (PDF 1153 kb)
382_2014_2154_MOESM2_ESM.pdf
First three leading EOF patterns, as in Fig. 10, but of high-pass SSTA in the North Atlantic for a selection of model simulations and observations. In addition to the observations, the CMIP3 and CMIP5 super models, the models with the largest (CSIRO-Mk3.0) and smallest (CGCM3.1 (T47)) RMSEEOF value are shown (PDF 751 kb)
382_2014_2154_MOESM3_ESM.pdf
First three leading EOF patterns, as in Fig. 10, but of high-pass SSTA in the Tropical Atlantic for a selection of model simulations and observations. In addition to the observations, the CMIP3 and CMIP5 super models, the models with the largest (INMCM4) and smallest (CCSM4) RMSEEOF value are shown (PDF 491 kb)
382_2014_2154_MOESM4_ESM.pdf
First three leading EOF patterns, as in Fig. 10, but of high-pass SSTA in the Southern Ocean for a selection of model simulations and observations. In addition to the observations, the CMIP3 and CMIP5 super models, the models with the largest (INMCM4) and smallest (CCSM4) RMSEEOF value are shown (PDF 999 kb)
382_2014_2154_MOESM5_ESM.pdf
Nspatial values for all models as shown in Fig. 4 (PDF 93 kb)
382_2014_2154_MOESM6_ESM.pdf
RMSEEOF values relative to the observations as shown in Fig. 9 for all models (PDF 81 kb)
382_2014_2154_MOESM7_ESM.pdf
RMSEEOF values of the pairwise model comparison as shown in Fig. 11 (PDF 82 kb)
Rights and permissions
About this article
Cite this article
Wang, G., Dommenget, D. & Frauen, C. An evaluation of the CMIP3 and CMIP5 simulations in their skill of simulating the spatial structure of SST variability. Clim Dyn 44, 95–114 (2015). https://doi.org/10.1007/s00382-014-2154-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-014-2154-0