Evaluating explanatory models of the spatial pattern of surface climate trends using model selection and bayesian averaging methods


We evaluate three categories of variables for explaining the spatial pattern of warming and cooling trends over land: predictions of general circulation models (GCMs) in response to observed forcings; geographical factors like latitude and pressure; and socioeconomic influences on the land surface and data quality. Spatial autocorrelation (SAC) in the observed trend pattern is removed from the residuals by a well-specified explanatory model. Encompassing tests show that none of the three classes of variables account for the contributions of the other two, though 20 of 22 GCMs individually contribute either no significant explanatory power or yield a trend pattern negatively correlated with observations. Non-nested testing rejects the null hypothesis that socioeconomic variables have no explanatory power. We apply a Bayesian Model Averaging (BMA) method to search over all possible linear combinations of explanatory variables and generate posterior coefficient distributions robust to model selection. These results, confirmed by classical encompassing tests, indicate that the geographical variables plus three of the 22 GCMs and three socioeconomic variables provide all the explanatory power in the data set. We conclude that the most valid model of the spatial pattern of trends in land surface temperature records over 1979–2002 requires a combination of the processes represented in some GCMs and certain socioeconomic measures that capture data quality variations and changes to the land surface.

This is a preview of subscription content, log in to check access.

Fig. 1


  1. Anselin L, Bera AK, Florax R, Yoon MJ (1996) Simple diagnostic tests for spatial dependence. Regional Science and Urban Economics 26:77–104

    Article  Google Scholar 

  2. Berk RA, Fovell RG, Schoenberg F, Weiss RE (2001) The use of statistical tools for evaluating computer simulations. Climatic Change 51(2):119–130

    Google Scholar 

  3. Brohan P, Kennedy JJ, Harris I, Tett SFB, Jones PD (2006) Uncertainty estimates in regional and global observed temperature changes: a new dataset from 1850. J Geophys Res 111:D12106. doi:10.1029/2005JD006548

    Article  Google Scholar 

  4. CCSP (2008) Climate models: an assessment of strengths and limitations. A report by the U.S. Climate Change Science Program and the Subcommittee on Global Change Research In: Bader DC, Covey C, Gutowski Jr WJ, Held IM, Kunkel KE, Miller RL, Tokmakian RT, Zhang MH (eds), Department of Energy, Office of Biological and Environmental Research, Washington, DC

  5. Covey C, AchutaRao KM, Cubasch U, Jones P, Lambert SJ, Mann ME, Phillips TJ, Taylor KE (2003) An overview of results from the Coupled Model Intercomparison Project. Global Planet Change 37:103–133

    Article  Google Scholar 

  6. Davidson R, MacKinnon JG (1981) Several tests for model specification in the presence of alternative hypotheses. Econometrica 49(3):781–793

    Article  Google Scholar 

  7. Davidson R, MacKinnon JG (2004) Econometric theory and methods. Toronto, Oxford

    Google Scholar 

  8. De Laat ATJ, Maurellis AN (2004) Industrial CO2 emissions as a proxy for anthropogenic influence on lower tropospheric temperature trends. Geophys Res Lett 31:L05204. doi:10.1029/2003GL019024

    Article  Google Scholar 

  9. De Laat ATJ, Maurellis AN (2006) Evidence for influence of anthropogenic surface processes on lower tropospheric and surface temperature trends. Int J Climatol 26:897–913

    Article  Google Scholar 

  10. Easterly W, Sewadeh M (2003) World Bank global development network growth data base. http://www.worldbank.org/research/growth/GDNdata.htm. Accessed fall 2003

  11. Fernandez C, Ley E, Steel M (2001) Benchmark priors for Bayesian model averaging. Journal of Econometrics 100:381–427

    Article  Google Scholar 

  12. Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113:D06104. doi:10.1029/2007JD008972

    Article  Google Scholar 

  13. Hegerl G, Hasselmann K, Cubash U, Mitchell J, Roeckner E, Voss R, Waszkewitz J (1997) Multi-fingerprint detection and attribution analysis of greenhouse gas, greenhouse gas-plus-aerosol and solar forced climate change. Clim Dyn 13(9):613–634

    Article  Google Scholar 

  14. Hegerl GC, Zwiers FW, Braconnot P, Gillett NP, Luo Y, Marengo Orsini JA, Nicholls N, Penner JE, Stott PA (2007) Under-standing and attributing climate change. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds), Climate Change 2007: the physical science basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge and New York

  15. Hoeting J, Madigan D, Raftery A, Volinsky C (1999) Bayesian model averaging: a tutorial. Statistical Science 14:382–417

    Article  Google Scholar 

  16. Jenne RL (1974) Jenne’s northern hemisphere climatology, monthly, 1950–64. National Center for Atmospheric Research Dataset DS205.0. National Center for Atmospheric Research, Boulder, CO

    Google Scholar 

  17. Jun M, Knutti R, Nychka DW (2008) Spatial analysis to quantify numerical model bias and dependence: how many climate models are there? Journal of the American Statistical Association 108(483):934–947. doi:10.1198/016214507000001265

    Article  Google Scholar 

  18. Kaufmann RK, Stern DI (2004) A statistical evaluation of atmosphere–ocean general circulation models: complexity vs. simplicity. Rensselaer Polytechnic Institute Department of Economics Working Paper 0411, May 2004

  19. Kiehl JT (2007) Twentieth century climate model response and climate sensitivity. Geophys Res Lett 34:L22710. doi:10.1029/2007GL031383

    Article  Google Scholar 

  20. Knutson TR, Delworth TL, Dixon KW, Held IM, Lu J, Ramaswamy V, Schwartzkopf MD, Stenchikov G, Stouffer RJ (2006) Assessment of twentieth-century regional surface temperature trends using the GFDL CM2 coupled models. J Clim. 19:1624–1651

    Google Scholar 

  21. Knutti R (2008) Why are climate models reproducing the observed global surface warming so well? Geophys Res Lett 35:L18704. doi:10.1029/2008GL034932

    Article  Google Scholar 

  22. Knutti R, Hegerl G (2008) The equilibrium sensitivity of the Earth’s temperature to radiation changes. Nat Geosci 1:735–743. doi:10.1038/ngeo337

    Article  Google Scholar 

  23. Madigan D, York J (1995) Bayesian graphical models for discrete data. International Statistical Review 63:215–232

    Article  Google Scholar 

  24. McKitrick RR (2010) Atmospheric oscillations do not explain the temperature-industrialization correlation. Stat Polit Policy 1(1)

  25. McKitrick R, Michaels PJ (2004) A Test of Corrections for Extraneous Signals in Gridded Surface Temperature Data. Climate Research 26:159–173

    Article  Google Scholar 

  26. McKitrick RR, Michaels PJ (2007) Quantifying the influence of anthropogenic surface processes and inhomogeneities on gridded global climate data. J Geophys Res 112:D24S09. doi:10.1029/2007JD008465

    Article  Google Scholar 

  27. McKitrick RR, Nierenberg N (2010) Socioeconomic patterns in climate data. J Econ Social Meas 35(3,4):149–175. doi:10.3233/JEM-2010-0336

    Google Scholar 

  28. McKitrick RR, McIntyre S, Herman C (2010) Panel and Multivariate Methods for Tests of Trend Equivalence in Climate Data Sets. Atmospheric Science Letters. doi:10.1002/asl.290

    Google Scholar 

  29. Mears CA, Schabel MC, Wentz FJ (2003) A reanalysis of the MSU channel 2 tropospheric temperature record. J Clim 16(22):3650–3664

    Article  Google Scholar 

  30. Michaels PJ, Knappenberger PC, Balling RC Jr, Davis RE (2000) Observed warming in cold anticyclones. Climate Research 14:1–6

    Article  Google Scholar 

  31. Mizon GE (1984) The encompassing approach in econometrics. In: Hendry DF, Wallis KF (eds) Econometrics and quantitative economics. Basil Blackwell, Oxford, pp 135–172

    Google Scholar 

  32. Parry ML, Canziani OF, Palutikof JP, van der Linden PJ, Hanson CE (eds) (2007) Contribution of working group II to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge and New York

    Google Scholar 

  33. Pisati M (2001) Tools for spatial data analysis. Stata Tech Bull STB-60, March 2001, pp 21–37

  34. Randall DA, Wood RA, Bony S, Colman R, Fichefet T, Fyfe J, Kattsov V, Pitman A, Shukla J, Srinivasan J, Stouffer RJ, Sumi A, Taylor KE (2007) Climate Models and Their Evaluation. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M Miller HL (eds) Climate change 2007: the physical science basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge and New York

  35. Santer BD, Thorne PW, Haimberger L, Taylor KE, Wigley TML, Lanzante JR, Solomon S, Free M, Gleckler PJ, Jones PD (2008) Consistency of modelled and observed temperature trends in the tropical troposphere. Int J Climatol. doi:10.1002/joc.1756

    Google Scholar 

  36. Schmidt G (2009) Spurious correlation between recent warming and indices of local economic activity. Int J Climatol. doi:10.1002/joc.1831

    Google Scholar 

  37. Schwartz SE, Charlson RJ, Rodhe H (2007) Quantifying climate change—too rosy a picture? Nat Rep Clim Change 2:23–24

    Article  Google Scholar 

  38. Shukla J, DelSole T, Fennessy M, Kinter J, Paolino D (2006) Climate model fidelity and projections of climate change. Geophy Res Lett 33:L07702, doi:10.1029/2005GL025579

  39. Spencer RW, Christy JC (1990) Precise monitoring of global temperature trends from satellites. Science 247:1558–1562

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Ross McKitrick.

Appendix: Further details on data set

Appendix: Further details on data set

Temperature Trends: The observed surface temperature trends y i are linear (Ordinary Least Squares) trends through monthly temperature anomalies (not subject to annual averaging) within 5 × 5 degree grid cells over 1979:1–2002:12 in the land-based grid cells in the CRU data, versions 2 and 3, as well as in the GCM-generated data and the tropospheric data. Because of the need for a trend across 23 years we required each cell to have data for at least ninety percent of the years, where a year is considered intact if at least 8 months are available. In the CRU version 2 data this left 451 usable locations. Antarctic cells were removed, leaving 440 observations in the final version 2 data set but only 428 in CRU version 3.

GCM Data: We used all available (55) runs from 22 GCMs used in the IPCC report (Hegerl et al. 2007). The archive is at http://www-pcmdi.llnl.gov. Multiple runs from a single model were averaged. HadCM3 wasn’t used because it did not represent its data in the required IPCC pressure levels. MUIB ECHO G wasn’t processed because no atmospheric temperature data was available, thus synthetic MSU brightness temperatures couldn’t be calculated. A single run is a deterministic computation of a climate model sing as inputs the observed climatic forcings over the historical interval, and solving for predicted climate fields including temperature, pressure and precipitation.

The calculation of tropospheric temperature from the models was done using the same algorithm and weighting functions implemented in Santer et al. (2008), which are designed to yield layer averages corresponding to observational ones measured by weather satellites. Model trend fields in degrees C/decade for the surface and lower tropospheric temperature were calculated as follows.

  1. 1.

    Extract all monthly GCM-generated data on temperature by grid cell from the surface through to the mid-troposphere from Jan 1979–Dec 2002.

  2. 2.

    Compute the climatology (gridcell averages by month) for the same period.

  3. 3.

    Subtract the climatology from the original data, yielding deviations or “anomalies.”

  4. 4.

    Calculate the least squares trend field for each grid point only if all the data points are valid.

  5. 5.

    Collect only the trends that correspond to the McKitrick and Michaels (2007) set of lat/lon coordinates.

  6. 6.

    Multiply the resulting annual trends by 10 to obtain decadal trends

There was no missing data for the surface temperature variable in models, but there was some missing data in some runs for the lower tropospheric (LT) temperatures. This is because the models originally didn’t represent the atmospheric temperature on the same set of pressure levels that the IPCC mandated. Interpolation was required and this resulted in some missing data points in the lower atmosphere. To calculate the LT temperature, the atmospheric temperature profile was multiplied by a set of weights specific to a given atmospheric layer (TLT, TMT, TLS). The weighted temperatures were then added up and divided by the sum of weights that correspond to non-missing temperature values. If this total weight did not equal or exceed 0.5 or 50 %, then the temperature at that grid point was flagged as missing.

Geographic Data: press i is the mean sea level air pressure in grid cell i. The source of the pressure data is the climatology of Jenne (1974). DRY i is a dummy variable denoting when a grid cell is characterized by predominantly dry conditions (which is indicated by the mean dewpoint being below 0 °C). DSLP i  = DRY i  × PRESS i . Surface warming due to greenhouse gases is hypothesized to occur faster in regions with relatively dry air and high atmospheric pressure (Michaels et al. 2000) so pressure enters the regression model as a linear spline function with a different intercept and slope in dry regions versus moist regions. WATER i is a dummy variable indicating the grid cell contains a major coastline. ABSLAT i denotes the absolute latitude of the grid cell.

Oscillation data: The measures for the Arctic Oscillation, North Atlantic Oscillation, Pacific Decadal Oscillation and Southern Oscillation are taken from McKitrick (2010), who obtained them in turn from http://www.cdc.noaa.gov/Correlation, the website of the National Oceanic and Atmospheric Administration). There is a single value of each oscillation index for the whole planet each period. What is reported at the grid cell level is the correlation between the temperatures in that grid cell and the index value over the 1979–2001 interval, thus representing a measure of the influence of the oscillation over space. The correlation can be computed in two ways, as simple Pearson correlation term, or as a regression coefficient. McKitrick (2010) reports that the latter formula yielded stronger results for the oscillation terms in the regression models so that is the form used herein.

Socioeconomic data: Each grid cell was assigned to a country. Annual real (inflation adjusted) GDP for 1979, 1989 and 1999 for each country was obtained primarily from Easterly and Sewadeh (2003) or the Central Intelligence Agency (CIA) World Fact Book. Conversions from local currency to US dollars was done using the purchasing power parity method. There were small adjustments made to the economic data for some countries to provide consistency in quantities where direct measures were unavailable. In most cases the adjustment took the form of using an available observation for 1 or 2 years after the desired year, and adjusting it backwards. Population data are obtained from Easterly and Sewadeh (2003) and the percent change p i is measured from 1979 to 1999. Income growth m i is the percentage change in real GDP per capita from 1979 to 1999. GDP growth y i is defined as the percentage change in real GDP from 1979 to 1999. National coal consumption data were obtained from the US Energy Information Administration and the coal growth measure is the percentage growth of short tons of coal consumed between 1980 and 2000. The 1999 (or closest year) national literacy rate and the percentage completing post-secondary education was obtained from UNESCO. The two measures are summed together to yield e i . Land area estimates (excluding water) for each country were obtained from the CIA World Fact Book. GDP density g i is measured as $million/km2. The 1979 value was used to help ensure the right-hand side variables are predetermined with respect to the dependent variable. x i is the number of months over the period 1979–2002 in which an observation was missing for a grid cell.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

McKitrick, R., Tole, L. Evaluating explanatory models of the spatial pattern of surface climate trends using model selection and bayesian averaging methods. Clim Dyn 39, 2867–2882 (2012). https://doi.org/10.1007/s00382-012-1418-9

Download citation


  • GCM testing
  • Spatial trend patterns
  • Climate data contamination, spatial autocorrelation
  • Non-nested tests
  • Encompassing tests
  • Bayesian model averaging