Hsiang and Burke (2014) examine 50 quantitative empirical studies and find a “remarkable convergence in findings” (p. 52) and “strong support for a causal association” (p. 42) between climatological changes and conflict at all scales and across all major regions of the world. A companion paper by Hsiang et al. (2013) that attempts to quantify the average effect from these studies indicates that a 1 standard deviation (σ) increase in temperature or rainfall anomaly is associated with an 11.1 % change in the risk of “intergroup conflict”.Footnote 1 Assuming that future societies respond similarly to climate variability as past populations, they warn that increased rates of human conflict might represent a “large and critical impact” of climate change.

What Hsiang and Burke set out to do – to synthesize scientific evidence on climate and conflict – is a timely task, and their study adds to an increasingly long list of scientific reviews published in the recent past (Adger et al. 2014; Bernauer et al. 2012; Gleditsch 2012; Klomp and Bulte 2013; Meierding 2013; Scheffran et al. 2012a,b; Theisen et al. 2013; Zografos et al. 2014). However, Hsiang and Burke depart from these other reviews in two important ways. First, their conclusion about a consistent and robust climate effect contrasts dominant observations, such as “researchers have failed to uncover consistent linkages between environmental shifts and intrastate contention” (Meierding 2013, 185); “we find little robust evidence linking weather shocks to the onset of conflict” (Klomp and Bulte 2013, 63); and, arguably most prominently, the comprehensive Human Security chapter of the UN’s Intergovernmental Panel on Climate Change’s Fifth Assessment Report: “collectively the research does not conclude that there is a strong positive relationship between warming and armed conflict” (Adger et al. 2014, 16). Second, Hsiang and Burke’s review stands out by being founded on an attempt to quantify the “true” climate effect on conflict. This is done through a meta-analysis of replication data from selected statistical articles and unpublished papers on climate and human conflict (Hsiang et al. 2013).

We believe an important reason for the notable discrepancy in views between Hsiang and Burke and the larger scholarly community can be traced back to problems related to the accompanying meta-analysis.Footnote 2 This commentary identifies three limitations with the meta-analysis, and implicitly with Hsiang and Burke’s review, and shows that well-justified modifications to the original setup result in a different conclusion. The problems that we consider here relate to three underlying assumptions of the meta-analysis: (1) cross-study independence, (2) causal homogeneity, and (3) sample representativeness. While Hsiang et al. consider these problems at length in the supplementary material, their portrayal of the statistical association and the unambiguous conclusion in both review articles reveal that they do not attribute much weight to these concerns when interpreting the results.

First, the underlying meta-analysis rests on the assumption that sample studies are fully independent, although it is clear that there is considerable overlap between them. Every civil conflict work considered by Hsiang and Burke (2014) and 19 of 22 studies on the modern climate-intergroup conflict inHsiang et al. (2013) include African countries, and more than half of these are limited to post-1980 Sub-Saharan Africa. In one case, the cross-study correlation is estimated at r = 0.6 (see supplementary information). Accordingly, the variance-weighted calculation of climate effects conducted by Hsiang et al. returns unrealistically precise estimates and the true uncertainty around the average climate effect is much larger than reported.Footnote 3 Instead of relying on the overly narrow variance-weighted confidence interval, Hsiang and coauthors would have done better (i.e., provided a more conservative test) by referring to the 95 % density interval of the distribution of the effect sizes.

Second, Hsiang and Burke’s sample of “intergroup conflict” studies covers a wide range of social phenomena, from non-violent land grabbing via urban riots to major civil war; a wide range of climatic events, from heat waves via excess rainfall to global ENSO cycles; and a wide range of spatial scales, from municipalities via countries to the entire world. Moreover, the surveyed studies differ widely in how the climate effect is theorized and modeled, which sometimes is linear, at other times parabolic; sometimes instant and at other times materialize after a distinct temporal lag. Claiming the same underlying climate effect across these heterogeneous studies is certainly a bold assumption, but it is essential for the meta-analysis to be meaningful. A careful reading of the literature, or inspection of Fig. 1 below, reveals a variation in findings that is inconsistent with the assumption of causal homogeneity.

Fig. 1
figure 1

Modern empirical estimates for the effect of climate variability on civil conflict. The markers illustrate the estimated percentage change in conflict with a 1σ increase in temperature (red), loss of rainfall (blue), increase in drought (orange), El Niño-like conditions (brown) or increase in severity of climatic natural disasters (gray). Whiskers denote the 95 % confidence interval. The solid horizontal line indicates the median climate effect with the 95 % highest density interval in grey, based on a Bayesian hierarchical model. The panels at the right show the distribution of results from all candidate studies (black) or those focusing squarely on temperature effects (red); solid lines represent the variance-weighted distribution while dashed lines depict the Bayesian hierarchical distribution. Studies listed alphabetically

Full size image

Third, aggregating and generalizing results from selected studies serve no larger purpose unless the sample constitutes a representative subset of all relevant scientific research. Yet, the sample inclusion strategy adopted by Hsiang and coauthors favors form over function by using selection criteria that explicitly disregard studies that revisit previously investigated climate-conflict associations, regardless of whether they complement, contrast or correct earlier findings. For example, the country-level relationship between rainfall and civil conflict is represented in the meta-analysis by a single peer-reviewed result (Miguel et al. 2004), omitting numerous more recent investigations that reach different conclusions (e.g., Buhaug 2010; Burke et al. 2009; Ciccone 2011; Couttenier and Soubeyran 2013; Koubi et al. 2012).Footnote 4 Moreover, the meta-analysis, and implicitly also the non-technical assessment in Hsiang and Burke, considers just one climate indicator from each study, in many cases the one that indicated the strongest effect, although many of the original studies actually explore multiple alternative and complementary climate measures that sometimes produce contrasting results.Footnote 5 We do not believe that oversampling of significant results is a fruitful way to provide a balanced and sound review of the scientific evidence base for a causal climate-conflict link.

These are not trivial concerns, and they add up. In an effort to address these issues, we replicated Hsiang et al.’s meta-analysis after implementing three changes. More specifically, we first streamlined the sample by limiting the selection of studies to those investigating violent political conflict between organized non-state actors and state military forces (i.e., civil conflicts). This is the form of conflict with the gravest social and political implications, and narrowing the scope of the dependent variable increases the plausibility of the assumed common causal effect. Second, to obtain a more representative sample and to avoid the temptation to selectively pick indicators that produce a desired result, we included all main climate variables investigated in the relevant studies. For the same reason, and to ensure analytical consistency and the correct sequence of events, we also applied a consistent lag of one time period (t–1) to all climate parameters across all models.Footnote 6

The updated meta-analysis with all three modifications implemented is visualized in Fig. 1. The result is striking. In contrast to Hsiang and coauthors, we find no evidence of a convergence of findings on climate variability and civil conflict. Recent studies disagree not only on the magnitude of the impact of climate variability but also on the direction of the effect. The aggregate median effect from these studies suggests that a one-standard deviation increase in temperature or loss of rainfall is associated with a 3.5 % increase in conflict risk, although the 95 % highest density area of the distribution of effects cannot exclude the possibility of large negative or positive effects. With all contemporaneous effects, the aggregate point estimate increases somewhat but remains statistically indistinguishable from zero.

Does our updated meta-analysis provide the true quantification of the average effect of climate on violent conflict? We dare say no. While the modifications we have implemented increase analytical consistency and representativeness and offer a meaningful alternative specification, a fundamental problem of this analysis remains largely unaddressed. Unlike meta-analyses of medical treatment studies, which are based on similar individual-level investigations from independent sample populations, this assessment bundles together partly overlapping observations at different spatial and temporal scales in an inconsistent and atheoretical fashion. Quantifying the “average” effect across this heterogeneous sample, weighting results by statistical precision but ignoring variations in substantive relevance and potential for generalization, is unlikely to be fruitful, unless there are good theoretical reasons to expect common causal effects. It is a bit like averaging the prices of apples and oranges, sampled in different locations and time periods, at different scales, using different metrics in the absence of a strong basis to expect that they are driven by the same factor(s). The exercise is mathematically feasible but the outcome may have no relevant meaning. Remedying this problem can only be done by taking into account the substantive content of candidate studies, ensuring similar units of analysis across studies based on unique samples, and parameterizing empirical indicators in a theoretically consistent manner.Footnote 7

To be clear, this commentary should not be taken to imply that climate has no influence on armed conflict. Rather, we argue – in line with recent scientific reviews (Adger et al. 2014; Bernauer et al. 2012; Gleditsch 2012; Klomp and Bulte 2013; Meierding 2013; Scheffran et al. 2012a,b; Theisen et al. 2013; Zografos et al. 2014) – that research to date has failed to converge on a specific and direct association between climate and violent conflict. In order to uncover more subtle and complex conditions under which climatic events plausibly may have a measurable impact on conflict dynamics, future research should continue the recent trend toward spatial and temporal disaggregation, focus on political violence beyond civil war, and investigate possible indirect mechanisms and intervening factors (for inspiration, see Fjelde and von Uexkull 2012; O′Loughlin et al. 2012; Theisen 2012; Wischnath and Buhaug 2014).