One effect to rule them all? A comment on climate and conflict
A recent Climatic Change review article reports a remarkable convergence of scientific evidence for a link between climatic events and violent intergroup conflict, thus departing markedly from other contemporary assessments of the empirical literature. This commentary revisits the review in order to understand the discrepancy. We believe the origins of the disagreement can be traced back to the review article’s underlying quantitative meta-analysis, which suffers from shortcomings with respect to sample selection and analytical coherence. A modified assessment that addresses some of these problems suggests that scientific research on climate and conflict to date has produced mixed and inconclusive results.
Hsiang and Burke (2014) examine 50 quantitative empirical studies and find a “remarkable convergence in findings” (p. 52) and “strong support for a causal association” (p. 42) between climatological changes and conflict at all scales and across all major regions of the world. A companion paper by Hsiang et al. (2013) that attempts to quantify the average effect from these studies indicates that a 1 standard deviation (σ) increase in temperature or rainfall anomaly is associated with an 11.1 % change in the risk of “intergroup conflict”.1 Assuming that future societies respond similarly to climate variability as past populations, they warn that increased rates of human conflict might represent a “large and critical impact” of climate change.
What Hsiang and Burke set out to do – to synthesize scientific evidence on climate and conflict – is a timely task, and their study adds to an increasingly long list of scientific reviews published in the recent past (Adger et al. 2014; Bernauer et al. 2012; Gleditsch 2012; Klomp and Bulte 2013; Meierding 2013; Scheffran et al. 2012a,b; Theisen et al. 2013; Zografos et al. 2014). However, Hsiang and Burke depart from these other reviews in two important ways. First, their conclusion about a consistent and robust climate effect contrasts dominant observations, such as “researchers have failed to uncover consistent linkages between environmental shifts and intrastate contention” (Meierding 2013, 185); “we find little robust evidence linking weather shocks to the onset of conflict” (Klomp and Bulte 2013, 63); and, arguably most prominently, the comprehensive Human Security chapter of the UN’s Intergovernmental Panel on Climate Change’s Fifth Assessment Report: “collectively the research does not conclude that there is a strong positive relationship between warming and armed conflict” (Adger et al. 2014, 16). Second, Hsiang and Burke’s review stands out by being founded on an attempt to quantify the “true” climate effect on conflict. This is done through a meta-analysis of replication data from selected statistical articles and unpublished papers on climate and human conflict (Hsiang et al. 2013).
We believe an important reason for the notable discrepancy in views between Hsiang and Burke and the larger scholarly community can be traced back to problems related to the accompanying meta-analysis.2 This commentary identifies three limitations with the meta-analysis, and implicitly with Hsiang and Burke’s review, and shows that well-justified modifications to the original setup result in a different conclusion. The problems that we consider here relate to three underlying assumptions of the meta-analysis: (1) cross-study independence, (2) causal homogeneity, and (3) sample representativeness. While Hsiang et al. consider these problems at length in the supplementary material, their portrayal of the statistical association and the unambiguous conclusion in both review articles reveal that they do not attribute much weight to these concerns when interpreting the results.
First, the underlying meta-analysis rests on the assumption that sample studies are fully independent, although it is clear that there is considerable overlap between them. Every civil conflict work considered by Hsiang and Burke (2014) and 19 of 22 studies on the modern climate-intergroup conflict inHsiang et al. (2013) include African countries, and more than half of these are limited to post-1980 Sub-Saharan Africa. In one case, the cross-study correlation is estimated at r = 0.6 (see supplementary information). Accordingly, the variance-weighted calculation of climate effects conducted by Hsiang et al. returns unrealistically precise estimates and the true uncertainty around the average climate effect is much larger than reported.3 Instead of relying on the overly narrow variance-weighted confidence interval, Hsiang and coauthors would have done better (i.e., provided a more conservative test) by referring to the 95 % density interval of the distribution of the effect sizes.
Third, aggregating and generalizing results from selected studies serve no larger purpose unless the sample constitutes a representative subset of all relevant scientific research. Yet, the sample inclusion strategy adopted by Hsiang and coauthors favors form over function by using selection criteria that explicitly disregard studies that revisit previously investigated climate-conflict associations, regardless of whether they complement, contrast or correct earlier findings. For example, the country-level relationship between rainfall and civil conflict is represented in the meta-analysis by a single peer-reviewed result (Miguel et al. 2004), omitting numerous more recent investigations that reach different conclusions (e.g., Buhaug 2010; Burke et al. 2009; Ciccone 2011; Couttenier and Soubeyran 2013; Koubi et al. 2012).4 Moreover, the meta-analysis, and implicitly also the non-technical assessment in Hsiang and Burke, considers just one climate indicator from each study, in many cases the one that indicated the strongest effect, although many of the original studies actually explore multiple alternative and complementary climate measures that sometimes produce contrasting results.5 We do not believe that oversampling of significant results is a fruitful way to provide a balanced and sound review of the scientific evidence base for a causal climate-conflict link.
These are not trivial concerns, and they add up. In an effort to address these issues, we replicated Hsiang et al.’s meta-analysis after implementing three changes. More specifically, we first streamlined the sample by limiting the selection of studies to those investigating violent political conflict between organized non-state actors and state military forces (i.e., civil conflicts). This is the form of conflict with the gravest social and political implications, and narrowing the scope of the dependent variable increases the plausibility of the assumed common causal effect. Second, to obtain a more representative sample and to avoid the temptation to selectively pick indicators that produce a desired result, we included all main climate variables investigated in the relevant studies. For the same reason, and to ensure analytical consistency and the correct sequence of events, we also applied a consistent lag of one time period (t–1) to all climate parameters across all models.6
The updated meta-analysis with all three modifications implemented is visualized in Fig. 1. The result is striking. In contrast to Hsiang and coauthors, we find no evidence of a convergence of findings on climate variability and civil conflict. Recent studies disagree not only on the magnitude of the impact of climate variability but also on the direction of the effect. The aggregate median effect from these studies suggests that a one-standard deviation increase in temperature or loss of rainfall is associated with a 3.5 % increase in conflict risk, although the 95 % highest density area of the distribution of effects cannot exclude the possibility of large negative or positive effects. With all contemporaneous effects, the aggregate point estimate increases somewhat but remains statistically indistinguishable from zero.
Does our updated meta-analysis provide the true quantification of the average effect of climate on violent conflict? We dare say no. While the modifications we have implemented increase analytical consistency and representativeness and offer a meaningful alternative specification, a fundamental problem of this analysis remains largely unaddressed. Unlike meta-analyses of medical treatment studies, which are based on similar individual-level investigations from independent sample populations, this assessment bundles together partly overlapping observations at different spatial and temporal scales in an inconsistent and atheoretical fashion. Quantifying the “average” effect across this heterogeneous sample, weighting results by statistical precision but ignoring variations in substantive relevance and potential for generalization, is unlikely to be fruitful, unless there are good theoretical reasons to expect common causal effects. It is a bit like averaging the prices of apples and oranges, sampled in different locations and time periods, at different scales, using different metrics in the absence of a strong basis to expect that they are driven by the same factor(s). The exercise is mathematically feasible but the outcome may have no relevant meaning. Remedying this problem can only be done by taking into account the substantive content of candidate studies, ensuring similar units of analysis across studies based on unique samples, and parameterizing empirical indicators in a theoretically consistent manner.7
To be clear, this commentary should not be taken to imply that climate has no influence on armed conflict. Rather, we argue – in line with recent scientific reviews (Adger et al. 2014; Bernauer et al. 2012; Gleditsch 2012; Klomp and Bulte 2013; Meierding 2013; Scheffran et al. 2012a,b; Theisen et al. 2013; Zografos et al. 2014) – that research to date has failed to converge on a specific and direct association between climate and violent conflict. In order to uncover more subtle and complex conditions under which climatic events plausibly may have a measurable impact on conflict dynamics, future research should continue the recent trend toward spatial and temporal disaggregation, focus on political violence beyond civil war, and investigate possible indirect mechanisms and intervening factors (for inspiration, see Fjelde and von Uexkull 2012; O′Loughlin et al. 2012; Theisen 2012; Wischnath and Buhaug 2014).
For reasons that are not clear, there is not perfect overlap between Hsiang and Burke’s sample of 50 “rigorous quantitative studies” and Hsiang et al.’s 60 “most rigorous quantitative studies”, and there are also some inconsistencies in how some studies are categorized. In this comment, we focus only on the subset of studies that concern modern (i.e., post-1950) “intergroup conflict”, comprising civil and communal conflict. Twenty of the 21 studies of this kind reviewed by Hsiang and Burke are also part of Hsiang et al.’s meta-analysis.
The noted discrepancy between Hsiang and Burke (and coauthors) and other scholars has triggered several commentaries and responses; see, for example, Buhaug (2014), Hsiang and Meng (2014), Raleigh et al. (2014), and O’Loughlin et al. (2014).
In Hsiang et al.’s supplementary material, cross-study correlation is discussed and alternative and more appropriate estimations of uncertainty are indicated, whereby the 95 % precision interval around the mean effect ranges from −7.8 to 40.1 (Hsiang et al. Table S1; see also right panel of Hsiang et al.’s Fig. 5). Crucially, the interpretation of this interval is different than for the variance-weighted interval, as it is here acknowledged that effects vary between studies (it is possible that climate effects sometimes are positive, and sometimes negative). It is not evident, however, what is gained by quantifying the variation in such a heterogeneous sample over and beyond what thorough qualitative reviews of the quantitative literature can (and do) offer.
Some of the studies that are omitted in Hsiang and colleagues’ assessment of the rainfall-conflict link are represented in the meta-analysis with other, typically more significant effects (e.g., temperature or drought).
To give one example, Hsiang and colleagues’ review of Couttenier and Soubeyran (2013) is limited to citing and replicating the result for the PDSI index, thus ignoring the study’s findings with regard to the other climatological indicators. Couttenier and Soubeyran (2013, 213) report that “[t]he effect of (log) rainfall is still surprisingly positive and the effect of temperature is still positive. Their effects are insignificant and the contemporaneous and lagged measures are jointly insignificant.” While we agree that the PDSI finding is important and deserves attention, ignoring results that contrast the general narrative inevitably provides a distorted view.
The surveyed studies are not consistent in their treatment of time, reflecting the lack of a precise and rigorous theoretical model that can be applied across cases, scales, and definitions of conflict. Some studies report contemporaneous effects, others report time-lagged effects, and several studies report both contemporaneous and lagged climate effects.
In other words, to facilitate a meta-analysis of this kind, one should focus on one specific form of outcome (e.g., fatal rebel-military conflict events), one specific form of treatment (e.g., high temperature anomaly), one common spatiotemporal scale (e.g., grid cells with monthly resolution), sampled in unique, non-overlapping locations, and use the exact same parameterization and model specification across datasets. Alternatively, one could conduct a meta-regression analysis to examine the impact of differences among studies on effect sizes.
This work has been supported in part by the Research Council of Norway, grant 217995/V10, and the U.S. Army Research Laboratory and the U.S. Army Research Office via the Minerva Initiative, grant W911NF-13-1-0307.
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.