Regional Environmental Change

, Volume 15, Issue 5, pp 755–769 | Cite as

Systematic review approaches for climate change adaptation research

Open Access
Review Article

Abstract

Recent controversy has led to calls for increased standardization and transparency in the methods used to synthesize climate change research. Though these debates have focused largely on the biophysical dimensions of climate change, human dimensions research is equally in need of improved methodological approaches for research synthesis. Systematic review approaches, and more recently realist review methods, have been used within the health sciences for decades to guide research synthesis. Despite this, penetration of these approaches into the social and environmental sciences has been limited. Here, we present an analysis of approaches for systematic review and research synthesis and examine their applicability in an adaptation context. Customized review frameworks informed by systematic approaches to research synthesis provide a conceptually appropriate and practical opportunity for increasing methodological transparency and rigor in synthesizing and tracking adaptation research. This review highlights innovative applications of systematic approaches, with a focus on the unique challenges of integrating multiple data sources and formats in reviewing climate change adaptation policy and practice. We present guidelines, key considerations, and recommendations for systematic review in the social sciences in general and adaptation research in particular. We conclude by calling for increased conceptual and methodological development of systematic review approaches to address the methodological challenges of synthesizing and tracking adaptation to climate change.

Keywords

Climate change Systematic review Human dimensions of climate change Vulnerability Adaptation Research synthesis Social sciences Realist review 

Introduction

With growing recognition of the inevitability of climate change, adaptation has become a core element of climate policy and research (Smith et al. 2011). Recent years have witnessed the commitment of unprecedented levels of adaptation finance through the Green Climate Fund and by multi/bi-lateral donors, and national governments have to varying degrees recognized the need for adaptation (Preston et al. 2011; Termeer et al. 2012). Yet, our knowledge of how human systems will adapt to climate change remains limited. The physical basis of climate change, though complex, can be evaluated vis a vis greenhouse gas emissions. Evaluating and understanding climate change adaptation is conceptually murkier, concerned with adjustments in human systems at different scales and by different actors, with success likely to be perceived differently among scholars, policy makers, and communities (Duerden 2004; Adger et al. 2005; Brooks et al. 2011).

As adaptation financing increases and initiatives are developed, the need for comprehensive syntheses of existing research and tools to evaluate progress on adaptation is increasingly needed. Criticism of existing IPCC assessment reports have called for research synthesis methods that are transparent, clearly defined, and limit reviewer/author bias (Ford and Pearce 2010; Petticrew and McCartney 2011). Though such debates have been primarily targeted at the physical basis of climate change, the adaptation literature is arguably in greater need of systematic synthesis of existing knowledge if we are to document if adaptation is taking place and respond to areas of highest impact and/or vulnerability, evaluate whether adaptation support is translating into actions, facilitate comparison of adaptations across regions and sectors, ensure resources are being appropriately invested, and inform governance systems on the current status and gaps in adaptation action (Pielke et al. 2007; Berrang-Ford et al. 2011; Biesbroek et al. 2013).

In this paper, we seek to contribute to the advancement of methodology for conducting research syntheses of climate change adaptation research, with particular emphasis on the challenge of reviewing adaptation policy and practice. In doing so, we hope to inform the development of systematic review frameworks applicable to a variety of ends, including adaptation assessments. We propose that systematic review approaches provide a conceptually appropriate and practical opportunity for increasing methodological transparency and rigor in synthesising and tracking adaptation research.

We first provide an overview of research synthesis methods used by researchers from varying disciplines. We then consider research questions from adaptation policy and practice, and critically assess the ways in which systematic methods can be adapted to address complex policy-relevant research questions, diverse and sometimes-sparse literature sources, and analysis of qualitative and heterogeneous information. Herein, we define adaptation policy and practice as per Berrang-Ford et al. (2011) and Lesnikowski et al. (2011) to include tangible and intentional actions to reduce vulnerability, increase resilience, and adapt to the impacts of climate change. This paper is part of a special edition commissioned by the UK Department for International Development (DFID) and Canada’s International Development Research Centre (IDRC) to inform the development of their Collaborative Adaptation Research Initiative in Africa and Asia (CARIAA) program. The paper compliments six articles in the special edition that apply to varying degrees systematic approaches to reviewing current knowledge on adaptation to climate change in global vulnerability hotspots (Berrang-Ford et al. this issue; Bizikova et al. 2015; De Souza et al. 2015; Ford et al. 2014; Kilroy 2015; Lwasa 2014; Sud et al. 2015; Tucker et al. 2014).

Systematic approaches to research synthesis

Systematic approaches to research synthesis have been used unevenly across disciplines and often use different terminology to describe the review process. The term ‘systematic review’ was first coined within the social sciences (Glass 1976; Waddington et al. 2012), but is predominantly used in current literature to refer to formal, strictly standardized review papers, primarily published in the health sciences. Fewer than 1 % of documents indexed in Web of Knowledge (218 of >60,000) with the title term ‘systematic review,’ for example, relate to research areas outside of the health or health-related sciences (Supplemental Materials, Search 1). Despite the predominance of systematic reviews in the health sciences, research syntheses guided by systematic methods are prevalent and diverse (Mays et al. 2005; Barnett-Page and Thomas 2009; Gough et al. 2012). Terminology used to describe literature reviews is murky, however, with different disciplines and publications referring to, for example, scoping or mapping reviews, narrative analysis, and conceptual synthesis, sometimes using systematic methods but differentiated from formal systematic review (Table 1) (Gough et al. 2012). Many reviews employ semi-systematic techniques, often implicitly, but without using the term ‘systematic’ (Berkhout 2012; McLeman 2013).
Table 1

A summary of approaches to research synthesis

Research synthesis method

Description

Quantitative or qualitative

References

Formal Systematic review

Predominantly used for analyses of quantitative empirical data in the health sciences, with an emphasis on randomized control trials. Numerous organizations (e.g., Cochrane) have published standardized methodological guidelines, coordinate working groups, and house libraries of approved systematic reviews. Methods are often strictly standardized, with statistical theory dominant. Integration of other guiding theories is possible but not common

Predominantly quantitative analysis of quantitative data, though emerging efforts to integrate qualitative evidence

Jadad et al. (1998), Klassen et al. (1998), Moher et al. (1998)

Meta-analysis

Considered a sub-set of systematic review that conducts formal statistical analysis of results from multiple quantitative studies. Published meta-analyses generally follow formal and strict methodological standards. Use of statistical theory is implicit

Quantitative analysis of quantitative data only

Espey (1996), Akobeng (2005), Furlan et al. (2009), Richardson and Loomis (2009), Higgins and Green (2011)

Best evidence synthesis

Similar to meta-analysis, but proposes that a stricter filter be applied to which studies and evidence are included in the analysis. Suggests using only the highest quality studies and excluding others. This contrasts with standard meta-analysis, which applies a broader inclusion criteria filter based on study quality and includes all eligible studies in analysis. Methods are generally strictly standardized, often with statistical theory dominant. Integration of other guiding theories is possible but not common

Quantitative analysis of quantitative data only

Slavin (1995), Letzel (1995), Cook et al. (1997), Cubbison (1999), Newhouse (2008), Cancelliere et al. (2011)

Quantitative comparative analysis (QCA)

Similar to meta-analysis, yet uses statistical techniques comparable to principal component analysis. Methods are explicitly articulated and guided by statistical theory. Integration of other guiding theories is possible but not common

Quantitative

Wagschal and Wenzelburger (2012), Macias-Chapula (2013)

Realist review

An approach to literature review designed to evaluate interventions by seeking to understand and explain complex causal contexts: ‘does a particular intervention work, for whom, why, and in what context?’ Methods are generally explicitly articulated, including detailed clarification of the scope of the research question and explicit use of theory to guide the review and analysis. Contrasts with traditional systematic review approaches predominantly in its focus on explaining causal processes and use with qualitative literature and questions. Was designed for explaining reasons for success or failure in intervention-based research, but has been adapted more broadly

Mixed, though generally includes a substantive qualitative component

Pawson et al. (2005), Mazzocato et al. (2010), Vassilev et al. (2011), DeBono et al. (2012)

Narrative review

Used predominantly in the health sciences literature to refer to reviews that apply systematic methods for document selection and inclusion, but evaluate mixed qualitative and quantitative literature or use predominantly descriptive analysis. Methods are often explicitly defined, but range in depth, and can include only brief reference to systematic document selection. Use of qualitative theory is possible and occurs occasionally

Mixed

Gibson et al. (2012), Kreichauf et al. (2012), Mwangi-Powell et al. (2013), Wong et al. (2013)

Meta-ethnography

Inductive approach to synthesis of qualitative literature using an interpretive rather than aggregative focus. Methods are often not explicit. Reviews are explicitly guided by qualitative theory. In some cases, authors outline methods by which the review is explicitly designed around the selected theoretical approach. Results are not replicable, and positionality may be considered

Qualitative

Atkins et al. (2008)

Scientometrics

Uses statistics based on citation index databases to measure scientific activity related to selected themes or topics. Methods are generally explicitly articulated and are usually not clearly guided by theory. Statistical theory is commonly used for analysis. Integration of other guiding theories is possible but not common

Quantitative

Bjurstrom and Polk (2011a, b), Janssen and Ostrom (2006) Stanhill (2001), Grieneisen and Zhang (2011)

Broadly, a systematic review refers to a focused review of the literature that seeks to answer a specific research question using pre-defined eligibility criteria for documents and explicitly outlined and reproducible methods (Cooper and Hedge 1994; Gough et al. 2012). They are distinct from other approaches to research synthesis in that they incorporate an explicit layer of methodological systematization, adding transparency and reproducibility to the review process. Though paralleled by other methodological approaches to research synthesis (Table 1), systematic review is notable for the degree to which the approach has been subject to formal standardization. While diverse in application, a systematic review process generally includes a number of formal methodological steps that a researcher follows to identify and analyze literature: (1) define the research question and scope of the study, (2) document selection, including development of inclusion and exclusion criteria, (3) critical appraisal of study quality, (4) analyze and synthesize evidence, quantitative and/or qualitative, and (5) present results (Petticrew and Roberts 2006; Higgins and Green 2011; Barth and Thomas 2012). The systematized review process is designed to ensure that the selection of documents and information sources included is based on a set of clearly defensible criteria rather than ad hoc selection or being subject to undisclosed researcher bias.

Standardized criteria for systematic review approaches have been criticized for a presumed bias toward analysis of primarily quantitative data and for their positivist approach to knowledge synthesis, restricting results based on pre-defined keywords and inclusion/exclusion criteria, and lacking the flexibility of more inductive inquiry approaches (Gough et al. 2012; Ansari and Moher 2013). Despite this and other critiques, researchers investigating adaptation have adapted systematic approaches to better meet the needs of their review questions. This includes combining quantitative and qualitative analyses, and designing complex literature searches, including iterative search methodologies to capture all relevant articles (Furgal et al. 2010; Pearce et al. 2011; Biesbroek et al. 2013). Thus, despite standardized guidelines for what constitutes a formal ‘systematic review’ in the health sciences, systematic approaches have been widely adapted. Here, we argue that the term ‘systematic review’ is more appropriately used to refer to a broad collection of research synthesis approaches that seek to apply systematic processes to review diverse and often complex literature bases; for systematic review, there is no ‘one size fits all.’

Some argue against the use of systematic review approaches for qualitative research, reasoning that such attempts have largely imposed structured templates designed for quantitative research and resulted in a watering down of critical analysis integral to qualitative research (Barbour and Barbour 2003). Research synthesis in these cases is thus highly inductive and iterative, eschewing systematization and prescriptive methods of review, while espousing critical inquiry, curiosity-driven research, and problematization of concepts. In contrast, increasing interest in systematic approaches to research synthesis of qualitative literature has led to ‘middle-ground’ approaches and the use of what is often termed meta-synthesis (Jensen and Allen 1996; Walsh and Downe 2005; Merten et al. 2010). Meta-synthesis includes a number of more specifically defined approaches, including secondary analysis, grounded meta-analysis, and meta-ethnography (Atkins et al. 2008; Suri and Clarke 2009; Barth and Thomas 2012; Gough et al. 2012). Many reviews employing a meta-synthesis approach include explicit description of methods for document selection, though apply a more iterative process less strictly defined than the formal inclusion/exclusion criteria endorsed for many quantitative systematic reviews. At the analysis stage, meta-synthesis approaches are theory-driven and focused on inductive and explanatory synthesis rather than aggregative analysis.

Sharing common characteristics with meta-synthesis, realist review approaches have been proposed as a model to address more complex and interdisciplinary research questions for which strict quantitative systematic review methods are ill-suited. While espousing the idea of systematization of the review process, the realist approach focuses on explanatory analysis and takes a more iterative approach to gathering evidence. For example, while aggregative reviews have generally sought to determine whether a particular intervention works, realist evaluation would aim to discern what works, for whom, why, and in what circumstances (Pawson et al. 2005). In realist review as for meta-synthesis, analysis is based on explicit integration of a theoretical framework to guide the review (Pawson et al. 2005; Gough et al. 2012). While realist approaches have been used predominantly to assess health-related interventions, the methods are relevant and applicable for adaptation policy and practice. For example, realist approaches are applicable when the aim of the research synthesis is to understand why and how a policy/practice works, for whom, and in what context, it is effective or ineffective. Why does a policy work in one country but not another? What are the conditions that affect the success or failure of this policy? Who wins and who loses in the context of a given adaptation practice? These questions often do not lend themselves to quantitative analysis, are likely to require in-depth contextual analysis; analytical reproducibility may be less relevant. Realist review and meta-synthesis are well suited to address research questions in the adaptation field that seek to synthesize conflicting evidence from literature that is epistemologically complex and methodologically diverse.

Systematic review for adaptation research: challenges and considerations for reviewing policy and practice literature

In this section, we first assess the extent to which the adaptation literature has engaged in the use of systematic review approaches. To do so, we apply systematic document selection to identify adaptation articles employing systematic review approaches. We then draw on this literature to discuss methodological challenges and considerations for reviewing adaptation policy and practice.

Climate change adaptation research: a systematic review of systematic reviews

We conducted a systematic literature review to identify peer-reviewed literature indexed in Web of Knowledge (WoK) that employed a systematic approach to climate change adaptation. Our aim herein was twofold, (1) to characterize the extent to which systematic approaches have been broadly applied within the adaptation literature, and (2) to identify a sample of adaptation literature meeting minimum criteria for systematic review. Definitions of minimum requirements for systematic review vary significantly outside of the health sciences review collaborations, and there is a notable absence of guidelines for applying systematic review approaches in the social sciences. To get a sense of the broad application of systematic approaches within adaptation research, we employed relatively liberal requirements for inclusion, focusing on the use and documentation of systematic approaches for document selection, or the expectation that selection of reviewed materials would be in some way replicable. Notably, we did not exclude based on systematization or description of analysis or presentation of results, though we would argue that these components should be mainstreamed as critical minimum components of systematic review in adaptation research. A paper was considered to be eligible for inclusion as a systematic review if it provided explicit description of methods for document selection, specifically articulating: (1) search terms, or at minimum a description of search strings and/or the search process, and (2) inclusion and exclusion criteria OR a list of reviewed documents. Details of the review process are provided in the Supplemental Materials (Search 2).

In Phase 1, we searched for all documents indexed in the search engine Web of Knowledge with topic words “adaptation” AND “climat* chang*” AND “review,” and excluding document types other than articles or reviews. No date or language restrictions were applied, though no non-English or pre-2009 articles met final inclusion criteria. A total of 720 articles were retrieved at this stage. Limiting the search to WoK means that our review is not exhaustive and provides only a proxy sample of the literature on systematic reviews of adaptation. We also intentionally focus only on literature self-identifying as climate change adaptation research. In Phase 2, we scanned titles and abstracts to select papers with clear relevance to climate change adaptation and with implied use of systematic or structured review methods. Papers related predominantly to climate impacts, vulnerability, mitigation, or general sustainability were excluded, as were papers focused on adaptation of biophysical rather than human systems. In order to be retained for full-text review, titles and abstracts were not required to self-identify as systematic, but needed to imply the use of some form of systematic or structured review approach (e.g., ‘comprehensive,’ ‘exhaustive’ or reference to number of documents reviewed). Eighty-two articles were retained and underwent full-text review, of which 27 met final inclusion criteria as systematic review articles addressing climate change adaptation. All included articles were full-text reviewed to record the systematic process employed in the review and interrogate trends in the application of systematic methods used in adaptation literature. This information was summarized in an excel spreadsheet and included the aim of the review, theoretical approach, document source, search terms, description of document selection, type and description of analysis, and presentation of results. Excluded articles with some evidence of systematic methods were also documented to more broadly characterize review approaches in adaptation research. Included articles are summarized in Table 2.
Table 2

Summary of systematic review articles on adaptation research indexed in Web of Knowledge

Source

Title

Aim

Data source

Methods explicit

Analysis

Berrang-Ford et al. (2011)

Are we adapting to climate change?

D

P

Y

Qn, Ql*

Biesbroek et al. (2013)

On the nature of barriers to climate change adaptation

D, X

P

Y

Qn, Ql

Bouzid et al. (2013)

The Effectiveness of Public Health Interventions to Reduce the Health Impact of Climate Change: A Systematic Review of Systematic Reviews

E

P

Y

Qn

Bowler et al. (2010)

Urban greening to cool towns and cities: A systematic review of the empirical evidence

E, X

P

Y

Qn

Charlton and Arnell (2011)

Adapting to climate change impacts on water resources in England-An assessment of draft Water Resources Management Plans

D

G

Y

Qn

Cheng and Berry (2013)

Health co-benefits and risks of public health adaptation strategies to climate change: a review of current literature

D

P

N

Ql

Eisenack et al. (2012)

Adaptation to climate change in the transport sector: a review of actions and actors

D

G

Y

Qn, Ql

Ford et al. (2012a)

Mapping Human Dimensions of Climate Change Research in the Canadian Arctic

D

P

Y

Qn

Ford et al. (2012b)

Research on the Human Dimensions of Climate Change in Nunavut, Nunavik, and Nunatsiavut: A Literature Review and Gap Analysis

D

P

Y

Ql

Ford et al. (2011)

A systematic review of observed climate change adaptation in developed nations A letter

D

P

Y

Qn, Ql

Hardee and Mutunga (2010)

Strengthening the link between climate change adaptation and national development plans: lessons from the case of population in National Adaptation Programmes of Action (NAPAs)

E

G

N

Qn, Ql

Heller and Zavaleta (2009)

Biodiversity management in the face of climate change: A review of 22 years of recommendations

E, X

P

Y

Qn, Ql*

Hosking and Campbell-Lendrum (2012)

How Well Does Climate Change and Human Health Research Match the Demands of Policymakers? A Scoping Review

D

P

Y

Qn

Kajan and Saarinen (2013)

Tourism, climate change and adaptation: a review

D, E

P

Y

Ql

Kamau and Mwaura (2013)

Climate change adaptation and EIA studies in Kenya

D, E

G, O

Y

Qn

Kolstrom et al. (2011)

Reviewing the Science and Implementation of Climate Change Adaptation Measures in European Forestry

D

P, O

N

Qn, Ql

Larsen et al. (2012)

Mind the gap in SEA: An institutional perspective on why assessment of synergies amongst climate change mitigation, adaptation and other policy areas are missing

E, X

G, O

Y

Ql

Linnenluecke et al. (2013)

Firm and industry adaptation to climate change: a review of climate adaptation studies in the business and management field

D

P

Y

Ql

McDowell (2013)

Climate-Change Adaptation and Mitigation: Implications for Land Acquisition and Population Relocation

E

G

N

Qn*, Ql

McLeman (2011)

Settlement abandonment in the context of global environmental change

X

P

Y

Qn, Ql

Murtinho and Hayes (2012)

Adaptation in Resource-Dependent Communities: A Call for Greater Methodological Clarity in Adaptation Field Research

D

G, O

Y

Ql

Pearce et al. (2011)

Advancing adaptation planning for climate change in the Inuvialuit Settlement Region (ISR): a review and critique

D

P, G

Y

Qn*, Ql

Poutiainen et al. (2013)

Civil society organizations and adaptation to the health effects of climate change in Canada

D

G

Y

Qn, Ql*

Shepard et al. (2011)

The Protective Role of Coastal Marshes: A Systematic Review and Meta-analysis

D

P

Y

Qn

Toloo et al. (2013)

Evaluating the effectiveness of heat warning systems: systematic review of epidemiological evidence

E, X

P

N

Qn, Ql*

Vink et al. (2013)

The role of knowledge and power in climate change adaptation governance: a systematic literature review

D

P

Y

Qn, Ql

Walker et al. (2011)

Health promotion interventions to address climate change using a primary health care approach: a literature review

D

P

Ql

Entries indicate where articles are consistent with different types or approaches to review aim, data source, or analysis. Aim: D descriptive, E evaluative, X explanatory; Data source: P peer-reviewed, G grey literature, O other participatory (interviews, workshops, stakeholder consultation); Methods explicitly described: Y yes, N no; Analysis: Qn quantitative, Ql qualitative, starred entry denote secondary or partial relevance

Of the 27 adaptation articles identified as meeting our requirements for systematic review, more than half (18) self-identified as systematic reviews, of which two were meta-analyses (Bowler et al. 2010; Shepard et al. 2011). Reflecting the tradition of systematic reviews in the health sciences, health-related adaptation was the focus of six articles (Walker et al. 2011; Hosking and Campbell-Lendrum 2012; Bouzid et al. 2013; Cheng and Berry 2013; Poutiainen et al. 2013; Toloo et al. 2013). The most dominant adaptation focus within which systematic reviews have penetrated has been reviewing lessons from, and trends in, adaptation governance (Hardee and Mutunga 2010; Berrang-Ford et al. 2011; Ford et al. 2011; Pearce et al. 2011; Ford et al. 2012a, b; Larsen et al. 2012; Murtinho and Hayes 2012; Biesbroek et al. 2013; Kamau and Mwaura 2013; Vink et al. 2013). Systematic reviews have not been restricted to this focus, however, with articles considering tourism (Kajan and Saarinen 2013), business management (Linnenluecke et al. 2013), transport (Eisenack et al. 2012), urban planning (Bowler et al. 2010), human displacement (McLeman 2011; McDowell 2013), human management of ecosystem services (Heller and Zavaleta 2009; Charlton and Arnell 2011; Kolstrom et al. 2011; Shepard et al. 2011), and case studies of generalized adaptation research (Murtinho and Hayes 2012; Ford et al. 2012a, b).

Despite the relatively low number of papers meeting our—admittedly already quite liberal—minimum criteria for inclusion, a number of excluded articles employed systematic approaches in their reviews. We excluded approximately 20–25 papers that employed some form of explicit systematic method but did not meet inclusion criteria. For example, a number of papers appeared to use highly systematic approaches to document selection, but these methods were not clearly or sufficiently articulated in the papers to validate or support inclusion (Mills 2009; Poyar and Beller-Simms 2010; Preet et al. 2010; Huang et al. 2011; Hunt and Watkiss 2011; Clarke and Berry 2012; Black et al. 2013; Sharmina et al. 2013; Wamsler et al. 2013). In many of these cases, we suspect the failure was not in the absence of systematization of the review process—many of these papers in fact implied highly systematic and comprehensive approaches—but rather in documentation of methods. With the rise of electronic and on-line journals in recent decades, and the associated opportunity to include Supplemental Materials with many journal submissions, it is now feasible to provide more detailed documentation of search methods. Had authors included Supplementary Materials in their methods, several of these papers would likely have met inclusion criteria. Other papers used some form of semi-systematic approach, but lacked articulation of keywords or document selection (Shepherd et al. 2011; Clarke and Berry 2012; Wilby and Keenan 2012; Bierbaum et al. 2013; Morrison and Pickering 2013; Weinhofer and Busch 2013), or lacked an explicit or implied list of documents or detail on inclusion/exclusion criteria (Ford et al. 2010). Of the 27 articles we reviewed, 21 also described to varying degrees their methods for analysis despite this not being included as a requirement for our inclusion criteria. Over half of the articles also systematically presented results based on articulated theories, or in clear alignment with their research questions.

There is evidence, therefore, that engagement with systematic review approaches has penetrated into adaptation research and is not limited to a particular sub-field or—notably—a handful of authors. This emergence is very recent, with the first included article from 2009 and all but three articles published in 2011 or later. Despite this emerging application of systematic approaches, there is negligible evidence of standardization, guidelines for review documentation, or a methodological baseline for what constitutes a ‘systematic review’ in the context of adaptation literature. The exclusion of numerous articles attempting systematic approaches based on liberal criteria applied in this paper implies that interest in systematic review approaches has not be met by guidance and best-practice consensus on what makes an adaptation review systematic. We herein hope to contribute to this gap, and propose guidelines for key components of a systematic review for adaptation research (Box 1).
Box 1

Proposed components of a systematic review in adaptation research

Research question/aim

 Explicit aim and objectives of review Including context and scoping of the research problem. This information should frame the research question and be directly aligned with inclusion and exclusion criteria, i.e., what literature is not considered? What questions are not asked? What is the spatial and temporal frame within which the review is conducted?

 Clear description of theoretical or conceptual approach used to guide the review Even predominantly quantitative reviews draw—whether explicitly or implicitly—on key theories or paradigms within their respective literature (e.g., public health intervention, epidemiologic causal theory, supply–demand economics, ecosystem services). Systematic reviews should summarize prevailing literature and concepts that inform the review, with this context clearly guiding the articulation of research questions and methods, and in many cases also the presentation of results

Data source and document selection

 Justification and description of literature source, and consideration of bias arising from the selection of literature source. This should consider literature type (peer-reviewed, grey, policy docs, other), language, search engines selected, and dates considered

 Articulation of search terms and/or detailed description of search process Ideally, explicit description of search strings, terms, and search criteria. Highly iterative search processes may be appropriate in some cases, but risk compromising theoretical reproducibility

 Description of criteria for inclusion and exclusion Inclusion and exclusion criteria (often summarized in tabular format) are generally directly inferred from, and aligned with, clearly identified objectives and detailed scoping of the research question. These criteria specify how the research question and frame are methodologically operationalized during the search and document selection process

 Documentation of literature included and excluded With the rise of electronic journal publishing, it is increasingly feasible to include more detailed documentation of literature searched and sorted within the review process. Diagrams of the document selection process (e.g., Berrang-Ford et al. 2011) are common, though at minimum systematic reviews should summarize key literatures excluded and provide a list or link to included documents

Analysis and presentation of results

 Description of methods for analysis This is perhaps the most neglected component of current systematic reviews, many of which do not clearly articulate how documents are reviewed or analyzed. Quantitative reviews are generally guided in part by inferential or descriptive statistical analysis. Qualitative analyses are often guided by thematic content analysis (manifest or latent), sometimes involving quantitative or qualitative coding. Many reviews implicitly use a theoretical approach or research questions to guide analysis. At minimum, systematic reviews should explicitly align analysis with the research aim and theoretical approach used for the review, and outline this approach, even if briefly, in the methods section of the text

 Critical appraisal of information quality It is standard in the health sciences to evaluate the methodological quality of included research and include a quality filter for inclusion. In realist review, a quality filter may not be used simply to exclude entire documents, but rather to consider the quality, relevance, and significance of different information in providing insight into the research question. The central tenant here is that not all documents or pieces of information are equally relevant or reliable in the context of a particular research question and should not be treated as such. While peer-reviewed literature is often considered more ‘rigorous’ or ‘reliable,’ for example, grey literature may provide valuable explanatory information, and stakeholder engagement might provide critical context for policy relevance of results. Particularly for evaluative and explanatory systematic reviews, consideration of the quality and relevance of different document, research, and data sources should be mainstreamed into the review process

Table 2 summarizes the 27 articles indexed in Web of Knowledge that met inclusion criteria. Articles are categorized based on the aim of review, the literature or information source, and analytical approach used to synthesize data. In the following sections, we draw on these reviews and categorizations, as well as criteria presented in Box 1, to discuss some of the challenges and consideration for the use of systematic approaches to adaptation research synthesis.

Aim of review

Systematic review is a process, not a static outcome, and the process can be dynamic, flexible, and adaptable to meet a variety of research questions and reviewer needs. At the most basic level, we might seek to get a broad sense of the literature by conducting a scoping review and describing patterns and trends: e.g., what adaptation policies or practices are being undertaken? (Eisenack et al. 2012; Ford et al. 2012a; Kamau and Mwaura 2013; Poutiainen et al. 2013). More complex research questions might seek to evaluate policy, explain why, how, and when practices are effective and how they operate, or to challenge and question theories, paradigms, or conceptual approaches; realist review frameworks are particularly appropriate for such questions. Biesbroek et al. (2013), for example, employ an explicit and clearly articulated systematic review approach to understand the conceptualizations and theoretical models used to understand barriers to adaptation. Hardee and Mutunga (2010) explore how NAPA documents submitted to the UNFCCC are integrated into national development processes.

Importantly, research questions must be answerable given the available literature or data. Given frequently fixed time and financial resources within a research team, there is an inevitable trade-off between depth and breadth (Gough et al. 2012). A key difference is whether one takes a positivist (reductionist) approach to aggregated information or an interpretive, explanatory approach using realist methods (Barth and Thomas 2012; Gough et al. 2012; Waddington et al. 2012). The former may increase simplicity and presentation of results and the perception of objectivity, often preferable in informing evidence-based decision-making (Barth and Thomas 2012). Reductionist analysis, however, assumes that key insights can be distilled from generalist trends in a few key measureable factors, and risks missing critical lessons to be learned from causes of variation, contextual significance, and unmeasured (or not easily quantified) variables. The two approaches are not mutually exclusive, however, with mixed methods approaches feasible (McLeman 2011; Eisenack et al. 2012; Biesbroek et al. 2013). Complex and context-dependent questions may in some cases be more policy relevant and of interest to researchers, but often require iterative and less systematic search strategies and must be feasible (Gough et al. 2012).

The choice of research question and scope of the topic will influence what type of review approach is most appropriate, and thus, the research question and review methods are often co-designed. In all cases, the research question must by necessity be aligned with an appropriate body of literature available for review. In cases where there are limited empirical data or studies to thoroughly assess the literature, emphasis on critical appraisal of existing research, state-of-knowledge scoping reviews, and identification of priority research gaps and directions remain important contributions. This is particularly relevant for rapidly growing fields such as adaptation research.

Literature and information sources

While we may develop policy-relevant and critically important research questions, the quality of a literature review will depend upon the quality and quantity of the available literature; in short, the review question must be answerable and this depends on where relevant information can be found and whether it is accessible to the reviewer (Pawson et al. 2005). In many cases, adaptation policy and practice research questions may be difficult to synthesize from existing literature, particularly in the case of evaluation or explanatory questions for which only sparse and diverse literature exists. In other cases, key information may be available outside of the peer-reviewed literature. Restricting a policy review to only peer-reviewed literature, for example, may miss key trends and insights with significant implications for biasing results. Ford et al. (2014) find that trends in reported adaptation policy and practice differ substantively by literature source, highlighting the extent to which results are highly sensitive to the choice of document types included in a review. Quantitative and even formal qualitative evaluations of adaptation policy and practice efficacy are likely to be limited. Research syntheses will often as a minimum necessitate going beyond the peer-reviewed literature to include extensive consideration of grey literature. In their review of current research on adaptation in the transport sector, for example, Eisenack et al. (2012) draw on peer-reviewed and grey literature, articulating a clear search strategy, to identify discrete transportation adaptations. Larsen et al. (2012) review 149 Strategic Environmental Assessment (SEA) reports from Denmark, and complement these with interviews in six municipalities to understand how climate change considerations are integrated in SEAs. Kamau and Mwaura (2013), meanwhile, similarly combine policy documents with interviews to assess climate change adaptation in Kenyan Environmental Impact Assessments.

In contrast, for a research question with a very large and diverse amount of information, the reviewer may need to identify ways of placing limits on the review so that it can be feasibly conducted. This might involve restricting the review to a single literature source (e.g., peer-reviewed articles, articles indexed only in Web of Knowledge), or selecting a sample of the literature (e.g., Ford et al. 2014; Kilroy 2015; Kamau and Mwaura 2013). A reviewer must place limits on the amount of literature that can be covered, and this may be done at the defining/scoping stage (Pawson et al. 2005). Consideration of context, processes, and mechanisms of causality may require more time commitment per document than extraction of key estimates or discrete items of information. Many realist or in-depth reviews, for example, are conducted using fewer than 50 articles, often 30–40 documents (Bouzid et al. 2013; Cheng and Berry 2013; Kajan and Saarinen 2013; Linnenluecke et al. 2013), and even in-depth qualitative systematic analyses with as few as 15–20 articles (Walker et al. 2011; Toloo et al. 2013).

Research has shown that systematic review of complex and heterogeneous literature bases cannot rely solely on strict keyword searches (Greenhalgh and Peacock 2005), and this is likely to apply to many complex adaptation policy and practice questions. While electronic keyword searches are the best recognized and standard method for document selection, there are a number of complimentary techniques a reviewer can consider using to access appropriate documents. Forward and backward citation tracking, ‘snowballing’ methods, and personal knowledge or contacts have been integrated into search methods (Pearce et al. 2011). The distinction here lies in the complexity of the topic, with more complex research questions often necessitating a combination of search approaches and more flexible or iterative search design. The use of expert-sourced information can be used both to compliment electronic searches and as a point of entry into the literature.

In some cases, research questions will not be answerable in existing literatures and may require a broader framework that seeks to include expert or public knowledge (acquired from interviews, workshops, and stakeholder consultations). At this point, the research ceases to be a standard literature review and transitions into the realm of broader knowledge synthesis frameworks that integrate primary research. Kolstrom et al. (2011) for example, combine data from a peer-reviewed literature search, a questionnaire carried out with key policy makers, and a participatory database assembled with country representatives to identify potential adaptation options in European forestry. Kamau and Mwaura (2013), Larsen et al. (2012), and Murtinho and Hayes (2012) all to some extent combine external consultation or interviews with literature sources to enhance depth of analysis. Though not always meeting our criteria for inclusion, integration of participatory approaches was not uncommon in adaptation reviews, including solicitation of feedback on early literature review results from policy makers (Adam-Poupart et al. 2013), engagement of community stakeholders in the adaptation planning (Pearce et al. 2012), and key informant interviews to assess adaptation mainstreaming in public health (Clarke and Berry 2012). We argue here that in the case of policy and practice, consideration of systematic approaches to synthesizing knowledge should not be restricted to reviews of existing literature only. There exists a growing literature on the systematic collection and use of expert-sourced or public knowledge (Davis and Wagner 2003; O’Neill et al. 2008; Taewoo 2012; Raymond and Robinson 2013) that might provide an innovative and potentially appropriate complement to standard sources of information in existing literature.

Analyzing and synthesizing evidence

We anticipate that many adaptation review questions will employ qualitative or mixed methods analysis, and employ theoretical models other than—or in additional to—statistical theory to guide synthesis of evidence. While quantitative—likely predominantly descriptive—analysis may be feasible as a component of scoping reviews, it is often poorly aligned with more explanatory or realist questions or for complex, diverse and sparse literature. Lack of quantitative analysis, however, does not imply lack of rigor, validity, or transparency in a review, particularly where analysis methods are clearly and explicitly documented and reported. Qualitative extraction and analyses are commonly guided by theoretical frameworks or conceptual models used to frame the review. Vink et al. (2013), for example, frame their review of adaptation governance using theoretical constructs of knowledge and power, with their theoretical framework explicitly described in the methods section and results summarized around these concepts. The predominant theories guiding the literature we reviewed included adaptation and vulnerability theory (e.g., Ford et al. 2012b), statistical theory in the case of meta-analyses (Bowler et al. 2010; Shepard et al. 2011) and public health (Walker et al. 2011; Hosking and Campbell-Lendrum 2012; Bouzid et al. 2013; Cheng and Berry 2013; Toloo et al. 2013).

Recalling the distinctions between aggregative and explanatory review objectives, it may be relevant and important here to seek out and explore contradictory evidence or outliers to provide insights into context (Forbes and Griffiths 2002; Barbour and Barbour 2003; Pawson et al. 2005): Why did this community successfully adapt to increasing extreme weather events while another did not? Is there evidence to explain why one household, community or nation is able to engage in adaptation practice more effectively than others? This type of information, critical for adaptation policy, may only be distilled in realist understanding of the context behind divergent evidence and results. Toloo et al. (2013), for example, investigate not only whether health warning systems (HWS) are effective in reducing heat-related health impacts associated with climate projections, but also what factors are critical in influencing the effectiveness of HWS. Biesbroek et al. (2013) similarly adopt realist thinking by considering not only questions of ‘if’ and ‘which’ barriers to adaptation exist, but also ‘how’ and ‘why’ barriers emerge.

In some cases, authors applying systematic review approaches have reported formal descriptions of review methods, yet not labeled or indexed their work as ‘systematic’ (Hardee and Mutunga 2010; Charlton and Arnell 2011; Kolstrom et al. 2011; Hosking and Campbell-Lendrum 2012; Larsen et al. 2012; Murtinho and Hayes 2012; Cheng and Berry 2013; Kamau and Mwaura 2013; McDowell 2013). In many more cases, methods are not explicitly reported at all, and it is presumed that authors have applied formal or informal thematic content analysis to extract key themes, either based on latent or manifest content analysis (Baxter and Eyles 1997). We argue here that more explicit and detailed reporting of analysis methods for qualitative reviews would contribute to both improved transparency and increased ability to critically assess the rigor of review methods.

Discussion

For a systematic review, there is no ‘one size fits all.’ Despite the perception that systematic reviews must follow strict guidelines, those guidelines must be developed and adapted for their application. Systematic review reflects efforts by researchers to increase the transparency and intentionality of the methods they use to identify, select and analyze available information. A strength of systematic approaches is that explicitly reported methods allow the quality and reliability of results to be reproduced and examined. The current de facto, if unintentional, monopoly of the use and application of systematic reviews within health and associated sciences has, however, restricted its use in other domains grappling with complex research questions, a range of conceptual and epistemological approaches, and diverse information sources. Reviews of adaptation policy and practice are unlikely to lend themselves to standard approaches to systematic review, and as a result, there has been limited penetration and use of systematic approaches in adaptation research. This is despite ongoing and increasing calls for transparency of review methods, particularly within the Intergovernmental Panel on Climate Change (IPCC)’s Assessment Reports (Petticrew and McCartney 2011; Ford et al. 2012c), and paralleled by calls in international development literature (Waddington et al. 2012). New methodological tools to support evidence-based reviews of adaptation policy are sorely needed.

Some, such as Barbour and Barbour (2003) argue against the use of systematic review approaches for qualitative research, noting that in many cases strict adherence to ill-suited guidelines have limited the scope of reviews and compromised critical analysis, consideration of contextual complexity, and quality of results. We argue that poorly designed reviews do not negate the potential for systematic approaches to provide meaningful and substantive contributions to the literature. Adaptation research, in particular, is inherently interdisciplinary and necessitates engagement across the physical and human sciences. Retreat from opportunities for integrative research synthesis risks deepening a polarization of methodologies between physical and social scientists. We propose increased engagement with the flexible and creative potential of systematic review approaches to address complex adaptation challenges using intentionally designed, transparent, reproducible, and explicitly documented methods of research synthesis.

Integrating literature and results from different research traditions and using different methods remain, however, a fundamental challenge of adaptation research synthesis. These include but are not restricted to the conceptual challenge of how to define adaptation in a way that can be applied consistently and comparably, identifying comparable and comprehensive sources of information, and the time and human resources—including appropriate expertise—required to synthesize large and heterogeneous information sources. We argue for two approaches to tackle this. First, systematic reviews should seek to review literature whose commonality is relevance to answering a particular research question rather than methodological or ontological similarity. This will require careful a priori consideration of search terms and literature sources that reflect a diversity of research traditions. Importantly, this will narrow and clarify the scope of the research question and allow the researcher to evaluate the feasibility of the review. Secondly, the analysis stage should apply a critical lens to judge the extent to which information is relevant to the research question and might be biased by the methods or theories guiding the research. This quality and relevance filter should be informed by a clear understanding of ontological and methodological differences in the literature and how they impact results. In this sense, we propose infusion of realist approaches into the application of systematic review methods in social sciences.

The greatest potential of systematic approaches for adaptation research may lie in their use beyond the traditional model of literature reviews. In some cases, reviews have used discrete events or adaptation actions as the unit of analysis rather than individual articles. Kolstrom et al. (2011) identify 444 forestry adaptation measures for 19 European countries, while Heller and Zavaleta (2009) extract 524 biodiversity management recommendations from 113 peer-reviewed articles. Similarly, Lesnikowski et al. (in press), and Poutiainen et al. (2013), code adaptation initiatives as a unit of analysis. McLeman (2011) uses systematic review methods to identify 246 examples of abandoned settlements arising from global environmental change. Similarly, many emerging reviews in the adaptation field are employing mixed methods, including quantitative and qualitative primary data and results, and often guided by both aggregative and theory-driven realist perspectives. This research reflects new and diverse applications of systematic approaches to research synthesis.

Of particular note are efforts to integrate different literature types and information sources. In the case of Tompkins et al. (2010), for example, they include information drawn from expert advisors, snowballing from government and private sector annual reports, and solicited feedback from a request for evidence of adaptation sent via monthly mail out to the UKCIP mailing list. Tompkins et al.’s work, indexed as primary research rather than a review, is notably absent from our systematic review list, highlighting the potential challenge of distinguishing research from review at the boundaries of integrative and participatory research synthesis. Such research crosses the boundary between standard literature reviews and primary research, yet is guided by attempts to systematize the process of research synthesis and integrate diverse information sources. Researchers interested in conducting rigorous and comprehensive syntheses of the state of knowledge for a particular research question might integrate a review of existing literatures with primary data collection (interviews, workshops, crowd-sourcing, expert opinion) within a comprehensive systematic information synthesis framework. There is potential therein to address criticisms of overly structured approaches within systematic frameworks by integrating new information sources and creating opportunities for participatory knowledge creation and social learning. It is here that knowledge synthesis, built within a framework guided by systematic approaches to reviewing diverse information, may be of most relevance and benefit to improving our understanding of the human dimensions of climate change.

We herein propose guidelines for systematic review in adaptation research (Box 1). In the short term, we seek to guide and inform adaptation researchers seeking more systematic approaches to research synthesis. In the longer term, we hope to stimulate collective development and consensus regarding minimum expectations for systematic review in adaptation research. We do not seek to impose methodological templates on adaptation research, but rather to provide current and future reviewers of the adaptation literature with a broader toolkit of ideas, approaches, methods, and tips to guide and promote adaptation research synthesis. Our recommendations in Box 1 imply significant investment of time, resources, and training, to conduct thorough and rigorous research synthesis, to develop evolving guidelines, and to build capacity and knowledge in systematic review approaches. We argue here for the value of that investment for research rigor, policy relevance, and evidence-based adaptation research.

Notes

Acknowledgments

This article was funded by the Collaborative Adaptation Research Initiative in Africa and Asia (CARIAA), supported by the International Development Research Center (IDRC) of Canada and the Department for International Development (DFID) in the UK. We would like to thank CARIAA team members for contributions to conceptual development and comments on early drafts. Thanks also to Dr. Nancy Ross and several cohorts of undergraduate and graduate students at McGill University over the years, whose efforts to apply systematic review approaches to socio-ecological determinants of health research in GEOG 503 have helped shape and evolve much of the thinking reflected in this paper.

Supplementary material

10113_2014_708_MOESM1_ESM.pdf (109 kb)
Supplementary material 1 (PDF 108 kb)

References

  1. Adam-Poupart A, Labreche F, Smargiassi A, Duguay P, Busque M-A, Gagne C, Rintamaki H, Kjellstrom T, Zayed J (2013) Climate change and occupational health and safety in a temperate climate: potential impacts and research priorities in Quebec, Canada. Ind Health 51:68–78CrossRefGoogle Scholar
  2. Adger WN, Arnell NW, Tompkins EL (2005) Successful adaptation to climate change across scales. Glob Environ Chang Hum Policy Dimens 15:77–86. doi:10.1016/j.gloenvcha.2004.12.005 CrossRefGoogle Scholar
  3. Akobeng AK (2005) Understanding systematic reviews and meta-analysis. Arch Dis Child 90:845–848CrossRefGoogle Scholar
  4. Ansari MT, Moher D (2013) Systematic reviews deserve more credit than they get. Nat Med 19:395–396. doi:10.1038/nm.3151 CrossRefGoogle Scholar
  5. Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J (2008) Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol 8. doi:10.1186/1471-2288-8-21
  6. Barbour RS, Barbour M (2003) Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract 9:179–186. doi:10.1046/j.1365-2753.2003.00371.x CrossRefGoogle Scholar
  7. Barnett-Page E, Thomas J (2009) Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 9:11. doi:10.1186/1471-2288-9-59 CrossRefGoogle Scholar
  8. Barth M, Thomas I (2012) Synthesising case-study research—ready for the next step? Environ Educ Res 18:751–764. doi:10.1080/13504622.2012.665849 CrossRefGoogle Scholar
  9. Baxter J, Eyles J (1997) Evaluating qualitative research in social geography: establishing ‘rigour’ in interview analysis. Trans Inst Br Geogr 22:505–525. doi:10.1111/j.0020-2754.1997.00505.x CrossRefGoogle Scholar
  10. Berkhout F (2012) Adaptation to climate change by organizations. Wiley Interdiscip Rev Clim Chang 3:91–106. doi:10.1002/wcc.154 CrossRefGoogle Scholar
  11. Berrang-Ford L, Ford JD, Paterson J (2011) Are we adapting to climate change? Glob Environ Chang Hum Policy Dimens 21:25–33. doi:10.1016/j.gloenvcha.2010.09.012 CrossRefGoogle Scholar
  12. Bierbaum R, Smith JB, Lee A, Blair M, Carter L, Chapin FS III, Fleming P, Ruffo S, Stults M, McNeeley S, Wasley E, Verduzco L (2013) A comprehensive review of climate adaptation in the United States: more than before, but less than needed. Mitig Adapt Strat Glob Chang 18:361–406. doi:10.1007/s11027-012-9423-1 CrossRefGoogle Scholar
  13. Biesbroek GR, Klostermann JEM, Termeer CJAM, Kabat P (2013) On the nature of barriers to climate change adaptation. Reg Environ Chang 13:1119–1129. doi:10.1007/s10113-013-0421-y CrossRefGoogle Scholar
  14. Bizikova L, Parry JE, Karami J, Echevierria D (2015) Review of key initiatives and approaches to adaptation planning at the national level in semi-arid areas. Reg Environ Change. doi:10.1007/s10113-014-0710-0
  15. Bjurstrom A, Polk M (2011a) Climate change and interdisciplinarity: a co-citation analysis of IPCC third assessment report. Scientometrics 87:525–550CrossRefGoogle Scholar
  16. Bjurstrom A, Polk M (2011b) Physical and economic bias in climate change research: a scientometric study of IPCC third assessment report. Clim Change 108:1–22CrossRefGoogle Scholar
  17. Black R, Arnell NW, Adger WN, Thomas D, Geddes A (2013) Migration, immobility and displacement outcomes following extreme events. Environ Sci Policy 27:S32–S43. doi:10.1016/j.envsci.2012.09.001 CrossRefGoogle Scholar
  18. Bouzid M, Hooper L, Hunter PR (2013) The effectiveness of public health interventions to reduce the health impact of climate change: a systematic review of systematic reviews. PLoS ONE 8:e62041. doi:10.1371/journal.pone.0062041 CrossRefGoogle Scholar
  19. Bowler DE, Buyung-Ali L, Knight TM, Pullin AS (2010) Urban greening to cool towns and cities: a systematic review of the empirical evidence. Landsc Urban Plan 97:147–155. doi:10.1016/j.landurbplan.2010.05.006 CrossRefGoogle Scholar
  20. Brooks N, Anderson S, Ayers J, Burton I, Tellam I (2011) Tracking adaptation and measuring development. In: Climate change working paper no 1Google Scholar
  21. Cancelliere C, Cassidy JD, Ammendolia C, Cote P (2011) Are workplace health promotion programs effective at improving presenteeism in workers? a systematic review and best evidence synthesis of the literature. BMC Public Health 11:395CrossRefGoogle Scholar
  22. Charlton MB, Arnell NW (2011) Adapting to climate change impacts on water resources in England—an assessment of draft water resources management plans. Glob Environ Chang Hum Policy Dimens 21:238–248. doi:10.1016/j.gloenvcha.2010.07.012 CrossRefGoogle Scholar
  23. Cheng JJ, Berry P (2013) Health co-benefits and risks of public health adaptation strategies to climate change: a review of current literature. Int J Public Health 58:305–311. doi:10.1007/s00038-012-0422-5 CrossRefGoogle Scholar
  24. Clarke K-L, Berry P (2012) From theory to practice: a Canadian case study of the utility of climate change adaptation frameworks to address health impacts. Int J Public Health 57:167–174. doi:10.1007/s00038-011-0292-2 CrossRefGoogle Scholar
  25. Cook DJ, Mulrow CD, Haynes RB (1997) Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 126:376–380CrossRefGoogle Scholar
  26. Cooper H, Hedge LV (eds) (1994) The handbook of research synthesis. Sage, New YorkGoogle Scholar
  27. Cubbison L (1999) Systematic reviews: synthesis of best evidence for health care decisions. J Bus Tech Commun 13:473–475CrossRefGoogle Scholar
  28. Davis A, Wagner JR (2003) Who knows? On the importance of identifying “experts” when researching local ecological knowledge. Hum Ecol 31:463–489. doi:10.1023/A:1025075923297 CrossRefGoogle Scholar
  29. De Souza K, Kituyi E, Leone M, Harvey B, Murali KS, Ford J (2015) Vulnerability to climate change in three hot spots in Africa and Asia: key issues for policy-relevant adaptation and resilience-building research. Reg Environ Change. doi:10.1007/s10113-015-0755-8
  30. DeBono NL, Ross NA, Berrang-Ford L (2012) Does the food stamp program cause obesity? A realist review and a call for place-based research. Health Place 18:747–756CrossRefGoogle Scholar
  31. Duerden F (2004) Translating climate change impacts at the community level. Arctic 57:204–212CrossRefGoogle Scholar
  32. Eisenack K, Stecker R, Reckien D, Hoffmann E (2012) Adaptation to climate change in the transport sector: a review of actions and actors. Mitig Adapt Strat Glob Chang 17:451–469. doi:10.1007/s11027-011-9336-4 CrossRefGoogle Scholar
  33. Espey M (1996) Explaining the variation in elasticity estimates of gasoline demand in the United States: a meta-analysis. Energy J 17:49–60CrossRefGoogle Scholar
  34. Forbes A, Griffiths P (2002) Methodological strategies for the identification and synthesis of ‘evidence’ to support decision-making in relation to complex healthcare systems and practices. Nurs Inq 9:141–155. doi:10.1046/j.1440-1800.2002.00146.x CrossRefGoogle Scholar
  35. Ford JD, Pearce T (2010) What we know, do not know, and need to know about climate change vulnerability in the western Canadian Arctic: a systematic literature review. Environ Res Lett 5. doi:10.1088/1748-9326/5/1/014008
  36. Ford JD, Berrang-Ford L, King M, Furgal C (2010) Vulnerability of aboriginal health systems in Canada to climate change. Glob Environ Chang Hum Policy Dimens 20:668–680. doi:10.1016/j.gloenvcha.2010.05.003 CrossRefGoogle Scholar
  37. Ford JD, Berrang-Ford L, Paterson J (2011) A systematic review of observed climate change adaptation in developed nations. Clim Chang 106:327–336. doi:10.1007/s10584-011-0045-5 CrossRefGoogle Scholar
  38. Ford JD, Bolton K, Shirley J, Pearce T, Tremblay M, Westlake M (2012a) Mapping human dimensions of climate change research in the Canadian Arctic. Ambio 41. doi:10.1007/s13280-012-0336-8
  39. Ford JD, Vanderbilt W, Berrang-Ford L (2012b) Authorship in IPCC AR5 and its implications for content: climate change and Indigenous populations in WGII. Clim Chang 113:201–213. doi:10.1007/s10584-011-0350-z CrossRefGoogle Scholar
  40. Ford JD, Bolton KC, Shirley J, Pearce T, Tremblay M, Westlake M (2012c) Research on the human dimensions of climate change in Nunavut, Nunavik, and Nunatsiavut: a literature review and gap analysis. Arctic 65:289–304CrossRefGoogle Scholar
  41. Ford JD, Berrang-Ford L, Bunce A, McKay C, Irwin M, Pearce T (2014) The status of climate change adaptation in Africa and Asia. Reg Environ Change. doi:10.1007/s10113-014-0648-2
  42. Furgal CM, Garvin TD, Jardine CG (2010) Trends in the study of aboriginal health risks in Canada. Int J Circumpolar Health 69:322–332CrossRefGoogle Scholar
  43. Furlan AD, Pennick V, Bombardier C, van Tulder M, Editorial Board Cochrane Back R (2009) 2009 updated method guidelines for systematic reviews in the cochrane back review group. Spine 34:1929–1941CrossRefGoogle Scholar
  44. Gibson EL, Kreichauf S, Wildgruber A, Voegele C, Summerbell CD, Nixon C, Moore H, Douthwaite W, Manios Y, ToyBox-Study G (2012) A narrative review of psychological and educational strategies applied to young children’s eating behaviours aimed at reducing obesity risk. Obes Rev 13:85–95CrossRefGoogle Scholar
  45. Glass G (1976) Primary, secondary and meta-analysis of research. Educ Res 10:3–8CrossRefGoogle Scholar
  46. Gough D, Thomas J, Oliver S (2012) Clarifying differences between review designs and methods. Syst Rev 1:28. doi:10.1186/2046-4053-1-28 CrossRefGoogle Scholar
  47. Greenhalgh T, Peacock R (2005) Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. Br Med J 331:1064–1065. doi:10.1136/bmj.38636.593461.68 CrossRefGoogle Scholar
  48. Grieneisen ML, Zhang M (2011) The current status of climate change research. Nat Clim Change 1:72–73CrossRefGoogle Scholar
  49. Hardee K, Mutunga C (2010) Strengthening the link between climate change adaptation and national development plans: lessons from the case of population in National Adaptation Programmes of Action (NAPAs). Mitig Adapt Strat Glob Chang 15:113–126. doi:10.1007/s11027-009-9208-3 CrossRefGoogle Scholar
  50. Heller NE, Zavaleta ES (2009) Biodiversity management in the face of climate change: a review of 22 years of recommendations. Biol Conserv 142:14–32. doi:10.1016/j.biocon.2008.10.006 CrossRefGoogle Scholar
  51. Higgins J, Green S (eds) (2011) Cochrane handbook for systematic reviews of interventions, v.5.1.0. The Cochrane Collaboration. http://handbook.cochrane.org/
  52. Hosking J, Campbell-Lendrum D (2012) How well does climate change and human health research match the demands of policymakers? A scoping review. Environ Health Perspect 120:1076–1082. doi:10.1289/ehp.1104093 CrossRefGoogle Scholar
  53. Huang C, Vaneckova P, Wang X, FitzGerald G, Guo Y, Tong S (2011) Constraints and barriers to public health adaptation to climate change a review of the literature. Am J Prev Med 40:183–190. doi:10.1016/j.amepre.2010.10.025 CrossRefGoogle Scholar
  54. Hunt A, Watkiss P (2011) Climate change impacts and adaptation in cities: a review of the literature. Clim Change 104. doi:10.1007/s10584-010-9975-6
  55. Jadad AR, Moher D, Klassen TP (1998) Guides for reading and interpreting systematic reviews - II. How did the authors find the studies and assess their quality? Arch Pediatr Adolesc Med 152:812–817CrossRefGoogle Scholar
  56. Janssen MA, Ostrom E (2006) Resilience, vulnerability, and adaptation: a cross-cutting theme of international human dimensions programme on global environmental change. Glob Environ Change Hum Policy Dimens 16:237–239CrossRefGoogle Scholar
  57. Jensen LA, Allen MN (1996) Meta-synthesis of qualitative findings. Qual Health Res 6:553–560. doi:10.1177/104973239600600407 CrossRefGoogle Scholar
  58. Kajan E, Saarinen J (2013) Tourism, climate change and adaptation: a review. Curr Issues Tour 16:167–195. doi:10.1080/13683500.2013.774323 CrossRefGoogle Scholar
  59. Kamau JW, Mwaura F (2013) Climate change adaptation and EIA studies in Kenya. Int J Clim Chang Strat Manag 5:152–165. doi:10.1108/17568691311327569 CrossRefGoogle Scholar
  60. Kilroy G (2015) A review of the biophysical impacts of climate change in three hotspot regions in Africa and Asia. Reg Environ Change. doi:10.1007/s10113-014-0709-6
  61. Klassen TP, Jadad AR, Moher D (1998) Guides for reading and interpreting systematic reviews - I. Getting started. Arch Pediatr Adolesc Med 152:700–704CrossRefGoogle Scholar
  62. Kolstrom M, Lindner M, Vilen T, Maroschek M, Seidl R, Lexer MJ, Netherer S, Kremer A, Delzon S, Barbati A, Marchetti M, Corona P (2011) Reviewing the science and implementation of climate change adaptation measures in European forestry. Forests 2:961–982. doi:10.3390/f2040961 CrossRefGoogle Scholar
  63. Kreichauf S, Wildgruber A, Krombholz H, Gibson EL, Voegele C, Nixon CA, Douthwaite W, Moore HJ, Manios Y, Summerbell CD, ToyBox-study G (2012) Critical narrative review to identify educational strategies promoting physical activity in preschool. Obes Rev 13:96–105CrossRefGoogle Scholar
  64. Larsen SV, Kornov L, Wejs A (2012) Mind the gap in SEA: an institutional perspective on why assessment of synergies amongst climate change mitigation, adaptation and other policy areas are missing. Environ Impact Assess Rev 33:32–40. doi:10.1016/j.eiar.2011.09.003 CrossRefGoogle Scholar
  65. Lesnikowski AC, Ford JD, Berrang-Ford L, Paterson JA, Barrera M, Heymann SJ (2011) Adapting to the health impacts of climate change: a study of UNFCCC Annex I parties. Environ Res Lett 6. doi:10.1088/1748-9326/6/4/044009
  66. Lesnikowski A, Ford JD, Berrang-Ford L, Barrera M, Heymann SJ (in press) How are we adapting to climate change? A global assessment of the state of adaptation. Mitig Adapt Strateg Glob Change. doi:10.1007/s11027-013-9491-x
  67. Letzel H (1995) “Best-evidence synthesis: an intelligent alternative to metaanalysis”: discussion. A case of “either-or” or “as well”. J Clin Epidemiol 48:19–21CrossRefGoogle Scholar
  68. Linnenluecke MK, Griffiths A, Winn MI (2013) Firm and industry adaptation to climate change: a review of climate adaptation studies in the business and management field. Wiley Interdiscip Rev Clim Chang 4:397–416. doi:10.1002/wcc.214 CrossRefGoogle Scholar
  69. Lwasa S (2014) A systematic review of research on climate change adaptation policy and practice in Africa and South Asia deltas. Reg Environ Change. doi:10.1007/s10113-014-0715-8
  70. Macias-Chapula CA (2013) Comparative analysis of health public policy research results among Mexico, Chile and Argentina. Scientometrics 95:615–628CrossRefGoogle Scholar
  71. Mays N, Pope C, Popay J (2005) Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 10(Suppl 1):6–20. doi:10.1258/1355819054308576 CrossRefGoogle Scholar
  72. Mazzocato P, Savage C, Brommels M, Aronsson H, Thor J (2010) Lean thinking in healthcare: a realist review of the literature. Qual Saf Health Care 19:376–382Google Scholar
  73. McDowell C (2013) Climate-change adaptation and mitigation: implications for land acquisition and population relocation. Dev Policy Rev 31:677–695. doi:10.1111/dpr.12030 Google Scholar
  74. McLeman RA (2011) Settlement abandonment in the context of global environmental change. Glob Environ Chang Hum Policy Dimens 21:S108–S120. doi:10.1016/j.gloenvcha.2011.08.004 CrossRefGoogle Scholar
  75. McLeman R (2013) Developments in modelling of climate change-related migration. Clim Chang 117:599–611. doi:10.1007/s10584-012-0578-2 CrossRefGoogle Scholar
  76. Merten S, Kenter E, McKenzie O, Musheke M, Ntalasha H, Martin-Hilber A (2010) Patient-reported barriers and drivers of adherence to antiretrovirals in sub-Saharan Africa: a meta-ethnography. Trop Med Int Health 15:16–33. doi:10.1111/j.1365-3156.2010.02510.x CrossRefGoogle Scholar
  77. Mills E (2009) A global review of insurance industry responses to climate change. Geneva Pap Risk Insur Issues Pract 34:323–359. doi:10.1057/gpp.2009.14 CrossRefGoogle Scholar
  78. Moher D, Jadad AR, Klassen TP (1998) Guides for reading and interpreting systematic reviews - III. How did the authors synthesize the data and make their conclusions? Arch Pediatr Adolesc Med 152:915–920CrossRefGoogle Scholar
  79. Morrison C, Pickering C (2013) Limits to climate change adaptation: case study of the Australian Alps. Geogr Res 51:11–25. doi:10.1111/j.1745-5871.2012.00758.x CrossRefGoogle Scholar
  80. Murtinho F, Hayes TM (2012) Adaptation in resource-dependent communities: a call for greater methodological clarity in adaptation field research. Soc Nat Resour 25:513–522. doi:10.1080/08941920.2011.604068 CrossRefGoogle Scholar
  81. Mwangi-Powell FN, Powell RA, Harding R (2013) Models of delivering palliative and end-of-life care in sub-Saharan Africa: a narrative review of the evidence. Curr Opin Supportive Palliat Care 7:223–228CrossRefGoogle Scholar
  82. Newhouse RP (2008) Evidence synthesis—the good, the bad, and the ugly. J Nurs Adm 38:107–111CrossRefGoogle Scholar
  83. O’Neill SJ, Osborn TJ, Hulme M, Lorenzoni I, Watkinson AR (2008) Using expert knowledge to assess uncertainties in future polar bear populations under climate change. J Appl Ecol 45:1649–1659. doi:10.1111/j.1365-2664.2008.01552.x CrossRefGoogle Scholar
  84. Pawson R, Greenhalgh T, Harvey G, Walshe K (2005) Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 10(Suppl 1):21–34. doi:10.1258/1355819054308530 CrossRefGoogle Scholar
  85. Pearce T, Ford JD, Duerden F, Smit B, Andrachuk M, Berrang-Ford L, Smith T (2011) Advancing adaptation planning for climate change in the Inuvialuit Settlement Region (ISR): a review and critique. Reg Environ Chang 11:1–17. doi:10.1007/s10113-010-0126-4 CrossRefGoogle Scholar
  86. Pearce T, Ford J, Caron A, Kudlak B (2012) Climate change adaptation planning in remote, resource-dependent communities: an Arctic example. Reg Environ Chang 12:825–837. doi:10.1007/s10113-012-0297-2 CrossRefGoogle Scholar
  87. Petticrew M, McCartney G (2011) Using systematic reviews to separate scientific from policy debate relevant to climate change. Am J Prev Med 40:576–578. doi:10.1016/j.amepre.2010.12.022 CrossRefGoogle Scholar
  88. Petticrew M, Roberts H (2006) Systematic reviews in the social sciences. Blackwell, OxfordCrossRefGoogle Scholar
  89. Pielke R Jr, Prins G, Rayner S, Sarewitz D (2007) Lifting the taboo on adaptation. Nature 445:597–598. doi:10.1038/445597a CrossRefGoogle Scholar
  90. Poutiainen C, Berrang-Ford L, Ford J, Heymann J (2013) Civil society organizations and adaptation to the health effects of climate change in Canada. Public Health 127:403–409. doi:10.1016/j.puhe.2013.02.004 CrossRefGoogle Scholar
  91. Poyar KA, Beller-Simms N (2010) Early responses to climate change: an analysis of seven US state and local climate adaptation planning initiatives. Weather Clim Soc 2:237–248. doi:10.1175/2010WCAS1047.1 CrossRefGoogle Scholar
  92. Preet R, Nilsson M, Schumann B, Evengard B (2010) The gender perspective in climate change and global health. Glob Health Action 3. doi:10.3402/gha.v3i0.5720
  93. Preston BL, Westaway RM, Yuen EJ (2011) Climate adaptation planning in practice: an evaluation of adaptation plans from three developed nations. Mitig Adapt Strat Glob Chang 16:407–438. doi:10.1007/s11027-010-9270-x CrossRefGoogle Scholar
  94. Raymond CM, Robinson GM (2013) Factors affecting rural landholders’ adaptation to climate change: insights from formal institutions and communities of practice. Glob Environ Chang Hum Policy Dimens 23:103–114. doi:10.1016/j.gloenvcha.2012.11.004 CrossRefGoogle Scholar
  95. Richardson L, Loomis J (2009) The total economic value of threatened, endangered and rare species: an updated meta-analysis. Ecol Econ 68:1535–1548CrossRefGoogle Scholar
  96. Sharmina M, Anderson K, Bows-Larkin A (2013) Climate change regional review: Russia. Wiley Interdiscip Rev Clim Chang 4:373–396. doi:10.1002/wcc.236 CrossRefGoogle Scholar
  97. Shepard CC, Crain CM, Beck MW (2011) The protective role of coastal marshes: a systematic review and meta-analysis. Plos One 6. doi:10.1371/journal.pone.0027374
  98. Shepherd A, Wu L, Chadwick D, Bol R (2011) A review of quantitative tools for assessing the diffuse pollution response to farmer adaptations and mitigation methods under climate change. Adv Agron 112:1–54. doi:10.1016/B978-0-12-385538-1.00001-9 CrossRefGoogle Scholar
  99. Slavin RE (1995) Best evidence synthesis—an intelligent alternative to metaanalysis. J Clin Epidemiol 48:9–18CrossRefGoogle Scholar
  100. Smith MS, Horrocks L, Harvey A, Hamilton C (2011) Rethinking adaptation for a 4 degrees C world. Philos Trans R Soc Math Phys Eng Sci 369:196–216. doi:10.1098/rsta.2010.0277 CrossRefGoogle Scholar
  101. Stanhill G (2001) The growth of climate change science: a scientometric study. Clim Change 48:515–524CrossRefGoogle Scholar
  102. Sud R, Mishra A, Varma N, Bhadwal S (2015) Adaptation policy and practice in densely populated glacier-fed river basins of South Asia: a systematic review. Reg Environ Change. doi:10.1007/s10113-014-0711-z
  103. Suri H, Clarke D (2009) Advancements in research synthesis methods: from a methodologically inclusive perspective. Rev Educ Res 79:395–430CrossRefGoogle Scholar
  104. Taewoo N (2012) Suggesting frameworks of citizen-sourcing via Government 2.0. Gov Inf Q 29:12–20. doi:10.1016/j.giq.2011.07.005 CrossRefGoogle Scholar
  105. Termeer C, Biesbroek R, Van den Brink M (2012) Institutions for adaptation to climate change: comparing national adaptation strategies in Europe. Eur Polit Sci 11:41–53. doi:10.1057/eps.2011.7 CrossRefGoogle Scholar
  106. Toloo G, FitzGerald G, Aitken P, Verrall K, Tong S (2013) Evaluating the effectiveness of heat warning systems: systematic review of epidemiological evidence. Int J Public Health 58:667–681. doi:10.1007/s00038-013-0465-2 CrossRefGoogle Scholar
  107. Tompkins EL, Adger WN, Boyd E, Nicholson-Cole S, Weatherhead K, Arnell N (2010) Observed adaptation to climate change: UK evidence of transition to a well-adapting society. Glob Environ Chang 20:627–635. doi:10.1016/j.gloenvcha.2010.05.001 CrossRefGoogle Scholar
  108. Tucker J, Daoud M, Oates N, Few R, Conway D, Mtisi S, Matheson S (2014) Social vulnerability in three high-poverty climate change hotspots: what does the climate change literature tell us? Reg Environ Change. doi:10.1007/s10113-014-0741-6
  109. Vassilev I, Rogers A, Sanders C, Kennedy A, Blickem C, Protheroe J, Bower P, Kirk S, Chew-Graham C, Morris R (2011) Social networks, social capital and chronic illness self-management: a realist review. Chronic illn 7:60-86CrossRefGoogle Scholar
  110. Vink MJ, Dewulf A, Termeer C (2013) The role of knowledge and power in climate change adaptation governance: a systematic literature review. Ecol Soc 18. doi:10.5751/ES-05897-180446
  111. Waddington H, White H, Snilstveit B, Hombrados JG, Vojtkova M, Davies P, Bhavsar A, Eyers J, Koehlmoos TP, Petticrew M, Valentine JC, Tugwell P (2012) How to do a good systematic review of effects in international development: a tool kit. J Dev Eff 4:359–387. doi:10.1080/19439342.2012.711765 CrossRefGoogle Scholar
  112. Wagschal U, Wenzelburger G (2012) When do governments consolidate? A quantitative comparative analysis of 23 OECD countries (1980-2005). J Comp Policy Anal 14:45–71Google Scholar
  113. Walker R, Hassall J, Chaplin S, Congues J, Bajayo R, Mason W (2011) Health promotion interventions to address climate change using a primary health care approach: a literature review. Health Promot J Aust 22:S6–S12Google Scholar
  114. Walsh D, Downe S (2005) Meta-synthesis method for qualitative research: a literature review. J Adv Nurs 50:204–211. doi:10.1111/j.1365-2648.2005.03380.x CrossRefGoogle Scholar
  115. Wamsler C, Brink E, Rivera C (2013) From theory to practice. J Clean Prod 50:68–81. doi:10.1016/j.jclepro.2012.12.008 CrossRefGoogle Scholar
  116. Weinhofer G, Busch T (2013) Corporate strategies for managing climate risks. Bus Strategy Environ 22:121–144. doi:10.1002/bse.1744 CrossRefGoogle Scholar
  117. Wilby RL, Keenan R (2012) Adapting to flood risk under climate change. Prog Phys Geogr 36:348–378. doi:10.1177/0309133312438908 CrossRefGoogle Scholar
  118. Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R (2013) RAMESES publication standards: meta-narrative reviews. J Adv Nurs 69:987–1004CrossRefGoogle Scholar

Copyright information

© The Author(s) 2015

Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Authors and Affiliations

  • Lea Berrang-Ford
    • 1
  • Tristan Pearce
    • 2
    • 3
  • James D. Ford
    • 1
  1. 1.Department of GeographyMcGill UniversityMontrealCanada
  2. 2.Faculty of Arts and Business, Sustainability Research CentreUniversity of the Sunshine CoastMaroochydoreAustralia
  3. 3.Department of GeographyUniversity of GuelphGuelphCanada

Personalised recommendations