Introduction

Changing climate conditions have been documented on both global and regional scales (IPCC 2014; Melillo et al. 2014). Although there is general agreement that temperatures in and around Colorado are projected to increase, precipitation models are more variable and tend to show increasing precipitation for most of Colorado (Lucas et al. 2014). However, when snowpacks are low, warmer spring temperatures and the effects of increased dust-on-snow have combined to shift the timing of snowmelt and peak runoff from 1 to 4 weeks earlier (Lucas et al. 2014). Earlier onset of spring-like conditions increases forest wildfire incidence (Westerling et al. 2006) and changes patterns of streamflow and soil moisture availability (Harpold and Molotch 2015). Both historic and pre-instrumental record droughts have had notable effects on vegetation patterns, with concomitant changes in patterns of human habitation and social interaction (Woodhouse 2004; Benson et al. 2007). The possibility of future droughts that exceed the most severe droughts of the past millennia cannot be excluded (Cook et al. 2015). These observations have heightened concern about the potential for future harm to species and ecosystems as well as people and their livelihoods under changing climatic conditions.

Current thought emphasizes the need to act now on climate change adaptation planning and implementation despite uncertainty in climate change projections. Understanding and planning for a likely range of future possibilities or even just a general direction of change will be more useful to land managers than simply maintaining the status quo. A key first step in developing adaptation strategies is the vulnerability assessment (Glick et al. 2011; Rowland et al. 2011; Stein et al. 2014). Such assessments are intended to shape policy development focused on the reduction of risks associated with climate change (Füssel and Klein 2006), as well as supporting prioritization of effort and resources. Vulnerability assessments of species and their habitats are expected to help land managers identify opportunities for action to mitigate the effects of climate change, recognize potential novel conditions that may require additional analysis or new methods, and characterize uncertainties inherent in the process.

The components of vulnerability were described by Glick et al. (2011) and consist of projected exposure to climate change, sensitivity of the species or ecosystem to those expected changes, and the adaptive capacity of the species or ecosystem to respond to changes. Exposure is the amount of change in climatic conditions that is projected for a particular area and is perhaps the most amenable to spatial analysis given the increased availability of downscaled global circulation models (GCMs) together with related derived datasets. Sensitivity reflects the degree to which organisms respond to changed climate-driven conditions (the response may be positive or negative). Sensitivity is difficult to characterize at a landscape scale, even when we have detailed physiological and life-history information about a species. The combined effects of exposure and sensitivity are the potential impacts from climate change and often are the primary focus of vulnerability assessments for elements of biodiversity. Adaptive capacity is a measure of whether a species or its habitat can persist under future conditions and is believed to be affected by both the inherent characteristics of the species as well as its disturbance history and current condition (Glick et al. 2011).

To date, a majority of natural resource-focused climate change vulnerability assessments have evaluated individual species (reviewed in Johnson 2014; Pacifici et al. 2015), although habitats, places, processes, and ecosystem services have also been addressed in a number of unpublished assessments (Johnson 2014). Approaches include a spectrum of methods from quantitative species response models incorporating spatially explicit distribution models, details of life-history characteristics, demography, and downscaled climate projections to the more general qualitative, index-based methods that are useful in the commonplace instance where detailed species- or habitat-specific information is lacking. In a discussion of qualitative and quantitative research approaches, Jick (1979) notes that the process of combining results from different method types is useful even without convergence and that in instances where outcomes do converge, confidence in the results is increased. This multi-method research approach has been called triangulation, after the surveying technique of using multiple reference points to locate an exact position (Denzin 1978). Similarly, the identification of climate change vulnerability levels for species and ecosystems through multiple studies using varied methods can be regarded as a form of triangulation to converge on key elements of vulnerability that can be addressed by land managers.

One method of triangulation uses multiple bearings from a single (unknown) location sited on several prominent landmarks (which must be known to the observer), in order to locate the observation point on a map. Conversely, from a series of known locations, an observer may take multiple bearings on a feature of interest, for instance, a mountain peak, or a strong radio signal. From each observation point, a line in the direction of the bearing is plotted on a map. A single bearing gives no distance information. Although two highly precise and accurate bearings may locate a point correctly, a correct location can’t be verified without visiting the true location. Three (or more) bearings are typical in field-based observations. With well-dispersed observation points and precise bearings, the object of interest can be located fairly accurately (Fig. 1a). Less precise observations still enable the object to be located within a comparatively small area (Fig. 1b and c), while inaccurate observations may miss the target altogether (Fig. 1d).

Fig. 1
figure 1

Some potential outcomes of triangulating with three bearings to locate a single point

Full size image

In estimating the vulnerability of ecosystems to future climate conditions, we are starting from a fairly well-known location (the present) and can be reasonably sure of direction for temperature (i.e., there are no models that project cooler conditions), but not for precipitation. In our assessments, we concentrated on identifying the area within bearings that represented a worst-case future scenario, with the assumption that it is more important to managers and policy-makers to identify the boundaries of uncertainty that can be addressed with no-regret actions, than it is to determine an exact degree of vulnerability, if such a thing were possible. During 2014–2015, the authors produced three climate change vulnerability assessments focused on major vegetation types (i.e., habitats or ecological systems) in Colorado. Differences in project objectives, funder goals, and available data resulted in a gradual evolution of assessment techniques from strictly qualitative to primarily quantitative. Our objective in this paper is to compare vulnerability assessments across a spectrum of techniques from qualitative at a sub-state regional scale to quantitative at a statewide scale and evaluate the consistency and utility of the results.

Methods

Climate change vulnerability assessment methods for three case studies differed in spatial extent, vegetation data, climate data, and methodology used (Table S1 in the Supplementary Materials). Distribution of the vegetation types evaluated in the three studies is shown in Fig. S1 of the Supplementary Materials. For cross-case study comparison, we reconciled differences in vegetation type grouping within studies to a common standard. All assessments were focused on vulnerability under projected mid-century (2050) conditions.

Sub-state region: southwestern Colorado

Our assessment for the US Forest Service and Bureau of Land Management’s San Juan and Tres Rios management area used methods equivalent to a companion project in the Gunnison Basin (Neely et al. 2012). The area represents 7.4% of the total area of Colorado. For 16 vegetation types represented in the area, we developed basic descriptive climatic information about the present and recent past, in order to characterize the type’s current climatic envelope. Dynamically downscaled temperature and precipitation projections under an A2 emissions scenario produced for the North American Regional Climate Change Assessment Program (NARCCAP) were used by Cozzetto et al. (2011) to assess potential exposure and impact from future climate conditions for the region in a narrative fashion. Narratives included discussion of best-case and worst-case scenarios. We did not make a detailed spatial analysis of future climate projections linked with mapped vegetation type locations, instead relying on the narrative projections.

Vulnerability assessment factors for all three assessments were based on methods developed by the Manomet Center for Conservation Sciences with the Massachusetts Division of Fisheries and Wildlife to evaluate important habitats identified for the state’s Wildlife Action Plan (MCCS and MAFW 2010). This protocol identified ten habitat variables likely to affect vulnerability to climate change (Table S2 in Supplementary Materials) and included an eight-level qualitative scoring system summarizing the likelihood of the evaluated habitat being eliminated from the area as a result of climate change. For each habitat, a narrative vulnerability evaluation was used to present the rationale used by evaluators in summarizing overall vulnerability. Our assessment for southwestern Colorado addressed nine of the original variables (Table S2 in Supplementary Materials), with some differences in focus from the original protocol. Each factor was assigned a qualitative score of high, medium, low, or no importance. No algorithm was used for the overall vulnerability score, but factors as a group were evaluated in light of projected future conditions to establish a logical overall estimate of the vulnerability of each vegetation type. Full methods and results are available in Decker and Rondeau (2014). An overall vulnerability rank in this assessment represents a qualitative, expert opinion-based version of the calculated resilience scores discussed below.

Colorado statewide I

Our initial statewide vulnerability assessment was completed for Colorado Parks and Wildlife as a part of the revision of the state’s Wildlife Action Plan. Our approach identified climate variables that were most influential in determining the current distribution of 18 vegetation types and evaluated degree and direction of projected future change by mid-century for each variable. We used three sources of information: (1) geospatial distribution modeling for a subset of vegetation types, (2) expert review of model factors, and (3) literature review. Spatial analysis methods focused on characterizing exposure of each vegetation type to change in climate variables to which it was likely to be most sensitive, thus combining the two factors contributing to impact.

We used bias corrected, statistically downscaled (BCCA) daily climate data at a resolution of 12 km (BOR 2013), averaged over 1980–2005 to represent “current” normals, and averaged over 2035–2060 to represent mid-century projections. Parks and Wildlife personnel preferred to use representative concentration pathway (RCP) 6.0, which is the second highest (van Vuuren et al. 2011) used in the Climate Model Diagnosis and Intercomparison Project, phase 5 (CMIP5), estimating the equivalent of 850 ppm of CO2 beyond the year 2100, making it loosely equivalent to the A2 emissions scenario from CMIP3 (830 ppm CO2 by 2100). To capture the range of model variation as best- and worst-case scenarios, the central 80% of the 12-model ensemble was chosen to represent a “reasonable range” of possible future climate for each metric. The top five climate variables per vegetation type (those totaling at least 75% of the relative influence in the final distribution model or identified as important by our other two sources) were used to evaluate exposure by assessing the degree to which future conditions within the current mapped distribution of a vegetation type were projected to be outside the core range of current conditions, i.e., the “range shift” (Fig. S2 in the Supplementary Materials). For each variable in each type, we calculated the percent of current mapped acreage that was projected to fall below (for precipitation) or above (for temperature) current conditions as the worst-case scenario. Exposure-sensitivity scores represent the average of the five range shift proportions. Resilience, and by implication, adaptive capacity were evaluated with a modified Manomet and MAFW (2010) methodology, where factors were combined into five groups covering eight of the original ten metrics (Table S2 in Supplementary Materials), using spatial data to generate scores. The exposure for sensitive variable score and the resilience score were combined according to the schema presented in Comer et al. (2012). Complete methods and results are available in Decker and Fink (2014).

Colorado statewide II

Our second statewide vulnerability assessment built on the previous analysis. In order to use more finely downscaled climate projections and simplify our exposure analysis, we used the 34-model CMIP5 ensemble averages of 800-m NASA Earth Exchange Downscaled Climate Projections (NEX-DCP30) for the Continental US. To address the variability of precipitation projections in Colorado’s highly complex terrain, we incorporated results from hydrologic modeling for the Colorado River and other basins (e.g., Nash and Gleick 1991) indicating that, as a generalized rule-of-thumb, for each 1 °C of warming, an approximate 5% increase in precipitation would be required for runoff levels (and therefore, moisture availability) to remain unchanged. With projected mid-century temperatures increasing 2 °C or more over current levels, no areas in Colorado are expected to receive sufficient compensatory precipitation. In order to account for the potential effects of warmer temperatures on soil moisture availability and determine the extent to which each ecosystem may be exposed to effectively drier conditions, we made a conservative application of the above rule, to evaluate how much of an ecosystem might receive at least a partial (50%) level of compensatory precipitation. For 20 vegetation types, we calculated the proportion of its current acreage where projected annual mean temperature for mid-century (2035–2060) under RCP 8.5 (our worst-case scenario) was greater than any annual mean temperatures currently experienced by that ecosystem within Colorado, and projected future precipitation changes were less than a 5% increase over current levels (Fig. S2 in the Supplementary Materials). The same resilience methods used in the previous statewide assessment were used for scores, with minor modification to accommodate the expanded number of vegetation types assessed (Table S2 in the Supplementary Materials). Details of methods and results are in CNHP (2015).

Comparison of assessment results

For all terrestrial vegetation types assessed in each of the three projects, final vulnerability ranks were reclassified to either vulnerable (moderate or higher vulnerability, Y in Table 1) or not vulnerable (low to no vulnerability, N in Table 1), following the methodology of Cruz et al. (2015). The percentage of vulnerable types was compared between projects. Where vegetation types were assessed in two or more projects, we compared vulnerability outcomes and identified the primary source of disagreement or uncertainty. Because area assessed, time frame, and vegetation mapping source are the same for the statewide assessments (Table S1 in the Supplementary Materials), we identified sources of disagreement between them by comparing individual exposure-sensitivity scores and resilience scores (Fig. 2) and summarized how scoring methodology affected the scores. Matched pairs were compared using Spearman’s rank correlation coefficient (ρ), calculated using cor.test in R (R Development Core Team 2016).

Table 1 Vegetation types assessed in each study and their overall vulnerability outcomes. Final vulnerability scores of moderate or higher are shown as Y (yes, vulnerable), or N (low or no vulnerability)
Full size table
Fig. 2
figure 2

Comparison of exposure-sensitivity scores (top) and resilience scores (bottom) for 16 vegetation types common to both statewide vulnerability assessments. Points closest to the diagonal are most in agreement between assessments

Full size image

Disagreement between the southwestern Colorado assessment and either of the two statewide assessments may arise because the extent and types of vegetation present in this region are not representative of the state as a whole (Fig. S1 in the Supplementary Materials) as well as from the qualitative techniques used in the assessment. Instances of disagreement between the southwestern Colorado assessment as compared to one or both statewide assessments were explicated by identifying the evaluation factors and uncertainties that most contributed to the regional vulnerability score and characterizing differences between the qualitative and quantitative scoring results.

Results

Although the proportion of vegetation types assessed as vulnerable in each study was similar (63–70%), this broad agreement did not translate to identical outcomes for the 11 vegetation types that were common to all three studies. Of those 11, only four outcomes were in agreement across all studies. Each statewide assessment was in agreement with the sub-state assessment on about half of shared vegetation types (Table 1). Within vegetation types, 55% of pairwise comparisons across all studies were in agreement on vulnerability status. In contrast, of the 16 vegetation types shared between the two statewide studies, the outcomes of 12 (75%) agreed.

Agreement between statewide assessments

Differences in vulnerability outcomes between the two statewide assessments can be attributed to methods used to assess exposure-sensitivity (Fig. S2 in the Supplementary Materials), identified as exposure method in Table 1. Ranked results of paired exposure-sensitivity scores in the two statewide assessments were not well correlated (ρ = 0.19, P = 0.600). In contrast, resilience scores were highly correlated (ρ = 0.94, P < <0.001), in spite of differences in expert input and vegetation type grouping. Four vegetation types (alpine areas and three of the six shared wetland/riparian sub-types) had different vulnerability outcomes. In the case of alpine vegetation, a projected increase in growing season length and temperature was sufficient to push this type into the vulnerable range for the statewide I assessment, while projected changes in precipitation using different climate data appeared to be adequate to mitigate against vulnerability in statewide II. In the three instances of disagreement in wetland or riparian vulnerability, projected precipitation amounts for the statewide II assessment were insufficient to maintain these vegetation types as they currently occur, which moved these types into the vulnerable range.

Agreement between statewide assessments and SW Colorado assessment

Six vegetation in common had a different outcome in the southwestern Colorado assessment compared to one or both statewide assessments (Table 1). For both ponderosa woodland and montane grassland, the condition of the type in southwestern Colorado was judged to be better than in the state as a whole, resulting in a “not vulnerable” rank. Sagebrush shrubland in southwestern Colorado has a much narrower bioclimatic envelope than it has statewide, which increased its local vulnerability in comparison with the larger area. We call these disagreements regional sampling bias, because the southwestern region is not representative of the state as a whole. In the case of alpine, montane riparian, and montane wetlands, the disagreement is due to qualitative analysis regarding the degree of exposure to change. In a statewide context, these types have low exposure to change, but this could not be determined through the entirely qualitative methods used for the southwestern assessment. We used the term exposure resolution to describe this type of disagreement. Finally, three types assessed only in southwestern Colorado and statewide II had different vulnerability outcomes. One disagreement was due to regional sampling bias (desert shrubland in better condition in the southwestern area) and one due to exposure resolution (semi-desert grassland with low exposure to change). In the case of mixed conifer, disagreement was the result of high uncertainty about response of this vegetation type to a number of evaluated factors, summarized as model uncertainty.

Discussion

Why are climate change vulnerability assessments conducted by the same researchers in related geographic areas not in better agreement? Are the assessments believable and/or useful to natural resources managers? In the course of completing the assessments discussed herein, we identified more than a dozen previous vulnerability assessments for species, ecosystems, or other natural resources that covered some portion of Colorado. These assessments were completed by a variety of federal, state, and non-profit entities. While it is clear that a number of organizations are engaged in vulnerability assessment, there is little exposure of this effort in the peer-reviewed literature (Johnson 2014). The proliferation of climate change vulnerability assessments in the “gray literature” highlights the need for moving forward with this work, even under conditions of imperfect outcomes.

It is important to remember that an assessment typically reports the relative vulnerability of a species or ecosystem (Glick et al. 2011), not an absolute measure of risk. Flagging the most vulnerable types should help land managers identify areas where action may mitigate the effects of climate change and recognize potential novel conditions that may require new approaches. Due to the comparative nature of assessment, we expect that the calculated vulnerability of an ecosystem or species may vary according to the choice of climate model dataset(s), emissions scenario/concentration pathway, focal timeframe, scale of analysis area used, quality and quantity of knowledge available about the assessed target, and decisions made by participants in the process. Land management agency directives, responsibilities, and corporate culture are key factors influencing the outcome of an assessment. The tendency of expert input to guide the outcomes toward pre-conceived priorities is difficult to avoid, but can be compensated for by repeated feedback and discussion cycles between subject experts and end users (Gregory et al. 2012). Uncertainty does not invalidate an assessment, but must be brought to the attention of end users to avoid confusion arising from differing results.

Uncertainty in assessment outcomes that are due to how the assessment goals are framed by participating entities prior to actual analysis falls within the context uncertainty category of Walker et al. (2003). This is the fundamental origin of most of the disagreements in our outcomes because it leads to differences in assessment targets, area assessed, and in the assessment methods employed, including the choice of climate data and expert-based input availability. Climate projection data is now widely available and constantly being refined and updated. However, results from the suite of emissions scenario/GCM combinations are highly variable from one to the other, with some factors more reliably modeled than others. The highly complex topography of much of Colorado, in particular, makes precipitation predictions problematic at scales which are likely to be important to local vegetation types over the relatively short-time period typically encompassed by natural resource management plans. Variation due to climate projection forcing choices is reduced by the short planning horizon (i.e., it is difficult to distinguish between scenarios until later in the century) but was still a source of disagreement in our assessments. In addition, we know that there are other (non-climate) factors that will continue to be important in determining future persistence or loss of a particular vegetation type. Data for these factors is often lacking, incomplete, or of poor resolution in comparison with the scale of the component organisms comprising the vegetation type. We refined qualitative inputs in all three assessments based on iterative expert review, progressively incorporating information from additional participant review with each assessment. Such methodological choices (input uncertainties of Walker et al. 2003) are generally based on what participants think they know about the system, in this case, a particular vegetation type. The use of qualitative methods allowed us to incorporate expert opinion into our overall model process. Finally, qualitative sources of variation (model structure uncertainty of Walker et al. 2003) in our assessments are primarily due to deficiencies in our understanding of the system (i.e., the behavior of individual species in the environment and other factors including time lags or anthropogenic influences).

In the end, our goal is to identify the future location of a vegetation type in relation to the vulnerability boundary (Fig. 3). We assume that possible future outcomes for a species (or a group of species) fall under one of three categories: conditions get worse, stay the same, or improve. The cumulative possible levels of these conditions when multiple species are combined into a vegetation type provide additional uncertainty about the outcome but should remain on the continuum. The transition from secure to vulnerable is probably not a narrow threshold from our current observation point, so we hope that multiple observations will give us more confidence in the fate of a vegetation type. It is also useful to know which types are furthest across the line in either direction as an aid to prioritization, and, if loss seems inevitable, as a cue to cut losses and turn to other actions. In each vulnerability assessment, we focused on worst-case outcomes, with weight of analysis tilted toward the vulnerable side. In qualitative methods, this meant regarding potential poor outcomes as of somewhat greater concern and ranking accordingly. In quantitative methods, we framed our assessment using more pessimistic climate trajectories. In this way, we hope to establish the limit of the type’s vulnerability (i.e., a form of triangulation equivalent to Fig. 1c).

Fig. 3
figure 3

Conceptual illustration of potential outcomes of a vulnerability assessment. The center of the continuum represents current conditions under which a vegetation type remains relatively stable. As conditions become worse for a type (to the left), it is increasingly vulnerable to loss, or, as conditions improve (to the right), it may be able to expand into newly suitable areas. Filled triangles represent VA ranks for a vegetation type that would be ranked as vulnerable, open triangles represent ranks for a vegetation type that is stable to increasing and would not be ranked as vulnerable

Full size image

In spite of apparently high levels of uncertainty, managers can still act on vulnerability concerns, especially with adaptive methods. Stein et al. (2014) outlined the climate-smart conservation cycle, a multi-step process in which vulnerability assessment is an early step in an iterative approach for climate change adaptation work. Within the cycle, vulnerability assessments form a link between planning objectives and the development of adaptation strategies but do not lead directly to on-the-ground action. The vulnerability assessment is most useful for prioritizing next steps when resources are limited. The key vulnerabilities, thresholds, and information gaps identified during the assessment may require additional work before adaptation strategies can be developed, although it is likely that high-priority targets will be clear enough to allow work to commence. Adaptation strategies are the step in the cycle which can lead to actual management implementation. The degree of alacrity in moving from the vulnerability assessment through the development and implementation of adaptation strategies depends on committed staff and adequate funding and is generally constrained by planning and funding cycles on a multi-year scale. Our vulnerability assessments have met with varying degrees of response. Our first statewide vulnerability assessment was incorporated into the state’s recent Wildlife Action Plan revision but has not yet lead to the next step. In southwestern Colorado, adaptation strategies have been developed for pinyon-juniper woodlands as part of a collaborative effort in the area. High-priority ecosystems identified in our second statewide vulnerability assessment are now the subject of work-in-progress for adaptation strategy development and an attempt to incorporate the information into a new land management plan.

Our work with an evolving set of vulnerability assessments has helped illuminate considerations for improving the utility of future assessments, data constraints, and sources of disagreement. Furthermore, our experience supports the conclusion that the successful use of climate knowledge is enhanced by a repeated collaborative process involving both knowledge producers and users (Dilling and Lemos 2011). Uncovering sources of disagreement helped increase acceptance of the results by clarifying how data selection and other participant choices shape the end result, mitigating against the common perception that climate science is too difficult for the lay person to use or understand.