As the recognition of the importance of biological diversity in biological conservation grows, an ongoing challenge is to develop metrics that can be used for effective conservation and management. The ecological integrity assessment has been proposed as such a metric. It is held by some to measure species composition, diversity, and habitat quality, as well as ecosystem structure, composition, and function. The methodology relies on proxy variables that include data on landscape characteristics such as patch size, abiotic factors such as hydrology, and some features of vegetation structure and composition. We suggest that the measure is flawed on four levels. First, its putative representation of general ecological form and function, and its lack of specific detail about how it actually represents those attributes, leaves the metric without the focus needed to be useful for measuring ecological features on the ground and testing associated hypotheses and predictions. Second, the proxy variables used to represent biological diversity, such as habitat (vegetation) metrics and vascular plant species diversity, are not empirically correlated with diversity of a range of taxa or of other components of the biota. Third, like other ecological indices that integrate many distinct features, the ecological integrity index is subject to the loss of information in its condensation of multi-dimensional variability into a one-dimensional index, and it may be subject to systematic bias from the conversion of raw data into categorical scores. Fourth, the sampling protocols are at risk of sampling bias, observer bias, and measurement error, any of which can confound the estimation of conservation value. In terms of biological diversity, the methodology produces an unreliable estimate of the number of vascular plant species and their relative percentages of occurrence, and an absence of any protocols for taxa other than plants. For these reasons we believe that ecological integrity assessment is currently of limited value as a measure of site-specific biological diversity and its change over time. A considerable amount of investigation is needed in order to have confidence in the results of an ecological integrity assessment, especially if it is to be used for regulatory purposes. We suggest further refinements and discuss alternative measures of biological diversity that provide reliable metrics for assessing change. A thoughtful choice among measures can help to identify the most appropriate assessment for conservation decisions.
The dynamics and functioning of ecosystems, and hence the ability of ecosystems to provide humans with essential goods and services, depends to a great extent on the diversity of life (Cardinale 2011; Hector 2011; Tscharntke et al. 2012; review by Cardinale et al. 2012). Diversity has other benefits as well, such as the reduction of disease prevalence in plants and animals when diversity is high (Civitello et al. 2015). But the diversity of species, genetics, and communities is being lost at an alarming rate (Cardinale et al. 2012). Human-induced species losses are arguably leading to a sixth mass extinction (Ceballos et al. 2015)—for example, the rate of vertebrate species loss over the past century is 100 times higher than the background rate (Ceballos et al. 2015).
The conservation and management of species, ecosystems, and diversity at various levels are crucially important in sustaining natural structures and functions. Conservation and management are likely to entail land-use decisions, for example, the design of a system of reserves, selection of areas for intensified agricultural production, or the choice among options for balancing human and ecological needs (Margules and Pressey 2000; Tscharntke et al. 2012). Some forms of management may involve making land-use decisions through such mechanisms as environmental markets (Pindilli and Casey 2015), which are often incorporated in regulatory frameworks that guide management of species and habitats (Salzman and Ruhl 2000).
The management of biological diversity on the landscape requires its accurate measurement, so as to compare alternatives, choose management actions, and monitor progress in achieving objectives. One approach to the measurement and tracking of biological diversity is use of a multi-metric site assessment to serve as a surrogate for surveys of species presence and abundance (Oliver et al. 2014). An example of this type of multi-metric index is the ecological integrity assessment, which combines measures of biotic and abiotic ecosystem features into a single index, as outlined by Andreasen et al. (2001). NatureServe has developed a version of ecological integrity assessment that includes geographic information system (GIS)–based landscape features, vegetation, and abiotic attributes. This framework has been used by NatureServe to build ecosystem-specific ecological integrity indices for wetlands (Faber-Langendoen et al. 2012a, p. 49) and northeastern temperate forests (Tierney et al. 2009), with the potential for application to other ecosystem types as well. The stated intention is to provide a standardized measure of outcomes of conservation programs. The proponents of ecological integrity assessment emphasize a number of desirable attributes such as convenience, cost-effectiveness, ease of use, measurability, flexibility, and sensitivity (Andreasen et al. 2001; Willamette Partnership 2011).
Given the relative scarcity of conservation funding and the increasingly urgent need for information about biological diversity, it is important to channel funds effectively for conservation value. Advocates of ecological integrity assessment explicitly claim that this approach can be used to measure and manage biological diversity efficiently (Andreasen et al. 2001; Willamette Partnership 2011; Vickerman and Kagan 2014), and hence this type of assessment is potentially very appealing to state and federal land-management agencies and other practitioners who are understandably looking for practical and inexpensive ways to generate useful information. However, ecological integrity assessment methodology so far lacks a thorough review in the refereed literature and a compilation of empirical evidence that it actually measures biological diversity. Without such a review and evidence, an ad hoc model can use up valuable time and resources but result in misleading metrics and misinformed decision making (Tulloch et al. 2013). These concerns are particularly relevant to broad-scale adoption of unvalidated methodology by government agencies, as was the case, for example, in 30 years of official commitment to ineffective tiger monitoring in India (Karanth et al. 2003).
This paper is an initial review and critique of the capacity of ecological integrity assessment to measure biological diversity. We first provide definitions and the general approach of ecological integrity assessment as described in the literature. Then we consider the ecological integrity index in relation to the rich scientific literature on biological diversity and its measurement, and examine the index’s reliability and robustness, its use for decision making, and robust alternative methods for measuring biological diversity. We conclude by discussing interpretation of ecological integrity assessment.
Ecological integrity assessment
An ecological integrity assessment is a multi-metric index in the form of an ecological scorecard (Faber-Langendoen et al. 2012a, b) that is intended to assess ecosystem structure, composition, function, species composition, diversity, and functional organization. It is held to be useful for measuring and monitoring biological diversity as well as ecosystem integrity (Table 1). For example, Parrish et al. (2003) claim that it can be used to track “important ecological characteristics of focal biodiversity” and synthesize their status into “a set of simple categorical ratings… of biodiversity status in an area” in order to “determine whether the status of biodiversity is responding to conservation investments and strategies.” Biota constitute the focus of ecosystem composition, one of the main conceptual aspects of integrity assessment (Andreasen et al. 2001). According to Unnasch et al. (2009), the concept of ecological integrity serves as a proxy for biological diversity, in that ecological integrity is said to be “the ability of an ecological system to support and maintain a community of organisms that has species composition, diversity, and functional organization comparable to those of natural habitats…” [italics added]. This definition clearly implies that an individual site with a high score for ecological integrity can be expected to host a typical array and abundance of biota for that site’s ecosystem type.
In the current ecological integrity assessment methodology as described by NatureServe (Faber-Langendoen et al. 2012a, b), an ecosystem’s “condition” at a particular site is expressed in terms of ranked scores for several spatial and ecological characteristics that are intended to represent ecological structure, function, and composition, in comparison to reference (benchmark) conditions for that ecosystem type (Faber-Langendoen 2012b, p. 1). The idea is that ecological integrity “can be effectively assessed using a suite of rapid assessment metrics, structured around our general ecological model” (Faber-Langendoen 2012b, p. 1). According to Faber-Langendoen et al. (2012b, p. 2), their general model incorporates three ecosystem attributes called primary attributes—size, condition, and landscape context—which are subdivided by a set of ecosystem attributes called major attributes, held to be key “components capturing the structure, composition, and processes of a system” (Faber-Langendoen et al. 2012b, p. 2). In other words, these attributes are assumed to represent reliably the biological patterns and processes of ecosystems (i.e., structure, function, and composition), including biological diversity. According to Faber-Langendoen et al. (2012b, p. 2), additional attributes of what is called biotic integrity, with birds, amphibians, and macroinvertebrates given as examples, can be included in the assessment where resources and time permit. Keystone species, rare/sensitive species, and guilds are suggested as potential indicator variables by Unnasch et al. (2009). After the set of major attributes has been chosen for a particular type of ecosystem, specific indicator or proxy variables are chosen for each of the attributes. The raw data collected for each variable or metric are converted as necessary into ordinal categories (scored excellent/good/fair/poor on a simple ranked scale, according to criteria established by NatureServe), and then weighted and combined into a single score (Parrish et al. 2003; Tierney et al. 2009; Faber-Langendoen et al. 2012a). This process is illustrated schematically in Fig. 1, and a specific example, the wetland ecological integrity index, is presented in detail in Appendix 2.
The pivotal role of vegetation in ecological integrity assessment
In their conceptual paper describing how a terrestrial ecological integrity index could be developed, Andreasen et al. (2001) stated that the focus of metrics for ecosystem composition is the biota, and therefore an important step is “selecting the biological entities to use as metrics for the index.” Plants and associated vegetation measures are at the core of the ecological integrity assessment version developed by NatureServe, whose Natural Heritage programs use vegetation as the primary means of assessing ecosystem condition (Faber-Langendoen et al. 2012a, p. 18). Vegetation thus serves as the key biological entity in ecological integrity assessment. The validity of this role is addressed in later sections that consider the empirical basis for the assumption that the chosen vegetation structure and composition attributes are consistently correlated with diversity of a range of taxa, and for the assumption that vegetation richness/diversity is correlated with richness/diversity of other groups.
Most of the vegetation metrics in the ecological integrity assessment (Faber-Langendoen et al. 2012b) are based on subjective visual estimates of vegetation structure or composition that can be recorded in a rapid site visit. Some vegetation methodology, such as the so-called floristic quality assessment, is intensive in terms of time and effort, and involves sampling in measured plots. Floristic quality assessment uses data on vascular plants, and can consist of one or more metrics that include species richness, other measures based on species richness (floristic quality index, mean coefficient of conservatism), and relative percentage of native vascular plant species at a site (Taft et al. 1997). These metrics are described in more detail in Appendix 3.
Ecological integrity assessment and the diversity of biota
The developers and advocates of ecological integrity assessment methodology have stated repeatedly that biodiversity is a major component of ecological integrity assessment (Andreasen et al. 2001; Parrish et al. 2003; Willamette Partnership 2011; Unnasch et al. 2009; Faber-Langendoen et al. 2012a, b; NatureServe 2012; Vickerman and Kagan 2014). Andreasen et al. (2001) explicitly state that ecological integrity encompasses biodiversity. Parrish et al. (2003) claim that the methodology can be used to measure the “status of biodiversity,” track “important ecological characteristics of focal biodiversity,” and determine whether the “status of biodiversity” is responding to investments (see Table 1). Unnasch et al. (2009) hold that ecological integrity assessment methodology supports conservation of native biological diversity. As described in the previous section, plant species, in themselves an important component of biological diversity, are the biological entities that play the most important role in the methodology. The assumption underlying such vegetation-based metrics is that the natural composition and structure of plant communities are optimal for supporting the range of naturally occurring wildlife (Willamette Partnership 2011, p. 10). Unimpaired ecosystem functions (which are assumed to be measured by the ecological indicators that are chosen) are also interpreted as providing habitats for naturally occurring biota (Willamette Partnership 2011, p. 11).
Given the emphasis in ecological integrity assessment on measurement and management of biological diversity (see Table 1), here we examine the methodology in relation to biological diversity. Because biological diversity has been a central theme in ecology for well over a century, a large body of work in both theoretical and applied ecology has resulted in a rich scientific literature on diversity and its measurement. Numerous standard biostatistics textbooks such as Krebs (1999), as well as more specialized texts such as Magurran (1988, 2004), cover the measurement of diversity. Thus, a wealth of information is available for examining the ecological integrity assessment methodology in light of existing literature on biological diversity.
Issues in biological diversity
In approaches to biological diversity, ecologists often study species diversity, partly because it is one of the most intuitive measures (Chiarucci et al. 2011). There are a number of statistically robust techniques for investigating species diversity. Most of the classical diversity measures are based on concepts of species richness (the number of species), evenness (the proportional abundance of each species), and differentiation (differences in species composition in an assemblage). According to Krebs (1999), the simplest measure of diversity is species richness, the number of species in an area or community. Several common statistical methods for estimating species richness include the rarefaction method, bootstrap procedures, and species–area curve (Krebs 1999). A new generation of statistical advances is based on capture–recapture methods (Williams et al. 2002a; MacKenzie et al. 2006). A common problem with species richness measures is that they are sensitive to sample size and the size of the sampled area. In particular, sampling must be random in order to avoid a biased result. Any method chosen for determining species richness must therefore control for sample size effects, species–area effects, and sampling effectiveness; the best ways to avoid the common pitfalls in measuring species richness are discussed by Gotelli and Colwell (2001). A more complex measure of diversity is heterogeneity, which combines richness and evenness in the distribution of the sampled species (the relative abundance of different species). The logarithmic series, Shannon’s index, and Simpson’s index are among the most widely used approaches (Krebs 1999). Again, samples must be random, and the sampling method chosen can strongly affect the results (Magurran 2004).
Some important considerations in conservation management that are not captured by traditional diversity measures are phylogenetic diversity (Magurran 2004; Chiarucci et al. 2011), as well as species intactness across a landscape and community structure relative to reference conditions (Lamb et al. 2009). Consideration of the mathematical properties of measures of diversity is also important in choosing an appropriate metric (Van Strien et al. 2012; Buckland et al. 2005).
Detecting trends over time in multiple taxa is especially challenging. For statistically robust longer-term monitoring, it often is better to focus on minimizing sampling error and designing robust monitoring (number of sites, duration of monitoring, sample variability, etc.) rather than monitoring a few sites intensively and many sites rapidly (Nielsen et al. 2009). MacKenzie et al. (2006) cover recent monitoring and estimation advances in depth. Magurran’s (1988, 2004) textbooks cover both traditional and new-generation methods for measuring numerous aspects of diversity.
Representing biological diversity with ecological integrity metrics
The foregoing brief overview raises questions about whether and how accurately the ecological integrity assessment measures key aspects of biological diversity. The ecological integrity assessment includes data on some remotely sensed landscape characteristics such as patch size and surrounding land use, some abiotic factors such as hydrology, and some attributes of vegetation structure and composition (see indicator variables in Table 2). The methodology relies almost entirely on proxy variables, such as structure of vegetation or the species richness of vascular plants as a proxy for diversity of a range of taxa. In this section we examine empirical evidence for the validity of these proxy relationships in representing biological diversity.
Vegetation attributes and diversity
One important question is whether the vegetation attributes as measured by indicator variables (Table 2) are correlated with biological diversity. In an ecological integrity assessment it is assumed that selected habitat measures are reliably correlated with biological diversity, and that easily measured vegetation characteristics (such as structure, composition, relative percent cover of native plant species, and other similar indicator variables [see Table 2]) can be used as surrogates for the diversity of biota. However, Faber-Langendoen et al. (2012a, b) present no evidence for a strong correlation between diversity across multiple taxa and the chosen habitat characteristics represented by indicator variables such as those in Table 2.
Empirical studies of the relationship between habitat structure and faunal diversity have had mixed success in finding any significant relationship (Williams et al. 2002 b). For example, Cushman et al. (2008) found that forest composition and structure variables and forest community type could not explain the majority of variation in the relative abundance of 53 bird species: forest vegetation measures were not reliable proxies for abundance or viability of animal populations. Psyllakis and Gillingham (2009) empirically tested how many of 55 vertebrate species could be predicted by various sets of forest structure measures (e.g., woody debris, tree and shrub species and size, stem numbers), and found that no set of structural attributes predicted multiple species. Barton et al. (2014) tested six vegetation variables (such as percent cover and species richness of different vegetation strata) and quantified their relationships to abundance, species richness, and composition of bird, mammal and reptile assemblages. They found strong and consistent relationships between vegetation overstory richness and cover and bird assemblages, but inconsistent relationships of varying strengths for any given vegetation attributes across all three study assemblages of bird, mammal and reptile taxa, suggesting that vegetation attributes were not reliable proxies for diversity across fauna taxa. Gollan et al. (2009) tested environmental factors (vegetation structure, soil) and found that none were reliably correlated with species diversity across four invertebrate orders, and hence could not be adequate proxies for the diversity of multiple invertebrate taxa. In another empirical study, Axmacher et al. (2009) found in a principal components analysis that vegetation structural attributes overall (cover, tree crown diameter, leaf shape, height, epiphyte cover) accounted for less than one-quarter of the variation in diversity patterns of 279 geometrid moth species. Because habitat-based measures tend to be specific to a given taxon, they may correlate well with occurrence of that particular species or group [such as birds and overstory vegetation richness (Barton et al. (2014); arboreal marsupials and hollow trees (Lindenmayer et al. 2014); see review by McElhinny et al. (2006) for fauna of Australian woodlands)] but are unlikely to be suitable as proxies for diversity of biota across multiple taxa. Different patterns result from the fact that different species groups perceive habitat differently and rely on different resources. Even for a species with well-known habitat requirements that include a readily measured structural or floristic element that is a limiting resource (Lindenmayer et al. 2014), accurate and reliable estimates of abundance and population trends depend on well-designed species sampling (MacKenzie et al. 2006), not simply habitat metrics (Lindenmayer et al. 2014; Pierson et al. 2016).
Vegetation richness and diversity
A second important question is whether vegetation richness/diversity is correlated with richness/diversity of other groups. In other words, is vegetation diversity a good proxy for diversity of biota across taxa and trophic levels? The floristic quality index and vascular plant species richness are considered by Wilhelm and Ladd (1988) and Taft et al. (1997) to be measures of vascular plant diversity, and their methodology is described in Faber-Langendoen et al. (2012a), but no evidence is presented in any of these sources for a consistent correlation between species richness of vascular plants and that of other biota. However, a number of studies in the refereed literature have found highly inconsistent relationships between species richness values for one taxon compared to others, including vascular plants. Studies of cross-taxon richness relationships (Prendergast and Eversham 1997; Wolters et al. 2006; Heino 2010; Eglington et al. 2012; Westgate et al. 2014) have shown that no taxon is particularly good for predicting the richness of other taxa. In other words no group—including vascular plants—has been shown empirically to be a good biodiversity indicator. For example, Kirkman et al. (2012) found that species richness of vascular plants at wetland sites was not a good predictor of species richness of the other groups, probably because biotic and abiotic processes act at different scales for different taxa, and other factors such as the number of species in neighboring patches and the number of neighboring patches also need to be accounted for. In another empirical study, Axmacher et al. (2009) found that alpha diversity of 279 geometrid moth species was not significantly correlated with overall vascular plant species richness, and hence “the diversity of vascular plants cannot universally be used as a suitable biodiversity indicator for diverse insect taxa” at the community level. While it is certain that animal species assemblages are influenced by vegetation, among other drivers, what is not so clear is the nature of the relationship between vegetation and diversity, and the degree to which the former can serve as a surrogate for the latter. The published evidence suggests that the relationship is at best scale-specific, taxon-specific, and of limited value at the scale at which habitat management is typically practiced.
Focal biodiversity and diversity
A third question is whether focal biodiversity is representative of biological diversity in an area. The use of so-called focal biodiversity to focus conservation planning efforts is intertwined with the issue of biodiversity indicators by Parrish et al. (2003) among others (see Lindenmayer et al. 2002). Parrish et al. (2003) claim that because focal biodiversity is “chosen to represent the biodiversity” of an area, an assessment of focal biodiversity is also a “measure of the status of biodiversity overall” (Parrish et al. 2003). But is the assumption that one or a small number of focal species reliably represents much of the regional biota empirically supported? Andelman and Fagan (2000) evaluated patterns of spatial co-occurrence between different biodiversity indicator species and regional biota in three conservation databases representing different scales and regions, and found that none of the various schemes (e.g., species most threatened; riparian species) captured ecologically associated species better than randomly selected indicator species. Prendergast et al. (1993), using empirical data on British plants and animals across many sites, found that species-rich areas frequently differ for different taxa, and many rare species do not occur in species-rich areas. Cushman et al. (2010) found that abundance patterns of multiple forest bird species were not consistently correlated in any of several species grouping schemes (e.g., by migratory status). The assumption that the response of indicator species will be typical of the response of many other species is not supported by evidence (Lindenmayer et al. 2002), for example because of limited species co-occurrence or varying responses to habitat disturbance. In order to use an indicator species or taxon as a reliable proxy for the presence, abundance, or richness of other taxa or for particular environmental conditions, it is essential to quantify the relationship between the indicator and what it is supposed to represent (Lindenmayer and Likens 2011). No such relationship has been demonstrated for vascular plants or other taxa at sites evaluated by ecological integrity assessment methods.
Multi-metric indices and diversity
Finally, there remains the question of whether multi-metric indices of site condition are correlated with diversity of a range of taxa. Site-condition multi-metric indices standardize, weight, and combine a variety of habitat-based variables in a single score, as distinct from habitat-based biological diversity surrogates comprising individual habitat measures (Lindenmayer et al. 2000; Psyllakis and Gillingham 2009; Oliver et al. 2014; see also foregoing section on vegetation characteristics and diversity). A reliable correlation between a multi-metric ecological score and biological diversity at a site is required for the score to be as useful in planning and management as claimed by proponents (see Table 1). There have been few tests of how diversity (Oliver et al. 2014), but empirical analyses so far have produced problematic results. Kwok et al. (2011) tested whether there was a predictable empirical relationship between either of two types of ecological scorecard for eucalypt woodlands (one based on measures of landscape function related to water run-off and soil condition; and one based on measures of vegetation structure and composition such as tree and shrub richness and cover, litter cover, log numbers) and patterns of diversity in three arthropod orders (species and family abundance, richness, and community composition). Index values from both ecological scorecards were weakly and inconsistently related to arthropod diversity in all three orders. Similarly, Oliver et al. (2014) tested the relationship of site-condition scores for three Australian multi-metric indices (based on vegetation composition and structure such as percent cover of native canopy and shrubs, litter cover, number of logs, as well as landscape-scale attributes) and species diversity data for 11 disparate taxa (1068 species of vertebrates, invertebrates, and plants). Site condition scores were not reliably related to diversity in most cases: of the 11 taxa, richness/diversity of only 2 taxa (birds and wasps) was significantly correlated with the site condition score. McGoff et al. (2013) used multivariate analyses in empirical tests of the relationship of two scoring protocols for lake shore habitat and patterns of diversity (taxon richness, Shannon-Wiener diversity index, and several macroinvertebrate metrics) in 14 macroinvertebrate taxa in 4 European regions. Although in some cases selected macroinvertebrate metrics were significantly correlated with habitat scores, there was no relationship between overall lakeshore index scores and macroinvertebrate diversity metrics across all the regions (McGoff et al. 2013). Taken together, these findings suggest that a scorecard approach is likely to be of limited use in representing patterns of biological diversity across multiple taxonomic groups, perhaps because biological processes operate at different scales for the different species using a site, and that empirical validation is essential.
Statistical robustness of ecological integrity assessment
According to Parrish et al. (2003), the ecological integrity assessment can “track important ecological characteristics” of focal biodiversity; Faber-Langendoen et al. (2012a, p. 7) state that it can be used to monitor status and trends; and Vickerman and Kagan (2014) state that it will “reveal trends.” Above and beyond the question of whether the indicators in an ecological integrity assessment actually represent diversity, there is a methodological question about the statistical reliability and repeatability of the indictors themselves. Substantial bias or high statistical variability could limit their usefulness, irrespective of their linkage to biodiversity. In this section we examine two issues related to statistical robustness, namely systematic bias and statistical power.
Systematic bias in index
Systematic bias is a concern with multi-metric indices, such as the ecological scorecard, that convert raw measurements of diverse variables into scores and weight and combine them into a single score. The indices can be subject not only to measurement error and observer bias in collecting the raw data (Gorrod et al. 2013; Dolph et al. 2010), but also to systematic bias when raw estimates are converted into categorical scores (Gorrod et al. 2013). In a study of uncertainty associated with site-condition scores for two Australian multi-metric indices based on vegetation composition and structure and used to predict value of sites for diversity of a range of taxa, Gorrod et al. (2013) found substantial and systematic underestimation of value, generated by sensitivity of the benchmark scoring intervals to observer error. The resulting bias in site scores clearly could have significant implications for conservation outcomes, especially in a market-based context (Gorrod et al. 2013). Dolph et al. (2010) used multi-metric indices of aquatic biotic integrity to demonstrate how their sensitivity to random sampling variability could lead to bias in scores and hence affect management decisions. In addition, as pointed out for multi-metric indices in general (Suter 1993; Efroymson et al. 2008), such indices do not measure real-world properties, and the index’s variance is not clearly related to a biological response; further, combining heterogeneous measures into a single index value implies that there is only a single linear scale and only a single type of response by ecosystems to disturbance. Thus, the inherent statistical properties of multi-metric indices in general tend to confound their results.
Statistical power to detect trends
A second issue concerns the statistical power needed to detect trends over time. According to its proponents, the ecological integrity assessment is useful to detect ecological change and reveal trends when repeated measures are made (Table 1). These statements notwithstanding, distinguishing real change from natural variation (e.g., spatial or temporal variation) and measurement or sampling error (e.g., detectability problems), requires a careful and sophisticated survey design. A critical issue is the level of survey effort required to achieve enough precision to identify (quantify) trends (MacKenzie et al. 2006; Magurran et al. 2010). For a single site, a key is the length of the time series and the precision of measurement at each point (Magurran et al. 2010). Power calculations or determination of confidence intervals can be used to estimate the level of survey effort needed (Magurran et al. 2010). Consideration of statistical power is not mentioned as part of ecological integrity assessment methodology in Faber-Langendoen et al. (2012a, b). Without explicit design for adequate statistical power, there is no assurance that the ecological integrity index can distinguish among directional change in diversity, natural variation over time, and measurement error.
Reliability of the vegetation metrics in ecological integrity assessment
In sampling plants and animals alike, detectability and spatial variation are well known issues in statistical methodology (Yoccoz et al. 2001), as is the need to account for them in sampling (Williams et al. 2002a; MacKenzie et al. 2006). Reliability of the vegetation metrics in ecological integrity analysis is important because measures of vascular plants are the main measures of biological diversity and of ecosystem composition in NatureServe’s characterization of the methodology. These metrics have been the core of the approach from its earliest development by Wilhelm (1977), Taft et al. (1997), and Wilhelm and Ladd (1988). In this section we examine concerns about various sources of error in the vegetation methodology of ecological integrity assessments.
Bias in richness estimates
One concern is bias in estimates of plant species richness. Species richness measures are sensitive to the number of individuals sampled, and to the number, size, and spatial arrangement of samples (Gotelli and Colwell 2011). The single-plot sampling of vascular plant species richness at a wetland site in Faber-Langendoen et al. (2012a) cannot provide any indication of variability across the site. Without multiple plots, it is impossible to determine the mean and variance of index values among plots (Magurran 1988) and thus to examine whether the sample data are actually representative. Any individual plot cannot characterize site variation in plant communities accurately (Bourdaghs et al. 2006). However, increasing the area sampled has its own pitfalls, because increasing the size of the sampling area also increases species richness estimates (Krebs 1999) as well as the mean coefficient of conservatism and floristic quality index (Matthews et al. 2005; Bourdaghs et al. 2006) which are based on species richness. Species richness of an entire site, rather than one plot in the site, is often used in floristic quality assessment (Nichols 1999), following the protocols defined by Wilhelm and Ladd (1988) and cited by Faber-Langendoen et al. (2012a) as the basis for floristic quality assessment. In such a case, variation in site size can lead to variation in estimates of species richness and related measures such as mean coefficient of conservatism. These sources of bias can easily confound the estimation of a site’s conservation value.
Species detection error
A second concern is sampling error in detecting plant species presence. Vegetation measures are highly susceptible to detectability problems. Numerous studies have empirically tested detectability of plants (Chen et al. 2013; Moore et al. 2011; Per et al. 2008; Archaux et al. 2006; Ringvall et al. 2005; Klimes 2003; Milberg et al. 2008) in vegetation monitoring, including the presence/absence sampling and visual estimates of cover that are used in ecological integrity assessment. In surveys of plant species presence and distribution, detection error is the norm rather than the exception: in some surveys a high percentage of species (e.g. 20–34 %) were overlooked, and detection probability varied with observer, sampling time, season, and spatial location. If survey designs do not explicitly incorporate detection probability, serious bias in estimates of species distributions and species richness can result. Accommodation for sampling error is not mentioned by Faber-Langendoen et al. (2012a), and hence the vegetation metrics are likely to be subject to bias.
Bias in coefficient of conservatism
Another concern is subjectivity (observer variability) in assigning coefficients of conservatism. Coefficients of conservatism assigned to plant species are qualitative and subjectively determined on the basis of professional judgment, rather than objectively assigned. The reason given is that the taxa “span a range that is too broad…for any objective natural sorting to serve as a guide to species rankings” (Taft et al. 1997). Because of this subjectivity, the metric is highly susceptible to inter-observer variability and bias. Land and Chiarucci (2010) specifically tested inter-observer variation in coefficients of conservatism, and found that scores given by different experts were not consistent and resulted in derived floristic quality indices that were statistically different. Inter-observer variation within NatureServe species databases that contain pre-assigned coefficients of conservatism apparently has not been examined by Faber-Langendoen et al. (2012a) and could result in bias.
There are other potential problems with the coefficient of conservatism in addition to those mentioned above. The metric is computed by averaging predetermined conservatism values for all the native species observed at a site, and then assigning the resulting average value to one of a few ordinal categories (see Appendix 3). One problem is that the coefficient of conservatism is based on species occurrence at a single site, which may or may not represent the larger area to which inference is made. The absence of any accounting of spatial variability associated with it (Magurran 1988) limits its usefulness for broader assessments. In addition, the metric is based on observed rather than actual occurrence, with no accommodation for sampling error in detecting plant species presence (see above discussion of this topic). Also, sensitivity of the metric to cut-off points used to distinguish categories is a concern that is yet to be investigated. Finally, even under ideal circumstances, the metric would be at best an ambiguous measure of native plant diversity: a large mean C at a site can be obtained when just one or a few high-value species are observed, and a much smaller mean C can be obtained from the range of conservatism values resulting when numerous species are observed. Among other things this means the coefficient of conservatism can actually be inversely related to diversity of native plants at a site.
Ecological integrity assessment and decision making
The foregoing issues affect the usefulness of ecological integrity assessment in decision making. Assessments are held by some to be useful in land-use planning, in the prioritization of sites for mitigation (Faber-Langendoen et al. 2012a, p. 7), in the identification of sites for “conservation/management actions” (Faber-Langendoen et al. 2012a, p. 9), in conservation planning (Unnasch et al. 2009), in assessment of management effectiveness (Tierney et al. 2009), and in the allocation of resources and management decisions (Vickerman and Kagan 2014). One example would be a comparative assessment of sites for land-use decisions in environmental markets and trading schemes, such as those described by Salzman and Ruhl (2000). It is unclear how an ecological integrity assessment could play these important roles. Whatever else is involved, decision making is based on a comparison of alternative management choices against value-based criteria, so as to allow one to recognize value differences among alternatives. From the regulatory point of view, it is important that objectives are clear and acceptable, and measures allow discrimination among alternative outcomes on the basis of the scientific method, including hypothesis testing. NatureServe’s ecological integrity assessment in its present form may be unable to satisfy these requirements, due to the many potential sources of error and sampling bias that are unaccounted for and the ambiguity about what ecological attributes are actually represented by the index. For example, any metric used by federal agencies in a context of the U.S. Endangered Species Act (ESA) would need to be able to withstand legal scrutiny during judicial review. In the ESA context, 25 of 32 listing decisions reviewed by the court in 2003 were set aside, many because the U.S. Fish and Wildlife Service did not use “best available science” (Wilde 2014). It is unlikely that an ecological integrity assessment as currently described and justified would meet the necessary criteria to withstand such scrutiny.
Robust alternative measures of biological diversity
Biodiversity measures can indeed provide valuable empirical metrics for assessing change (Buckland et al. 2005). But one size definitely does not fit all—there is no universally appropriate generic monitoring program, and an optimal design must always be tailored to a particular situation and purpose (Ferraz et al. 2008). Different indices measure different aspects of diversity, so it is essential to define the objectives of any monitoring program clearly in order to choose an appropriate index (Yoccoz et al. 2001; Pollock et al. 2002). Recent biometric research on how to measure and monitor biological diversity has proliferated, and includes work on appropriate statistical criteria, new indices and methods, design of monitoring programs, and methodological comparisons (Yoccoz et al. 2001; Magurran 2004; Lamb et al. 2009; Buckland et al. 2005; MacKenzie et al. 2006; Magurran et al. 2010; Magurran and McGill 2011; Dornelas et al. 2013). While it is beyond the scope of this paper to review the statistical literature, here we cover some important considerations in designing statistically robust surveys of biological diversity.
The size of the area and the number of taxa of interest affect the design of a monitoring program. For monitoring change in biological diversity over time in a wide heterogeneous region, Buckland et al. (2005) set out six criteria to evaluate indices for detecting temporal change. They reviewed performance of several measures (Shannon’s index, Simpson’s index, arithmetic mean of relative abundance indices, geometric mean of relative abundance indices) against the criteria and found that Shannon’s index and the geometric mean performed best. Yoccoz et al. (2001) and Pollock et al. (2002) suggested that in certain circumstances species richness is an adequate measure, absent an accounting for abundance. Lamb et al. (2009) examined diversity indices for monitoring temporal change over large areas. They evaluated 13 diversity indices of 3 types (traditional, community [species] intactness based on occurrence, community [species] intactness based on abundance) against several criteria such as sensitivity to detection error and power to detect trends, in six ecological scenarios. They found that the intactness index based on Buckland’s (2005) arithmetic mean of relative abundance indices performed best (Lamb et al. 2009). Monitoring of single sites with unique habitats or rare species could be treated by collecting data only on the specialist species of interest (Buckland et al. 2005) or by conducting single-species surveys (Magurran et al. 2010).
Any serious attempt to monitor biological diversity should address the two main sources of error, detectability and spatial variation (or environmental heterogeneity) (Yoccoz et al. 2001; Williams et al. 2002a; Buckland et al. 2005). Detection error results when some individual organisms or species evade detection during a survey. Distance sampling and capture–recapture are two types of methodology that can be used to estimate detection probabilities associated with count statistics (Yoccoz et al. 2001; Williams et al. 2002a; MacKenzie et al. 2006). While these methods may be more appropriate for one or a few sites, Pollock et al. (2002) discussed methods to measure and incorporate detectability in large-scale studies at multiple points in space and time. Survey errors result when inferences about a larger area are not based on an appropriate spatial design for sampling smaller sites. A particular problem is the use of subjectively chosen sampling sites, which can lead to biased estimates of diversity at the larger scale (Yoccoz et al. 2001). Yoccoz et al. (2001) and Williams et al. (2002a) discussed some of the many recent statistical advances in sampling design, with survey features that can be customized for the taxa of interest.
A survey that is properly designed can quantify the uncertainty and precision of diversity measures, and thus their reliability (Buckland et al. 2005; MacKenzie et al. 2006). Determining the survey effort needed for sufficient precision to quantify change over time is key to producing unbiased results (Buckland et al. 2005; Magurran et al. 2010). The effort needed for a given level of precision can be estimated in advance by power calculations. How long the time series is, and how precise the measurements are at each time, are important factors at a single site (Magurran et al. 2010). In addition, the number of plots, plot size, and frequency of sampling also determine precision for multi-site sampling of a larger area (Magurran et al. 2010). In a case study, Nielsen et al. (2009) examined statistical power for detecting simulated species declines in several monitoring scenarios, given different numbers of sites coupled with different sampling intervals. Dornelas et al. (2014) reviewed other general issues involved in quantifying trends in biological diversity over time.
Broad applicability and cost-effectiveness are both important considerations in implementing large-scale biological diversity monitoring programs. Use of common monitoring designs across global regions was recommended by Buckland et al. (2005), in order to provide greater scope in measuring changes, as well as economies of scale. They suggested designing surveys such that entry at various levels is possible, thus allowing nations with fewer resources to take part, perhaps with design modifications such as a subset of species, lower sampling rates, and simpler methods (Buckland et al. 2005). If the monitoring program and sampling protocols are well-designed and well-coordinated, professionals are not necessarily needed to collect all data (Magurran et al. 2010; Tulloch et al. 2013). With adequate training and supervision, non-professional volunteers can be as proficient as professionals in many tasks, as shown by a number of quantitative evaluations of volunteer-collected data [e.g., in mammal surveys (Newman et al. 2003), amphibian call surveys (Genet and Sargent 2003), or shark counts (Ward-Paige and Lotze 2011)]. However, generating high-quality data from “citizen science” involves other costs associated with coordination, communication, and data quality control (Tulloch et al. 2013), and it is important to keep in mind that adhering to statistical and ecological principles remains essential (Buckland et al. 2005; Lamb et al. 2009; Magurran et al. 2010; Tulloch et al. 2013).
The investigation of ecological integrity addresses a critical need for usable information to help stem the accelerating loss of biological diversity. But we believe there are serious and unaddressed concerns about the suitability of ecological integrity assessment as described by Faber-Langendoen et al. (2012a, b) for measuring diversity and detecting trends over time. For example, the vegetation sampling methods—including protocols for estimating plant species richness—are susceptible to sampling error and observer bias. Vascular plant diversity, which is used as a key proxy for biological diversity, is not a reliable indicator of diversity of other taxa and has no demonstrated relationship to measures of cross-taxon diversity at a site. The empirical studies discussed earlier point to serious difficulties in using these indicators as surrogates for biological diversity. In fact, patterns of biological diversity in landscapes, however they are represented, are too complex to be represented effectively with these indicators. And there is no evidence that the ecological integrity assessment protocols as currently designed can resolve problems of detectability and environmental heterogeneity in distinguishing natural variation from ecological change over time.
A related issue that merits discussion is whether expansion of the current index to include taxa other than vascular plants would improve its suitability for measuring biological diversity. Proponents hold that the ecological integrity index can be extended as necessary to include other components of the biota such as birds or amphibians (Faber-Langendoen et al. 2012b, p. 2); keystone species, rare/sensitive species or guilds (Unnasch et al. 2009, p. 27); and unique native species or vertebrate species (Vickerman and Kagan 2014, p. 16). In our opinion, the inclusion of additional taxa would still leave ecological integrity assessment in its present form an inadequate measure of biological diversity. First, none of the current sampling protocols described in Faber-Langendoen et al. (2012a, b) address the issues of detectability and environmental heterogeneity in sampling for plants, and these problems are even harder to resolve when sampling animal populations (Williams et al. 2002a). Second, as discussed previously, no taxon or group has been shown empirically to be a reliable proxy for diversity of a range of taxa. Third, the process of converting raw data to numerical scores and then weighting them in an assessment can introduce bias (Gorrod et al. 2013). A noteworthy point is that in the various publications describing ecological integrity assessment, proponents of the methodology (Parrish et al. 2003; Willamette Partnership 2011; Unnasch et al. 2009; Faber-Langendoen et al. 2012a, b; NatureServe 2012; Vickerman and Kagan 2014) have yet to include a worked example for any species other than vascular plants, although the word biodiversity is frequently mentioned.
Measuring diversity of biota and estimating the conservation importance of any given site requires more than a few proxy variables or indicator species. There is a large body of recent literature on diversity theory (reviewed by Bestelmeyer et al. 2003), indicating that the biological processes creating biological patterns operate at different scales for different species. Many factors interact to determine animal diversity patterns, including competition, territoriality, dispersal, predation, physical environmental variation (especially landscape-scale gradients and patchiness), and historical variation in biogeography (Bestelmeyer et al. 2003). For individual species, distribution patterns across scales are determined by habitat requirements, dispersal capabilities, and the size and location of the geographic range. Thus, measuring habitat variables at a given site is not sufficient for monitoring population viability or abundance of even a single species (MacKenzie et al. 2006). By extension, scorecards at individual sites are not sufficient to explain distribution patterns across sites. For biological diversity as a whole, patterns of species diversity are strongly influenced by spatial heterogeneity in a scale-dependent way (Williams et al. 2002b), which can potentially result in a strong association of habitat features with one or a few species but a weak association with diversity of multiple taxonomic groups. However, ecological integrity assessment largely ignores these issues, and assumes instead that habitat and landscape features at individual sites fully account for diversity of biota in a predictable way. Inaccurate estimates of the conservation value of a site could easily result from this unproven assumption.
While we applaud efforts to address important environmental issues with an ecological integrity assessment, further development is needed, especially in the areas of technical refinements and validation. In its current form the methodology is of limited use in providing meaningful metrics of biological diversity, and lacks a foundation in ecological and statistical principles. At a minimum, sources of sampling error—especially organism detectability and spatial variation—should be investigated and accounted for, along with the potential for bias and loss of essential information due to condensing such a large amount of disparate data into a single index. Further empirical investigation is needed to quantify how well the indicator variables and metrics are correlated with the particular ecological processes or environmental conditions they are supposed to represent (Lindenmayer and Likens 2011; Lindenmayer et al. 2014). Relationships between indicator variables and biological diversity attributes should be quantified to determine the transferability of a given indicator to the biotic component for which it is used as a proxy (Lindenmayer and Likens 2011). More effective development of a metric will require greater collaboration of statisticians, landscape ecologists, and theoretical ecologists.
A broader acceptance will necessitate evidence from the refereed scientific literature that ecological integrity assessment actually measures biological diversity. In particular, the methodology should be subjected to ongoing critical review in the literature to a much greater extent than it has been to date. Empirical studies such as those suggested in the previous paragraph should be undertaken, with results published in peer-reviewed journals. In addition to a more thorough investigation of the assessment in relation to biological diversity than this paper permits, the linkage between ecological integrity assessment and ecological structure and function should also be investigated, for example by identifying criteria with respect to the use of various proxy variables, by which to assess index performance; conducting a comprehensive literature review to identify evidence for each criterion; and subjecting the evidence to formal analysis.
With improvements in methodology and thoughtful choices, measuring biological diversity can produce unbiased results that reflect real change rather than sources of error, and provide the accurate assessments necessary for effective conservation decisions.
Andelman SA, Fagan WF (2000) Umbrellas and flagships: efficient conservation surrogates or expensive mistakes? Proc Natl Acad Sci 97:5954–5959
Andreasen JK, O’Neill RV, Noss R, Slosser NC (2001) Considerations for the development of a terrestrial index of ecological interity. Ecol Ind 1:21–35
Archaux F, Gosselin F, Berges L, Chevalier R (2006) Effects of sampling time, species richness and observer on the exhaustiveness of plant censuses. J Veg Sci 17:299–306
Axmacher JC, Brehm G, Hemp A, Tuente H, Lyaruu HVM, Mueller-Hohenstein K, Fiedler K (2009) Determinants of diversity in afrotropical herbivorous insects (Lepidoptera: Geometridae): plant diversity, vegetation structure or abiotic factors? J Biogeogr 36:337–349
Barton PS, Westgate MJ, Lane PW, MacGregor C, Lindenmayer DB (2014) Robustness of habitat-based surrogates of animal diversity: a multitaxa comparison over time. J Appl Ecol 51:1434–1443
Bestelmeyer BT, Miller JR, Wiens JA (2003) Applying species diversity theory to land management. Ecol Appl 13:1750–1761
Bourdaghs M, Johnston CA, Regal RR (2006) Properties and performance of floristic quality index in Great Lakes coastal wetlands. Wetlands 26:718–735
Buckland ST, Magurran AE, Green RE, Fewster RM (2005) Monitoring change in biodiversity through composite indices. Philos Trans R Soc B 360:243–254
Cardinale B (2011) Biodiversity improves water quality through niche partitioning. Nature 472:86–89
Cardinale B, Duffy JE, Gonzalez A, Hooper DU, Perrings C, Vanail P, Narwani A, Mace GM, Tilman D, Wardle DA, Kinzig AP, Daily GC, Loreau M, Grace JB, Larigauderie A, Srivastave DS, Naeem S (2012) Biodiversity loss and its impact on humanity. Nature 485:59–67
Ceballos G, Ehrlich PR, Barnosky AD, Garcia A, Pringle RM, Palmer TM (2015) Accelerated modern human-induced species losses: entering the sixth mass extinction. Sci Adv 1: 1–5. http://advances.sciencemag.org/content/1/5/e1400253.full
Chen G, Kery M, Plattner M, Ma K, Gardner B (2013) Imperfect detection is the rule rather than the exception in plant distribution studies. J Ecol 101:183–191
Chiarucci A, Bacaro G, Scheiner SM (2011) Old and new challenges in using species diversity for assessing biodiversity. Philos Trans R Soc B 366:2426–2437
Civitello DJ, Cohen C, Fatima H, Halstead NT, Liriano J, McMahob TA, Ortega CN, Sauer EL, Sehgal T, Young S, Rohr JR (2015) Biodiversity inhibits parasites: broad evidence for the dilution effect. Proc Natl Acad Sci 112:8667–8671
Cushman SA, McKelvey KS, Flather CH, McGarigal K (2008) Do forest community types provide a sufficient basis to evaluate biological diversity? Front Ecol Environ 6:13–17
Cushman SA, McKelvey KS, Noon BR, McGarigal K (2010) Use of abundance of one species as a surrogate for abundance of others. Conserv Biol 24:830–840
Dolph CL, Sheshukov AY, Chizinski CJ, Vondracek B, Wilson B (2010) The index of biological integrity and the bootstrap: can random sampling error affect stream impairment decisions? Ecol Ind 10:527–537
Dornelas M, Magurran AE, Buckland ST, Chao A, Chazdon RL, Colwell RK, Curtis T, Gaston KJ, Gotelli NJ, Kosnik MA, McCune JL, Morlon H, Mumby PJ, Ovreas L, Studeny A, Vellend M (2013) Quantifying temporal change in biodiversity: challenges and opportunities. Proc R Soc B 280:1–10
Efroymson RA, Peterson MJ, Welsh CJ, Druckenbrod DL, Ryon MG, Smith JG, Hargrove WW, Giffen NR, Roy WK, Quarles HD (2008) Investigating habitat value to inform contaminant remediation options: approach. J Environ Manag 88:1436–1451
Eglington SM, Noble DG, Fuller RJ (2012) A meta-analysis of spatial relationships in species richness across taxa: birds as indicators of wider biodiversity in temperate regions. J Nat Conserv 20:301–309
Faber-Langendoen D, Hedge C, Kost M, Thomas S, Smart L, Smyth R, Drake J, Menard S (2012a) Assessment of wetland ecosystem condition across landscape regions: a multi-metric approach. Part A. Ecological integrity assessment overview and field study in Michigan and Indiana. U.S. Environmental Protection Agency report EPA/600/R-12/021a, Washington, DC
Faber-Langendoen D, Rocchio J, Thomas S, Kost M, Hedge C, Nichols B, Walz K, Kittel G, Menard S, Drake J, Muldavin E (2012b) Assessment of wetland ecosystem condition across landscape regions: a multi-metric approach. Part B. Ecological integrity assessment protocols for rapid field methods (L2). U.S. Environmental Protection Agency report EPA/600/R-12/021b, Washington, DC
Federal Geographic Data Committee (2008) National vegetation classification standard. FGDC-STD-005-2008. https://www.fgdc.gov/standards/projects/FGDC-standards-projects/vegetation/NVCS_V2_FINAL_2008-02.pdf. Accessed February 2016
Ferraz G, Marinelli CE, Lovejoy TE (2008) Biological monitoring in the Amazon: recent progress and future needs. Biotropica 40:7–10
Genet KS, Sargent LG (2003) Evaluation of methods and data quality from a volunteer-based amphibian call survey. Wildl Soc Bull 31:703–714
Gollan JR, Ashcroft MB, Cassis G, Donnelly AP, Lassau SA (2009) Testing common habitat-based surrogates of invertebrate diversity in a semi-arid rangeland. Biodivers Conserv 18:1147–1159
Gorrod EJ, Bedward M, Keith DA, Ellis MV (2013) Systematic underestimation resulting from measurement error in score-based ecological indices. Biol Conserv 157:266–276
Gotelli NJ, Colwell RK (2001) Quantifying biodiversity: procedures and pitfalls in the measurement and comparison of species richness. Ecol Lett 4:379–391
Gotelli NJ, Colwell RK (2011) Estimating species richness. In: Magurran AE, McGill BJ (eds) Biological diversity: frontiers in measurement and assessment. Oxford University Press, Oxford, pp 39–54
Hector A (2011) Diversity favours productivity. Nature 472:45–46
Heino J (2010) Are indicator groups and cross-taxon congruence useful for predicting biodiversity in aquatic ecosystems? Ecol Ind 10:112–117
Karanth KU, Nichols JD, Seidensticker J, Dinerstein E, David Smith JL, McDougal C, Johnsingh AJT, Chundawat RS, Thapar V (2003) Science deficiency in conservation practice: the monitoring of tiger populations in India. Anim Conserv 6:141–146
Kirkman LK, Smith LA, Quintana-Ascencio PF, Kaeser MJ, Golladay SW, Farmer AL (2012) Is species richness congruent among taxa? Surrogacy, complementarity, and environmental correlates among three disparate taxa in geographically isolated wetlands. Ecol Ind 18:131–139
Klimes L (2003) Scale-dependent variation in visual estimates of grassland plant cover. J Veg Sci 14:815–821
Krebs CJ (1999) Ecological methodology, 2nd edn. Benjamin/Cummings, Menlo Park
Kwok AB, Eldridge DJ, Oliver I (2011) Do landscape health indices reflect arthropod diversity status in the eucalypt woodlands of eastern Australia? Aust Ecol 36:800–813
Lamb EG, Bayne E, Holloway G, Schieck J, Boutin S, Herbers J, Haughland DL (2009) Indices for monitoring biodiversity change: are some more effective than others? Ecol Ind 9:432–444
Land S, Chiarucci A (2010) Is floristic quality assessment reliable in human-managed ecosystems? Syst Biodivers 8:269–280
Lindenmayer DB, Likens GE (2011) Direct measurement versus surrogate indicator species for evaluating environmental change and biodiversity loss. Ecosystems 14:47–59
Lindenmayer DB, Margules CR, Botkin DB (2000) Indicators of biodiversity for ecologically sustainable forest management. Conserv Biol 14:941–950
Lindenmayer DB, Manning AD, Smith PL, Possingham HP, Fischer J, Oliver I, McCarthy MA (2002) The focal-species approach and landscape restoration: a critique. Conserv Biol 16:338–345
Lindenmayer DB, Barton PS, Lane PW, Westgate MJ, McBurney L, Blair D, Gibbons P, Likens GE (2014) An empirical assessment and comparison of species-based and habitat-based surrogates: a case study of forest vertebrates and large old trees. PLoS One 9:1–10
MacKenzie DI, Nichols JD, Royle JA, Pollock KH, Bailey LL, Hines JE (2006) Occupancy estimation and modeling: inferring patterns and dynamics of species occurrence. Academic Press, Oxford
Magurran AE (1988) Ecological diversity and its measurement. Princeton University Press, Princeton
Magurran AE (2004) Measuring biological diversity. Blackwell, Oxford
Magurran AE, McGill BJ (eds) (2011) Biological diversity: frontiers in measurement and assessment. Oxford University Press, Oxford
Magurran AE, Baillie SR, Buckland ST, McP Dick J, Elston DA, Scott EM, Smith RI, Somerfield PJ, Watt AD (2010) Long-term datasets in biodiversity research and monitoring: assessing change in ecological communities through time. Trends Ecol Evol 25:574–582
Margules CR, Pressey RL (2000) Systematic conservation planning. Nature 405:243–253
Matthews JW, Tessene RA, Wiesbrook SM, Zercher BW (2005) Effect of area and isolation on species richness and indices of floristic quality in Illinois, USA wetlands. Wetlands 25:607–615
McGoff M, Aroviita J, Pilotto F, Miler O, Solimini AG, Porst G, Jurca T, Donohue L, Sandin L (2013) Assessing the relationship between the Lake Habitat Survey and littoral macroinvertebrate communities in European lakes. Ecol Ind 25:205–214
McIlhinny C, Gibbons P, Brack C, Bauhus J (2006) Fauna–habitat relationships: a basis for identifying key stand structural attributes in temperate Australian eucalypt forests and woodlands. Pac Conserv Biol 12:89–110
Milberg P, Bergstedt J, Fridman J, Odell G, Westerberg L (2008) Observer bias and random variation in vegetation monitoring data. J Veg Sci 19:633–644
Moore JL, Hauser CF, Bear JL, Williams NSG, McCarthy MA (2011) Estimating detection–effort curves for plants using search experiments. Ecol Appl 21:601–607
NatureServe (2012) Rapid measurement of ecological integrity: measuring habitat/biodiversity outcomes across jurisdictions and scales. Powerpoint presentation “Ecological integrity monitoring framework” http://conservationregistry.org/projects/217753. Accessed February 2016
Newman C, Buesching CD, Macdonald DW (2003) Validating mammal monitoring methods and assessing the performance of volunteers in wildlife conservation—“Sed quis custodiet ipsos custodies?” Biol Conserv 113:189–197
Nichols SA (1999) Floristic quality assessment of Wisconsin lake plant communities with example applications. J Lake Reserv Manag 15:133–141
Nielsen SE, Haughland DL, Bayne E, Shieck J (2009) Capacity of long-term biodiversity monitoring programmes to detect trends in species prevalence. Biodivers Conserv 18:2961–2978
Oliver I, Eldridge DJ, Nadolny C, Martin WK (2014) What do site condition multi-metrics tell us about species biodiversity? Ecol Ind 38:262–271
Parrish JD, Braun DP, Unnasch RS (2003) Are we conserving what we say we are? Measuring ecological integrity within protected areas. Bioscience 53:851–860
Pierson JS, Mortelliti A, Barton PS, Lane PW, Lindenmayer DB (2016) Evaluating the effectiveness of overstory cover as a surrogate for bird community diversity and population trends. Ecol Ind 61:790–798
Pindilli E, Casey F (2015) Biodiversity and habitat markets—policy, economic, and ecological implications of market-based conservation. U.S. Geological Survey circular 1414. http://pubs.usgs.gov/circ/1414/circ1414.pdf. Accessed March 2016
Pollock KH, Nichols JD, Simons TR, Farnsworth GL, Bailey LL, Sauer JR (2002) Large scale wildlife monitoring studies: statistical methods for design and analysis. Environmetrics 13:105–119
Prendergast JR, Eversham BC (1997) Species richness covariance in higher taxa: empirical tests of the biodiversity indicator concept. Ecography 20:210–216
Prendergast JR, Quinn RM, Lawton JH, Eversham BC, Gibbons DW (1993) Rare species, the coincidence of biodiversity hotspots and conservation strategies. Nature 365:335–337
Psyllakis JM, Gillingham MP (2009) Using forest structure and composition to predict the occurrence of vertebrate species in Douglas-Fir forests of British Columbia. Biol Conserv 142:1427–1441
Ringvall A, Petersson H, Stahl G, Lamas T (2005) Surveyor consistency in presence/absence sampling for monitoring vegetation in a boreal forest. For Ecol Manag 212:109–117
Salzman J, Ruhl JB (2000) Currencies and the commodification of environmental law. Stanf Law Rev 53:607–694
Suter GW II (1993) A critique of ecosystem health concepts and indexes. Environ Toxicol Chem 12:1533–1539
Swink F, Wilhelm G (1994) Plants of the Chicago region. Indiana Academy of Science, Indianapolis
Taft JB, Wilhelm GS, Ladd DM, Masters LA (1997) Floristic quality assessment for vegetation in Illinois: a method for assessing vegetation integrity. Erigenia J Ill Nativ Plant Soc J 15:3–95
Tierney GL, Faber-Langendoen D, Mitchell BR, Shriver WG, Gibbs JP (2009) Monitoring and evaluating the ecological integrity of forest ecosystems. Front Ecol Environ 7:308–316
Tscharntke T, Clough Y, Wanger TC, Jackson L, Motzke I, Perfecto I, Vandermeer J, Whitbread A (2012) Global food security, biodiversity conservation and the future of agricultural intensification. Biol Conserv 151:53–59
Tulloch AIT, Possingham HP, Joseph LN, Szabo J, Martin TG (2013) Realising the full potential of citizen science monitoring programs. Biol Conserv 165:128–138
Unnasch RS, Braun DP, Comer PJ, Eckert GE (2009) The Ecological Integrity Assessment framework: assessing the ecological integrity of biological and ecological resources of the national park system. Report to the National Park Service. http://www.sound-science.org/NPS_Ecological%20Integrity%20Framework.pdf. Accessed February 2016
U.S. Senate Finance Committee (2005) Report on staff investigation of The Nature Conservancy. Senate Print 109-27. U.S. Government Printing Office, Washington, DC
Van Strien AJ, Soldaat LL, Gregory RD (2012) Desirable mathematical properties of indicators for biodiversity change. Ecol Ind 14:202–208
Vickerman S and Kagan J (2014) Assessing ecological integrity across jurisdictions and scales. http://www.defenders.org/sites/default/files/publications/assessing-ecological-integrity-across-jurisdictions-and-scales.pdf. Accessed February 2016
Ward-Paige CA, Lotze HK (2011) Assessing the value of recreational divers for censusing elasmobranchs. PLoS One 6:e25609
Westgate MJ, Barton PS, Lane PW, Lindenmayer DB (2014) Global meta-analysis reveals low consistency of biodiversity congruence relationships. Nat Commun 5:3899
Wilde C (2014) Evaluating the Endangered Species Act: trends in mega-petitions, judicial review, and budget constraints reveal a costly dilemma for species conservation. Villanova Environ Law J 25:307–349
Wilhelm GS (1977) Ecological assessment of open land areas in Kane County, Illinois. Kane County Urban Development Division, Geneva, IL
Wilhelm G, Ladd D (1988) Natural area assessment in the Chicago region. In: Transactions of the 53rd North American Wildlife and Natural Resources Conference, pp. 361–375
Willamette Partnership (2011) Measuring up: synchronizing biodiversity measurement systems for markets and other incentive programs. http://willamettepartnership.org/wp-content/uploads/2015/04/Measuring-Up-w-appendices-final.pdf. Accessed February 2016
Williams BK, Nichols JD, Conroy MJ (2002a) Analysis and management of animal populations. Academic Press, San Diego
Williams SE, Marsh H, Winter J (2002b) Spatial scale, species diversity, and habitat structure: small mammals in Australian tropical rain forest. Ecology 83:1317–1329
Wolters V, Bengtsson J, Zaitsev A (2006) Relationship among the species richness of different taxa. Ecology 87:1886–1895
Yoccoz NG, Nichols JD, Boulinier T (2001) Monitoring of biological diversity in space and time. Trends Ecol Evol 16:446–453
We thank the USGS Science and Decisions Center for support for B.K.W. during preparation of this paper. John Sauer provided an incisive review of the manuscript, and two anonymous reviewers provided useful comments.
Communicated by Karen E. Hodges.
Appendix 1: Data sources and development of methodology
Andreasen et al. (2001) described development of a terrestrial index of ecological integrity in concept, including many of the basic ideas underpinning the latest version of ecological integrity assessment. Much of the methodology in NatureServe’s current ecological integrity assessment was developed from databases and previous assessments in The Nature Conservancy’s Natural Heritage Network program (Faber-Langendoen et al. 2012a). These databases, containing information about the existence and location of species and natural communities in the U.S., originated from a biological inventory conducted by The Nature Conservancy in the 1970s (U.S. Senate Finance Committee 2005). In 2000, the Natural Heritage Network became a separate and independent organization, renamed NatureServe (U.S. Senate Finance Committee 2005), and now maintains the databases of native ecosystems. It also served as the source of the “best professional judgment and with a minimal amount of quantitative information” by which to evaluate and rate the ecological integrity of individual sites in the databases (Faber-Langendoen et al. 2012a, p. 3). During subsequent development of an ecological integrity assessment for wetlands (Faber-Langendoen et al. 2012a, b) the original subjective site assessments were combined with other assessment information and GIS and remotely sensed data. A number of field-based measures were defined, along with specific protocols for data collection in the office and in the field. Floristic quality assessment metrics (see Appendix 3), originally incorporated in Natural Heritage Network databases, were also used because they putatively “provide a powerful and relatively easy assessment of the integrity of both biotic and abiotic processes” (Faber-Langendoen et al. 2012a, p. 119).
According to Faber-Langendoen et al. (2012b, p. 1), methodology is being developed by NatureServe for different ecosystem-specific ecological integrity assessments for forests, grasslands, wetlands, and other ecosystems, including the full protocols (sampling design, field methods, calculations, ranking criteria, etc.) for producing and applying the measures. Major ecosystem attributes and/or their indicator variables can be added for specific ecological types, depending on available time and funding. For example, some of the ecosystem attributes and indicator variables used to express “condition” of temperate forests would be different from those used for wetlands. The Environmental Protection Agency (EPA) Office of Research and Development has funded development of protocols for the latest version of ecological integrity assessment, which involves assessment of Midwestern wetlands (Faber-Langendoen et al. 2012a, b). The National Park Service (NPS) previously funded development of monitoring frameworks for ecological integrity of some forest ecosystems in the NPS Inventory and Monitoring Program (Tierney et al. 2009), as well as an overview of how ecological integrity assessment methodology could be used in the context of biological resource conservation and management by the NPS (Unnasch et al. 2009).
Appendix 2: Specific example—wetland ecological integrity assessment
NatureServe’s recent two-part report to the EPA (Faber-Langendoen et al. 2012a, b) states that “here…we are publishing the latest version (version 3.0) of our Ecological Integrity Assessment” (Faber-Langendoen et al. 2012b, p. iv). In particular, this report contains a description of the assessment methodology and an accompanying field study. According to Faber-Langendoen et al. (2012a), wetland ecosystem integrity can be assessed by measuring indicators of 6 abiotic and biotic attributes that are held to express an ecosystem’s landscape context, size, and condition, and are intended to represent wetland ecosystem structure, composition, and function. The attributes are (1) landscape context, (2) surrounding buffer, (3) patch size, (4) vegetation condition, (5) hydrology condition, and (6) soil condition (see Table 2). Each attribute is assessed by collecting data on various indicator variables chosen to serve as proxies for the attribute. For the EPA project, numerous indicator variables were developed for wetlands, with variants for specific types of wetlands such as marsh, bog, fen, or swamp forest. Most indicator variables were designed to be used in rapid assessments that can be carried out in a short site visit (2–4 h). For example, a standard set of ecosystem attributes and their indicator variables as recommended by Faber-Langendoen et al. (2012a, p. 5) for rapid assessment of wetlands is shown in Table 2.
Generically speaking, an ecological integrity assessment as described by Faber-Langendoen et al. (2012a, b) can be carried out at any, or all, of three levels of intensity, as needed. Some data can be collected in the office (level 1), because they consist of information from remote sensing and geographic information systems (GIS). Rapid-assessment data (level 2) come from a rapid on-site assessment, with simple field measures that can be carried out in 2–4 h. Finally, intensive metrics (level 3) are based on more rigorous methods such as collection of quantitative data from a measured plot, according to a sampling design. Specific protocols have been developed for collecting and analyzing data for each indicator variable (Faber-Langendoen et al. 2012b).
To determine reference (benchmark) conditions for the wetland ecological integrity index, NatureServe used their extensive Natural Heritage Program databases of the location and classification of natural areas in the United States, as well as other detailed classifications of ecosystem types [e.g., the U.S. National Vegetation Classification System (Federal Geographic Data Committee 2008)], to construct reference gradients for various wetland types (bog, fen, marsh, wet meadow, swamp, etc.) against which to compare the ecological integrity of any particular site in question (Faber-Langendoen et al. 2012a, pp. 47–54). Each reference gradient identified by NatureServe contained multiple occurrences of a given wetland type, with each occurrence rated (four categories, A–D) according to its degree of disturbance/degradation (Faber-Langendoen et al. 2012a, pp. 1–3, 40–41).
To compare a site of a specific wetland type against the reference gradient for that type, data were collected on indicator variables for the so-called major ecological attributes, which are said by NatureServe to “capture the structure, composition, and processes” of an ecosystem (Faber-Langendoen et al. 2012b, p. 2)—i.e., in the wetland case, these major attributes were the six mentioned previously: vegetation condition, hydrology condition, soil condition, size, buffer, and landscape context (Faber-Langendoen et al. 2012a, pp. 4–5), as shown in our Table 2. The data collected usually consisted of categorical values (i.e., ratings) rather than continuous values, as follows: “field observations typically involved assigning a rating of A, B, or C, and sometimes D or E to a metric [indicator variable], rather than a numeric score” (Faber-Langendoen et al. 2012a, p. 76). These ratings were calibrated to a scale pre-determined by NatureServe (Faber-Langendoen et al. 2012a, pp. 37–42), as were the benchmark reference sites that provided the standard of comparison. After collection, the data for all indicator variables for each major ecological attribute were entered in an ecological observations database.
From the ecological observations database, an index of ecological integrity was generated and presented as a summary scorecard (see Table 3) showing the score (weighted rating) of indicator variables and ecological attributes (Faber-Langendoen et al. 2012a, p. 76). To get an indicator’s score, its rating was converted to a point value, multiplied by the weight (Faber-Langendoen et al. 2012a, p. 128). Scores were summed within attributes, then divided by the summed weight in order to get a weighted average attribute score and letter (A–D) rank (Faber-Langendoen et al. 2012a, p. 128). In generating this summary scorecard, varying weights were applied at three stages of the process (Faber-Langendoen et al. 2012a, p. 76). In addition, the overall index of ecological integrity was itself assigned a rating of A to D on a ranked 4-point scale (Faber-Langendoen et al. 2012a, p. 76).
Appendix 3: Vegetation metrics
In NatureServe’s version of the ecological integrity assessment, the central concept of so-called vegetation quality as a gauge of the “ecological health” of an ecosystem, and some associated metrics, were originated by Wilhelm (1977) and Swink and Wilhelm (1994), further developed by Wilhelm and Ladd (1988) and Taft et al. (1997), and incorporated by Faber-Langendoen et al. (2012a) in the current methodology, all in non-refereed publications. The following metrics are floristic quality assessment methods used in ecological integrity assessment, as described in Faber-Langendoen et al. (2012a).
Vascular plant species richness S (=number of species) in the wetland study (Faber-Langendoen et al. 2012a, pp. 72–74) was determined on the basis of species presence/absence in an unreplicated single plot. Crews recorded all vascular plant species present in one core module (10 × 10 m), then searched for any new species in the remaining modules and plot. Relative percent cover was roughly characterized by rapid visual estimation of the percentage of cover of each species in the module, and recorded as 1 of 10 categories.
The mean coefficient of conservatism as described in the wetland study (Faber-Langendoen et al. 2012a, Appendix 4) was the mean conservatism of all native vascular plant species at the site, stated to be a measure of vegetation condition. The idea is that less disturbed sites will have more plants with higher measures of conservatism, and hence be more valuable for conservation. In this method, a preassigned coefficient of conservatism (C) is a numerical score between 1 and 10, which is said to “represent an estimated probability that a plant is likely to occur” in an undisturbed habitat (Faber-Langendoen et al. 2012a, p. 119). In the wetland project, each native plant species sampled at a site was given its preassigned coefficient of conservatism. Then the coefficients of conservatism for all native species were summed and the sum divided by the total number (i.e., species richness) of native species at the plot (or site) to obtain the mean (i.e., mean C = ∑C/Snative). Finally, the mean coefficient of conservatism for a given site was ranked in one of four categories (A–D, excellent through poor). This rank value was entered on the ecological integrity scorecard.
The floristic quality index is a similar measure that is derived from the coefficients of conservatism (Faber-Langendoen et al. 2012a, Appendix 4). After each native species observed has been given its preassigned coefficient of conservatism, the mean coefficient of conservatism is calculated for all native species (i.e., mean C = ∑C/Snative) as described previously. To obtain the floristic quality index FQI, the mean coefficient of conservatism is multiplied by the square root √S of site or plot native species richness (FQI = mean C × √Snative). Scaling by √Snative is asserted by Swink and Wilhelm (1994), Taft et al. (1997), and Faber-Langendoen et al. (2012a, p. 119) to limit the variable influence of area on species richness.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Brown, E.D., Williams, B.K. Ecological integrity assessment as a metric of biodiversity: are we measuring what we say we are?. Biodivers Conserv 25, 1011–1035 (2016). https://doi.org/10.1007/s10531-016-1111-0
- Biodiversity conservation
- Ecological integrity assessment
- Biodiversity indicators
- Ecosystem integrity