Abstract
Screening-level risk assessment tools for non-indigenous species are useful to inform management and policy decisions. While a number of tools have been developed to evaluate the risk of introductions in terrestrial and freshwater environments, only one tool is available for marine invertebrates [Marine Invertebrate Invasiveness Scoring Kit (MI-ISK)] and it has not been calibrated or evaluated previously. This paper introduces the Canadian Marine Invasive Screening Tool (CMIST), a tool based on a series of questions related to the sequence of events in the invasion process. We assessed the performance of both CMIST and MI-ISK using 60 species/ecosystem combinations of non-indigenous marine invertebrate species known to have been introduced to three Canadian marine ecoregions. Assessment scores were compared to the results of an expert opinion survey about the level of ecological risk posed by these species in these ecoregions. In addition, a simple way to quantify uncertainty and compute confidence limits surrounding expert evaluations and CMIST scores is presented; the technique could be incorporated into other existing tools. The risk scores returned by the two tools generally were similar, had comparable levels of inter-assessor variability, and correlated well with the results of the expert opinion survey for two of the three ecoregions. In the third ecoregion, we believe the lack of highly problematic species and greater uncertainty (linked to less reliable information) contributed to the weaker correlation. For all ecoregions, CMIST scores were more strongly correlated with expert opinion scores when compared to MI-ISK scores. Finally, we demonstrate how CMIST can be used for new incursions; for each ecoregion we scored 15 additional species not yet introduced and compared these scores with those of species already present. Overall CMIST is a promising screening-level tool; it distinguishes between elements of invasion risk (likelihood and impact), has fewer questions to score than similar tools, and could easily be adapted for other taxa.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
A major challenge for invasion biologists has been to determine which species can complete the invasion process (arrive, survive, establish, spread), and eventually have negative impacts in an invaded area. To this end, research has focused on three main invasion aspects: propagule pressure (the number of individuals arriving within a given time interval; Lockwood et al. 2005; Simberloff 2009), species traits influencing invasiveness (Rejmanek and Richardson 1996; Kolar and Lodge 2001, 2002), and characteristics of the receiving environment influencing invasibility (e.g., Lonsdale 1999; Levine et al. 2004). Many tools have been developed to evaluate the risk a species poses to a particular area using various combinations of factors thought to influence the success of a species outside of its native range (reviewed in Kumschick and Richardson 2013). Such tools, if accurate, have clear benefits from a policy or management perspective, such as rapidly screening species, denying risky proposed intentional introductions, and focusing limited resources on species posing the greatest risk.
Screening-level tools for non-indigenous species are generally based on the answers to a series of questions to determine if a species is a threat (high risk) or not. Common screening-level tool types include decision trees (Reichard and Hamilton 1997; Kolar and Lodge 2002; Caley and Kuhnert 2006) and scoring systems (Pheloung et al. 1999; Daehler et al. 2004; Copp et al. 2009). Most scoring systems that have been calibrated and tested to date are derivatives of the Australia Weed Risk Assessment model (AuWRA; Pheloung 1995), a tool designed to evaluate proposed intentional plant introductions. The AuWRA has been adapted successfully for terrestrial plants internationally (Daehler et al. 2004; Kato et al. 2006; Gordon et al. 2008a, b; Gordon and Gantz 2008), aquatic plants (Gordon and Gantz 2011), and some animal taxa (Copp et al. 2009; Tricarico et al. 2010).
In the marine realm, risk assessment for non-indigenous species has mostly focused on vectors and pathways of introduction (Floerl et al. 2005; Weigle et al. 2005; Barry et al. 2008; Acosta et al. 2010; Clarke Murray et al. 2013); much less has been done on developing approaches to rapidly screen species based on the risk they pose (but see Hayes and Sliwa 2003; Nyberg and Wallentinus 2005; Miller et al. 2007; Locke 2009). To our knowledge, only one screening-level risk assessment tool specific to marine invertebrates is available, the Marine Invertebrate Invasiveness Scoring Kit (MI-ISK). This tool is an adaptation of the AuWRA and has yet to be calibrated and tested (GH Copp, pers. comm.). Given the sheer number and impact of recent marine invertebrate introductions (Ruiz et al. 2000; Grosholz 2002), managers would benefit from tools that allow a rapid evaluation of the risk posed by a species in a particular area.
The testing and calibration of risk assessment tools typically involve scoring species known to have been introduced to an area and relating those scores to the actual outcome of the introductions. However, quantification of impacts of non-indigenous species can be problematic (e.g., Nentwig et al. 2010; Blackburn et al. 2014). Thus, most risk assessment studies generally identify categorical outcomes (e.g., non-pest/minor pest/major pest, or invasive/not invasive; often referred to as a priori categories) that are based on the opinion of experts (Pheloung et al. 1999; Daehler et al. 2004; Gordon et al. 2012), on databases (Daehler and Carino 2000; Gordon et al. 2008a; Copp et al. 2009; Gordon and Gantz 2011), or literature accounts of whether species became established following documented introductions (Bomford and Glover 2004; Bomford et al. 2005). Threshold values can then be determined as the assessment scores that best assign species to the correct outcome category (e.g., Copp et al. 2009). This allows categorizing a species in relation to the risk posed and associated management decisions (e.g., in the case of proposed introductions: ‘accept’ if low risk, ‘evaluate further’ if ambiguous, or ‘reject’ if high risk). However, the realized impact of a non-indigenous species in an area is not categorical; species categorized as invaders will invariably result in a gradient of impacts. A continuous scale for the expert rankings may thus be more appropriate as it allows: (1) ranking of relative impact within categories (i.e., some invaders have greater impacts or pose higher risk than others), (2) an evaluation of strength of association between assessment score and realized impact (e.g., using correlation or regression analysis), and (3) a quantification of uncertainty. However, few studies (Pheloung et al. 1999; Daehler et al. 2004; Crosti et al. 2010; McClay et al. 2010) have transformed categorical scores to a continuous scale by using the average of semi-quantitative expert classifications.
Answers to questions included in screening-level tools and the information used to test these tools (i.e., the measure of realized impacts/risk) both contain uncertainties. Uncertainty may arise from the quality of information used or its interpretation (judgement subjectivity, sensu Regan et al. 2002) or the interpretation of the language used in assessment tool questions or expert surveys (linguistic uncertainty, sensu Regan et al. 2002), resulting in both intra- and inter-assessor/expert uncertainty. Few studies have addressed these issues directly (Kumschick and Richardson 2013). Blackburn et al. (2014) proposed a qualitative way to describe uncertainty surrounding realized impacts. Holt et al. (2012, 2014) devised a way to visualize uncertainty surrounding risk components and developed a scheme to combine two risk components (e.g., risk of entry and risk of establishment) under different levels of uncertainty. Copp et al. (2009) and Tricarico et al. (2010) added an uncertainty score associated with the answer to each question; these scores are then averaged to provide a relative measure of intra-assessor uncertainty for the score assigned to each species. Copp et al. (2009) reported the variability in scores assigned by different assessors for the same species, thus quantifying inter-assessor uncertainty. To date, none of the risk assessment tools developed has included a way to quantify uncertainty to compute confidence limits surrounding either risk or impact scores (Koop et al. 2012).
Sanctioned introductions of novel species in the marine environment have been greatly reduced in many parts of the world (ICES 2005), and most recent introductions are the result of either illegal or accidental releases. Thus, a screening-level risk assessment tool adapted to these means of introduction is needed to inform legislations (i.e., populate lists of species to be regulated) and to prioritize intervention (i.e., focus resources on riskiest species when a choice needs to be made). Here we present a new tool (Canadian Marine Invasive Screening Tool; CMIST) that follows the sequence of events in the invasion process of marine invertebrates (arrival, survival, establishment, spread, and impact), but that is general enough to be adapted to any taxon. Further, we provide the first evaluation of screening-level risk assessment tools for the marine environment by comparing predictions made by CMIST and MI-ISK against expert evaluations of risk posed by species known to have been introduced to three Canadian marine ecoregions. Since these tools are ultimately designed to evaluate risk posed by species not already present in an area, we also evaluated 45 potential invasive species/ecoregion combinations and compared their risk scores with those of species already present. Lastly, we present and employ a simple way to quantify uncertainty in both the expert evaluations and assessment scores.
Methods
Tools evaluated
The Canadian Marine Invasive Screening Tool (CMIST) is a modification of the Alberta Risk Assessment Tool (version 3; IASWG 2009; a general risk assessment tool to evaluate the risk associated with terrestrial and aquatic organisms, developed for use by the province of AB, Canada). It focuses on the different steps of the invasion process and explicitly distinguishes the two risk components: ‘Likelihood of invasion’ and ‘Impact of invasion’ (Kumschick and Richardson 2013). To this end, CMIST asks 17 questions pertaining to a species’ present status, rate of introduction, probability of survival, establishment, and spread in the assessment area, and ecological impacts in the assessment area and elsewhere (Table 1; guidance for each question is available as supplemental material). The answer to each question (‘Low’, ‘Moderate’, or ‘High’) is converted into a numerical score of 1, 2, or 3. A mean score is calculated for likelihood of invasion (i.e., questions 1–8) and potential impacts (i.e., questions 9–17). These two mean scores are then multiplied to obtain a final risk score ranging from 1 to 9 (a spreadsheet to calculate species assessment scores is available as supplemental material); this results in an equal contribution of each question to the final score, but we recognize accuracy or precision could be increased with a weighting scheme (Drolet et al. in prep). Assessors also assign a qualitative certainty score to the answer provided for each question. Certainty assigned as ‘Low’, ‘Moderate’, or ‘High’ is hereafter referred to as uncertainty being ‘High’, ‘Moderate’, or ‘Low’, respectively, to standardize with published literature. The questions are phrased to be possible to answer even in the absence of information, and all questions need to be answered to calculate a final score.
The second tool evaluated, MI-ISK, is an adaptation of the AuWRA specific to marine invertebrates and is available online (www.cefas.defra.gov.uk/media/621525/decisiontools__background&guidance_v4_oct13.pdf). It asks 49 questions (most responses require a choice among yes/no/don’t know and some are qualitative-ordinal) pertaining to species domestication, climate requirements, distribution, history of invasion, biological traits, feeding biology, reproduction biology, dispersal, and persistence attributes. A qualitative level of confidence (0 = very uncertain, 1 = mostly uncertain, 2 = mostly certain, 3 = very certain) is recorded for each question. Using the answers provided by the assessor, this system returns scores ranging from −12 to 57. A minimum of 10 questions need to be answered for score calculation and some questions are given greater weight based on their perceived importance and answers to some questions influence the weighting of subsequent questions.
Species assessments
Risk scores for non-indigenous invertebrate species known to have been introduced in Canadian marine ecosystems were calculated using both CMIST and MI-ISK. Specifically, we focused on species introduced to three marine ecoregions [DFO 2009: Strait of Georgia on Canada’s west coast (30 species), and Gulf of St. Lawrence (15 species) and Scotian Shelf (15 species) on Canada’s east coast; Table 2]. Some species were introduced to more than one ecoregion (Table 2). Two biologists with good knowledge of non-indigenous and indigenous marine fauna independently scored each species-ecoregion combination using both tools. Scoring was done through searching for information needed to answer each question from various sources (e.g., primary publications, reports, databases) available via the internet, the idea being that any new species could be evaluated in a day or two using currently available tools. The justification and sources of information used to answer each question were noted to leave a record justifying potential decisions based on assessment (accountability). The same procedure was used to assess 15 species not already present in each of the three ecoregions, but that have the potential to arrive and establish in the future (i.e., species with a history of establishment outside their native range in broadly similar environmental conditions).
Expert opinion survey
Ideally tool performance would be evaluated against actual outcomes, here impacts in the new ecosystem. Other similar studies have been able to test performance using species for which the realized impact has been documented, i.e., species that have been present for a long time and for which the invasion outcome (impacts) are evident (e.g., McClay et al. 2010). Unfortunately, for marine invertebrates in Canadian waters, too few species meet these criteria to generate a sufficient dataset for evaluation; most introductions are recent, the species are still spreading, and impacts have not been realized or fully documented. Thus, we conducted an expert opinion survey to obtain a baseline against which CMIST and MI-ISK tool performance could be evaluated. Experts have knowledge of the species (vectors, abundance, spread, impacts, etc.) and environmental/habitat characteristics in areas where they have been introduced; thereby providing a reasonable proxy for risk to an ecoregion. This survey was conducted using the web-based platform SurveyMonkey®. A list of 70 potential respondents, consisting of biologists with extensive experience and knowledge of non-indigenous marine species in Canadian and bordering American waters, were contacted by e-mail and invited to complete the survey. Details about the objectives of the survey were explained and potential respondents were asked to provide answers only for species/ecoregions for which they felt they had sufficient expertise. Experts were asked to qualify the level of risk a species poses to an ecoregion, and their level of certainty, as ‘Low’, ‘Moderate’, or ‘High’.
Testing of tools
The precision and accuracy of the assessment scores returned by CMIST and MI-ISK were evaluated for each ecoregion. Precision was evaluated as the between-assessor variability using correlation analyses of the overall risk scores assigned by each assessor; relative precision of the two tools was evaluated by statistically comparing the correlation coefficients (Zar 2010). To determine if scores returned by the two tools were in agreement, average scores (of the two assessments) were compared using correlation analyses. These two analyses were conducted separately for species known to have been introduced to an ecoregion and species not already present. Tool accuracy was evaluated using linear regression analyses with the expert opinion scores as the dependent variable and the average assessment scores as the independent variable (only species already introduced). We used the average of the expert risk ratings (‘Low’ = 1, ‘Moderate’ = 2, and ‘High’ = 3) to produce a continuous expert opinion score to use in the linear regression. Finally, corrected Akaike Information Criterion (AICc) values (a measure of model fit) were used to compare the accuracy of both tools (a lower relative AICc values represents better model accuracy) and evaluate the likelihood that each tool provides the best fit to the expert opinion scores.
We conducted a second independent evaluation of the accuracy of CMIST using the results of detailed risk assessments. So far, in Canada, such information is available for five tunicate taxa (Styela clava, Ciona intestinalis, Botrylloides violaceus, Botryllus schlosseri, and Didemnum vexillum.; Therriault and Herborg 2007), the European green crab (Carcinus maenas; Therriault et al. 2008a), and the Chinese mitten crab (Eriocheir sinensis; Therriault et al. 2008b). These assessments, which often take over a year to produce, classify species as posing ‘Low’, ‘Moderate’, or ‘High’ risk for both the Atlantic and Pacific Coasts. To visualize how the results of these detailed risk analyses compare to the CMIST scores adjusted for uncertainty (see below), species-ecoregion plots were produced. CMIST scores for each species were sorted in ascending order (but no threshold between risk categories was determined); note that the first question (about current status in the ecoregion) was ignored in score calculation to allow a comparison between species present and not present. We then added the results of the available detailed risk assessments to evaluate where species classified by the detailed assessments as moderate and high risk (no species were evaluated as low risk) fall along the spectrum of CMIST scores.
Quantification of uncertainty
A simple way to quantify uncertainty around the expert opinion scores and the CMIST species assessment scores was developed. The idea is similar in concept to fuzzy logic (previously used in risk assessment for non-indigenous species when dealing with subjective data; Acosta et al. 2010) and captures the probabilities that an expert/assessor would have provided a different answer if they had to answer a question several times. For example, given a question for which the answer is obvious (clearly falls within a category based on the guidance) and the evidence is strong (several peer-reviewed sources with similar conclusions), an assessor would probably always answer the question the same way. In contrast, if the sources of information are weak (e.g., anecdotal evidence and/or similar studies reaching different conclusions) assessors might answer a question differently if they had to repeat the process because the available evidence makes several answers equally possible. Thus, we developed probability distributions of answers under different levels of uncertainty (Low, Moderate, or High) and used them to compute the range of possible outcomes. Four authors (CDB, AL, CWM, and TWT) independently drew these distributions to reflect how, on average, it was felt the scores would be distributed for the nine possible combinations of score and uncertainty levels. The probabilities returned by this group were then averaged to produce the final distributions in Fig. 1. Confidence limits for the expert opinion scores were calculated using Monte Carlo procedures. Specifically, we used the distributions in Fig. 1 and the answers returned by experts, i.e., combination of risk and uncertainty levels for a species, to produce a range of possible scores. As a simplified example, assume two experts returned risk scores for a species: one scored the species as being high risk with low uncertainty and the other, high risk with high uncertainty. For each Monte Carlo simulation, the species risk score would be calculated by drawing numbers from the bottom right distribution in Fig. 1 for the first expert (score would always be a 3) and from the bottom left distribution for the second expert (score would be 3 for ~60 %, 2 for ~35 %, and 1 for ~5 % of the simulations). The drawn scores would then be averaged among experts and the process would be repeated 1000 times; using the 2.5th and 97.5th percentiles as the 95 % confidence limits. A similar approach was used for the CMIST assessment scores: values for each question were drawn for the probability distributions in Fig. 1 based on the assessor’s answer and level of uncertainty. The adjusted CMIST scores and expert opinion scores were plotted with their associated confidence limits, and linear regression analyses used to determine if the inclusion of uncertainty changed the fit between the two variables (when compared to the raw scores). A vector field showing how the approach changed the results for each species was produced.
Results
Overall, the two assessors returned similar scores. Correlation coefficients for species already introduced ranged from 0.51 to 0.85 among ecoregions and tools, and 0.72–0.83 for species not present (Fig. 2). The linear relationships generally had similar slopes and intercepts when comparing species already present and not present. However, the intercept for CMIST scores was slightly smaller for species not present in the Gulf of St. Lawrence and Scotian Shelf (Fig. 2). The precision of CMIST and MI-ISK was similar for the Gulf of St. Lawrence (comparison of correlation coefficient: p = 0.88 and 0.82 for species already present and not present, respectively), the Scotian Shelf (p = 0.61 and 0.78), and the Strait of Georgia (p = 0. 35 and 0.50). The scores returned by the two tools for species already present were highly correlated for the Gulf of St. Lawrence and the Scotian Shelf but less so for the Strait of Georgia, and scores were moderately correlated for all ecoregions when considering species not present (Fig. 3).
A total of 43 experts returned the survey and results are presented in Table 2. In general, there was a positive relationship between expert opinion scores and the risk assessment scores (Fig. 4). For CMIST, the slope of the best-fit regression line was significant for all ecoregions (p = 0.02 for the Gulf of St. Lawrence, p < 0.001 for the Scotian Shelf, and p = 0.01 for the Strait of Georgia). For MI-ISK, the relationship was significant for the Scotian Shelf (p = 0.005) but not for the other ecoregions (p = 0.11 for the Gulf of St. Lawrence and p = 0.15 for the Strait of Georgia). For all ecoregions, the AICc value for CMIST was smaller than that for MI-ISK, translating into likelihoods that CMIST provides a better fit than MI-ISK ranging from 0.82 to 0.95 (Table 3). Incorporating uncertainty in the calculations of expert opinion and CMIST scores improved the fit for the Gulf of St. Lawrence and the Scotian Shelf, but not the Strait of Georgia (Figs. 4, 5). At the species level, incorporation of uncertainty generally brought scores closer to the middle of the plot; species whose scores changed the most were generally lower risk (Fig. 5).
The range of CMIST scores for species not present in the ecoregions was comparable to the range of scores for species already introduced (Fig. 6). Among species evaluated, the Pacific oyster (Crassostrea gigas) was identified as the highest-risk species on the east coast, followed by the veined rapa whelk (Rapana venosa). In the Strait of Georgia, the European green crab (C. maenas) was the highest-risk species. The CMIST scores were in agreement with the results of previously conducted detailed-level risk assessments. For tunicates, risk for all species was considered ‘high’ in east coast ecoregions (Therriault and Herborg 2007) and CMIST ranked these species among the riskiest. On the west coast (of which the Strait of Georgia is a small portion), all tunicate species were considered ‘high’ risk, except C. intestinalis that was considered ‘moderate’ risk (Therriault and Herborg 2007). CMIST ranked all tunicate species among the riskiest, with the exception of C. intestinalis which received a moderate score. For C. maenas, risk was considered ‘very high’ for both coasts (Therriault et al. 2008a), which is well reflected by the high scores returned by CMIST for all ecoregions. Finally, E. sinensis was considered to pose ‘moderate’ risk to the marine environment on both coasts (Therriault et al. 2008b) and CMIST returned moderate scores for this species. For all ecoregions, CMIST perfectly discriminated among moderate and high risk species classified by detailed risk assessment. Admittedly, the number of species for which detailed risk assessments are available is very low.
Discussion
Screening-level risk assessment tools are imperfect, but of great utility to quantify risk and inform management of non-indigenous species. Scoring schemes provide a relatively quick and accurate way to screen and rank species without conducting time- and data-intensive formal quantitative risk analyses (e.g., Leung et al. 2012; Therriault et al. 2008a, b; Therriault and Herborg 2007). Most screening tools currently used to evaluate risks from non-indigenous species are derived from the AuWRA (Pheloung et al. 1999) which was designed to evaluate proposed intentional plant introductions (i.e., to recommend acceptance or rejection). Thus, by definition the introduction step is almost certain and the tool was designed to assess risk in the context of probability of persistence and spread outside cultivation. As such, this tool does not include questions about probability of introduction (arrival) and includes few about probable ecological impacts. It is thus difficult to decompose risk in terms of likelihood and impact of invasion as recommended in Kumschick and Richardson (2013; but see Daehler and Virtue 2010). CMIST was designed to follow the sequence of events in the invasion process (including potential to be introduced to a new area), and thus asks questions directly related to probability of arrival, survival, establishment, spread, and impacts. Thus, it is better suited for assessing risk of unintentional (accidental) introductions (in addition to intentional ones); the most prevalent type of invasions in marine coastal waters and elsewhere.
CMIST uses generalized questions, which could be considered more difficult to answer or subject to greater interpretation than questions about specific life-history traits, such that greater inter-assessor variability in scores might be expected. However, no notable differences in precision were observed between CMIST and MI-ISK. In addition, CMIST inter-assessor variability was smaller than that of the other tool for which similar information was available [mean absolute difference in scores assigned by two assessors divided by mean score for all species assessed; freshwater Fish Invasiveness Scoring Kit (FISK): 0.51, derived from Fig. 1 in Copp et al. (2009), CMIST: 0.16]. Finally, an analysis of individual questions (data not presented), showed the two assessors answered 63 % of the 1785 questions the same way and when answers differed, it was by a score of just 1 in 97 % of these cases. Therefore it appears CMIST is not more prone to high inter-assessor variability in scores when compared to other tools, but admittedly very few tools have actually been evaluated.
The assessment scores returned by the tools were well correlated and provided a good approximation of the expert opinion scores, with the notable exception of the Strait of Georgia (see discussion below). Accuracy of CMIST was slightly lower (Gulf of St. Lawrence; R2 = 0.33) or comparable (Scotian Shelf; R2 = 0.66) to that reported for the WRA in the other studies for which a linear relationship was reported [R2 = 0.47 (Pheloung et al. 1999), 0.52 (Daehler et al. 2004), 0.67 (Crosti et al. 2010), and 0.52 (McClay et al. 2010)]. Comparison of CMIST results to those for which time intensive detailed-level risk assessments have been conducted for the same geographic areas (five tunicate and two crab species) further support the premise that CMIST generally returns reliable risk scores. Other studies generally use Receiver Operating Characteristics (ROC) curves (Hughes and Madden 2003) and report correct classification rates to evaluate accuracy. While this technique is appropriate for evaluation of intentional introductions for which the management consequence of classification is obvious (accept non-pests and reject pests), we felt that such an approach was not appropriate for unintentional introductions. To appropriately evaluate unintentional introductions, it is essential to retain the continuous nature of the severity of realized impacts. This allows: (1) a direct comparison of risk posed by several species in a situation where management resources need to be prioritized, (2) an assessment of the expected risk posed by a potential novel non-indigenous species introduction, in relation to known past experiences (e.g., if species A arrives in an area, we may expect an impact similar to already-established species B), and (3) a better representation of reality because species classified as invaders will invariably differ in their magnitude of impacts within and between area(s).
Kumschick and Richardson (2013) identified the lack of techniques to quantify uncertainty as one of the main weaknesses of screening-level risk assessment tools. The technique we developed to quantify uncertainty is very similar to the one independently developed by the United States Department of Agriculture (USDA 2015; Anthony Koop, Pers. Comm.). They also use a Monte Carlo procedure to generate potential scores based on the level of uncertainty associated with each question; the main difference lies in the probability distributions used (an area of future research). These simple techniques are a significant advancement that could be applied to other risk assessment tools (although challenges may exist for tools with unequal questions/scoring or with feedback among questions). While these systems do not take into account natural variation (peculiarities and chance events influencing each individual invasion), we believe they encompass the uncertainty related to the quality of information available and/or used and language interpretation. In our system, the calculated confidence limits incorporate intra-individual uncertainty, inter-individual disagreements, and sample size (number of individuals that participate in the evaluations). The technique adjusts the influence of an individual response based on the level of certainty; uncertain answers are given less weight than more certain ones. This usually resulted in scores moving closer to the center of the plot and in a notable improvement in fit between expert and assessment scores for two of three assessed ecoregions. Species predicted to have the lowest risk seem to be the most affected (larger change in scores) by this procedure. This is logical as species with greater impacts are often better studied and thus, assessors and experts are more certain of potential effects from these species than those from less well-studied species with potentially fewer impacts.
Studies testing and calibrating risk assessment tools typically use categorical outcomes for realized impacts. Once a species is categorized, this category is considered to be the ‘true value’ of impact. However, despite recent progress (Nentwig et al. 2010; Kumschick et al. 2012; Blackburn et al. 2014), it is still difficult to quantify impacts of non-native species, especially for the less studied species. Therefore, uncertainty exists in the data being used to test these tools. To our knowledge, our study is the first to quantify this uncertainty. It revealed that sometimes, discrepancies between risk assessment scores and an indicator of realized impact (expert opinion) may be the result of high uncertainty on behalf of the expert. Future studies should consider this source of uncertainty and evaluate the potential consequences of misclassification in impact categories.
Both CMIST and MI-ISK scores were more weakly related to expert opinion for species in the SoG than in the other ecoregions considered. There may be several reasons for this. First, the impact of non-indigenous marine invertebrates is often considered less significant on the Canadian west coast compared to the east coast. In fact, the expert scores were significantly lower in the Strait of Georgia than the other two ecoregions (Tukey post hoc tests following significant one-way ANOVA; results not presented). The absence (or lack of realization of potential impacts) of highly problematic species might prevent detection of a statistical relationship (this is particularly likely for MI-ISK, for which high risk scores in the Strait of Georgia were rare). Second, the lack of relationship might be related to higher uncertainty for this region. There are few species with a high negative economic impact in the Strait of Georgia, and these are species for which ecological studies are usually urged. Thus, the quality of information for this ecoregion might be lower than for the east coast ecoregions where several high profile invasive species were included in our analyses (e.g., several tunicate species, green crab). Also, fewer expert evaluations were completed for the Strait of Georgia, and the scores were more uncertain. These factors resulted in significantly larger confidence limits in the Strait of Georgia than in the other two ecoregions (Tukey post hoc tests following significant one-way ANOVA; results not presented).
In conclusion, we recommend the use of CMIST as a screening-level risk assessment tools for non-indigenous marine invertebrate species. This tool reflects the invasion cycle, the scores relate well with expert opinion scores, and uncertainty can be quantified. The technique developed to quantify uncertainty should be incorporated in existing tools designed to evaluate intentional introductions. Since the CMIST questions are generalized to the invasion process and resulting impacts, CMIST could easily be adapted to other taxa simply by modifying the guidance for each question.
References
Acosta H, Wu D, Forrest BM (2010) Fuzzy experts on recreational vessels, a risk modelling approach for marine invasions. Ecol Model 221:850–863. doi:10.1016/j.ecolmodel.2009.11.025
Barry SC, Hayes KR, Hewitt CL, Dragsund E, Bakke SM (2008) Ballast water risk assessment: principles, processes, and methods. ICES J Mar Sci 65:121–131. doi:10.1093/icesjms/fsn004
Blackburn TM, Essl F, Evans T, Hulme PE, Jeschke JM, Kühn I, Kumschick S, Marková Z, Mrugala A, Nentwig W, Pergl J, Pyšek P, Rabitsch W, Ricciardi A, Richardson DM, Sendek A, Vilà M, Wilson JRU, Winter M, Genovesi P, Bacher S (2014) A unified classification of alien species based on the magnitude of their environmental impacts. PLoS Biol 12:e1001850. doi:10.1371/journal.pbio.1001850
Bomford M, Glover J (2004) Risk assessment model for the import and keeping of exotic freshwater and estuarine finfish. Report of the Australian Bureau of Rural Sciences, p 125
Bomford M, Kraus F, Braysher M, Walter L, Brown L (2005) Risk assessment model for the import and keeping of exotic reptiles and amphibians. Report of the Australian Bureau of Rural Sciences, p 110
Caley P, Kuhnert PM (2006) Application and evaluations of classification trees for screening unwanted plants. Austral Ecol 31:647–655. doi:10.1111/j.1442-9993.2006.01617.x
Clarke Murray C, Therriault TW, Pakhomov E (2013) What lies beneath? An evaluation of rapid assessment tools for management of hull fouling. Environ Manag 52:374–384. doi:10.1007/s00267-013-0085-x
Copp GH, Vilizzi L, Mumford J, Fenwick GV, Godard MJ, Gozlan RE (2009) Calibration of FISK, an invasiveness screening tool for nonnative freshwater fishes. Risk Anal 29:457–467. doi:10.1111/j.1539-6924.2008.01159.x
Crosti R, Cascone C, Cipollaro S (2010) Use of a weed risk assessment for the Mediterranean region of Central Italy to prevent loss of functionality and biodiversity in agro-ecosystems. Biol Invasions 12:1607–1616. doi:10.1007/s10530-009-9573-6
Daehler CC, Carino DA (2000) Predicting invasive plants: prospects for a general screening system based on current regional models. Biol Invasions 2:93–102. doi:10.1023/A:1010002005024
Daehler CC, Virtue JG (2010) Likelihood and consequences: reframing the Australian weed risk assessment to reflect a standard model of risk. Plant Prot Q 25:52–55
Daehler CC, Denslow J, Ansari S, Kuo H-C (2004) A risk-assessment system for screening out invasive pest plants from Hawaii and other Pacific islands. Conserv Biol 18:360–368. doi:10.1111/j.1523-1739.2004.00066.x
DFO (2009) Development of a framework and principles for the biogeographic classification of Canadian marine areas. DFO Canadian Science Advisory Secretariat Science Advisory Report 2009/056, p 17
Floerl O, Inglis GJ, Hayden BJ (2005) A risk-based predictive tool to prevent accidental introductions of nonindigenous marine species. Environ Manag 35:765–778. doi:10.1007/s00267-004-0193-8
Gordon DR, Gantz CA (2008) Screening new plant introductions for potential invasiveness: a test of impact for the United States. Conserv Lett 1:227–235. doi:10.1111/j.1755-263X.2008.00032.x
Gordon DR, Gantz CA (2011) Risk assessment for invasiveness differs for aquatic and terrestrial plant species. Biol Invasions 13:1829–1842. doi:10.1007/s10530-011-0002-2
Gordon DR, Onderdonk DA, Fox AM, Stocker RK, Gantz C (2008a) Predicting invasive plants in Florida using the Australian weed risk assessment. Invasive Plant Sci Manag 1:178–195. doi:10.1614/IPSM-07-037.1
Gordon DR, Onderdonk DA, Fox AM, Stocker RK (2008b) Consistent accuracy of the Australian weed risk assessment system across varied geographies. Divers Distrib 14:234–242. doi:10.1111/j.1472-4642.2007.00460.x
Gordon DR, Gantz CA, Jerde CL, Chadderton WL, Keller RP, Champion PD (2012) Weed risk assessment for aquatic plants: modification of a New Zealand system for the United States. PLoS ONE 7:e40031. doi:10.1371/journal.pone.0040031
Grosholz E (2002) Ecological and evolutionary consequences of coastal invasions. Trends Ecol Evol 17:22–27
Hayes RH, Sliwa C (2003) Identifying potential marine pests: a deductive approach applied to Australia. Mar Pollut Bull 46:91–98. doi:10.1016/S0025-326X(02)00321-1
Holt J, Leach AW, Knight JD, Griessinger D, MacLeod A, van der Gaag DJ, Schrader G, Mumford JD (2012) Tools for visualizing and integrating pest risk assessment ratings and uncertainties. EPPO Bull 42:35–41. doi:10.1111/j.1365-2338.2012.02548.x
Holt J, Leach AW, Schrader G, Petter F, MacLeod A, van der Gaag DJ, Baker RHA, Mumford JD (2014) Eliciting and combining decision criteria using a limited palette of utility functions and uncertainty distributions: illustrated by application to pest risk analysis. Risk Anal 34:4–16. doi:10.1111/risa.12089
Hughes G, Madden LV (2003) Evaluating predictive models with application in regulatory policy for invasive weeds. Agric Syst 76:755–774. doi:10.1016/S0308-521X(02)00164-6
IASWG (Invasive Alien Species Working Group) (2009) Alberta invasive alien species risk assessment tool. Alberta Agriculture and Rural Development, Edmonton, p 65
ICES (International Council for the Exploration of the Sea) (2005) ICES code of practice on the introductions and transfers of marine organisms. ICES, Copenhagen, p 30
Kato H, Hata K, Yamamoto H, Yoshioka T (2006) Effectiveness of the weed risk assessment system for the Bonin Islands. In: Koike F, Clout MN, Kawamichi M, De Poorter M, Iwatsuki K (eds) Assessment and control of biological invasion risks. Shoukadoh Book Sellers, Kyoto, and IUCN, Gland, pp 65–72
Kolar CS, Lodge DM (2001) Progress in invasion biology: predicting invaders. Trends Ecol Evol 16:199–204. doi:10.1016/S0169-5347(01)02101-2
Kolar CS, Lodge DM (2002) Ecological predictions and risk assessment for alien fishes in North America. Science 298:1233–1236. doi:10.1126/science.1075753
Koop AL, Fowler L, Newton LP, Caton BP (2012) Development and validation of a weed screening tool for the United States. Biol Invasions 14:273–294. doi:10.1007/s10530-011-0061-4
Kumschick S, Richardson DM (2013) Species-based risk assessments for biological invasions: advances and challenges. Divers Distrib 19:1095–1105. doi:10.1111/ddi.12110
Kumschick S, Bacher S, Dawson W, Heikkilä J, Sendek A, Pluess T, Robinson TB, Kühn I (2012) A conceptual framework for prioritization of invasive alien species for management according to their impact. NeoBiota 15:69–100. doi:10.3897/neobiota.15.3323
Leung B, Roura-Pascual N, Bacher S, Heikkilä J, Brotons L, Burbman MA, Dehnen-Schmutz K, Essl F, Hulme PE, Richardson DM, Sol D, Vilà M, Rejmanek M (2012) TEASIng apart alien species risk assessments: a framework for best practices. Ecol Lett 15:1475–1493. doi:10.1111/ele.12003
Levine JM, Adler PB, Yelenik SG (2004) A meta-analysis of biotic resistance to exotic plant invasions. Ecol Lett 7:975–989. doi:10.1111/j.1461-0248.2004.00657.x
Locke A (2009) A screening procedure for potential tunicate invaders of Atlantic Canada. Aquat Invasions 4:71–79. doi:10.3391/ai.2009.4.1.7
Lockwood JL, Cassey P, Blackburn T (2005) The role of propagule pressure in explaining species invasions. Trends Ecol Evol 20:223–228. doi:10.1016/j.tree.2005.02.004
Lonsdale WM (1999) Global patterns of plant invasions and the concept of invasibility. Ecology 80:1522–1536. doi:10.1890/0012-9658(1999)080[1522:GPOPIA]2.0.CO;2
McClay A, Sissons A, Wilson C, David S (2010) Evaluation of the Australian weed risk assessment system for the prediction of plant invasiveness in Canada. Biol Invasions 12:4085–4098. doi:10.1007/s10530-010-9819-3
Miller AW, Ruiz GM, Minton MS, Ambrose RF (2007) Differentiating successful and failed molluscan invaders in estuarine ecosystems. Mar Ecol Prog Ser 332:41–51. doi:10.3354/meps332041
Nentwig W, Kühnel E, Bacher S (2010) A generic impact-scoring system applied to alien mammals in Europe. Conserv Biol 24:302–311. doi:10.1111/j.1523-1739.2009.01289.x
Nyberg CD, Wallentinus I (2005) Can species traits be used to predict marine macroalgal introductions? Biol Invasions 7:265–279. doi:10.1007/s10530-004-0738-z
Pheloung PC (1995) Determining the weed potential of new plant introductions to Australia: a report on the development of a weed risk assessment system commissioned by the Australian Weeds Commission and the Plant Industries Committee. Agricultural Protection Board, Western Australia, p 36
Pheloung PC, Williams PA, Halloy SR (1999) A weed risk assessment model for use as a biosecurity tool evaluating plant introductions. J Eviron Manag 57:239–251. doi:10.1006/jema.1999.0297
Regan HM, Colyvan M, Burgman MA (2002) A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecol Appl 12:618–628. doi:10.1890/1051-0761(2002)012[0618:ATATOU]2.0.CO;2
Reichard SH, Hamilton CW (1997) Predicting invasions of woody plants introduced into North America. Conserv Biol 11:193–203. doi:10.1046/j.1523-1739.1997.95473.x
Rejmanek M, Richardson DM (1996) What attributes make some plant species more invasive? Ecology 77:1655–1661. doi:10.2307/2265768
Ruiz GM, Fofonoff PW, Carlton JT, Wonham MJ, Hines AH (2000) Invasion of coastal marine communities in North America: apparent patterns, processes, and biases. Annu Rev Ecol Syst 31:481–531. http://www.jstor.org/stable/221741
Simberloff D (2009) The role of propagule pressure in biological invasions. Annu Rev Ecol Evol Syst 40:81–102. doi:10.1146/annurev.ecolsys.110308.120304
Therriault TW, Herborg L-M (2007) Risk assessment for two solitary and three colonial tunicates in both Atlantic and Pacific Canadian Waters. Canadian Science Advisory Secretariat Science Research Document 2007/063, p 64
Therriault TW, Herborg L-M, Locke A, McKindsey CW (2008a) Risk Assessment for European green crab (Carcinus maenas) in Canadian Waters. Canadian Science Advisory Secretariat Science Research Document 2008/042, p 40
Therriault TW, Herborg L-M, Locke A, McKindsey CW (2008b) Risk Assessment for Chinese mitten crab (Eriocheir sinensis) in Canadian Waters. Canadian Science Advisory Secretariat Science Research Document 2008/041, p 33
Tricarico E, Vilizzi L, Gherardi F, Copp GH (2010) Calibration of FI-ISK, an invasiveness screening tool for nonnative freshwater invertebrates. Risk Anal 30:285–292. doi:10.1111/j.1539-6924.2009.01255.x
USDA (2015) Guidelines for the USDA-APHIS-PPQ weed risk assessment process. United States Department of Agriculture (USDA), Animal and Plant Health Inspection Service (APHIS), Plant Protection and Quarantine (PPQ), 125 pp
Weigle SM, Smith LD, Carlton JT, Pederson J (2005) Assessing the risk of introducing exotic species via the live marine species trade. Conserv Biol 19:213–223. doi:10.1111/j.1523-1739.2005.00412.x
Zar JH (2010) Biostatistical analysis, 5th edn. Prentice Hall, Upper Saddle River
Acknowledgments
We are grateful to Debbie Paltzat for completing species assessments, to Thomas Landry and the participants of a National DFO risk assessment workshop for helpful discussions, and Fisheries and Oceans Canada for funding provided through its Aquatic Invasive Species (AIS) program. DD received financial support from the Canadian Aquatic Invasive Species Network (CAISN II) during the preparation of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Drolet, D., DiBacco, C., Locke, A. et al. Evaluation of a new screening-level risk assessment tool applied to non-indigenous marine invertebrates in Canadian coastal waters. Biol Invasions 18, 279–294 (2016). https://doi.org/10.1007/s10530-015-1008-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10530-015-1008-y