Instituting a national early detection and rapid response program: needs for building federal risk screening capacity

The invasive species issue is inherently a matter of risk; what is the risk that an invasive species will adversely impact valued assets? The early detection of and rapid response to invasive species (EDRR) requires that an assessment of risk is conducted as rapidly as possible. We define risk screening as rapid characterization of the types and degree of risks posed by a population of non-native species in a particular spatio-temporal context. Risk screening is used to evaluate the degree to which various response measures are warranted and justifiable. In this paper, we evaluate the US government’s risk screening programs with a view towards advancing national EDRR capacity. Our survey-based findings, consistent with prior analyses, indicate that risk evaluation by federal agencies has largely been a reactive, ad hoc process, and there is a need to improve information sharing, risk evaluation tools, and staff capacity for risk screening. We provide an overview of the US Department of Agriculture’s Tiered Weed Risk Evaluation and US Fish and Wildlife Service’s Ecological Risk Screening Summaries, two relatively new approaches to invasive species risk screening that hold promise as the basis for future work. We emphasize the need for a clearinghouse of risk evaluation protocols, tools, completed assessments and associated information; development of performance metrics and standardized protocols for risk screening; as well as support for complementary, science-based tools to facilitate and validate risk screening.


Introduction
Executive Order 13751 defines invasive species to mean, ''with regard to a particular ecosystem, a nonnative organism whose introduction causes or is likely to cause economic or environmental harm, or harm to human, animal, or plant health'' (Executive Office of the President 2016). Inherent in this definition are questions about risk: What is the risk that a non-native species will be introduced into a new ecosystem? What is the risk that it will cause harm to certain valued assets if introduced into said ecosystem? The answer to both of these questions is the same-it depends. It depends on variables that are contextspecific and dynamic, changing in ways that can be difficult to predict with high certainty. The severity and cumulative nature of invasive species impacts are often substantial and require action despite uncertainties. Risks need to be assessed and response options weighed against the costs of inaction.
Definitions of the term ''risk analysis'' vary widely and thus may instill confusion when protocols are not transparent and/or are inconsistently applied (Roy et al. 2018). The 2016-2018 National Invasive Species Council (NISC) Management Plan defines risk analysis as the set of tools or processes incorporating risk assessment, risk management, and risk communication, which are used to evaluate the potential risks associated with a non-native species or invasion pathway, possible mitigation measures to address the risk, and the information to be shared with decisionmakers and other stakeholders (NISC 2016). Ideally, risk analyses provide a framework for considering the costs (harm) and benefits of the movement of a particular species via a particular pathway (mode of transport) to a particular locality in the near-term, as well as into the future (i.e., when considering climate change) (US Environmental Protection Agency 1992, 1998Anderson et al. 2004;Lodge et al. 2016). Increasingly, risk analyses are mandated by international, national, and sub-national policies to improve measures to prevent invasive species from entry into across jurisdication borders and/or introduction into a novel ecosystem (Burgiel et al. 2006;Burgos-Rodríguez and Burgiel 2019, this issue). Risk analyses function as decision support systems for developing and enforcing laws and regulations, justifying restrictions on certain commodities (trade, as well as industry and consumer behavior), informing environmentally responsible and sustainable decisions by industries, hobbyists, and consumers, and prioritizing detection (surveillance) and post-detection response measures (Roy et al. 2017).
In the United States, ecologically-oriented risk analysis frameworks emerged out of the need to implement the National Environmental Policy Act (NEPA) adopted in 1969 (Burgos-Rodríguez and Burgiel 2019, this issue). In 1993, the US Congress Office of Technology Assessment's (OTA) review of invasive species issues (termed ''harmful, non-indigenous species'' in the report) identified risk analysis as a critical tool for federal decision-making. The authors concluded that the US Department of Agriculture (USDA) and US Department of the Interior (DOI) needed to strengthen their capacities for taking a science-based approach to risk analysis (OTA 1993). In 1999, the National Science and Technology Council's (NSTC) Committee on Environment and Natural Resources (CENR) undertook a review of ecological risk assessment across the federal government (NSTC 1999). The study included a focus on invasive species (termed ''nonindigenous species'' in the report) and, among other things, recommended that federal interagency cooperation be improved to help reduce redundancy and focus limited resources.
The criteria for undertaking a comprehensive risk analysis to address invasive species consistent with regulatory frameworks have been proposed and reviewed in detail elsewhere (e.g., Meyerson and Reaser 2003;Anderson et al. 2004;Stohlgren and Schnase 2006;Lodge et al. 2016;Roy et al. 2018). It is important to note that holistic and comprehensive risk analyses can be expensive and time-consuming, particularly if relevant biological and/or socio-economic data are not readily available. This reality runs counter to one of the most important premises in invasive species management: response measures are far more likely to be cost-efficient and effective if taken either prior to an organism's introduction into the United States or early in the invasion process (as soon as feasible after the non-native species has been detected and accurately identified) (Leung et al. 2002). Fundamentally, we need to determine how to balance the mandate to assess risk as accurately as possible with the imperative to act as quickly as possible.
In general terms, risk screening has been regarded as the use of simple tests to identify individual species that have risk factors or are at the early stages of exhibiting an adverse impact for which there is not yet clear symptomology or other evidence of harm. For the purposes of our paper, we regard risk screening as a rapid characterization of the types and degree of risks posed by a population of non-native species in a particular spatio-temporal context. Risk screening is employed to efficiently ascertain if the likelihood and scale of impacts are (a) ''low,'' warranting no response measures other than making these findings publicly available (per discussion in Reaser et al. 2019a, this issue); (b) ''high,'' warranting immediate, priority action as feasible, including consistency with regulatory frameworks that might require more detailed risk analyses as a next step (Burgos-Rodríguez and Burgiel 2019, this issue); or (c) ''uncertain,'' due to a paucity of reliable information, which necessitates more extensive data collection and analysis before response measures are considered.
At a minimum, science-based risk screening requires accurate taxonomic identification of the species (Lyal and Miller 2019, this issue) and the best available data on the biology of the species, habitat associations, distributions in native and introduced ranges, and the species' impact in similar ecological contexts. A wide range of specific biological and environmental parameters have been used by risk assessors, including diet, fecundity, competitiveness, propagule pressure, dispersal patterns (pathways), detectability, and longevity (Ruesink et al. 1995;Kolar and Lodge 2002;Bartell and Nair 2003;Fujisaki et al. 2010;Reed et al. 2012). Risk screening can function as a meta-analysis. If sufficient data are available, various analytical tools (e.g., climate matching, niche modeling) can be used to improve the confidence level of risk screening outputs. Specific examples of risk screening processes and their informational components are described later in this paper.
Influenced by the aforementioned OTA and NSTC assessments, NISC's first national management plan stated that NISC would develop ''a fair, feasible, and risk-based screening system'' for intentional nonnative species introductions by 2003 (NISC 2001). Although a resulting guidance document emphasized the importance of conducting ''rapid risk assessments'' to inform decision-makers of potential management decisions (NISC 2003), development of a screening system was deferred to NISC's second national management plan (NISC 2008). In response to this direction, USDA and DOI independently developed risk screening systems, most notably the Weed Risk Assessment Process (USDA 2016) and Ecological Risk Screening Summaries (US Fish and Wildlife Service 2016).
Recognizing the need for the federal government and its partners to adopt and implement a standardized risk screeding framework consistent with federal regulatory requirements (see Burgos-Rodríguez and Burgiel 2019, this issue), the 2016-2018 NISC Management Plan called for an assessment of ''the capacity of the Federal government to conduct the risk analyses and horizon scanning necessary to produce timely and well-informed watch lists of potentially harmful species'' (NISC 2016). For the purpose of this paper, we use the term ''risk screening'' (per the earlier definition) to stress the inherently rapid nature of the evaluation. Ideally, risk screening is conducted in hours to days in order to minimize conflicts with the commerce, tourism, and other economic activities that, in accordance with relevant laws and policies, may need to be put on hold until risk levels and appropriate mitigation measures are determined. However, a lack of the necessary data inputs, sufficient staff to conduct the analyses, and coordination among federal and non-federal institutions frequently result in delays that negate the opportunity for ''rapid'' response. Reaser et al. (2019a, this issue) illustrate where risk screening fits into a comprehensive EDRR system. Complementary guidance for the application of target analysis (Morisette et al. 2019, this issue) and watch lists (Reaser et al. 2019c, this issue) to EDRR are addressed elsewhere in this special issue. The remainder of this paper explores the current capacities and needs for establishing a science-based invasive species risk screening system. Although this paper is focused on risk screening in the federal context, a standardized risk screening system would ideally meet the needs of states, territories, tribes, and regional collaboratives of those entities, as well as serve as a decision-support tool for the private sector.

Approach
In response to the 2016-2018 NISC Management Plan directive to assess federal capacity for EDRR, the NISC Secretariat invited the sixteen federal agency members represented by Council leadership (https:// www.doi.gov/invasivespecies/about-nisc, accessed 12 February 2019) to respond to a survey on federal EDRR implementation capacity (SI 1, Reaser et al. 2019a, this issue), including their ability to enact a range of tools and processes to evaluate invasive species risk. In the survey's glossary of terms, risk screening was described as ''a preliminary assessment of the consequences of the introduction and of the likelihood of establishment of an alien species using science-based information. Identification of potential adverse consequences in the risk screening could lead to a full risk assessment.'' Although this definition is not identical to the one used herein, we regard them as identical in intent and thus comparable for the purposes of analysis and reporting (Reaser et al. 2019a, this issue).
The findings, discussion, and recommendations presented herein are based on the survey responses provided by the federal agencies, augmented by discussions with risk assessment experts (within and outside the federal government), programmatic information available online, and the expertise of the authors. The findings need to be considered in light of the fact that (1) the management plan directive and data call used the term ''risk analysis'' when a focus on ''risk screening'' was more appropriate in the EDRR context (terminology might have limited explicit reporting on risk screening activities conducted by federal agencies), (2) not all agencies responded (risk screening may not be applicable to their mission), and (3) the depth of responses varied substantially among agencies (see Reaser et al. 2019a, this issue, for further discussion on agency responses).  Table 4 in Reaser et al. 2019a, this issue). Those responses, in combination with the information gathered to augment them, revealed that numerous semi-quantitative, quantitative, and qualitative risk evaluation protocols have been employed by federal agencies. Many of the protocols are permutations of each other rather than unique protocols. While similar in intent, they differ in various parameters, including the content and scope of questions asked by subject matter experts, scoring methodologies, and species traits being assessed. Although these differences may be viewed as minor in concept, any difference in protocol design could result in differences in protocol outputs, and thus variations in the interpretation of risk and appropriateness of response measures.

Survey responses
Our evaluation of the survey results led to the following major observations regarding the federal government's current capacities to enact ''a fair, feasible, and risk-based screening system'' for invasive species: 1. Mandates and terminology for invasive species risk evaluation vary among the agencies, potentially leading to differences in risk communication and management. 2. In general, agencies are not sufficiently supported (including funding, staffing, training, and guidelines) to meet risk evaluation needs. This can result in substantial time delays (potentially years) for outputs, and thus well-informed and timely action. 3. Due to resource limitations, many federal agencies frequently collaborate with, or rely wholly upon, state agencies' or nongovernmental organizations' risk evaluations. 4. There is no standardized approach to invasive species risk evaluation within or among federal agencies, and approaches may even differ at the site level (e.g., among DOD installations). 5. There is a need for adequate information inputs.
See Reaser et al. (2019b, this issue), for further discussion of information management capacity needs. 6. The technical competency of risk screening personnel is essential to ensure timely, accurate outputs. Training needs to be standardized, routine, and include assessment measures. 7. Risk evaluation tools are frequently developed in response to funding opportunities rather than in response to specific agency mandates and criteria. 8. Public access to agency risk evaluation frameworks and/or risk evaluation outputs via agency websites is mostly poor (authors frequently encountered outdated information, inconsistent information, and broken links to documents). 9. While some federal agencies have a track record of applying risk analyses or risk assessments in their efforts to prevent the introduction or spread of invasive species, far less attention has been given to the development of risk screening protocols and tools for EDRR. 10. Invasive species risk screening has mostly focused on species characterization. Very limited pathway risk screening has been conducted by federal agencies.
The lack of consistency in risk evaluation approaches across the federal government contributes to operational inefficiencies: miscommunication, time delays, and duplicative effort. A detailed operational evaluation is needed to document case studies and ascertain how to better standardize risk evaluation while maintaining process requirements that, in some cases, are directed by long-standing regulations and/or enculturated practices. Below we highlight two agency invasive species risk evaluation approaches that we believe hold promise for application within a national EDRR program, if modified as necessary to function as compatable, standardized risk screening tools.

Ecological risk screening summaries
The USFWS' Fish and Aquatic Conservation (FAC) invasive species program comprises 65 offices in eight regions that collaborate with other federal agencies, tribes, states, territories, Canada, Mexico, private landowners, and nongovernmental organizations. To advance the development of risk screening tools called for in the 2008-2010 NISC Management Plan, the USFWS Midwest Region developed a model rapid screening process. During development of that process, USFWS received input from the Aquatic Nuisance Species Task Force's (ANSTF) Mississippi River Basin Regional Panel on Aquatic Nuisance Species (MRBP). Later, MRBP peer-reviewed the model process. Following that review, the Nonnative Wildlife Screening Workgroup, which operated under the joint aegis of ANSTF and NISC also peerreviewed the model rapid screening process. A final peer-review of that process and associated products were conducted pursuant to the Office of Management and Budget's policies for influential science. This work provided a foundation for developing the Ecological Risk Screening Summaries (ERSS), a protocol designed to provide an efficient and costeffective, rapid risk assessment for non-native species, In particular, the ERSS process was developed to evaluate the risks posed to US assests by (1) species native to parts of the United States but established outside their native range within the United States (e.g., red swamp crayfish [Procambarus clarkii]); (2) species in trade in the US but not known to be established in the US; (3) species not known to be in trade within the US, but known to be in one or more trade pathways globally or continentally (e.g., golden mussel [Limnoperna fortunei]); and (4) species not known to be in trade within the United States and not known to be in global trade, but of some concern because of well-documented impacts elsewhere (e.g., certain freshwater stingrays).
The ERSS quantitatively compares the location (or likely location) of species introduction in the United States to the present climate niche elsewhere in the world. It also evaluates the historical invasiveness of the subject species (Hayes and Barry 2008). The ERSS process thus combines relevant databases, scientific literature, and either of two climate-niche matching tools: (1) CLIMATCH, a long-established Australian model that evaluates risk that the subject species may become established outside the species' native range (https://climatch.cp1.agriculture.gov.au/climatch. jsp), accessed 16 September 2019); or (2) a faster and more user-friendly Risk Assessment Mapping Tool (RAMP) that is based on the CLIMATCH algorithm and was developed by USFWS (Sanders et al. 2014). Ultimately, the ERSS is a decision support tool; it enables a wide variety of policy and management decisions, particularly when a rapid response is required (See Fig. 1).
The results of both climate-niche matching tools provide an approximate geographic range in the United States within which climate is similar to other locations where the subject species is established (Bomford 2008;Bomford et al. 2010;Howeth et al. 2016). Evaluation of the history of invasiveness determines if there is clear, convincing, scientifically-reliable evidence of adverse impacts by the subject species outside its native range. Sources of information used to categorize history of invasiveness are either prescribed lists of scientific resources or other reliable information. Significant adverse effects and detailed descriptions of the impacts are listed in the ERSS. An ERSS is developed either in response to requests from jurisdictions or based on prioritized species lists resulting from global horizon scanning. It may be used by governments, industries, and other stakeholders to characterize risks associated with species, and to support either sustainable supply chains nationally and regionally, or appropriate regulatory or non-regulatory risk management decisions and subsequent actions.
The USFWS, in consultations with the US Forest Service, has developed a second peer-reviewed tool to be used when the overall risk level of a freshwater fish species assessed through the ERSS process is uncertain. The tool is a Bayesian risk assessment model called the Freshwater Fish Injurious Species Risk Assessment Model (FISRAM), which predicts invasiveness of a species based on the known and projected characteristics of the species that may not be accounted for in the ERSS (Marcot et al. 2019). FISRAM was used to assess risk of African longfin eel (Anguilla mossambica) aquaculture in Michigan (Wyman-Grothem et al. 2018). The USFWS is developing a similar Bayesian Crayfish Invasiveness Risk Assessment Model (CIRAM) (C. Martin, pers. comm).
To facilitate an assessor's use of the ERSS process, the USFWS published a detailed standard operating procedures (SOP) manual containing a template for report completion, data sources for aquatic and terrestrial animals and plants, details of how to complete the report template, and a comprehensive quality assurance and control checklist to enable a reviewer to determine if the ERSS process has been completed properly (USFWS 2016). The SOP requires inserting scientific data, information, and summaries into the ERSS template. Completed ERSS templates document the best available scientific information on native species range and status in the United States, biological and ecological information, adverse impacts of any introductions, global distribution, US distribution, and results of climate-niche matching. The materials in each completed ERSS categorize risk as low, high, or uncertain. Each completed ERSS template is accompanied by a completed quality assurance/quality control checklist to ensure the ERSS product is in compliance with the SOP. Where climate matching is either high or medium within the United States, habitat matching may be completed for portions of the United States. Habitat matching approaches may be qualitative or Fig. 1 a Illustrates the two key information inputs (data needs) to determine the overall risk of a specific non-native species. b illustrates the three options qualifying the certainty of a risk screen determination: high, uncertain, and low. The quality and quantity of data available to determine the history of invasiveness and climate match will have a strong influence on output certainty quantitative and depend on the availability of quantitative habitat matching tools for the subject species. Integrated species, pathway, and locational risk analyses have been completed for Asian carp in the Great Lakes and other situations. The integrated risk analysis for Asian carp in the Great Lakes was the foundation for the early detection program developed and implemented by the Midwest Region of the USFWS. User manuals for RAMP (USFWS 2019a) and FISRAM (USFWS 2019b) are available to facilitate implementation of the suite of ERSS assessment tools.
After an ERSS has undergone internal USFWS review, the ERSS is posted online (https://www.fws. gov/fisheries/ANS/species_erss_reports.html, accessed 27 September 2019) and catalogued by taxonomic groupings and level of assessed risk. The USFWS website enables public comment on individual ERSSs. The ERSS is a screening process that can either stand alone or serve as the basis for further risk evaluation. Completed ERSSs are intended to identify high-risk species for which preventative measures could be implemented, and low-risk species (nationally, regionally, by jurisdiction) for which outreach to the supply chain can categorize a species, region, and pathway as sustainable. A categorization of high risk can be used to inform states and the private sector (such as importers of live animals) of the risks of importing or transporting assessed species, as well as USFWS's injurious wildlife listing process under the authority of the Lacey Act (18 USC § 42). ERSSs can be utilized to prioritize the higher risk species for initiating the highly detailed and lengthy injurious wildlife listing process, which also includes opportunities for public comment. Prioritization for each species can be assigned using ERSS outputs and a USFWS scoring system.
As of the end of 2017, USFWS had conducted an initial evaluation of about 40,000 species, including 33,500 fish species from Fishbase (http://www. fishbase.org/search.php, accessed 27 September 2019). Criteria for including non-native species in the risk screening process included one or more of the following: (1) currently established in the US, (2) presence in US trade but not yet established in the US, (3) not established or in trade in the US but in trade pathways elsewhere that present concern for US entry, and (4) not known to be in trade but of concern for other reasons (e.g., known impacts elsewhere or associations with invasive species). The scanning process involved detailed evaluation of invasive species databases, review of scientific literature, and consultations with scientists from around the world. Based on evidence and expert opinion, USFWS utilized the ERSS approach to select approximately 1800 species from the initial scan for further risk screening. Some of those species have also been evaluated using the FISRAM tool. FISRAM model outputs are typically completed in a day or less. A trained assessor with a sufficient expertise in the ecology of the taxa being evaluated, as well as familiarity with operating climate-niche matching software, may be able to use the USFWS SOP to complete a draft ERSS assessment in 2-4 h. However, when the acquisition of data, information, and expert input are insufficient, the process may take substantially longer.
Following internal agency review, these ERSS products are posted online as publicly available watch lists (see Reaser et al. 2019c, this issue for discussion of watch lists in EDRR context). USFWS noted, in its response to the NISC data call, that ERSS-determined high-priority watch list species are identified at national, regional, and state scales, while other watch list species are identified by a number of states as part of their state aquatic nuisance species initiatives (Reaser et al. 2019c, this issue). A central clearinghouse of watch lists is needed to support rapid response actions; a clearinghouse would make watch lists more readily accessible, facilitate survey prioritization, and, if the watch lists were standardized, enable spatio-temporal risk comparison.

APHIS-PPQ tiered weed risk evaluation
Plant Protection and Quarantine (PPQ), a program within the USDA's Animal and Plant Health Inspection Service (APHIS), is responsible for safeguarding American agriculture and natural resources from the introduction and spread of plant pests, including weeds and invasive plants (Groves et al. 2001;Magarey et al. 2017 To determine whether a plant species poses a noxious weed risk, PPQ evaluates weeds using a tiered approach (A. Koop, pers. comm). PPQ may initiate an evaluation for a variety of reasons, including (1) detection of a species that is new to the United States, (2) change in the invasive status of a species present in the United States or elsewhere in the world, (3) evaluation of a market access request for a plant for planting, or (4) evaluation of a pathway that may allow the entry of a potential weed (USDA 2016). Plants are initially screened to determine if they pose a potential threat and that they are not yet present in the US, or if present, are not widely established there. These plants then undergo a weed characterization evaluation, which is based on basic biological information about the species' identity, distribution, invasive status, and impacts. PPQ policy managers use this information to determine whether the plants are potential targets for exclusion from importation. Plants that are potential candidates for regulation under NAPPRA are formally evaluated through the NAPPRA process (7 CFR § 319.37-4), whereas plants that are potential candidates for regulation as a federal noxious weed are evaluated under PPQ's weed risk assessment (WRA) process (USDA 2016; see Fig. 2).
PPQ's WRA process is the agency's most detailed weed evaluation tool (USDA 2016). It consists of approximately 90 questions that address a species' ability to enter, establish, spread, and harm US agricultural and natural resources. The assessment includes three major analytical components. The first component is a predictive tool that uses a logisticregression model, which was validated with 204 known US noninvaders, minor-invaders, and majorinvaders (Koop et al. 2012). The outcome of the model is expressed in qualitative terms such as low risk, high risk, and evaluate further. It also expresses probabilities that a plant will become an invader. The second component of the WRA process evaluates the sensitivity of the predicted risk scores to uncertainty through  Koop, pers. comm) illustrates the flow of information from process initiation to management decision (output). Species classified for ''No Action'' either widely naturalized, widely cultivated and economically beneficial, or are native to a wide region of the United States. These species do not meet the criteria of being a quarantine pest. Species that are placed ''On Hold'' could be quarantine pests, but without enough evidence to confirm invasiveness or potential to cause harm. Assessed species that obtain a high risk result (or at times moderate) during characterization are evaluated further and are potential candidates for regulatory action. Depending on the outcome of these evaluations, these species may be considered for regulation as a Federal Noxious Weed (FNW) or under the Not Authorized Pending Pest Risk Analysis (NAPPRA) category. For additional information on the weed risk assessment or NAPPRA processes, go to www.aphis.usda.gov an analysis, which generates a range of other possible scores if some of the answers in the WRA were to change slightly. The higher the uncertainty surrounding the evidence considered in the WRA, the more likely the answers to the individual questions will vary (Caton et al. 2018). The third component of the WRA process incorporates a climate-matching analysis to determine which US areas are suitable for species establishment (Magarey et al. 2017). PPQ developed detailed guidelines to allow anyone to conduct a WRA using its process (USDA 2016). By design, PPQ WRAs do not include any policy decisions, which allows any US stakeholder to use them as a resource for their own decision-making processes (A. Koop, pers. comm).
Prior to PPQ's adoption of its predictive model in 2012 (Koop et al. 2012), a WRA using the previous, narrative-based process (Lehtonen 2001) took 2-8 weeks to complete (Parker et al. 2007). With the 2012 WRA model, which is compliant with PPQ's analytical, regulatory, and management requirements, an analyst may complete an evaluation in 1-2 weeks depending on the species and the assessor's expertise (A. Koop, pers. comm). Additional time is needed for internal review of the assessment. Despite the relative efficiency of the WRA and NAPPRA processes, decisions to add a species to one or more regulatory lists may take years due to required compliance with other processes mandated by federal statutes and regulations, including economic impact assessment, NEPA compliance, and public comment periods. For example, it took 4 years to add 22 species of plants, which are potentially invasive, under the NAPPRA regulations (82 Fed Reg 116: 22786-77792, June 19, 2017). PPQ publishes on its website all NAPPRA evaluations used to support rulemaking (https:// www.aphis.usda.gov/aphis/ourfocus/planthealth/importinformation/permits/plants-and-plant-products-permits/ plants-for-planting/CT_NAPPRA, accessed 27 September 2019) and all weed risk assessments it conducts (https://www.aphis.usda.gov/aphis/ourfocus/ planthealth/plant-pest-and-disease-programs/pests-anddiseases/sa_weeds/sa_noxious_weeds_program/ct_ riskassessments, accessed 27 September 2019).

Key findings and conclusion
The need for the United States to develop a fair, feasible, science-based risk screening framework has been recognized for decades. Our findings indicate that the US approach to risk evaluation across the federal government remains largely ad hoc and underresourced, particularly with regard to meeting staffing needs. The demand for risk evaluation exceeds current agency capacities (as also noted by Reaser and Waugh 2007), resulting in a backlog of risk assessment and reporting. In the context of risk screening for EDRR, time is of the essence. Some combination of staff expansion and technology application (e.g., developing machine learning tools; Martinez et al. 2019, this issue) will likely be necessary to support risk screening.
Risk screening capacity is also limited by information system capacity. Reaser et al. (2019b, this issue) explicitly address federal information system needs in the EDRR context. From a risk screening perspective, there is a clear need to incorporate wildlife and plant import data into open-access information systems so that species, quantities of imports, and country of origin data are readily available. Federal prioritization of risk screening for species new to trade or increasing in trade popularity could help reduce propagule pressure. Priorization could be accomplished through new or component databases that curate species-intrade data that is contributed by industries or by harvesting non-native species trade data from social media via webcrawling tools (Reaser et al. 2008).
Poorly coordinated efforts can facilitate duplication of programs and processes, policy and programmatic confusion, inefficient use of limited resources, gaps in information sharing, and inconsistencies in regulatory enforcement. Barriers to the development of a federalwide risk screening framework have included differences in agency missions and cultures, as well as a lack of funding and personnel. Any effort to build a multiagency information framework to serve EDRR nationwide will need to carefully consider the differences in agency culture and how to foster a unified mission (Reaser et al. 2019a, this issue). Agency responses also suggest there is a lingering need to enhance collaborative, voluntary partnerships with states, tribes, industries, and other stakeholders that need to utilize risk screening frameworks and tools.
Our assessment also reveals that there are at least two US-developed risk screening tools available on which to build future efforts. Although, these tools are relatively new and thus have not yet been used widely for EDRR decision support, they show promise for broad application. They also demonstrate the need for the United States to focus on the development of a flexible, risk screening toolkit to support a standardized, transparent risk screening framework consistent with regulatory requirements (e.g., trade rule compliance). Other governments have come to similar conclusions; different taxonomic groups may require individualized, yet complementary variations in risk evaluation strategy (e.g., D'hondt et al. 2015;Mumford et al. 2010;Baker et al. 2008). Risk screening of pathogens and parasites may be particularly challenging, but the Belgian Pandora system provides a useful reference point for US consideration (https://ias.biodiversity.be/ harmoniaplus, accessed 30 October 2019).
As the high-level policy and planning body with a ''whole of government'' mandate, NISC has the role and responsibility to facilitate the development of federal risk screening protocols and associated tools. Ideally, such work would proceed with strong input and cooperation from other government entities, the private sector, and technical experts from various scientific disciplines. In addition to the priorities already set forth in NISC management plans, our assessment indicates that the following actions would advance federal risk screening capacities, particularly within the EDRR context: 1. Creation of a user-friendly, open access, centralized, searchable clearinghouse of risk evaluation protocols, tools, and completed assessments available in the United States, with links to those produced elsewhere (e.g., D'hondt et al. 2015). The clearinghouse should incorporate and/or interface with similarly intended projects at the subnational and multi-national level, as discussed previously in this paper. 2. Timely, ongoing submission of information on risk evalution approaches and outputs into the clearinghouse. Ideally, contributions would not be limited to federal agencies, but be open to all parties conducting invasive species risk evaluation. 3. Development of performance metrics (e.g., reliability/verifiability, timeliness, transparency, replicability, accessability, cost effectiveness) for the risk screening capacities needed to support a national EDRR program and evaluation of the risk screening protocols and tools in the clearinghouse according to these metrics, with the findings, including data relied upon, made publicly available. 4. Based on the findings of (3), development of one or more invasive species risk screening protocols and associated tools for standardized use by federal agencies and their partners, keeping in mind that protocols and tools may need to be delineated by taxonomic group, as discussed previously in this paper. Where a rapid screening process characterizes a species risk as uncertain, supplemental tools may be needed to better evaluate that risk (e.g., FISRAM). 5. Development of complementary, science-based risk analytical protocols and tools (e.g., pathwayoriented risk screening, habitat-matching, climate-matching, horizon scanning) to facilitate and validate risk screening, as well as to assist with EDRR target analysis (per Morisette et al. 2019, this issue). A useful resource could include a model currently under development by the USFWS Midwest Region that promotes an integrated approach to species, pathways, and habitat/ locational risk analyses for Asian carp in the Great Lakes.
Risk screening should be considered a key component of any EDRR program. Review articles indicate that an increasing number of evidence-based risk analyses are available for potential uses in the United States and abroad (e.g., Lodge et al. 2016). Although these tools can provide a useful conceptual framework for decisionmaking, they are not designed as an initial filter or risk screen and may necessitate significant resources (training, time, staffing, funding, data) that make them impractical for broad application.
There are clear benefits to ensuring that risk screening tools are standardized (such as those within the USFWS and APHIS systems, allowing for consistent and comparable outcomes) and made available through an open-access information system for reference. In building a national EDRR program, improved coordination and collaboration among relevant government agencies is essential for identifying programmatic challenges, developing minimum standards, publishing SOPs for risk screening protocols and tools, and launching a user-friendly, open access clearinghouse. A lack of sufficient data, including empirical evidence of harm, may be the greatest constraint to risk screening (as well as robust risk analyses; Roy et al. 2018). More studies that focus on evaluating impacts of non-native species established in the United States will provide evidence of harm for some species that have been screened and assessed as uncertain risk by one or more scientific tools. Federal capacities to enact sufficient risk screening capacities are thus also inextricably linked to federal support of a national invasive species information infrastructure (Reaser et al. 2019b, this issue).
Effective implementation of the recommendations contained herein will significantly improve coordination, cost-efficiencies, and collaboration across agencies with responsibilities for the detection and/or management of invasive species. Ultimately, this will improve the US government's ability to prevent the further introduction and spread of invasive species, thus protecting national assets.