Biological Invasions

, Volume 14, Issue 2, pp 273–294 | Cite as

Development and validation of a weed screening tool for the United States

  • Anthony L. KoopEmail author
  • Larry Fowler
  • Leslie P. Newton
  • Barney P. Caton
Original Paper


The Australian weed risk assessment has been promoted as a simple and effective screening tool that can help prevent the entry of weeds and invasive plants into new areas. On average, the Australian model identifies major-invaders more accurately than it does non-invaders (90% vs. 70% accuracy). While this difference in performance emphasizes protection, the overall accuracy of the model will be determined by its performance with non-invaders because the frequency of invasive species among new plant introductions is relatively low. In this study, we develop a new weed risk assessment model for the entire United States that increases non-invader accuracy. The new screening tool uses two elements of risk, establishment/spread potential and impact potential, in a logistic regression model to evaluate the invasive/weedy potential of a species. We selected 204 non-invaders, minor-invaders, and major-invaders to develop and validate the new model, and compare its performance to the Australian model using the same set of species. Performing better than the Australian model, our new model accurately identified 94.1% of major-invaders and 97.1% of non-invaders, without committing any false positives or false negatives. The new secondary screening tool we developed reduced the number of species requiring secondary evaluation from 22 to 12%. We expect that the new weed risk assessment model should significantly enhance the United State’s timeliness and accuracy in regulating potential weeds.


Weed risk assessment ROC analysis Predictive screening tool Base-rate effect Australian WRA 



We would like to thank the following risk analysts who contributed to the assessment of the 204 species: Stephen McLean, Lisa Kohl, Sharon Talley, Sarah Marnell, Stephanie Dubon, Robert Schall, and Sherrie Emerine. Edward Jones provided guidance on statistical analyses. Doria Gordon, Curtis Daehler, and Tomoko Nishida shared their data from their tests of the Australian WRA. Doria Gordon also provided some guidance on modification of the Australian WRA for use in the entire United States and shared the guidance that she and others developed for implementing the Australian WRA. We would like to thank Rob Ahern, Ashley Jackson, and Al Tasker, who provided comments on earlier versions of the manuscript and two anonymous reviewers who provided useful comments.


  1. APHIS (2009) Importation of plants for planting: establishing a category of plants for planting not authorized for importation pending pest risk analysis. Fed Regist 74:36403–36414Google Scholar
  2. Bailey LH, Bailey EZ (1930) Hortus: a concise dictionary of gardening, general horticulture and cultivated plants in North America. The MacMillan Company, New YorkGoogle Scholar
  3. Baker HG (1965) Characteristics and modes of origin of weeds. In: Baker HG, Stebbins GL (eds) The genetics of colonizing species. Academic Press, New York, pp 147–172Google Scholar
  4. Barney JN, DiTomaso JM (2008) Nonnative species and bioenergy: are we cultivating the next invader? Bioscience 58:64–70CrossRefGoogle Scholar
  5. Bewick V, Cheek L, Ball J (2004) Statistics review 13: receiver operating characteristic curves. Crit Care 8:508–512PubMedCrossRefGoogle Scholar
  6. Bridges DC (ed) (1992) Crop losses due to weeds in the United States—1992. Weed Science Society of America, ChampaignGoogle Scholar
  7. Byrd DM, Cothern CR (2005) Introduction to risk analysis: a systematic approach to science-based decision making. Government Institutes, LanhamGoogle Scholar
  8. Crosti R, Cascone C, Cipollaro S (2010) Use of a weed risk assessment for the Mediterranean region of Central Italy to prevent loss of functionality and biodiversity in agro-ecosystems. Biol Invasions 12:1607–1616CrossRefGoogle Scholar
  9. Daehler CC, Carino DA (2000) Predicting invasive plants: prospects for a general screening system based on current regional models. Biol Invasions 2:93–102CrossRefGoogle Scholar
  10. Daehler CC, Strong DR (1993) Prediction and biological invasions. Trends Ecol Evol 8:380PubMedCrossRefGoogle Scholar
  11. Daehler CC, Denslow JS, Ansari S, Kuo HC (2004) A risk-assessment system for screening out invasive pest plants from Hawaii and other Pacific Islands. Conserv Biol 18:360–368CrossRefGoogle Scholar
  12. Dawson W, Burslem DFRP, Hulme PE (2009) The suitability of weed risk assessment as a conservation tool to identify invasive plant threats in East African rainforests. Biol Conserv 142:1018–1024CrossRefGoogle Scholar
  13. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845PubMedCrossRefGoogle Scholar
  14. DPI (2008) Victoria weed risk assessment (WRA) method. Department of Primary Industries (DPI), Victoria, Australia. Accessed 22 March 2011
  15. Fawcett T (2004) ROC graphs: notes and practical considerations for researchers. HP Laboratories. Accessed 4 Oct 2010
  16. FICMNEW (2003) A national early detection and rapid response system for invasive plant species in the United States: conceptual design. Federal Interagency Committee for the Management of Noxious and Exotic Weeds, Washington, DC, USA. Accessed 04 Oct 2010
  17. Fluss R, Faraggi D, Reiser B (2005) Estimation of the Youden index and its associated cutoff point. Biom J 47:458–472PubMedCrossRefGoogle Scholar
  18. Fox AM, Gordon DR, Dusky JA, Tyson L, Stocker RK (2005) IFAS assessment of the status of non-native plants in Florida’s natural areas. University of Florida, Institute of Food and Agricultural Sciences, Florida Cooperative Extension Service, Agronomy Department. Accessed 18 March 2008
  19. Gassó N, Basnou C, Vilà M (2010) Predicting plant invaders in the Mediterranean through a weed risk assessment system. Biol Invasions 12:463–476CrossRefGoogle Scholar
  20. Gordon DR, Gantz CA (2008) Screening new plant introductions for potential invasiveness: a test of impacts for the United States. Conserv Lett 1:227–235CrossRefGoogle Scholar
  21. Gordon DR, Gantz C, Onderdonk DA (2008a) Assessment and impact of the Australian weed risk assessment modified for the US. Hortscience 43:1051Google Scholar
  22. Gordon DR, Onderdonk DA, Fox AM, Stocker RK (2008b) Consistent accuracy of the Australian weed risk assessment system across varied geographies. Divers Distrib 14:234–242CrossRefGoogle Scholar
  23. Gordon DR, Onderdonk DA, Fox AM, Stocker RK, Gantz C (2008c) Predicting invasive plants in Florida using the Australian weed risk assessment. Invasive Plant Sci Manag 1:178–195CrossRefGoogle Scholar
  24. Gordon DR, Mitterdorfer B, Pheloung PC, Ansari S, Buddenhagen C, Chimera C, Daehler CC, Dawson W, Denslow JS, LaRosa A, Nishida T, Onderdonk DA, Panetta FD, Pyšek P, Randall RP, Richardson DM, Tshidada NJ, Virtue JG, Williams PA (2010) Guidance for addressing the Australian weed risk assessment questions. Plant Protec Q 25:56–74Google Scholar
  25. Greiner M, Pfeiffer D, Smith RD (2000) Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med 45:23–41PubMedCrossRefGoogle Scholar
  26. Groves RH, Panetta FD, Virtue JG (2001) Weed risk assessment. CSIRO Publishing, CollingwoodGoogle Scholar
  27. Holm LG, Pancho JV, Herberger JP, Plucknett DL (1979) A geographical atlas of world weeds. Krieger Publishing Company, MalabarGoogle Scholar
  28. Hughes G, Madden LV (2003) Evaluating predictive models with application in regulatory policy for invasive weeds. Agric Syst 76(2):755–774CrossRefGoogle Scholar
  29. IPPC (2009) International standards for phytosanitary measures: 1–32. International plant protection convention (IPPC) and the Food and Agriculture Organization of the United Nations, Rome. Accessed 4 Oct 2010
  30. Jefferson L, Havens K, Ault J (2004) Implementing invasive screening procedures: the Chicago Botanic Garden model. Weed Technol 18:1434–1440CrossRefGoogle Scholar
  31. Kato H, Hata K, Yamamoto H, Yoshioka T (2006) Effectiveness of the weed risk assessment system for the Bonin Islands. In: Koike F, Clout MN, Kawamichi M, De Poorter M, Iwatsuki K (eds) Assessment and control of biological invasion risk. Shoukadoh Book Sellers and IUCN, Kyoto, pp 65–72Google Scholar
  32. Kolar CS, Lodge DM (2001) Progress in invasion biology: predicting invaders. Trends Ecol Evol 16:199–204PubMedCrossRefGoogle Scholar
  33. Kowarik I (1995) Time lags in biological invasion with regard to the success and failure of alien species. In: Pyšek P, Prach K, Rejmánek M, Wade M (eds) Plant invasions—general aspects and special problems. SPB Academic Publishing, Amsterdam, pp 15–38Google Scholar
  34. Kriticos DJ, Sutherst RW, Brown JR, Adkins SW, Maywald GF (2003) Climate change and the potential distribution of an invasive alien plant: Acacia nilotica ssp indica in Australia. J Appl Ecol 40(1):111–124CrossRefGoogle Scholar
  35. Křivánek M, Pyšek P (2006) Predicting invasions by woody species in a temperate zone: a test of three risk assessment schemes in the Czech Republic (Central Europe). Divers Distrib 12:319–327CrossRefGoogle Scholar
  36. Lasko TA, Bhagwat JG, Zou KH, Ohno-Machado L (2005) The use of receiver operating characteristic curves in biomedical informatics. J Biomed Inform 38:404–415PubMedCrossRefGoogle Scholar
  37. Lehtonen PP (2001) Pest risk assessment in the United States: guidelines for qualitative assessments for weeds. In: Groves RH, Panetta FD, Virtue JG (eds) Weed risk assessment. CSIRO, Collingwood, pp 117–123Google Scholar
  38. Lodge DM (1993) Biological invasions: lessons for ecology. Trends Ecol Evol 8:133–137PubMedCrossRefGoogle Scholar
  39. Lonsdale M (2010) Pest risk assessment and invasion pathways: invasive weeds. NZ J For Sci 40:S73–S76Google Scholar
  40. Mack RN (1996) Predicting the identity and fate of plant invaders: emergent and emerging approaches. Biol Conserv 78:107–121CrossRefGoogle Scholar
  41. Mack RN (2005) Predicting the identity of plant invaders: future contributions from horticulture. Hortscience 40:1168–1174Google Scholar
  42. McClay A, Sissons A, Wilson C, Davis S (2010) Evaluation of the Australian weed risk assessment system for the prediction of plant invasiveness in Canada. Biol Invasions. doi: 10.1007/s10530-010-9819-3
  43. Metz CE (1978) Basic principles of ROC analysis. Semin Nucl Med 8:283–298PubMedCrossRefGoogle Scholar
  44. Moles AT, Gruber MAM, Bonser SP (2008) A new framework for predicting invasive plant species. J Ecol 96:13–17Google Scholar
  45. NAPPO (2008) NAPPO regional standards for phytosanitary measures: RSPM#32: pest risk assessment for plants for planting as quarantine pests. North American Plant Protection Organization (NAPPO), Ottawa, Canada. Accessed 4 Oct 2010
  46. NISC (2008) 2008–2012 National Invasive Species Management Plan. National Invasive Species Council (NISC), Washington, D.C. Accessed 11 Aug 2010
  47. Nishida T, Yamashita N, Asai M, Kurokawa S, Enomoto T, Pheloung PC, Groves RH (2009) Developing a pre-entry weed risk assessment system for use in Japan. Biol Invasions 11:1319–1333CrossRefGoogle Scholar
  48. NRCS (1999) Soil taxonomy: a basic system of soil classification for making and interpreting soil surveys. National Resource Conservation Service (NRCS), United States Department of Agriculture, Washington, DC. Accessed 17 March 2008
  49. Onderdonk DA, Gordon DR, Fox AM, Stocker RK (2010) Lessons learned from testing the Australian weed risk assessment system: the devil is in the details. Plant Protec Q 25:79–85Google Scholar
  50. Parker C, Caton BP, Fowler L (2007) Ranking nonindigenous weed species by their potential to invade the United States. Weed Sci 55:386–397CrossRefGoogle Scholar
  51. Peel MC, Finlayson BL, McMahon TA (2007) Updated world map of the Köppen-Geiger climate classification. Hydrol Earth Syst Sci 11:1633–1644CrossRefGoogle Scholar
  52. Peters WL, Meyer MH, Anderson NO (2006) Minnesota horticultural industry survey on invasive plants. Euphytica 148:75–86CrossRefGoogle Scholar
  53. Pheloung PC, Williams PA, Halloy SR (1999) A weed risk assessment model for use as a biosecurity tool evaluating plant introductions. J Environ Manag 57:239–251CrossRefGoogle Scholar
  54. Pimentel D, Lach L, Zuniga R, Morrison D (2000) Environmental and economic costs of nonindigenous species in the United States. Bioscience 50:53–65CrossRefGoogle Scholar
  55. Randall JM, Morse LE, Benton N, Hiebert R, Lu S, Killeffer T (2008) The invasive species assessment protocol: a tool for creating regional and national lists of invasive nonnative plants that negatively impact biodiversity. Invasive Plant Sci Manag 1:36–49CrossRefGoogle Scholar
  56. Reichard SH (2004) Conflicting values and common goals: codes of conduct to reduce the threat invasive species. Weed Technol 18:1503–1507CrossRefGoogle Scholar
  57. Reichard SH, Hamilton CW (1997) Predicting invasions of woody plants introduced into North America. Conserv Biol 11:193–203CrossRefGoogle Scholar
  58. Rejmánek M (2000) Invasive plants: approaches and predictions. Austral Ecol 25:497–506CrossRefGoogle Scholar
  59. Richardson DM, Pyšek P, Rejmánek M, Barbour MG, Panetta FD, West CJ (2000) Naturalization and invasion of alien plants: concepts and definitions. Divers Distrib 6:93–107CrossRefGoogle Scholar
  60. Ruesink JL, Parker I, Groom M, Kareiva P (1995) Reducing the risks of nonindigenous species introductions. Bioscience 45:465–477CrossRefGoogle Scholar
  61. Scott JK, Panetta FD (1993) Predicting the Australian weed status of southern African plants. J Biogeogr 20:87–93CrossRefGoogle Scholar
  62. Smith CS, Lonsdale WM, Fortune J (1999) When to ignore advice: invasion predictions and decision theory. Biol Invasions 1:89–96CrossRefGoogle Scholar
  63. Stohlgren TJ, Schnase JL (2006) Risk analysis for biological hazards: what we need to know about invasive species. Risk Anal. doi: 10.1111/j.1539-6924.2006.00707.x
  64. USDA (2004) Weed-initiated pest risk assessment guidelines for qualitative assessments (v. 5.3). United States Department of Agriculture (USDA), Animal and Plant Health Inspection Service. Accessed 6 Oct 2010
  65. Weber E, Gut D (2004) Assessing the risk of potentially invasive plant species in central Europe. J for Nat Conserv 12:171–179CrossRefGoogle Scholar
  66. White PS, Schwarz AE (1998) Where do we go from here? The challenges of risk assessment for invasive plants. Weed Technol 12:744–751Google Scholar
  67. Williamson M, Fitter A (1996) The varying success of invaders. Ecol 77:1661–1666CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V.(outside the USA) 2011

Authors and Affiliations

  • Anthony L. Koop
    • 1
    Email author
  • Larry Fowler
    • 1
  • Leslie P. Newton
    • 1
  • Barney P. Caton
    • 1
  1. 1.United States Department of Agriculture, Plant Protection and QuarantinePlant Epidemiology and Risk Analysis LaboratoryRaleighUSA

Personalised recommendations