Biological Invasions

, Volume 20, Issue 8, pp 2107–2119 | Cite as

Comparison of model selection technique performance in predicting the spread of newly invasive species: a case study with Batrachochytrium salamandrivorans

  • Tatum S. Katz
  • Amanda J. ZellmerEmail author
Original Paper


Species distribution models (SDMs) increasingly have been used to anticipate the spread of invasive species. However, these models are powerful conservation tools only if they are biologically relevant, and thus validation of these models is essential. Here, we evaluate four model selection frameworks for their ability to identify a best fit model of spread under low data conditions early in an invasion, specifically testing the efficacy of methods that utilize absence data in addition to presence data in evaluating models. We test this question using a simulation where we generated data with varying confidence in the accuracy of the absence data, as absences in early invasions may become presences in the future, and increasing quantity of observation data to test the models. We create these simulations based on a real-world example of a newly emergent, invasive fungal pathogen, Batrachochytrium salamandrivorans (Bsal). The simulation demonstrated that AIC and Likelihood consistently outperform both Kappa and AUC in selecting the true model as the best model when data are limited and absence data are low quality, with AIC providing the most conservative results due to penalties for overparameterization. With these results, we then used these techniques to compare five candidate models for predicting the spread of Bsal. Consistent with the simulation, the best fit model of the candidate models for Bsal was inconsistent across the four metrics. However, AIC, which performed best in the simulation study, suggested that the spread of Bsal into Western Europe was best predicted by a combination of bioclimatic suitability, salamander richness, and number of salamander imports. Our results highlight the difficulty in evaluating predictive models when data are limited and of low quality, as with a newly invasive species, but show that these challenges can be partially addressed with the appropriate model selection approach. Use of this approach is critical as SDMs of invasive species are often used to inform conservation policy and efforts before the invasion spreads, when limited data are available.


AUC AIC Kappa Likelihood Amphibian conservation Chytrid Maxent Species distribution model 



We thank the Occidental College Undergraduate Research Center for funding. Thanks to J. McCormack and E. Braker for helpful comments on the manuscript. Thanks also to T. Yap for additional information on previously proposed bioclimatic suitability models.

Supplementary material

10530_2018_1690_MOESM1_ESM.docx (3 mb)
Supplementary material 1 (DOCX 3120 kb)


  1. Allouche O, Tsoar A, Kadmon R (2006) Assessing the accuracy of species distribution models: prevalence, Kappa and the true skill statistic (TSS). J Appl Ecol 43:1223–1232. CrossRefGoogle Scholar
  2. Anderson RP, Lew D, Peterson AT (2003) Evaluating predictive models of species’ distributions: criteria for selecting optimal models. Ecol Model 162:211–232. CrossRefGoogle Scholar
  3. Bales EK, Hyman OJ, Loudon AH et al (2015) Pathogenic chytrid fungus Batrachochytrium dendrobatidis, but not B. salamandrivorans, detected on eastern hellbenders. PLoS ONE 10:e0116405. CrossRefPubMedPubMedCentralGoogle Scholar
  4. Baums J (2015a) project_maxent.R. Accessed 1 Jan 2017
  5. Baums J (2015b) parse_lambdas.R. Accessed 1 Jan 2017
  6. Bosso L, De Conno C, Russo D (2017) Modelling the risk posed by the Zebra Mussel Dreissena polymorpha: Italy as a case study. Environ Manage 60:304–313. CrossRefPubMedGoogle Scholar
  7. Burnham KP, Anderson DR (2003) Model selection and multimodel inference. Springer, New YorkGoogle Scholar
  8. Carlos-Júnior LA, Barbosa NPU, Moulton TP, Creed JC (2015) Ecological niche model used to examine the distribution of an invasive, non-indigenous coral. Mar Environ Res 103:115–124. CrossRefPubMedGoogle Scholar
  9. CITES (2016) CITES trade database. Accessed 8 May 2017
  10. Diao C, Wang L (2014) Development of an invasive species distribution model with fine-resolution remote sensing. Int J Appl Earth Obs Geoinf 30:65–75. CrossRefGoogle Scholar
  11. Elith J, Graham CH, Anderson RP et al (2006) Novel methods improve prediction of species’ distributions from occurrence data. Ecography (Cop) 29:129–151. CrossRefGoogle Scholar
  12. Elith J, Kearney M, Phillips S (2010) The art of modelling range-shifting species. Methods Ecol Evol 1:330–342. CrossRefGoogle Scholar
  13. Elith J, Phillips SJ, Hastie T et al (2011) A statistical explanation of MaxEnt for ecologists. Divers Distrib 17:43–57. CrossRefGoogle Scholar
  14. Feldmeier S, Schefczyk L, Wagner N et al (2016) Exploring the distribution of the spreading lethal salamander chytrid fungus in its invasive range in Europe—a macroecological approach. PLoS ONE 11:e0165682. CrossRefPubMedPubMedCentralGoogle Scholar
  15. Ficetola GF, Thuiller W, Miaud C (2007) Prediction and validation of the potential global distribution of a problematic alien invasive species—the American bullfrog. Divers Distrib 13:476–485. CrossRefGoogle Scholar
  16. Fielding AH, Bell JF (1997) A review of methods for the assessment of prediction errors in conservation presence/absence models. Environ Conserv 24:38–49CrossRefGoogle Scholar
  17. Freeman EA, Moisen GG (2008) A comparison of the performance of threshold criteria for binary classification in terms of predicted prevalence and Kappa. Ecol Model 217:48–58. CrossRefGoogle Scholar
  18. Golicher D, Ford A, Cayuela L, Newton A (2012) Pseudo-absences, pseudo-models and pseudo-niches: pitfalls of model selection based on the area under the curve. Int J Geogr Inf Sci 26:2049–2063. CrossRefGoogle Scholar
  19. Gray MJ, Lewis JP, Nanjappa P et al (2015) Batrachochytrium salamandrivorans: the North American response and a call for action. PLoS Pathog 11:e1005251. CrossRefPubMedPubMedCentralGoogle Scholar
  20. Hastie T, Tibshirani R, Friedman JH (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New YorkCrossRefGoogle Scholar
  21. Hijmans RJ, Cameron SE, Parra JL et al (2005) Very high resolution interpolated climate surfaces for global land areas. Int J Climatol 25:1965–1978. CrossRefGoogle Scholar
  22. IUCN (2017) The IUCN red list of threatened species. Accessed 8 May 2017
  23. Kramer AM, Annis G, Wittmann ME et al (2017) Suitability of Laurentian Great Lakes for invasive species based on global species distribution models and local habitat. Ecosphere 8:e01883. CrossRefGoogle Scholar
  24. Lobo JM, Jiménez-Valverde A, Real R (2008) AUC: a misleading measure of the performance of predictive distribution models. Glob Ecol Biogeogr 17:145–151. CrossRefGoogle Scholar
  25. Martel A, Spitzen-van der Sluijs A, Blooi M et al (2013) Batrachochytrium salamandrivorans sp. nov. causes lethal chytridiomycosis in amphibians. Proc Natl Acad Sci USA 110:15325–15329. CrossRefPubMedGoogle Scholar
  26. Martel A, Blooi M, Adriaensen C et al (2014) Recent introduction of a chytrid fungus endangers Western Palearctic salamanders. Science 346:630–631. CrossRefPubMedPubMedCentralGoogle Scholar
  27. Merow C, Smith MJ, Silander JA (2013) A practical guide to MaxEnt for modeling species’ distributions: what it does, and why inputs and settings matter. Ecography (Cop) 36:1058–1069. CrossRefGoogle Scholar
  28. Muscarella R, Galante PJ, Soley-Guardia M et al (2014) ENMeval: an R package for conducting spatially independent evaluations and estimating optimal model complexity for MAXENT ecological niche models. Methods Ecol Evol 5:1198–1205. CrossRefGoogle Scholar
  29. Nakazawa M (2017) fmsb: functions for medical statistics book with some demographic data. R package version 0.6.1Google Scholar
  30. Nguyen TT, Van Nguyen T, Ziegler T et al (2017) Trade in wild anurans vectors the urodelan pathogen Batrachochytrium salamandrivorans into Europe. Amphibia Reptil. CrossRefGoogle Scholar
  31. Parrott JC, Shepack A, Burkart D et al (2016) Survey of pathogenic chytrid fungi (Batrachochytrium dendrobatidis and B. salamandrivorans) in salamanders from three mountain ranges in Europe and the Americas. EcoHealth 14:296–302. CrossRefPubMedGoogle Scholar
  32. Peterson AT, Vieglais D (2001) Predicting species invasions using ecological niche modeling: new approaches from bioinformatics attack a pressing problem. Bioscience 51:363–371.[0363:PSIUEN]2.0.CO;2 CrossRefGoogle Scholar
  33. Phillips SJ (2008) Transferability, sample selection bias and background data in presence-only modelling: a response to Peterson et al. (2007). Ecography (Cop) 31:272–278. CrossRefGoogle Scholar
  34. Phillips SJ, Anderson RP, Schapire RE (2006) Maximum entropy modeling of species geographic distributions. Ecol Model 190:231–259CrossRefGoogle Scholar
  35. Phillips SJ, Dudík M, Schapire RE (2010) [Internet] Maxent software for modeling species niches and distributions (Version 3.3.3k). Available from url: Accessed 1 May 2017
  36. Phillips SJ, Anderson RP, Dudík M et al (2017) Opening the black box: an open-source release of Maxent. Ecography (Cop) 40:887–893. CrossRefGoogle Scholar
  37. R Development Core Team (2014) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.
  38. Richgels KLD, Russell RE, Adams J et al (2016) Spatial variation in risk and consequence of Batrachochytrium salamandrivorans introduction in the USA. R Soc Open Sci 3:150616. CrossRefPubMedPubMedCentralGoogle Scholar
  39. Robin X, Turck N, Hainard A et al (2011) pROC: an open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics 12:77CrossRefPubMedPubMedCentralGoogle Scholar
  40. Robinson TP, van Klinken RD, Metternicht G (2010) Comparison of alternative strategies for invasive species distribution modeling. Ecol Model 221:2261–2269. CrossRefGoogle Scholar
  41. Rodríguez JP, Brotons L, Bustamante J, Seoane J (2007) The application of predictive modelling of species distribution to biodiversity conservation. Divers Distrib 13:243–251. CrossRefGoogle Scholar
  42. Somodi I, Lepesi N, Botta-Dukát Z (2017) Prevalence dependence in model goodness measures with special emphasis on true skill statistics. Ecol Evol 7:863–872. CrossRefPubMedPubMedCentralGoogle Scholar
  43. Spitzen-van der Sluijs A, Martel A, Asselberghs J et al (2016) Expanding distribution of lethal amphibian fungus Batrachochytrium salamandrivorans in Europe. Emerg Infect Dis 22:1286–1288. CrossRefPubMedPubMedCentralGoogle Scholar
  44. Stegen G, Pasmans F, Schmidt BR et al (2017) Drivers of salamander extirpation mediated by Batrachochytrium salamandrivorans. Nature 544:353–356. CrossRefPubMedGoogle Scholar
  45. Václavík T, Meentemeyer RK (2009) Invasive species distribution modeling (iSDM): are absence data and dispersal constraints needed to predict actual distributions? Ecol Model 220:3248–3258. CrossRefGoogle Scholar
  46. Václavík T, Kupfer JA, Meentemeyer RK (2012) Accounting for multi-scale spatial autocorrelation improves performance of invasive species distribution modeling (iSDM). J Biogeogr 39:42–55. CrossRefGoogle Scholar
  47. Vall-llosera M, Woolnough AP, Anderson D, Cassey P (2017) Improved surveillance for early detection of a potential invasive species: the alien Rose-ringed parakeet Psittacula krameri in Australia. Biol Invasions 19:1273–1284. CrossRefGoogle Scholar
  48. VanDerWal J, Falconi L, Januchowski S et al (2015) SDMTools: species distribution modelling tools: tools for processing data associated with species distribution modelling exercises. R package version 1.1–221Google Scholar
  49. Warren DL, Seifert SN (2011) Ecological niche modeling in Maxent: the importance of model complexity and the performance of model selection criteria. Ecol Appl 21:335–342CrossRefPubMedGoogle Scholar
  50. Warren DL, Wright AN, Seifert SN, Shaffer HB (2014) Incorporating model complexity and spatial sampling bias into ecological niche models of climate change risks faced by 90 California vertebrate species of concern. Divers Distrib 20:334–343. CrossRefGoogle Scholar
  51. Yap TA, Koo MS, Ambrose RF et al (2015) Averting a North American biodiversity crisis. Science 349:481–482. CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of BiologyOccidental CollegeLos AngelesUSA
  2. 2.Department of Ecology, Evolution, and Marine BiologyUniversity of CaliforniaSanta BarbaraUSA

Personalised recommendations