Abstract
Species distribution models (SDMs) increasingly have been used to anticipate the spread of invasive species. However, these models are powerful conservation tools only if they are biologically relevant, and thus validation of these models is essential. Here, we evaluate four model selection frameworks for their ability to identify a best fit model of spread under low data conditions early in an invasion, specifically testing the efficacy of methods that utilize absence data in addition to presence data in evaluating models. We test this question using a simulation where we generated data with varying confidence in the accuracy of the absence data, as absences in early invasions may become presences in the future, and increasing quantity of observation data to test the models. We create these simulations based on a real-world example of a newly emergent, invasive fungal pathogen, Batrachochytrium salamandrivorans (Bsal). The simulation demonstrated that AIC and Likelihood consistently outperform both Kappa and AUC in selecting the true model as the best model when data are limited and absence data are low quality, with AIC providing the most conservative results due to penalties for overparameterization. With these results, we then used these techniques to compare five candidate models for predicting the spread of Bsal. Consistent with the simulation, the best fit model of the candidate models for Bsal was inconsistent across the four metrics. However, AIC, which performed best in the simulation study, suggested that the spread of Bsal into Western Europe was best predicted by a combination of bioclimatic suitability, salamander richness, and number of salamander imports. Our results highlight the difficulty in evaluating predictive models when data are limited and of low quality, as with a newly invasive species, but show that these challenges can be partially addressed with the appropriate model selection approach. Use of this approach is critical as SDMs of invasive species are often used to inform conservation policy and efforts before the invasion spreads, when limited data are available.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allouche O, Tsoar A, Kadmon R (2006) Assessing the accuracy of species distribution models: prevalence, Kappa and the true skill statistic (TSS). J Appl Ecol 43:1223–1232. https://doi.org/10.1111/j.1365-2664.2006.01214.x
Anderson RP, Lew D, Peterson AT (2003) Evaluating predictive models of species’ distributions: criteria for selecting optimal models. Ecol Model 162:211–232. https://doi.org/10.1016/S0304-3800(02)00349-6
Bales EK, Hyman OJ, Loudon AH et al (2015) Pathogenic chytrid fungus Batrachochytrium dendrobatidis, but not B. salamandrivorans, detected on eastern hellbenders. PLoS ONE 10:e0116405. https://doi.org/10.1371/journal.pone.0116405
Baums J (2015a) project_maxent.R. https://github.com/johnbaums/things/blob/master/R/project_maxent.R. Accessed 1 Jan 2017
Baums J (2015b) parse_lambdas.R. https://github.com/johnbaums/things/blob/master/R/parse_lambdas.R. Accessed 1 Jan 2017
Bosso L, De Conno C, Russo D (2017) Modelling the risk posed by the Zebra Mussel Dreissena polymorpha: Italy as a case study. Environ Manage 60:304–313. https://doi.org/10.1007/s00267-017-0882-8
Burnham KP, Anderson DR (2003) Model selection and multimodel inference. Springer, New York
Carlos-Júnior LA, Barbosa NPU, Moulton TP, Creed JC (2015) Ecological niche model used to examine the distribution of an invasive, non-indigenous coral. Mar Environ Res 103:115–124. https://doi.org/10.1016/j.marenvres.2014.10.004
CITES (2016) CITES trade database. https://trade.cites.org/. Accessed 8 May 2017
Diao C, Wang L (2014) Development of an invasive species distribution model with fine-resolution remote sensing. Int J Appl Earth Obs Geoinf 30:65–75. https://doi.org/10.1016/j.jag.2014.01.015
Elith J, Graham CH, Anderson RP et al (2006) Novel methods improve prediction of species’ distributions from occurrence data. Ecography (Cop) 29:129–151. https://doi.org/10.1111/j.2006.0906-7590.04596.x
Elith J, Kearney M, Phillips S (2010) The art of modelling range-shifting species. Methods Ecol Evol 1:330–342. https://doi.org/10.1111/j.2041-210X.2010.00036.x
Elith J, Phillips SJ, Hastie T et al (2011) A statistical explanation of MaxEnt for ecologists. Divers Distrib 17:43–57. https://doi.org/10.1111/j.1472-4642.2010.00725.x
Feldmeier S, Schefczyk L, Wagner N et al (2016) Exploring the distribution of the spreading lethal salamander chytrid fungus in its invasive range in Europe—a macroecological approach. PLoS ONE 11:e0165682. https://doi.org/10.1371/journal.pone.0165682
Ficetola GF, Thuiller W, Miaud C (2007) Prediction and validation of the potential global distribution of a problematic alien invasive species—the American bullfrog. Divers Distrib 13:476–485. https://doi.org/10.1111/j.1472-4642.2007.00377.x
Fielding AH, Bell JF (1997) A review of methods for the assessment of prediction errors in conservation presence/absence models. Environ Conserv 24:38–49
Freeman EA, Moisen GG (2008) A comparison of the performance of threshold criteria for binary classification in terms of predicted prevalence and Kappa. Ecol Model 217:48–58. https://doi.org/10.1016/j.ecolmodel.2008.05.015
Golicher D, Ford A, Cayuela L, Newton A (2012) Pseudo-absences, pseudo-models and pseudo-niches: pitfalls of model selection based on the area under the curve. Int J Geogr Inf Sci 26:2049–2063. https://doi.org/10.1080/13658816.2012.719626
Gray MJ, Lewis JP, Nanjappa P et al (2015) Batrachochytrium salamandrivorans: the North American response and a call for action. PLoS Pathog 11:e1005251. https://doi.org/10.1371/journal.ppat.1005251
Hastie T, Tibshirani R, Friedman JH (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York
Hijmans RJ, Cameron SE, Parra JL et al (2005) Very high resolution interpolated climate surfaces for global land areas. Int J Climatol 25:1965–1978. https://doi.org/10.1002/joc.1276
IUCN (2017) The IUCN red list of threatened species. https://www.iucn.org/. Accessed 8 May 2017
Kramer AM, Annis G, Wittmann ME et al (2017) Suitability of Laurentian Great Lakes for invasive species based on global species distribution models and local habitat. Ecosphere 8:e01883. https://doi.org/10.1002/ecs2.1883
Lobo JM, Jiménez-Valverde A, Real R (2008) AUC: a misleading measure of the performance of predictive distribution models. Glob Ecol Biogeogr 17:145–151. https://doi.org/10.1111/j.1466-8238.2007.00358.x
Martel A, Spitzen-van der Sluijs A, Blooi M et al (2013) Batrachochytrium salamandrivorans sp. nov. causes lethal chytridiomycosis in amphibians. Proc Natl Acad Sci USA 110:15325–15329. https://doi.org/10.1073/pnas.1307356110
Martel A, Blooi M, Adriaensen C et al (2014) Recent introduction of a chytrid fungus endangers Western Palearctic salamanders. Science 346:630–631. https://doi.org/10.1126/science.1258268
Merow C, Smith MJ, Silander JA (2013) A practical guide to MaxEnt for modeling species’ distributions: what it does, and why inputs and settings matter. Ecography (Cop) 36:1058–1069. https://doi.org/10.1111/j.1600-0587.2013.07872.x
Muscarella R, Galante PJ, Soley-Guardia M et al (2014) ENMeval: an R package for conducting spatially independent evaluations and estimating optimal model complexity for MAXENT ecological niche models. Methods Ecol Evol 5:1198–1205. https://doi.org/10.1111/2041-210X.12261
Nakazawa M (2017) fmsb: functions for medical statistics book with some demographic data. R package version 0.6.1
Nguyen TT, Van Nguyen T, Ziegler T et al (2017) Trade in wild anurans vectors the urodelan pathogen Batrachochytrium salamandrivorans into Europe. Amphibia Reptil. https://doi.org/10.1038/srep44443.Martel
Parrott JC, Shepack A, Burkart D et al (2016) Survey of pathogenic chytrid fungi (Batrachochytrium dendrobatidis and B. salamandrivorans) in salamanders from three mountain ranges in Europe and the Americas. EcoHealth 14:296–302. https://doi.org/10.1007/s10393-016-1188-7
Peterson AT, Vieglais D (2001) Predicting species invasions using ecological niche modeling: new approaches from bioinformatics attack a pressing problem. Bioscience 51:363–371. https://doi.org/10.1641/0006-3568(2001)051[0363:PSIUEN]2.0.CO;2
Phillips SJ (2008) Transferability, sample selection bias and background data in presence-only modelling: a response to Peterson et al. (2007). Ecography (Cop) 31:272–278. https://doi.org/10.1111/j.2007.0906-7590.05378.x
Phillips SJ, Anderson RP, Schapire RE (2006) Maximum entropy modeling of species geographic distributions. Ecol Model 190:231–259
Phillips SJ, Dudík M, Schapire RE (2010) [Internet] Maxent software for modeling species niches and distributions (Version 3.3.3k). Available from url: http://biodiversityinformatics.amnh.org/open_source/maxent/. Accessed 1 May 2017
Phillips SJ, Anderson RP, Dudík M et al (2017) Opening the black box: an open-source release of Maxent. Ecography (Cop) 40:887–893. https://doi.org/10.1111/ecog.03049
R Development Core Team (2014) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/
RAVON (2017) RAVON. http://www.ravon.nl/English/Research/Bsal/tabid/3820/Default.aspx. Accessed 13 Jan 2018
Richgels KLD, Russell RE, Adams J et al (2016) Spatial variation in risk and consequence of Batrachochytrium salamandrivorans introduction in the USA. R Soc Open Sci 3:150616. https://doi.org/10.1098/rsos.150616
Robin X, Turck N, Hainard A et al (2011) pROC: an open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics 12:77
Robinson TP, van Klinken RD, Metternicht G (2010) Comparison of alternative strategies for invasive species distribution modeling. Ecol Model 221:2261–2269. https://doi.org/10.1016/j.ecolmodel.2010.04.018
Rodríguez JP, Brotons L, Bustamante J, Seoane J (2007) The application of predictive modelling of species distribution to biodiversity conservation. Divers Distrib 13:243–251. https://doi.org/10.1111/j.1472-4642.2007.00356.x
Somodi I, Lepesi N, Botta-Dukát Z (2017) Prevalence dependence in model goodness measures with special emphasis on true skill statistics. Ecol Evol 7:863–872. https://doi.org/10.1002/ece3.2654
Spitzen-van der Sluijs A, Martel A, Asselberghs J et al (2016) Expanding distribution of lethal amphibian fungus Batrachochytrium salamandrivorans in Europe. Emerg Infect Dis 22:1286–1288. https://doi.org/10.3201/eid2207.160109
Stegen G, Pasmans F, Schmidt BR et al (2017) Drivers of salamander extirpation mediated by Batrachochytrium salamandrivorans. Nature 544:353–356. https://doi.org/10.1038/nature22059
Václavík T, Meentemeyer RK (2009) Invasive species distribution modeling (iSDM): are absence data and dispersal constraints needed to predict actual distributions? Ecol Model 220:3248–3258. https://doi.org/10.1016/j.ecolmodel.2009.08.013
Václavík T, Kupfer JA, Meentemeyer RK (2012) Accounting for multi-scale spatial autocorrelation improves performance of invasive species distribution modeling (iSDM). J Biogeogr 39:42–55. https://doi.org/10.1111/j.1365-2699.2011.02589.x
Vall-llosera M, Woolnough AP, Anderson D, Cassey P (2017) Improved surveillance for early detection of a potential invasive species: the alien Rose-ringed parakeet Psittacula krameri in Australia. Biol Invasions 19:1273–1284. https://doi.org/10.1007/s10530-016-1332-x
VanDerWal J, Falconi L, Januchowski S et al (2015) SDMTools: species distribution modelling tools: tools for processing data associated with species distribution modelling exercises. R package version 1.1–221
Warren DL, Seifert SN (2011) Ecological niche modeling in Maxent: the importance of model complexity and the performance of model selection criteria. Ecol Appl 21:335–342
Warren DL, Wright AN, Seifert SN, Shaffer HB (2014) Incorporating model complexity and spatial sampling bias into ecological niche models of climate change risks faced by 90 California vertebrate species of concern. Divers Distrib 20:334–343. https://doi.org/10.1111/ddi.12160
Yap TA, Koo MS, Ambrose RF et al (2015) Averting a North American biodiversity crisis. Science 349:481–482. https://doi.org/10.1126/science.aab1052
Acknowledgements
We thank the Occidental College Undergraduate Research Center for funding. Thanks to J. McCormack and E. Braker for helpful comments on the manuscript. Thanks also to T. Yap for additional information on previously proposed bioclimatic suitability models.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Katz, T.S., Zellmer, A.J. Comparison of model selection technique performance in predicting the spread of newly invasive species: a case study with Batrachochytrium salamandrivorans. Biol Invasions 20, 2107–2119 (2018). https://doi.org/10.1007/s10530-018-1690-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10530-018-1690-7