Skip to main content
SpringerLink
Log in

Meta-models as a straightforward approach to the sensitivity analysis of complex models

  • Special feature: Original article
  • Mathematical Models for Effective Environmental Management
  • Published:
Population Ecology

Abstract

Complex simulation models are important tools in applied ecological and conservation research. However sensitivity analysis of this important class of models can be difficult to conduct. High level interactions and non-linear responses are common in complex simulations, and this necessitates a global sensitivity analysis, where each parameter is tested at a range of values, and in combination with changes in many other parameters. We reviewed the literature, searching for population viability analyses that used simulation models. We found only 9 out of the 122 simulation population viability analysis used global sensitivity analysis. This result is typical of other simulation models in applied ecology, where global sensitivity analysis is rare. We then demonstrate how to conduct a meta-modeling sensitivity analysis, where a simpler statistically fit function (the meta-model, also known as the surrogate model or emulator) is used to approximate the behavior of the complicated simulation. This simpler meta-model is interrogated to inform on the behavior of simulation model. We fit two example meta-models, a generalized linear model and a boosted regression tree, to exemplify the approach. Our hope is that by going through these techniques thoroughly they will become more widely adopted.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Bode M, Brennan KEC (2011) Using population viability analysis to guide research and conservation actions for Australia’s threatened malleefowl Leipoa ocellata. Oryx 45:513–521

    Article  Google Scholar 

  • Bonesi L, Rushton SP, Macdonald DW (2007) Trapping for mink control and water vole survival: identifying key criteria using a spatially explicit individual based model. Biol Conserv 136:636–650

    Article  Google Scholar 

  • Borgonovo E, Castaings W, Tarantola S (2012) Model emulation and moment-independent sensitivity analysis: an application to environmental modelling. Environ Modell Softw 34:105–115

    Article  Google Scholar 

  • Buckley YM, Briese DT, Rees M (2003) Demography and management of the invasive plant species Hypericum perforatum. II. Construction and use of an individual-based model to predict population dynamics and the effects of management strategies. J Appl Ecol 40:494–507

    Article  Google Scholar 

  • Coutts SR, van Klinken RD, Yokomizo H, Buckley YM (2011) What are the key drivers of spread in invasive plants: dispersal, demography or landscape: and how can we use this knowledge to aid management? Biol Invasions 13:1649–1661

    Article  Google Scholar 

  • Cross PC, Beissinger SR (2001) Using logistic regression to analyze the sensitivity of PVA models: a comparison of methods based on African wild dog models. Conserv Biol 15:1335–1346

    Article  Google Scholar 

  • Curtis JMR, Naujokaitis-Lewis I (2008) Sensitivity of population viability to spatial and nonspatial parameters using grip. Ecol Appl 18:1002–1013

    Article  CAS  PubMed  Google Scholar 

  • Dalerum F, Shults B, Kunkel K (2008) Estimating sustainable harvest in wolverine populations using logistic regression. J Wildlife Manage 72:1125–1132

    Article  Google Scholar 

  • Decesare NJ, Whittington J, Hebblewhite M, Robinson H, Bradley M, Neufeld L, Musiani M (2011) The role of translocation in recovery of woodland caribou populations. Conserv Biol 25:365–373

    PubMed  Google Scholar 

  • Elith J, Leathwick JR, Hastie T (2008) A working guide to boosted regression trees. J Anim Ecol 77:802–813

    Article  CAS  PubMed  Google Scholar 

  • Ellner SP, Fieberg J (2003) Using PVA for management despite uncertainty: effects of habitat, hatcheries, and harvest on salmon. Ecology 84:1359–1369

    Article  Google Scholar 

  • Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29:1189–1232

    Article  Google Scholar 

  • Girvetz EH (2010) Removing erosion control projects increases bank swallow (Riparia riparia) population viability modeled along the Sacramento River, California, USA. Biol Conserv 143:828–838

    Article  Google Scholar 

  • Gramacy RB, Taddy M (2010) Categorical inputs, sensitivity analysis, optimization and importance tempering with tgp version 2, an R package for treed Gaussian process models. J Stat Softw 33:1–48

    Google Scholar 

  • Haines AM, Tewes ME, Laack LL, Grant WE, Young J (2005) Evaluating recovery strategies for an ocelot (Leopardus pardalis) population in the United States. Biol Conserv 126:512–522

    Article  Google Scholar 

  • Hunter CM, Moller H, Fletcher D (2000) Parameter uncertainty and elasticity analyses of a population model: setting research priorities for shearwaters. Ecol Model 134:299–323

    Article  Google Scholar 

  • Johnson LR (2010) Implications of dispersal and life history strategies for the persistence of Linyphiid spider populations. Ecol Model 221:1138–1147

    Article  Google Scholar 

  • Kleijnen JPC, Noordegraaf AV, Nielen M (2001) Sensitivity analysis of censored output through polynomial, logistic, and tobit regression meta-models: Theory and case study. Proc 2001 Winter Simul Conf 1:486–491

    Google Scholar 

  • Lacy RC, Lindenmayer DB (1995) A simulation study of the impacts of population subdivision on the mountain brushtail possum Trichosurus caninus Ogilby (Phalangeridae: Marsupialia), in south-eastern Australia. II. Loss of genetic variation within and between subpopulations. Biol Conserv 73:131–142

    Google Scholar 

  • Lookingbill TR, Gardner RH, Ferrari JR, Keller CE (2010) Combining a dispersal model with network theory to assess habitat connectivity. Ecol Appl 20:427–441

    Article  PubMed  Google Scholar 

  • McCarthy MA, Burgman MA, Ferson S (1995) Sensitivity analysis for models of population viability. Biol Conserv 73:93–100

    Google Scholar 

  • McCarthy MA, Andelman SJ, Possingham HP (2003) Reliability of relative predictions in population viability analysis. Conserv Biol 17:982–989

    Article  Google Scholar 

  • Meyer KM, Gimpel K, Brandl R (2007) Viability analysis of endangered crayfish populations. J Zool 273:364–371

    Article  Google Scholar 

  • Ratto M, Castelletti A, Pagano A (2012) Emulation techniques for the reduction and sensitivity analysis of complex environmental models. Environ Modell Softw 34:1–4

    Article  Google Scholar 

  • Ricker WE (1954) Stock and recruitment. J Fish Res Board Canada 11:559–623

    Article  Google Scholar 

  • Ridgeway G (2006) gbm: Generalized Boosted Regression Models. R package version 1.5–7. http://CRAN.R-project.org/package=gbm

  • Rohmer J, Foerster E (2011) Global sensitivity analysis of large-scale numerical landslide models based on Gaussian-Process meta-modeling. Comput Geosci 37:917–927

    Article  Google Scholar 

  • Saltelli A, Scott M (1997) Guest editorial: the role of sensitivity analysis in the corroboration of models and its link to model structural and parametric uncertainty. Reliab Eng Syst Safe 57:1–4

    Article  Google Scholar 

  • Saltelli A, Tarantola S, Chan KPS (1999) A quantitative model-independent method for global sensitivity analysis of model output. Technometrics 41:39–56

    Article  Google Scholar 

  • Sawchik J, Dufrene M, Lebrun P, Schtickzelle N, Baguette M (2002) Metapopulation dynamics of the bog fritillary butterfly: modelling the effect of habitat fragmentation. Acta Oecol 23:287–296

    Article  Google Scholar 

  • Schtickzelle N, Wallis De Vries MF, Baguette M (2005) Using surrogate data in population viability analysis: the case of the critically endangered cranberry fritillary butterfly. Oikos 109:89–100

    Article  Google Scholar 

  • Storlie CB, Swiler LP, Helton JC, Sallaberry CJ (2009) Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models. Reliab Eng Syst Safe 94:1735–1763

    Article  Google Scholar 

  • Sun XY, Newham LTH, Croke BFW, Norton JP (2012) Three complementary methods for sensitivity analysis of a water quality model. Environ Modell Softw 37:19–29

    Article  Google Scholar 

  • Urban MC, Tewksbury JJ, Sheldon KS (2012) On a collision course: competition and dispersal differences create no-analogue communities and cause extinctions during climate change. Proc R Soc B Biol Sci 279:2072–2080

    Article  Google Scholar 

  • Vincenzi S, Crivelli AJ, Jesensek D, Rubin JF, Poizat G, De Leo GA (2008) Potential factors controlling the population viability of newly introduced endangered marble trout populations. Biol Conserv 141:198–210

    Article  Google Scholar 

  • Zabel RW, Scheuerell MD, McClure MM, Williams JG (2006) The interplay between climate variability and density dependence in the population viability of Chinook salmon. Conserv Biol 20:190–200

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

SRC is supported by the Australian Government’s National Environmental Research Program. HY is supported by JSPS KAKENHI Grant Number 25281057.

Author information

Authors and Affiliations

  1. Environmental Decisions Group, The School of Biological Sciences, The University of Queensland, Building 8, Brisbane, QLD, 4072, Australia

    Shaun R. Coutts

  2. Center for Environmental Risk Research, National Institute for Environmental Studies, Onogawa 16-2, Tsukuba, 305-8506, Japan

    Hiroyuki Yokomizo

Authors
  1. Shaun R. Coutts
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hiroyuki Yokomizo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaun R. Coutts.

Additional information

This manuscript was submitted for the special feature based on a symposium in Chiba, Japan, held on 21 October 2012.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 296 kb)

Supplementary material 2 (R 31 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Coutts, S.R., Yokomizo, H. Meta-models as a straightforward approach to the sensitivity analysis of complex models. Popul Ecol 56, 7–19 (2014). https://doi.org/10.1007/s10144-013-0422-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10144-013-0422-1

Keywords

Search

Navigation