Skip to main content
SpringerLink
Log in

Model structure adequacy analysis: selecting models on the basis of their ability to answer scientific questions

  • Published:
Synthese Aims and scope Submit manuscript

Abstract

Models carry the meaning of science. This puts a tremendous burden on the process of model selection. In general practice, models are selected on the basis of their relative goodness of fit to data penalized by model complexity. However, this may not be the most effective approach for selecting models to answer a specific scientific question because model fit is sensitive to all aspects of a model, not just those relevant to the question. Model Structural Adequacy analysis is proposed as a means to select models based on their ability to answer specific scientific questions given the current understanding of the relevant aspects of the real world.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Akaike H. (1973). Information theory and an extension of the maximum likelihood. In: Petrov B.N., Cs’aki F. (eds). Proceedings of the 2nd International Symposium on Information theory. Budapest, Akademia Kaido, pp. 267–281

    Google Scholar 

  • Akaike H. (1987). Factor-analysis and AIC. Psychometrika 52:317–332

    Article  Google Scholar 

  • Berryman A.A. (1991). Population theory: An essential ingredient in pest prediction, management, and policy making. American Entomologist 37:138–142

    Google Scholar 

  • Bozdogan H. (1987). Model selection and Akaike information criterion (AIC)—the general-theory and its analytical extensions. Psychometrika 52:345–370

    Article  Google Scholar 

  • Caswell H. (2001). Matrix population models: Construction, analysis, and interpretation. Sunderland, MA, Sinauer Associates

    Google Scholar 

  • Chatfield C. (1995). Model uncertainty, data mining and statistical inference. Journal of the Royal Statistical Society A 158: 419–466

    Article  Google Scholar 

  • Clark W.R., Schmitz R.A. (2001). When modelers and field biologists interact: Progress in resource science. In: Shenk T., Franklin A. (eds). Modeling in natural resource management: Development, interpretation, and application. Washington, DC: Island Press

    Google Scholar 

  • De Valpine P., Hastings A. (2002). Fitting population models incorporating process noise and observation error. Ecological Monographs 72:57–76

    Google Scholar 

  • Deleray, M., Knotek, L., Rumsey, S., & Weaver, T. (1999). Flathead lake and river fisheries status report: DJ Report No. F-78-R-1 through 5. Montana Department of Fish, Wildlife and Parks, Kalispell.

  • Dennis B., Munholland P.L., Scott J.M. (1991). Estimation of growth and extinction parameters for endangered species. Ecological Monographs 61:115–143

    Article  Google Scholar 

  • Dennis B., Ponciano J.M., Lele S.R., Taper M.L., Staples D.F. (2006). Estimating density dependence, process noise, and observation error. Ecological Monographs 76:323–341

    Article  Google Scholar 

  • Dennis B., Taper M.L. (1994). Density-dependence in time-series observations of natural-populations—estimation and testing. Ecological Monographs 64:205–224

    Article  Google Scholar 

  • Giere R.N. (2004). How models are used to represent reality. Philosophy of Science 71:742–752

    Article  Google Scholar 

  • Hilborn R., Mangel M. (1997). The ecological detective: Confronting models with data. Princeton, Princeton University Press

    Google Scholar 

  • Hilborn R., Walters C.J. (1992). Quantitative fisheries stock assessment—choice, dynamics and uncertainty. Routledge, Chapman & Hall

    Google Scholar 

  • Hobbs N.T., Hilborn R. (2006). Alternatives to statistical hypothesis testing in ecology: A guide to self teaching. Ecological Applications 16:5–19

    Article  Google Scholar 

  • Hooten, M. (1995). Distinguishing forms of statistical density dependence and independence in animal time series data using Information Criterion. Bozeman, Montana: Department of Biology, Montana State University.

  • Johnson J.B., Omland K.S. (2004). Model selection in ecology and evolution. Trends in Ecology & Evolution 19:101–108

    Article  Google Scholar 

  • Lele S.R. (2004). Evidence functions and the optimality of the law of likelihood. In: Taper M.L., Lele S.R. (eds). The nature of scientific evidence: Statistical, philosophical and empirical considerations. Chicago, The University of Chicago Press

    Google Scholar 

  • Liebhold, A. M. (1994). Use and abuse of insect and disease models in forest pest management: Past, present, and future. In W. W. Covington & L. F. DeBano (Eds.), Sustainable ecological systems: Implementing an ecological approach to land management. U.S. Forest Service Technical Report RM-247.

  • Lindsay B.G. (2004). Statistical distances as loss functions in assessing model adequacy. In: Taper M.L., Lele S.R. (eds). The nature of scientific evidence: Statistical, philosophical and empirical considerations. Chicago, The University of Chicago Press, pp. 439–488

    Google Scholar 

  • Logan J.A. (1994). In defense of big ugly models. American Entomologist 40:202–207

    Google Scholar 

  • Mangel M., Fiksen O., Giske J. (2001). Theoretical and statistical models in natural resource management and research. In: Shenk T., Franklin A. (eds). Modeling in natural resource management: Development, interpretation, and application. Washington, DC: Island Press

    Google Scholar 

  • Mayo D.G. (1996). Error and the growth of experimental knowledge. Chicago, The University of Chicago Press

    Google Scholar 

  • Mayo D.G. (2004). An error-statistical philosophy of evidence. In: Taper M.L., Lele S.R. (eds). The nature of scientific evidence: Statistical, philosophical and empirical considerations. Chicago, The University of Chicago Press, pp. 79–118

    Google Scholar 

  • Minin V., Abdo Z., Joyce P., Sullivan J. (2003). Performance-based selection of likelihood models for phylogeny estimation. Systematic Biology 52:674–683

    Article  Google Scholar 

  • Muhlfeld C.C., Taper M.L., Staples D.F., Shepard B.B. (2006). Observer error structure in bull trout redd counts in Montana streams: Implications for inference on true redd numbers. Transactions of the American Fisheries Society 135:643–654

    Article  Google Scholar 

  • Nichols J.D. (2001). Using models in the conduct of science and management of natural resources. In: Shenk T., Franklin A. (eds). Modeling in natural resource management: Development, interpretation, and application. Washington, DC: Island Press

    Google Scholar 

  • Reynolds J.H., Ford E.D. (1999). Multi-criteria assessment of ecological process models. Ecology 80:538–553

    Google Scholar 

  • Rieman B., Lee D.C., Thurow R.F. (1997). Distribution, status, and likely furture trends of bull trout within the Columbia River and Klamath Basins. North American Journal of Fisheries Management 17:1111–1125

    Article  Google Scholar 

  • Staples, D. F. (2006). Viable population monitoring: Risk-based population monitoring for threatened and endangered species with application to bull trout, salvelinus confluentus. Ph.D. Dissertation, Department of Ecology, Montana State University, Bozeman, MT.

  • Staples D.F., Taper M.L., Dennis B. (2004). Estimating population trend and process variation for PVA in the presence of sampling error. Ecology 85:923–929

    Article  Google Scholar 

  • Staples D.F., Taper M.L., Shepard B.B. (2005). Risk-based viable population monitoring. Conservation Biology 19:1908–1916

    Article  Google Scholar 

  • Stephens P.A., Buskirk S.W., Hayward G.D., Del Rio C.M. (2005). Information theory and hypothesis testing: A call for pluralism. Journal of Applied Ecology 42:4–12

    Article  Google Scholar 

  • Taper M.L. (2004). Model identification from many candidates. In: Taper M.L., Lele S.R. (eds). The nature of scientific evidence: Statistical, philosophical and empirical considerations. Chicago, The University of Chicago Press, pp. 448–524

    Google Scholar 

  • Taper M.L., Gogan P.J.P. (2002). The Northern Yellowstone elk: Density dependence and climatic conditions. Journal of Wildlife Management 66:106–122

    Article  Google Scholar 

  • Taper M.L., Lele S.R. (2004a). Dynamical models as paths to evidence in ecology. In: Taper M.L., Lele S.R. (eds). The nature of scientific evidence: Statistical, philosophical and empirical considerations. Chicago, The University of Chicago Press, pp. 275–297

    Google Scholar 

  • Taper M.L., Lele S.R. (2004b). The nature of scientific evidence: A forward-looking synthesis. In: Taper M.L., Lele S.R. (eds). The nature of scientific evidence: Statistical, philosophical and empirical considerations. Chicago, The University of Chicago Press, pp. 527–551

    Google Scholar 

  • Wiegand T., Jeltsch F., Hanski I., Grimm V. (2003). Using pattern-oriented modeling for revealing hidden information: A key for reconciling ecological theory and application. Oikos 100:209–222

    Article  Google Scholar 

  • Wisdom M.J., Mills L.S., Doak D.F. (2000). Life stage simulation analysis: Estimating vital-rate effects on population growth for conservation. Ecology 81:628–641

    Google Scholar 

  • Zeng Z., Nowierski R.M., Taper M.L., Dennis B., Kemp W.P. (1998). Complex population dynamics in the real world: Modeling the influence of time-varying parameters and time lags. Ecology 79:2193–2209

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Ecology, Montana State University, 310 Lewis Hall, Bozeman, MT, 59717-3460, USA

    Mark L. Taper & Bradley B. Shepard

  2. Fish & Wildlife Biometrics Unit, Minnesota Department of Natural Resources, 5463-C West Broadway, Forest Lake, MN, 55025, USA

    David F. Staples

  3. Montana Cooperative Fishery Research Unit, Montana Fish, Wildlife & Parks, 1400 South 19th Avenue, Bozeman, MT, 59718-5417, USA

    Bradley B. Shepard

Authors
  1. Mark L. Taper
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David F. Staples
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bradley B. Shepard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark L. Taper.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Taper, M.L., Staples, D.F. & Shepard, B.B. Model structure adequacy analysis: selecting models on the basis of their ability to answer scientific questions. Synthese 163, 357–370 (2008). https://doi.org/10.1007/s11229-007-9299-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11229-007-9299-x

Keywords

Search

Navigation