# Evidential statistics as a statistical modern synthesis to support 21st century science

- 758 Downloads
- 11 Citations

## Abstract

During the 20th century, population ecology and science in general relied on two very different statistical paradigms to solve its inferential problems: error statistics (also referred to as classical statistics and frequentist statistics) and Bayesian statistics. A great deal of good science was done using these tools, but both schools suffer from technical and philosophical difficulties. At the turning of the 21st century (Royall in Statistical evidence: a likelihood paradigm. Chapman & Hall, London, 1997; Lele in The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, Chicago, pp 191–216, 2004a), evidential statistics emerged as a seriously contending paradigm. Drawing on and refining elements from error statistics, likelihoodism, Bayesian statistics, information criteria, and robust methods, evidential statistics is a statistical modern synthesis that smoothly incorporates model identification, model uncertainty, model comparison, parameter estimation, parameter uncertainty, pre-data control of error, and post-data strength of evidence into a single coherent framework. We argue that evidential statistics is currently the most effective statistical paradigm to support 21st century science. Despite the power of the evidential paradigm, we think that there is no substitute for learning how to clarify scientific arguments with statistical arguments. In this paper we sketch and relate the conceptual bases of error statistics, Bayesian statistics and evidential statistics. We also discuss a number of misconceptions about the paradigms that have hindered practitioners, as well as some real problems with the error and Bayesian statistical paradigms solved by evidential statistics.

## Keywords

Bayesian statistics Error statistics Evidential statistics Information criteria Likelihoodism Statistical inference## Notes

### Acknowledgments

We thank Dr. Yukihiko Toquenaga for inviting MLT to present in a plenary symposium of the 30th Annual Meeting of the Society of Population Ecology in Tsukuba, Japan. We are also grateful to the Society and to the Japan Society for the Promotion of Science for providing funding. MLT was partially supported by US National Science Foundation grant # DUE-1432577. JMP was partially supported by US National Institute of Health grant # R01 GM103604. We thank Ian Ausprey, Juan Pablo Gomez, Brian Dennis, and Robert Holt for insightful comments and useful suggestion helping to improve this manuscript. We also would like to thank Jack Sullivan for his questions about information criteria, and Tessa Barton for her questions about the subjectivity of model choice. MLT would like to thank Prasanta Bandyopadhyay and Gordon Brittan for many discussions on the philosophy of statistics during the production of Bandyopadhyay et al. 2016. This paper and that work were produced simultaneously and ideas have filtered between the two. The authors wish to thank also the constructive critiques of Michael J. Lew and another anonymous reviewer. We thank the University of California Press for permission to reprint Subhash R. Lele, Mark L. Taper “Information Criteria in Ecology”, 371–375, Figure 1, by Hastings, Alan and Louis J. Gross in Encyclopedia of Theoretical Ecology (c) 2012 by the Regents of the University of California. Published by the University of California Press. We thank the Journal of the American Statistical Association for permission to reprint Richard M. Royall (2000). On the probability of observing misleading statistical evidence. J Amer Stat Assoc 95:760–780, Figure 2. We are grateful to Mayuko Tanigawa and Yukihiko Toquenaga for editorial patience in helping us get things right despite extreme platform translation problems.

## References

- Aho K, Derryberry D, Peterson T (2014) Model selection for ecologists: the worldviews of AIC and BIC. Ecology 95:631–636PubMedCrossRefGoogle Scholar
- Akaike H (1973) Information theory as an extension of the maximum likelihood principle. In: Petrov BN, Csaki F (eds) second international symposium on information theory. Akademiai Kiado, Budapest, pp 267–281Google Scholar
- Akaike H (1974) A new look at statistical-model identification. IEEE Trans Autom Control 19:716–723CrossRefGoogle Scholar
- Bandyopadhyay PS, Brittan G, Taper ML (2016) Belief, evidence, and uncertainty: problems of epistemic inference. SpringerBriefs in Philosophy of Science. Springer, Cham (in press)Google Scholar
- Barnard GA (1949) Statistical inference. J R Stat Soc Ser B 11:115–149Google Scholar
- Barnard GA (1967) The use of the likelihood function in statistical practice. In: Le Cam L, Neyman J (eds) Proceedings of the fifth berkeley symposium. University of California Press, Berkeley, pp 27–40Google Scholar
- Barnett V (1999) Comparative statistical inference, 3rd edn. Wiley, ChinchesterCrossRefGoogle Scholar
- Basu A, Shioya H, Park C (2011) Statistical inference: the minimum distance approach. CRC Press, Boca RatonGoogle Scholar
- Beaumont MA, Rannala B (2004) The Bayesian revolution in genetics. Nat Rev Genet 5:251–261PubMedCrossRefGoogle Scholar
- Blume J, Peipert JF (2003) What your statistician never told you about
*P*-values. J Am Assoc Gynecol Laparosc 10:439–444PubMedCrossRefGoogle Scholar - Box GEP (1976) Science and statistics. J Am Stat Assoc 71:791–799CrossRefGoogle Scholar
- Bozdogan H (1987) Model selection and Akaike information criterion (AIC): the general theory and its analytical extensions. Psychometrika 52:345–370CrossRefGoogle Scholar
- Burnham KP, Anderson DR (2002) Model selection and multi-model inference: a practical information-theoretic approach, 2nd edn. Springer, New YorkGoogle Scholar
- Burnham KP, Anderson DR (2004) Multimodel inference—understanding AIC and BIC in model selection. Soc Method Res 33:261–304CrossRefGoogle Scholar
- Chatfield C (1995) Model uncertainty, data mining and statistical inference. J R Stat Soc Ser A 158:419–466CrossRefGoogle Scholar
- Christen JA, Nakamura M (2000) On the analysis of accumulation curves. Biometrics 56:748–754PubMedCrossRefGoogle Scholar
- Clark JS (2005) Why environmental scientists are becoming Bayesians. Ecol Lett 8:2–14CrossRefGoogle Scholar
- Cohen JE (2004) Mathematics is biology’s next microscope, only better; biology is mathematics’ next physics, only better. PLoS Biol 2:2017–2023Google Scholar
- Dennis B, Ponciano JM (2014) Density-dependent state-space model for population-abundance data with unequal time intervals. Ecology 95:2069–2076PubMedPubMedCentralCrossRefGoogle Scholar
- Dennis B, Ponciano JM, Lele SR, Taper ML, Staples DF (2006) Estimating density dependence, process noise, and observation error. Ecol Monogr 76:323–341CrossRefGoogle Scholar
- Dorazio RM (2015) Bayesian data analysis in population ecology: motivations, methods, and benefits. Popul Ecol. doi: 10.1007/s10144-015-0503-4 Google Scholar
- Edwards AWF (1992) Likelihood, expanded. Johns Hopkins University Press, BaltimoreGoogle Scholar
- Efron B (2010) Large-scale inference: empirical Bayes methods for estimation, testing, and prediction Institute of mathematical statistics monographs. Cambridge Univ. Press, CambridgeCrossRefGoogle Scholar
- Efron B (2013) Bayes’ theorem in the 21st century. Science 340:1177–1178PubMedCrossRefGoogle Scholar
- Ellison AM (2004) Bayesian inference in ecology. Ecol Lett 7:509–520CrossRefGoogle Scholar
- Fisher RA (1922) On the mathematical foundations of theoretical statistics. Philos Trans R Soc Lond Ser A 222:309–368CrossRefGoogle Scholar
- Fisher RA (1956) Statistical methods and scientific inference. Oliver and Boyd, LondonGoogle Scholar
- Fisher RA (1971) The design of experiments, 8th edn. Hafner Publishing Company, New YorkGoogle Scholar
- Frigg R (2006) Scientific representation and the semantic view of theories. Theoria 55:49–65Google Scholar
- Gause GF (1934) The struggle for existence. Williams and Wilkins, BaltimoreCrossRefGoogle Scholar
- Gelman A, Hennig C (2015) Beyond subjective and objective in statistics. Columbia University Department of Statistics technical report. http://www.stat.columbia.edu/~gelman/research/unpublished/objectivity13.pdf. Accessed 24 Feb 2015
- Gelman A, Shalizi CR (2013) Philosophy and the practice of Bayesian statistics. Br J Math Stat Psychol 66:8–38PubMedPubMedCentralCrossRefGoogle Scholar
- Giere R (1988) Explaining science. University of Chicago Press, ChicagoCrossRefGoogle Scholar
- Giere RN (1999) Science without laws (science and its conceptual foundations). University of Chicago Press, ChicagoGoogle Scholar
- Giere RN (2004) How models are used to represent reality. Philos Sci 71:742–752CrossRefGoogle Scholar
- Giere RN (2008) Models, metaphysics, and methodology. In: Hartmann S, Bovens L, Hoefer C (eds) Nancy Cartwright’s philosophy of science. Routledge, New YorkGoogle Scholar
- Gimenez O, Buckland ST, Morgan BJT, Bez N, Bertrand S, Choquet R, Dray S, Etienne M-P, Fewster R, Gosselin F, Merigot B, Monestiez P, Morales JM, Mortier F, Munoz F, Ovaskainen O, Pavoine S, Pradel R, Schurr FM, Thomas L, Thuiller W, Trenkel V, de Valpine P, Rexstad E (2014) Statistical ecology comes of age. Biol Lett 10:20140698PubMedPubMedCentralCrossRefGoogle Scholar
- Guttorp P (1995) Stochastic modeling of scientific data. Chapman & Hall, LondonCrossRefGoogle Scholar
- Hacking I (1965) Logic of statistical inference. Cambridge University Press, CambridgeGoogle Scholar
- Hájek A (2012) Interpretations of probability. In: Zalta EN (ed) The Stanford encyclopedia of philosophy (winter 2012 edition). http://plato.Stanford.Edu/archives/win2012/entries/probability-interpret/. Stanford
- Hannan EJ, Quinn BG (1979) Determination of the order of an autoregression. J R Stat Soc Ser B 41:190–195Google Scholar
- Hughes RIG (1997) Models and representation. Philos Sci Proc 64:325–336CrossRefGoogle Scholar
- Hurvich CM, Tsai CL (1989) Regression and time series model selection in small samples. Biometrika 76:297–307CrossRefGoogle Scholar
- Kalbfleisch JG (1985) Probability and statistical inference. Volume ii: statistical inference, 2nd edn. Springer, New YorkGoogle Scholar
- Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90:773–795CrossRefGoogle Scholar
- Kolmogorov AN (1933) Grundbegriffe der wahrscheinlichkeitrechnung, ergebnisse der mathematik; translated as Foundations of probability (1950). Chelsea Publishing Company, New YorkGoogle Scholar
- Konishi S, Kitagawa G (2008) Information criteria and statistical modeling. Springer, New YorkCrossRefGoogle Scholar
- Kuhnert PM, Martin TG, Griffiths SP (2010) A guide to eliciting and using expert knowledge in Bayesian ecological models. Ecol Lett 13:900–914PubMedCrossRefGoogle Scholar
- Lele SR (2004a) Evidence functions and the optimality of the law of likelihood. In: Taper ML, Lele SR (eds) The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, Chicago, pp 191–216CrossRefGoogle Scholar
- Lele SR (2004b) Elicit data, not prior: on using expert opinion in ecological studies. In: Taper ML, Lele SR (eds) The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, Chicago, pp 410–435CrossRefGoogle Scholar
- Lele SR (2015) Is non-informative Bayesian analysis appropriate for wildlife management: survival of San Joaquin kit fox and declines in amphibian populations. arXiv preprint arXiv:150200483
- Lele SR, Allen KL (2006) On using expert opinion in ecological analyses: a frequentist approach. Environmetrics 17:683–704CrossRefGoogle Scholar
- Lele SR, Taper ML (2012) Information criteria in ecology. In: Hastings A, Gross L (eds) Encyclopedia of theoretical ecology. University of California Press, Berkley, pp 371–376Google Scholar
- Lele SR, Dennis B, Lutscher F (2007) Data cloning: easy maximum likelihood estimation for complex ecological models using Bayesian Markov chain Monte Carlo methods. Ecol Lett 10:551–563PubMedCrossRefGoogle Scholar
- Lele SR, Nadeem K, Schmuland B (2010) Estimability and likelihood inference for generalized linear mixed models using data cloning. J Am Stat Assoc 105:1617–1625CrossRefGoogle Scholar
- Lindblom CE (1959) The science of muddling through. Publ Admin Rev 19:79–88CrossRefGoogle Scholar
- Lindley DV (1957) A statistical paradox. Biometrika 44:187–192CrossRefGoogle Scholar
- Lindley DV (2000) The philosophy of statistics. J R Stat Soc Ser D 49:293–337CrossRefGoogle Scholar
- Lindsay BG (2004) Statistical distances as loss functions in assessing model adequacy. In: Taper ML, Lele SR (eds) The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, Chicago, pp 439–488CrossRefGoogle Scholar
- Mayo DG (1996) Error and the growth of experimental knowledge. University of Chicago Press, ChicagoCrossRefGoogle Scholar
- Mayo DG, Cox DR (2006) Frequentist statistics as a theory of inductive inference. In: Optimality: The 2nd Lehmann symposium. Institute of Mathematical Statistics, Lecture Notes—Monograph Series, vol 57, pp 77–97Google Scholar
- Mayo DG, Spanos A (2006) Severe testing as a basic concept in a Neyman–Pearson philosophy of induction. Br J Philos Sci 57:323–357CrossRefGoogle Scholar
- Montoya JA (2008) La verosimilitud perfil en la inferencia estadística. Doctoral Dissertation, Center for Research in Mathematics, Guanajuato, México (
**in Spanish**)Google Scholar - Montoya JA, Diaz-Frances E, Sprott DA (2009) On a criticism of the profile likelihood function. Stat Papers 50:195–202CrossRefGoogle Scholar
- Moreno M, Lele SR (2010) Improved estimation of site occupancy using penalized likelihood. Ecology 91:341–346PubMedCrossRefGoogle Scholar
- Morgan M (1999) Learning from models. In: Morrison M, Morgan M (eds) Models as mediators: perspectives on natural and social science. Cambridge University Press, Cambridge, pp 347–388CrossRefGoogle Scholar
- Newman KB, Buckland ST, Morgan BJT, King RS, Borchers DL, Cole DJ, Thomas L (2014) Modeling population dynamics. Springer, New YorkGoogle Scholar
- Neyman J, Pearson ES (1933) On the problem of the most efficient tests of statistical hypostheses. Philos Trans R Soc Lond Ser A 231:289–337CrossRefGoogle Scholar
- Pawitan Y (2001) In all likelihood: statistical modeling and inference using likelihood. Oxford University Press, OxfordGoogle Scholar
- Peirce CS (1878) Illustrations of the logic of science iii—the doctrine of chances. Popul Sci Mon 12:604–615Google Scholar
- Ponciano JM, Taper ML, Dennis B, Lele SR (2009) Hierarchical models in ecology: confidence intervals, hypothesis testing, and model selection using data cloning. Ecology 90:356–362PubMedCrossRefGoogle Scholar
- Ponciano JM, Burleigh G, Braun EL, Taper ML (2012) Assessing parameter identifiability in phylogenetic models using data cloning. Syst Biol 61:955–972PubMedPubMedCentralCrossRefGoogle Scholar
- Popper KR (1959) The propensity interpretation of probability. Br J Philos Sci 10:25–42CrossRefGoogle Scholar
- Raftery AE (1995) Bayesian model selection in social research. Sociol Methodol 25:111–163CrossRefGoogle Scholar
- Rannala B (2002) Identifiability of parameters in MCMC Bayesian inference of phylogeny. Syst Biol 51:754–760PubMedCrossRefGoogle Scholar
- Rice JA (1995) Mathematical statistics and data analysis, 2nd edn. Duxbury Press, BelmontGoogle Scholar
- Royall RM (1986) The effect of sample-size on the meaning of significance tests. Am Stat 40:313–315Google Scholar
- Royall RM (1997) Statistical evidence: a likelihood paradigm. Chapman & Hall, LondonGoogle Scholar
- Royall RM (2000) On the probability of observing misleading statistical evidence. J Am Stat Assoc 95:760–780CrossRefGoogle Scholar
- Royall RM (2004) The likelihood paradigm for statistical evidence. In: Taper ML, Lele SR (eds) The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, Chicago, pp 119–152CrossRefGoogle Scholar
- Royle JA, Dorazio RM (2008) Hierarchical modeling and inference in ecology: the analysis of data from populations, metapopulations and communities. Academic Press, San DeigoGoogle Scholar
- Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464CrossRefGoogle Scholar
- Solymos P (2010) dClone: data cloning in R. R J 2:29–37Google Scholar
- Sprott DA (2000) Statistical inference in science. Springer, New YorkGoogle Scholar
- Sprott DA (2004) What is optimality in scientific inference? In: Rojo J, PerezAbreu V (eds) First Erich L. Lehmann symposium—optimality. Institute of Mathematical Statistics, Lecture Notes—Monograph Series, vol 44, pp 133–152Google Scholar
- Strevens M (2010) Reconsidering authority: scientific expertise, bounded rationality, and epistemic backtracking, Chap. 13. In: Gendler TS, Hawthorne J (eds) Oxford studies in epistemology, vol 3. Oxford University Press, Oxford, pp 294–330Google Scholar
- Suppe F (1989) The semantic conception of theories and scientific realism. University of Chicago Press, ChicagoGoogle Scholar
- Taper ML (2004) Model identification from many candidates. In: Taper ML, Lele SR (eds) The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, Chicago, pp 448–524CrossRefGoogle Scholar
- Taper ML, Lele SR (eds) (2004) The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, ChicagoGoogle Scholar
- Taper ML, Lele SR (2011) Evidence, evidence functions, and error probabilities. In: Bandyopadhyay PS, Forster MR (eds) Philosophy of statistics. Elsevier, Oxford, pp 513–532CrossRefGoogle Scholar
- Taper ML, Ponciano JM (2016) Projections in model space: multimodel inference beyond model averaging. In: Bandyopadhyay P, Brittan G, Taper ML (eds) Belief, evidence, and uncertainty: problems of epistemic inference. SpringerBriefs in Philosophy of Science. Springer, Cham (in press)Google Scholar
- Taper ML, Staples DF, Shepard BB (2008) Model structure adequacy analysis: selecting models on the basis of their ability to answer scientific questions. Synthese 163:357–370CrossRefGoogle Scholar
- Thompson B (2007) The nature of statistical evidence. Springer, New YorkGoogle Scholar
- Underwood AJ (1997) Experiments in ecology: their logical design and interpretation using analysis of variance. Cambridge University Press, CambridgeGoogle Scholar
- van der Tweel I (2005) Repeated looks at accumulating data: To correct or not to correct? Eur J Epidemiol 20:205–211PubMedCrossRefGoogle Scholar
- van Fraassen B (2002) The empirical stance. Yale University Press, New HavenGoogle Scholar
- Venn J (1876) The logic of chance, 2nd edn reprinted 1962. Chelsea Publishing Co., New YorkGoogle Scholar
- von Mises R (1951) Probability, statistics, and truth, 3rd edn (English translation 1957). George Allen & Unwin Ltd, LondonGoogle Scholar
- Walker AM (1969) On asymptotic behaviour of posterior distributions. J Roy Stat Soc Ser B 31:80–88Google Scholar
- Wang JPZ, Lindsay BG (2005) A penalized nonparametric maximum likelihood approach to species richness estimation. J Am Stat Assoc 100:942–959CrossRefGoogle Scholar
- Wilks SS (1938) The large-sample distribution of the likelihood ratio for testing composite hypotheses. Ann Math Stat 9:60–62CrossRefGoogle Scholar
- Yamamura K (2015) Bayes estimates as an approximation to maximum likelihood estimates. Popul Ecol. doi: 10.1007/s10144-015-0526-x Google Scholar