Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Model-averaged confidence distributions

Abstract

Model averaging is commonly used to allow for model uncertainty in parameter estimation. As well as providing a point estimate that is a natural compromise between the estimates from different models, it also provides confidence intervals with better coverage properties, compared to those based on a single best model. In recent years, the concept of a confidence distribution has been promoted as a frequentist analogue of a Bayesian posterior distribution. The confidence distribution for a parameter is a visual representation of the set of \(100(1-\alpha )\%\) confidence intervals for all possible \(\alpha \), and was first proposed over 60 years ago. The purpose of this paper is to promote the use of model-averaged confidence distributions. One of the advantages of doing so is the ability to see unusual shapes in the distribution, such as multi-modality. This allows a more comprehensive assessment of the uncertainty about the parameter of interest, in exactly the same way that a model-averaged posterior distribution can be more useful than a model-averaged credible interval. We show that the model-averaged tail-area (MATA) method for calculating a model-averaged confidence interval leads to the corresponding MATA confidence distribution being a mixture of the confidence distributions associated with the individual models, the mixing being determined by the model weights. We consider two ecological examples that illustrate the advantages of a model-averaged confidence distribution over a model-averaged confidence interval.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Banner KM, Higgs MD (2017) Considerations for assessing model averaging of regression coefficients. Ecol Appl 27(1):78–93

  2. Barker RJ, Link WA (2015) Truth, models, model sets, aic, and multimodel inference: a Bayesian perspective. J Wildl Manag 79(5):730–738

  3. Buckland ST, Burnham KP, Augustin NH (1997) Model selection: an integral part of inference. Biometrics 53(2):603–618

  4. Cade BS (2015) Model averaging and muddled multimodel inferences. Ecology 96(9):2370–2382

  5. Claeskens G, Hjort NL (2008) Model selection and model averaging, vol xvii. Cambridge series on statistical and probabilistic mathematics 27. Cambridge University Press, Cambridge, p 312

  6. Cox DR (1958) Some problems connected with statistical inference. Ann Math Stat 29(2):357–372

  7. Crainiceanu CM, Dominici F, Parmigiani G (2008) Adjustment uncertainty in effect estimation. Biometrika 95(3):635–651

  8. Dormann CF, Calabrese JM, Guillera-Arroita G, Matechou E, Bahn V, Bartoń K, Beale CM, Ciuti S, Elith J, Gerstner K et al (2018) Model averaging in ecology: a review of bayesian, information-theoretic, and tactical approaches for predictive inference. Ecol Monogr 88(4):485–504

  9. Efron B (1993) Bayes and likelihood calculations from confidence intervals. Biometrika 80(1):3–26

  10. Fletcher D (2018) Model averaging. Springer, New York

  11. Fletcher D, Turek D (2011) Model-averaged profile likelihood intervals. JABES 17(1):38–51

  12. Fletcher D, MacKenzie D, Villouta E (2005) Modelling skewed data with many zeros: a simple approach combining ordinary and logistic regression. Environ Ecol Stat 12(1):45–54

  13. Fletcher D, Moller H, Clucas R, Bragg C, Scott D, Scofield P, Hunter CM, Win I, Newman J, McKechnie S et al (2013) Age at first return to the breeding colony and juvenile survival of sooty shearwaters. Condor 115(3):465–476

  14. Fraser DAS (2011) Is Bayes posterior just quick and dirty confidence? Statist Sci 26(3):299–316

  15. Hoeting J, Madigan D, Raftery A, Volinsky C (1999) Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and EI George, and a rejoinder by the authors. Stat Sci 14(4):382–417

  16. Kabaila P (2018) On the minimum coverage probability of model averaged tail area confidence intervals. Can J Stat 46(2):279–297

  17. Kabaila P, Welsh A, Abeysekera W (2016) Model-averaged confidence intervals. Scand J Stat 43(1):35–48

  18. Kabaila P, Welsh A, Mainzer R (2017) The performance of model averaged tail area confidence intervals. Commun Stat Theory Methods 46(21):10718–10732

  19. Kamary K, Mengersen K, Robert CP, Rousseau J (2014) Testing hypotheses via a mixture estimation model. arXiv preprint arXiv:1412.2044

  20. Link W, Barker R (2006) Model weights and the foundations of multimodel inference. Ecology 87(10):2626–2635

  21. Little RJ (2006) Calibrated Bayes: a Bayes/frequentist roadmap. Am Stat 60(3):213–223

  22. Pradel R (1993) Flexibility in survival analysis from recapture data: handling trap-dependence. Mark Ind Study Bird Popul 1993:29–37

  23. Pradel R, Hines JE, Lebreton J-D, Nichols JD (1997) Capture-recapture survival models taking account of transients. Biometrics 1997:60–72

  24. Schweder T (2003) Abundance estimation from multiple photo surveys: confidence distributions and reduced likelihoods for bowhead whales off alaska. Biometrics 59(4):974–983

  25. Schweder T (2018) Confidence is epistemic probability for empirical science. J Stat Plan Inference 195:116–125

  26. Schweder T, Hjort NL (2016) Confidence, likelihood, probability, vol 41. Cambridge University Press, Cambridge

  27. Singh K, Xie M, Strawderman WE, et al (2007) Confidence distribution (cd)–distribution estimator of a parameter. In: Complex datasets and inverse problems. Institute of Mathematical Statistics, p 132–150

  28. Turek D (2015) Comparison of the frequentist mata confidence interval with Bayesian model-averaged confidence intervals. J Probab Stat. https://doi.org/10.1155/2015/420483

  29. Turek D, Fletcher D (2012) Model-averaged wald confidence intervals. Comput Stat Data Anal 56(9):2809–2815

  30. Xie M-G, Singh K (2013) Confidence distribution, the frequentist distribution estimator of a parameter: a review. Int Stat Rev 81(1):3–39

  31. Yang Y (2005) Can the strengths of aic and bic be shared? A conflict between model indentification and regression estimation. Biometrika 92(4):937–950

  32. Yao Y, Vehtari A, Simpson D, Gelman A (2018) Using stacking to average bayesian predictive distributions (with discussion). Bayesian Anal 13(3):917–1007

  33. Zeng J, Fletcher D, Dillingham PW, Cornwall CE (2019) Studentized bootstrap model-averaged tail area intervals. PLoS ONE 14(3):1–16

Download references

Author information

All authors contributed to the development of the ideas and methods. DF wrote the R code for the examples and the initial draft of the manuscript. PWD and JZ contributed to the final draft.

Correspondence to David Fletcher.

Additional information

Communicated by Pierre Dutilleul.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fletcher, D., Dillingham, P.W. & Zeng, J. Model-averaged confidence distributions. Environ Ecol Stat 26, 367–384 (2019). https://doi.org/10.1007/s10651-019-00432-5

Download citation

Keywords

  • Confidence distribution
  • Model averaging
  • Model-averaged posterior distribution
  • Model uncertainty