Abstract
This paper examines, from a historical and model-based Bayesian perspective, two inferential issues: (1) the relation between confidence coverage and credibility of interval statements about model parameters; (2) the prediction of new values of a random variable. Confidence and credible intervals have different properties. This worries some statisticians, who want them to have the same properties, in frequentist repeated sampling coverage and Bayesian credibility content. Research continues on the conditions under which intervals have the same repeated sampling coverage and credibility content. We conclude that these two inferential approaches generally have incommensurable properties, which converge only asymptotically in sample size, and in a restricted class of samples. The repeated confusion, by new students of statistics, of the coverage of a confidence interval with the credibility of the observed interval, suggests that in general the credibility of the interval is the more important inferential aim. We show how the credibility of a confidence interval can be assessed generally. Bayesian prediction of new values is well established using the posterior predictive distribution. This has some curious features, known since Laplace but not well understood. The prediction of new Bernoullis highlights these features. We suggest that the credibility of predictive intervals needs to be reassessed.
Similar content being viewed by others
Notes
The small expected frequencies in the extreme bins affected the validity of the asymptotic distribution, and so he suggested pooling of the extreme bins.
References
Agresti, A., Caffo, B.: Simple and effective confidence intervals for proportions and differences of proportions. Am. Stat. 54, 280–288 (2000)
Anscombe, F.J.: Normal likelihood functions. Ann. Inst. Stat. Math. 16, 1–19 (1964)
Aitchison, J., Dunsmore, I.R.: Statistical Prediction Analysis. Cambridge University Press, Cambridge (1975)
Bayes, T.: An essay towards solving a problem in the doctrine of chances. Philos. Trans. R. Soc. Lond. 53, 370–418 (1764)
Berger, J.O., Bernardo, J.M., Sun, D.: The formal definition of reference priors. Ann. Stat. 37, 905–938 (2009)
Bjornstad, J.F.: Predictive likelihood: a review. Stat. Sci. 5, 242–265 (1990)
Cox, D.R.: Principles of Statistical Inference. Cambridge University Press, Cambridge (2006)
Datta, G.S., Sweeting, T.J.: Probability matching priors. In: Dey, D.K., Rao, C.R. (eds.) Handbook of Statistics, 25: Bayesian Thinking: Modeling and Computation, pp. 91–114. Elsevier, Amsterdam (2005)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39, 1–38 (1977)
Diaz-Francés, E.: Simple estimation intervals for Poisson, exponential and inverse Gaussian means obtained by symmetrizing the likelihood function. Am. Stat. 70, 171–180 (2016)
Fisher, R.A.: On an absolute criterion for fitting frequency curves. Messenger Math. 41, 155–160 (1912)
Fisher, R.A.: On the mathematical foundations of theoretical statistics. Philos. Trans. R. Soc. Lond. A 222, 309–368 (1922)
Fisher, R.A.: Theory of statistical estimation. Proc. Camb. Philos. Soc. 22, 700–725 (1925)
Fraser, D.A.S.: Is Bayes just quick and dirty confidence? (with discussion). Stat. Sci. 26, 299–316 (2011)
Gelman, A., Meng, X.L., Stern, H.S.: Posterior predictive assessment of model fitness via realised discrepancies (with discussion). Stat. Sin. 6, 733–807 (1996)
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian date analysis, 3rd edn. CRC Press, Boca Raton (2014)
Jeffreys, H.: Theory of Probability (3rd edn. 1961, reissued 1998). Oxford University Press, Oxford (1939)
Laplace, P.-S.: Essai philosophique sur les probabilités (1814). English translation A Philosophical Essay on Probabilities. Wiley (1902) and Dover, New York (1950)
Little, R.J.A.: Calibrated Bayes: a Bayes/frequentist roadmap. Am. Stat. 60, 213–223 (2006)
Little, R.J.A.: Calibrated Bayes, for statistics in general, and missing data in particular. Stat. Sci. 26, 162–174 (2011)
Little, R.J.A.: Calibrated Bayes, an inferential paradigm for official statistics in the era of big data. Stat. J. IAOS 31, 555–563 (2015)
Müller, U.K., Noretz, A.: Coverage inducing priors in nonstandard inference problems. J. Am. Stat. Assoc. 111, 1233–1241 (2016)
Pearson, K.: Contribution to the mathematical theory of evolution. Philos. Trans. R. Soc. Lond. A 185, 71–110 (1894)
Pearson, K.: On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philos. Mag. 50, 157–175 (1900)
Pearson, K.: On the systematic fitting of curves to observations and measurements, Parts I. Biometrika I, 265–303 (1902)
Pearson, K.: On the systematic fitting of curves to observations and measurements, Parts II. Biometrika II, 1–23 (1902)
Raiffa, H., Schlaifer, R.: Applied statistical decision theory. Harward Business School, Boston (1961)
Rubin, D.B.: Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Stat. 12, 1151–1172 (1984)
Tanner, M., Wong, W.: The calculation of posterior distributions by data augmentation. J. Am. Stat. Assoc. 82, 528–550 (1987)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Aitkin, M., Liu, C. Confidence, credibility and prediction. METRON 76, 251–268 (2018). https://doi.org/10.1007/s40300-018-0139-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40300-018-0139-1