Advertisement

TEST

, Volume 14, Issue 2, pp 317–384 | Cite as

Intrinsic credible regions: An objective Bayesian approach to interval estimation

  • José M. BernardoEmail author
Article

Abstract

This paper definesintrinsic credible regions, a method to produce objective Bayesian credible regions which only depends on the assumed model and the available data.Lowest posterior loss (LPL) regions are defined as Bayesian credible regions which contain values of minimum posterior expected loss: they depend both on the loss function and on the prior specification. An invariant, information-theory based loss function, theintrinsic discrepancy is argued to be appropriate for scientific communication. Intrinsic credible regions are the lowest posterior loss regions with respect to the intrinsic discrepancy loss and the appropriate reference prior. The proposed procedure is completely general, and it is invariant under both reparametrization and marginalization. The exact derivation of intrinsic credible regions often requires numerical integration, but good analytical approximations are provided. Special attention is given to one-dimensional intrinsic credible intervals; their coverage properties show that they are always approximate (and sometimes exact) frequentist confidence intervals. The method is illustrated with a number of examples.

Key Words

Amount of information intrinsic discrepancy Bayesian asymptotics confidence intervals Fisher information HPD regions interval estimation Jeffreys priors LPL regions objective priors reference priors point estimation probability centred intervals region estimation 

AMS subject classification

Primary 62F15 Secondary 62F25 62B10 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barnard, G. A. (1980). Pivotal inference and the Bayesian controversy (with discussion). In J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith, eds.,Bayesian Statistics 1, pp. 295–318, University Press, Valencia.Google Scholar
  2. Berger, J. O. (2000). Bayesian analysis: A look at today and thoughts of tomorrow.Journal of the American Statististical Association, 95:1269–1276.zbMATHCrossRefGoogle Scholar
  3. Berger, J. O. andBernardo, J. M. (1989). Estimating a product of means: Bayesian analysis with reference priors.Journal of the American Statististical Association, 84:200–207.zbMATHCrossRefMathSciNetGoogle Scholar
  4. Berger, J. O. andBernardo, J. M. (1992a). On the development of reference priors (with discussion). In J. M. Bernardo J. O. Berger, A. P. Dawid, and A. F. M. Smith, eds.,Bayesian Statistics 4, pp. 35–60. Oxford University Press, Oxford.Google Scholar
  5. Berger, J. O. andBernardo, J. M. (1992b). Ordered group reference priors with applications to a multinomial problem.Biometrika, 79:25–37.zbMATHCrossRefMathSciNetGoogle Scholar
  6. Berger, J. O. andBernardo, J. M. (1992c). Reference priors in a variance components problem In P. K. Goel and N. S. Iyengar, eds.,Bayesian Analysis in Statistics and Econometrics, pp. 323–340. Springer-Verlag, Berlin.Google Scholar
  7. Berger, J. O. andPericchi, L. R. (1996). The intrinsic Bayes factor for model selection and prediction.Journal of the American Statististical Association, 91:109–122.zbMATHCrossRefMathSciNetGoogle Scholar
  8. Bernardo, J. M. (1979a). Expected information as expected utility.The Annals of Statistics, 7:686–690.zbMATHMathSciNetGoogle Scholar
  9. Bernardo, J. M. (1979b). Reference posterior distributions for Bayesian inference (with discussion).Journal of the Royal Statistical Society, Series B, 41:113–147. Reprinted inBayesian Inference (N. G. Polson and G. C. Tiao, eds.), Edward Elgar, Brookfield, VT, 1995, pp. 229–263.zbMATHMathSciNetGoogle Scholar
  10. Bernardo, J. M. (1997). Non-informative priors do not exist (with discussion).Journal of Statistical Planning and Inference, 65:159–189.CrossRefMathSciNetGoogle Scholar
  11. Bernardo, J. M. (2001). Un programa de síntesis para la enseñanza universitaria de la estadística matemática contemporánea.Revista de la Real Academia de Ciencias Exactas, Físicas y Naturales (España), 95(1–2):87–105.Google Scholar
  12. Bernardo, J. M. (2005a). Intrinsic point estimation of the normal variance. In S. K. Upadhyay, U. Singh, and D. K. Dey, eds.,Bayesian Statistics and its Applications. Anamaya Pub, New Delhi. In press.Google Scholar
  13. Bernardo, J. M. (2005b). Reference analysis. In D. Dey and C. R. Rao, eds.,Bayesian Thinking, Modeling and Computation, vol. 25 ofHandbook of Statistics. North Holland, Amsterdam. In pressGoogle Scholar
  14. Bernardo, J. M. andJuárez, M. (2003). Intrinsic estimation. In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, eds.,Bayesian Statistics 7, pp. 465–476. Oxford University Press, Oxford.Google Scholar
  15. Bernardo, J. M. andRamón, J. M. (1998). An introduction to Bayesian reference analysis: Inference on the ration of multinomial parameters.Journal of the Royal Statistical Society, Series D. The Statistician, 47:1–35.CrossRefGoogle Scholar
  16. Bernardo, J. M. andRueda, R. (2002). Bayesian hypothesis testing: A reference approach.International Statistical Review, 70:351–372.zbMATHCrossRefGoogle Scholar
  17. Bernardo, J. M. andSmith, A. F. M. (1994).Bayesian Theory, John Wiley & Sons, Chichester.zbMATHGoogle Scholar
  18. Blyth, C. R. (1986). Approximate binomial confidence limits.Journal of the American Statististical Association, 81:843–855.zbMATHCrossRefMathSciNetGoogle Scholar
  19. Blyth, C. R. andStill, H. A. (1983). Binomial confidence intervals.Journal of the American Statististical Association, 78:108–116.zbMATHCrossRefMathSciNetGoogle Scholar
  20. Brown, L. (1968). Inadmissibility of the usual estimators of scale parameters in problems with unknown location and scale parameters.Annals of Mathematics and Statistics 39:29–48.zbMATHGoogle Scholar
  21. Brown, L. D., Cai, T. T., andDasGupta, A. (2001). Interval estimation for a binomial proportion (with discussion).Statistical Science, 16:101–133.zbMATHMathSciNetGoogle Scholar
  22. Brown, L. D., Cai, T. T., andDasGupta, A. (2002). Confidence intervals for a binomial proportion and Edgeworth expansion.The Annals of Statistics, 30(1):160–201.zbMATHCrossRefMathSciNetGoogle Scholar
  23. Burdick, R. K. andGraybill, F. A. (1992).Confidence Intervals on Variance Components. Marcel Dekker, New York.zbMATHGoogle Scholar
  24. Cano, J. A., Hernández, A., andMoreno, E. (1987). A note on maximized likelihood sets.European Journal of Operational Research, 32:291–293.zbMATHCrossRefMathSciNetGoogle Scholar
  25. Casella, G. (1986). Refining binomial confidence intervals.The Canadian Journal of Statistics, 14:113–129.zbMATHMathSciNetGoogle Scholar
  26. Casella, G., Hwang, J. T., andRobert, C. P. (1993). A paradox in decision-theoretic set estimation.Statistica Sinica, 3:141–155.zbMATHMathSciNetGoogle Scholar
  27. Casella, G., Hwang, J. T. andRobert, C. P. (1994). Loss function for set estimation. In J. O. Berger and S. S. Gupta, eds.,Statistical Decision Theory and Related Topics, V, pp. 237–252. Springer Verlag, New York.Google Scholar
  28. Datta, G. S. andMukerjee, R. (2004).Probability Matching Priors: Higher Order Asymptotics vol. 178 ofLecture Notes in Statistics. Springer Verlag, New York.Google Scholar
  29. Data, G. S. andSweeting, T. (2005). Probability matching priors. In D. Dey and C. R. Rao, eds.,Bayesian Thinking, Modeling and Computation, vol. 25 ofHandbook of Statistics, North Holland, Amsterdam. In press.Google Scholar
  30. de Finetti, B. (1970).Teoria delle Probabilità. Einaudi, Turin, English translation asTheory of Probability in 1974. John Wiley & Sons, Chichester.Google Scholar
  31. Eberly, L. E. andCasella, G. (2003). Estimating Bayesian credible intervals.Journal of Statistical Planning and Inference, 112:115–132.zbMATHCrossRefMathSciNetGoogle Scholar
  32. Efron, B. (1987). Better bootstrap confidence intervals (with discussion).Journal of the American Statististical Association, 82:171–200.zbMATHCrossRefMathSciNetGoogle Scholar
  33. George, E. I. andMcCulloch, R. E. (1993). On obtaining invariant prior distributions.Journal of Statistical Planning and Inference, 37:169–179.zbMATHCrossRefMathSciNetGoogle Scholar
  34. Geyer, C. J. andMeeden, G. D. (2005) Fuzzy and randomized confidence intervals andp-values (with discussion).Statistical Science. To appear.Google Scholar
  35. Gleser, L. andHwang, J. T. (1987). The nonexistence of 100(1−α) confidence sets of finite expected diameter in errors-in-variable and related models.The Annals of Statistics, 15:1351–1362.zbMATHMathSciNetGoogle Scholar
  36. Granger, C. andLin, J.-L. (1994). Using the mutual information coefficient to identify lags in nonlinear models.The Journal of Time series Analysis, 15(4):371–384.zbMATHMathSciNetGoogle Scholar
  37. Gutiérrez-Peña, E. (1992). Expected logarithmic divergence for exponential families. In J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, eds.,Bayesian Statistics 4, pp. 669–674. Oxford University Press, Oxford.Google Scholar
  38. Guttman, I. (1970).Statistical Tolerance Regions: Classical and Bayesian. Griffin, London.zbMATHGoogle Scholar
  39. Hahn, G. J. andMeeker, W. Q. (1991).Statistical Intervals. John Wiley & Sons, Chichester.zbMATHGoogle Scholar
  40. Hartigan, J. A. (1966). Note on the confidence prior of Welch and Peers.Journal of the Royal Statistical Society. Series B, 28:55–56.zbMATHMathSciNetGoogle Scholar
  41. Hartigan, J. A. (1983).Bayes Theory. Springer-Verlag. Berlin.zbMATHGoogle Scholar
  42. Joe, H. (1989). Relative entropy measures of multivariate dependence.The Journal of American Statistical Association, 405:157–164.CrossRefMathSciNetGoogle Scholar
  43. Juárez, M. (2004).Métodos Bayesianos Objetivos de Estimación y Contraste de Hipótesis. PhD Thesis, Universidad de Valencia, Spain.Google Scholar
  44. Kullback, S. (1968).Information Theory and Statistics. Dover, New York, 2nd ed. Reprinted in 1978, Peter Smith, Gloucester, MA.Google Scholar
  45. Kullback, S. andLeibler, R. A. (1951). On information and sufficiency.Annals of Mathematics and Statistics, 22:79–86.MathSciNetzbMATHGoogle Scholar
  46. Lindley, D. V. (1958). Fiducial distribution and Bayes' Theorem.Journal of the Royal Statistical Society. Series B, 20:102–107.zbMATHMathSciNetGoogle Scholar
  47. Lindley, D. V. (1970).Introduction to Probability and Statistics from a Bayesian Viewpoint. Part 2: Inference, Cambridge University Press, Cambridge.Google Scholar
  48. Peña, D. andvan der Linde, A. (2005). General measures of variability and dependence for multivariate continuous distribution. Manuscript submitted for publication.Google Scholar
  49. Piccinato, L. (1984). A Bayesian property of the likelihood sets.Statistica, 3:367–372.MathSciNetGoogle Scholar
  50. Robert, C. P. (1996). Intrinsic loss functionsTheory and Decision, 40:192–214.CrossRefGoogle Scholar
  51. Robert, C. P. (2001).The Bayesian Choice, Springer Verlag, New York, 2nd ed.zbMATHGoogle Scholar
  52. Robert, C. P. andCasella, G. (1994). Distance penalized losses for testing and confidence set evaluation.Test, 3(1):163–182.zbMATHCrossRefMathSciNetGoogle Scholar
  53. Robinson, P. (1991). Consistent nonparametric entropy-based testing.Review of Economic Studies, 58:437–453.zbMATHCrossRefMathSciNetGoogle Scholar
  54. Rousseau, J. (1997). Expansions of penalized likelihood ratio statistics and consequences on matching priors for HPD regions. Tech. rep., CREST, INSEE, Paris.Google Scholar
  55. Savage, L. J. (1954).The Foundations of Statistics. John Wiley & Sons, New York. Second edition in 1972, Dover, New York.zbMATHGoogle Scholar
  56. Schervish, M. J. (1995).Theory of Statistics, Springer-Verlag, Berlin.zbMATHGoogle Scholar
  57. Wasserman, L. (1989). A robust Bayesian interpretation of likelihood regions.The Annals of Statistics, 17:1387–1393.zbMATHMathSciNetGoogle Scholar
  58. Welch, B. L. andPeers, H. W. (1963). On formulae for confidence points based on intervals of weighted likelihoods.Journal of the Royal Statistical Society. Series B, 25:318–329.zbMATHMathSciNetGoogle Scholar

Copyright information

© Sociedad Española de Estadistica e Investigacion Operativa 2005

Authors and Affiliations

  1. 1.Departamento de Estadística e I. O.Universidad de ValenciaSpain

Personalised recommendations