Abstract
The paper revisits the problem of selection of priors for regular one-parameter family of distributions. The goal is to find some “objective” or “default” prior by approximate maximization of the distance between the prior and the posterior under a general divergence criterion as introduced by Amari (Ann Stat 10:357–387, 1982) and Cressie and Read (J R Stat Soc Ser B 46:440–464, 1984). The maximization is based on an asymptotic expansion of this distance. The Kullback–Leibler, Bhattacharyya–Hellinger and Chi-square divergence are special cases of this general divergence criterion. It is shown that with the exception of one particular case, namely the Chi-square divergence, the general divergence criterion yields Jeffreys’ prior. For the Chi-square divergence, we obtain a prior different from that of Jeffreys and also from that of Clarke and Sun (Sankhya Ser A 59:215–231, 1997).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amari S. (1982) Differential geometry of curved exponential families—curvatures and information loss. The Annals of Statistics 10: 357–387
Bernardo J.M. (1979) Reference posterior distributions for Bayesian inference. Journal of the Royal Statistical Society, Series B 41: 113–147
Bhattacharyya A.K. (1943) On a measure of divergence between two statistical populations defined by their probability distributions. Bulletin of the Calcutta Mathematical Society 35: 99–109
Clarke B., Barron A. (1990) Information-theoretic asymptotics of Bayes methods. IEEE Transactions on Information Theory 36: 453–471
Clarke B., Barron A. (1994) Jeffreys’ prior is asymptotically least favorable under entropy risk. Journal of Statistical Planning and Inference 41: 37–60
Clarke B., Sun D. (1997) Reference priors under the chi-square distance. Sankhya, Series A 59: 215–231
Clarke B., Sun D. (1999) Asymptotics of the expected posterior. Annals of the Institute of Statistical Mathematics 51: 163–185
Cressie N., Read T.R.C. (1984) Multinomial goodness-of-fit tests. Journal of the Royal Statistical Society, Series B 46: 440–464
Datta G.S., Mukerjee R. (2004) Probability matching priors: higher order asymptotics. Springer, New York
Ghosal S. (1997) Reference prior in multiparameter nonregular cases. Test 6(1): 159–186
Ghosal S., Samanta T. (1997a) Asymptotic expansion of posterior distributions in nonregular cases. Annals of the Institute of Statistical Mathematics 49(1): 181–197
Ghosal S., Samanta T. (1997b) Expansion of Bayes risk for entropy loss and refernce prior in nonregular cases. Statistics and Decisions 15: 129–140
Ghosh J.K., Mukerjee R. (1992) Non-informative priors (with discussion). In: Bernardo J.M., Berger J.O., David A.P., Smith A.F.M. (eds) Bayesian statistics 4. Oxford University Press, London, pp 195–210
Hartigan J.A. (1998) The maximum likelihood prior. The Annals of Statistics 26(6): 2083–2103
Hellinger E. (1909) Neue begründung der theorie quadratischen formen von unendlichen vielen veränderlichen. Journal Für Reine und Angewandte Mathematik 136: 210–271
Hewitt E., Stromberg K. (1969) Real and abstract analysis. A modern treatment of the theory of functions of a real variable (second printing corrected). Springer, Berlin
Jeffreys H. (1961) Theory of probability (3rd ed.). Oxford University Press, London
Kass R.E., Wasserman L. (1996) The selection of prior distributions by formal rules. Journal of the American Statistical Association 91: 1343–1370
Laplace P.S. (1812) Theorie analytique des probabilities. Courceir, Paris
Lindley D.V. (1956) On the measure of the information provided by an experiment. The Annals of Mathematical Statistics 27: 986–1005
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was partially supported by NSF Grants SES-0317589, SES-0631426 and NSA Grant MSPF-076-097.
About this article
Cite this article
Ghosh, M., Mergel, V. & Liu, R. A general divergence criterion for prior selection. Ann Inst Stat Math 63, 43–58 (2011). https://doi.org/10.1007/s10463-009-0226-4
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-009-0226-4