Summary
The notion of a conjugate family of distributions plays a very important role in the Bayesian approach to parametric inference. One of the main features of such a family is that it is closed under sampling, but a conjugate family often provides prior distributions which are tractable in various other respects. This paper is concerned with the properties of conjugate families for exponential family models. Special attention is given to the class of natural exponential families having a quadratic variance function, for which the theory is particularly fruitful. Several classes of conjugate families have been considered in the literature and here we describe some of their most interesting features. Relationships between such classes are also discussed. Our aim is to provide a unified approach to the theory of conjugate families for exponential family likelihoods. An important aspect of the theory concerns reparameterisations of the exponential family under consideration. We briefly review the concept of a conjugate parameterisation, which provides further insight into many of the properties discussed throughout the paper. Finally, further implications of these results for Bayesian conjugate analysis of exponential families are investigated.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramowitz, M. and Stegun, I.A. (1965).Handbook of Mathematical Functions. New York: Dover.
Arnold, B. C., Castillo, E. and Sarabia, J. M. (1993). Conjugate exponetial family priors for exponential family likelihoods.Statistics 25, 71–77.
Bar-Lev, S., Bshouty, D., Enis, P., Letac, G., Lu, I. and Richards, D. (1995). The diagonal multivariate natural exponential families and their classification.Journal of Theoretical Probability. (To be published).
Barndorff-Nielsen, O. (1978).Information and Exponential Families in Statistical Theory. Chichester: Wiley.
Bernardo, J. M. and Smith, A. F. M. (1994).Bayesian Theory. Chichester: Wiley.
Brown, L. D. (1986).Fundamentals of Statistical Exponential Families, with Applications in Statistical Decision Theory. Lecture Notes,9. Hayward: Institute of Mathematical Statistics.
Cacoullos, T. (1987). Characterizing priors by posterior expectations in multiparameter exponential families.Ann. Inst. Statist. Math. 39, 399–405.
Casalis, M. (1990).Familles exponentielles naturelles invariantes par un groupe. Ph.D. Thesis, P.aul Sabatier University, Toulouse.
Casalis, M. (1991). Les familles exponentielles à variance quadratique homogène sont des lois de Wishart sur un cône symétrique.Comptes Rendus de l’Academie des Sciences, Paris t.312, Série I, 537–540.
Casalis, M. (1994). The 2d+4 simple quadratic natural exponential families on ℝd Submitted toAnn. Statist.
Consonni, G. and Veronese, P. (1992). Conjugate priors for exponential families having quadratic variance functions.J. Amer. Statist. Assoc. 87, 1123–1127.
Dalal, S. R. and Hall, W. J. (1983). Approximating priors by mixtures of natural conjugate priors.J. Roy. Statist. Soc. B 45, 278–286.
Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families.Ann. Statist. 7, 269–281.
Diaconis, P. and Ylvisaker, D. (1985). Quantifying prior, opinion.Bayesian Statistics 2 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.), Amsterdam: North-Holland, 133–156 (with discussion).
Dickey, J. M. (1982). Conjugate families of distributions.Encyclopedia of Statistical Sciences (S. Kotz and N. Johnson, eds.), Chichester: Wiley, 135–145.
Eguchi, S. and Yanagimoto, T. (1995). Improvement of maximum likelihood estimators using relative entropy risk.Unpublished Technical Report.
Fienberg, S. E. (1980). Linear and quasi-linear Bayes estimators.Bayesian Analysis in Econometrics and Statistics (A. Zellner, ed.), Amsterdam: North-Holland, 355–361.
Gutiérrez-Peña, E. (1992). Expected logarithmic divergence for exponential families.Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.). Oxford: University Press, 669–674.
Gutiérrez-Peña, E. (1995a). Moments for the canonical parameter of an exponential family under a conjugate distribution.Tech. Rep. 95-01, Department of Mathematics, Imperial College London.
Gutiérrez-Peña, E. (1995b).Bayesian Topics Relating to the Exponential Family.Ph.D. Thesis, London: Imperial College London.
Gutiérrez-Peña, E. and Smith, A. F. M. (1995). Conjugate parameterisations for natural exponential families.J. Amer. Statist. Assoc. 90, 1347–1356.
Hills, S. E. and Smith, A. F. M. (1992). Parameterisations issues in Bayesian inference.Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.). Oxford: University Press, 227–246.
Jeffreys, H. (1961).Theory of Probability, 3rd. edn. Oxford: Oxford University Press.
Jewell, W. S. (1974). Exact multidimensional credibility.Mitteilungen der Vereinigung Schweizerischer Versicherungsmathematiker,74, 193–214.
Jorgensen, B. (1987). Exponential dispersion models.J. Roy. Statist. Soc. B 49, 127–162 (with discussion).
Jorgensen, B., Letac, G. and Seshadri, V. (1989). Sur une propriété des familles exponentielles naturelles de variance quadratique.Canadian J. Statist. 17, 1–8.
Letac, G. (1989). A characterization of the Wishart exponential families by an invariance property.J. Theoretical Prob. 2, 71–86.
Letac, G. (1991). The classification of the natural exponential families by their variance functions.Proceedings of the 48th Session of the International Statistical Institute 54, Book 3, 1–18.
Letac, G. and Mora, M. (1990). Natural real exponential families with cubic variance functions.Ann. Statist. 18, 1–37.
Magnus, J. R. and Neudecker, H. (1988)Matrix Differential Calculus, with Applications in Statistics and Econometrics. Chichester: Wiley.
Morris, C. N. (1982). Natural exponential families with quadratic variance functions.Ann. Statist. 10, 65–80.
Morris, C. N. (1983). Natural exponential families with quadratic variance functions: Statistical Theory.Ann. Statist. 11, 515–529.
Raiffa, H. and Schlaifer, R. (1961)Applied Statistical Decision Theory. Boston: Harvard Business School.
Additional References in the Discussion
Arnold, B. C., Castillo, E. and Sarabia, J. M. (1992). Conditionally Specified Distributions.Lecture Notes in Statistics 73. Berlin: Springer-Verlag, U.S.A.
Bernardo, J. M. (1997). Noninformative priors do not exist: a discussion.J. Statist. Plann. Inference (to appear).
Bar-Lev, S. K., Enis, P. and Letac, G. (1994). Sampling in models which admit a given exponential family as a conjugate family of priors.Ann. of Statist. 22, 1555–1586.
Bernardo, J. M. (1997). Noninformative priors do not exist: a discussion.J. Statis. Planning and Inference (to appear, with discussion).
Calabi, E. (1958) Improper affine hyperspheres of convex type and a generalization of a theorem by K. Jorgens.Michigan Math. J. 5, 105–126.
Caro, E., Domínguez, J. I. and Girón, F. J. (1994). A conjugate family forAR(1) processes with exponential errors.Commun. Statist.-Theory Meth. 23, (6), 1771–1784.
Casalis, M. (1996) The 2d+4 simple quadratic natural exponential families on ℝd.Ann. Statist. 24, 1828–1854.
Faraut, J. and Koranyi, A. (1994).Analysis on Symmetric Cones, Oxford: Clarendon Press.
Geiger, D. and Heckerman, D. (1994). A characterization of the Dirichlet distribution through global and local independence.Tech. Rep. 94-16, Microsoft Research, Redmond, WA, USA.
Geiger, D. and Heckerman, D. (1995). A characterization of the bivariate normal-wishart distribution.Tech. Rep. 95-53. Microsoft Research, Redmond, WA, USA.
Girón, F.J. and Moreno, E. (1997). Estimating with incomplete count data: A Bayesian approach.J. Statist. Planning and Inference, (to appear).
Gutiérrez-Peña, E. and Smith, A.F.M. (1995). Conjugate parameterizations for natural exponential families.J. Amer. Statist. Assoc. 90, 1347–1356.
Jackson, D. A., O’Donovan, T. M., Zimmer, W. J. and Deely, J. J. (1970). Minimax estimators in the exponential family.Biometrika 57, 439–443.
Jorgens, K. (1954). Uber die Losungen der Differentialgleichung.Math. Annalen 127, 130–134.
Kadane, J. B., Chan, N. H. and Wolfson, L. J. (1996). Priors for Unit-root Models.Journal of Econometrics 75, 99–111.
Magiera, R. and Wilczyński, M. (1991). Conjugate Priors for Exponential-type Processes.Statistics and Probability Letters 12, 379–384.
Morris, C. N. (1982) Natural exponential families with quadratic variance functions.Ann. Statist. 10, 65–80.
Pogorelov, A. V. (1978)The Minkowski Multidimensional Problem. New York: Wiley
Spiegelhalter, D. J., Dawid, A. P., Lauritzen, S. L. and Cowell, R. G. (1993). Bayesian analysis in expert systems.Statistical Science 8, 219–282.
Walter, G. G., Hamedani, G. G. (1991). Bayes empirical Bayes estimation for natural exponential families with quadratic variance functions.Annals of Statistics 19, 1191–1224.
Author information
Authors and Affiliations
Additional information
Research partially funded by ONR Grant N00014-96-1-0192
Research funded by NSF Grant DMS 94.04408 and Texas ARP Grant 003658130
Rights and permissions
About this article
Cite this article
Gutiérrez-Peña, E., Smith, A.F.M., Bernardo, J.M. et al. Exponential and bayesian conjugate families: Review and extensions. Test 6, 1–90 (1997). https://doi.org/10.1007/BF02564426
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02564426