Statistics and Computing

, Volume 19, Issue 3, pp 303–316 | Cite as

Bayesian covariance matrix estimation using a mixture of decomposable graphical models

  • Helen Armstrong
  • Christopher K. Carter
  • Kin Foon Kevin Wong
  • Robert KohnEmail author


We present a Bayesian approach to estimating a covariance matrix by using a prior that is a mixture over all decomposable graphs, with the probability of each graph size specified by the user and graphs of equal size assigned equal probability. Most previous approaches assume that all graphs are equally probable. We show empirically that the prior that assigns equal probability over graph sizes outperforms the prior that assigns equal probability over all graphs in more efficiently estimating the covariance matrix. The prior requires knowing the number of decomposable graphs for each graph size and we give a simulation method for estimating these counts. We also present a Markov chain Monte Carlo method for estimating the posterior distribution of the covariance matrix that is much more efficient than current methods. Both the prior and the simulation method to evaluate the prior apply generally to any decomposable graphical model.


Covariance selection Reduced conditional sampling Variable selection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Altham, P.: Improving the precision of estimation by fitting a model. J. R. Stat. Soc. B 46, 118–119 (1984) MathSciNetGoogle Scholar
  2. Atay-Kayis, A., Massam, H.: A Monte Carlo method to compute the marginal likelihood in non decomposable graphical Gaussian models. Biometrika 92(2), 317–335 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  3. Barnard, J., McCulloch, R., Meng, X.: Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Stat. Sin. 10, 1281–1311 (2000) zbMATHMathSciNetGoogle Scholar
  4. Brooks, S., Giudici, P., Roberts, G.O.: Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions (with discussion). J. R. Stat. Soc. B 65(1), 3–55 (2003) zbMATHCrossRefMathSciNetGoogle Scholar
  5. Castelo, R., Wormald, N.: Enumeration of p4-free chordal graphs. J. Graphs Comb. 19, 467–474 (2001) CrossRefMathSciNetGoogle Scholar
  6. Dawid, A.: Some matrix-variate distribution theory: notational considerations and a Bayesian application. Biometrika 68(1), 265–274 (1981) zbMATHCrossRefMathSciNetGoogle Scholar
  7. Dawid, A.P., Lauritzen, S.: Hyper Markov laws in the statistical analysis of decomposable graphical models. Ann. Stat. 21(3), 1272–1317 (1993) zbMATHCrossRefMathSciNetGoogle Scholar
  8. Dellaportas, P., Forster, J.: Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models. Biometrika 86(3), 615–633 (1999) zbMATHCrossRefMathSciNetGoogle Scholar
  9. Dellaportas, P., Giudici, P., Roberts, G.: Bayesian inference for non-decomposable graphical Gaussian models. Sankyha Ser. A 65, 43–55 (2003) MathSciNetGoogle Scholar
  10. Dempster, A.: Covariance selection. Biometrics 28, 157–175 (1972) CrossRefGoogle Scholar
  11. Drton, M., Perlman, M.D.: Model selection for Gaussian concentration graphs. Biometrika 91(3), 591–602 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  12. Efron, B., Morris, C.: Multivariate empirical Bayes estimation of covariance matrices. Ann. Stat. 4, 22–32 (1976) zbMATHCrossRefMathSciNetGoogle Scholar
  13. Frydenberg, M., Lauritzen, S.: Decomposition of maximum likelihood in mixed interaction models. Biometrika 76(3), 539–555 (1989) zbMATHCrossRefMathSciNetGoogle Scholar
  14. Geiger, D., Heckerman, D.: Parameter priors for directed acyclic graphical models and the characterization of several probability distributions. Ann. Stat. 30(5), 1412–1440 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  15. Giudici, P.: Learning in graphical Gaussian models. In: Berger, A.P.D.J., Bernardo, J.M., Smith, A.F.M. (eds.) Bayesian Statistics 5: Proceedings of the Fifth Valencia International Meeting, June 5–9, 1994, vol. 5, pp. 621–628. Oxford University Press, London (1996) Google Scholar
  16. Giudici, P., Castelo, R.: Improving Markov chain Monte Carlo model search for data mining. Mach. Learn. 50, 127–158 (2003) zbMATHCrossRefGoogle Scholar
  17. Giudici, P., Green, P.J.: Decomposable graphical Gaussian model determination. Biometrika 86(4), 785–801 (1999) zbMATHCrossRefMathSciNetGoogle Scholar
  18. Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., West, M.: Experiments in stochastic computation for high-dimensional graphical models. Stat. Sci. 20(4), 388–400 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  19. Larner, M.: Mass and its relationship to physical measurements. Technical Report, Department of Mathematics, University of Queensland, Australia (1996) Google Scholar
  20. Lauritzen, S.L.: Graphical Models. Oxford University Press, London (1996) Google Scholar
  21. Letac, G., Massam, H.: Wishart distributions for decomposable graphs. Ann. Stat. 35, 1278–1323 (2007) zbMATHCrossRefMathSciNetGoogle Scholar
  22. Liang, F., Rui, P., Molina, G., Clyde, M., Berger, J.: Mixtures of g priors for Bayesian variable selection. J. Am. Stat. Assoc. 103, 410–423 (2008) zbMATHCrossRefGoogle Scholar
  23. Liechty, J.C., Liechty, M.W., Müller, P.: Bayesian correlation estimation. Biometrika 91(1), 1–14 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  24. Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Academic Press, San Diego (1979) zbMATHGoogle Scholar
  25. Muirhead, R.: Aspects of Multivariate Statistical Theory. Wiley, New York (1982) zbMATHCrossRefGoogle Scholar
  26. Roverato, A.: Cholesky decomposition of a hyper inverse Wishart matrix. Biometrika 87, 99–112 (2000) zbMATHCrossRefMathSciNetGoogle Scholar
  27. Roverato, A.: Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models. Scand. J. Stat. 29, 391–411 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  28. Sargent, D., Hodges, J., Carlin, B.: Structured Markov chain Monte Carlo. J. Comput. Graph. Stat. 9, 217–234 (2000) CrossRefMathSciNetGoogle Scholar
  29. Smith, M., Kohn, R.: Bayesian parsimonious covariance matrix estimation for longitudinal data. J. Am. Stat. Assoc. 87, 1141–1153 (2002) CrossRefMathSciNetGoogle Scholar
  30. Tierney, L.: Markov chains for exploring posterior distributions. Ann. Stat. 22(1), 1701–1728 (1994) zbMATHCrossRefMathSciNetGoogle Scholar
  31. Whittaker, J.: Graphical Models in Applied Mathematical Analysis. Wiley, New York (1990) Google Scholar
  32. Wong, F., Carter, C., Kohn, R.: Efficient estimation of covariance selection models. Biometrika 90, 809–830 (2003) CrossRefMathSciNetGoogle Scholar
  33. Wormald, N.: Counting labelled chordal graphs. Graphs Comb. 1, 193–200 (1985) zbMATHCrossRefMathSciNetGoogle Scholar
  34. Yang, R., Berger, J.: Estimation of a covariance matrix using the reference prior. Ann. Stat. 22, 1195–1211 (1994) zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Helen Armstrong
    • 1
  • Christopher K. Carter
    • 2
  • Kin Foon Kevin Wong
    • 3
  • Robert Kohn
    • 2
    Email author
  1. 1.School of Mathematics and StatisticsUniversity of New South WalesSydneyAustralia
  2. 2.Australian School of BusinessUniversity of New South WalesSydneyAustralia
  3. 3.Neuroscience Statistics Research LaboratoryMassachusetts General HospitalBostonUSA

Personalised recommendations