Statistics and Computing

, Volume 19, Issue 3, pp 303–316 | Cite as

Bayesian covariance matrix estimation using a mixture of decomposable graphical models

  • Helen Armstrong
  • Christopher K. Carter
  • Kin Foon Kevin Wong
  • Robert Kohn


We present a Bayesian approach to estimating a covariance matrix by using a prior that is a mixture over all decomposable graphs, with the probability of each graph size specified by the user and graphs of equal size assigned equal probability. Most previous approaches assume that all graphs are equally probable. We show empirically that the prior that assigns equal probability over graph sizes outperforms the prior that assigns equal probability over all graphs in more efficiently estimating the covariance matrix. The prior requires knowing the number of decomposable graphs for each graph size and we give a simulation method for estimating these counts. We also present a Markov chain Monte Carlo method for estimating the posterior distribution of the covariance matrix that is much more efficient than current methods. Both the prior and the simulation method to evaluate the prior apply generally to any decomposable graphical model.


Covariance selection Reduced conditional sampling Variable selection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Altham, P.: Improving the precision of estimation by fitting a model. J. R. Stat. Soc. B 46, 118–119 (1984) MathSciNetGoogle Scholar
  2. Atay-Kayis, A., Massam, H.: A Monte Carlo method to compute the marginal likelihood in non decomposable graphical Gaussian models. Biometrika 92(2), 317–335 (2005) MATHCrossRefMathSciNetGoogle Scholar
  3. Barnard, J., McCulloch, R., Meng, X.: Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Stat. Sin. 10, 1281–1311 (2000) MATHMathSciNetGoogle Scholar
  4. Brooks, S., Giudici, P., Roberts, G.O.: Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions (with discussion). J. R. Stat. Soc. B 65(1), 3–55 (2003) MATHCrossRefMathSciNetGoogle Scholar
  5. Castelo, R., Wormald, N.: Enumeration of p4-free chordal graphs. J. Graphs Comb. 19, 467–474 (2001) CrossRefMathSciNetGoogle Scholar
  6. Dawid, A.: Some matrix-variate distribution theory: notational considerations and a Bayesian application. Biometrika 68(1), 265–274 (1981) MATHCrossRefMathSciNetGoogle Scholar
  7. Dawid, A.P., Lauritzen, S.: Hyper Markov laws in the statistical analysis of decomposable graphical models. Ann. Stat. 21(3), 1272–1317 (1993) MATHCrossRefMathSciNetGoogle Scholar
  8. Dellaportas, P., Forster, J.: Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models. Biometrika 86(3), 615–633 (1999) MATHCrossRefMathSciNetGoogle Scholar
  9. Dellaportas, P., Giudici, P., Roberts, G.: Bayesian inference for non-decomposable graphical Gaussian models. Sankyha Ser. A 65, 43–55 (2003) MathSciNetGoogle Scholar
  10. Dempster, A.: Covariance selection. Biometrics 28, 157–175 (1972) CrossRefGoogle Scholar
  11. Drton, M., Perlman, M.D.: Model selection for Gaussian concentration graphs. Biometrika 91(3), 591–602 (2004) MATHCrossRefMathSciNetGoogle Scholar
  12. Efron, B., Morris, C.: Multivariate empirical Bayes estimation of covariance matrices. Ann. Stat. 4, 22–32 (1976) MATHCrossRefMathSciNetGoogle Scholar
  13. Frydenberg, M., Lauritzen, S.: Decomposition of maximum likelihood in mixed interaction models. Biometrika 76(3), 539–555 (1989) MATHCrossRefMathSciNetGoogle Scholar
  14. Geiger, D., Heckerman, D.: Parameter priors for directed acyclic graphical models and the characterization of several probability distributions. Ann. Stat. 30(5), 1412–1440 (2002) MATHCrossRefMathSciNetGoogle Scholar
  15. Giudici, P.: Learning in graphical Gaussian models. In: Berger, A.P.D.J., Bernardo, J.M., Smith, A.F.M. (eds.) Bayesian Statistics 5: Proceedings of the Fifth Valencia International Meeting, June 5–9, 1994, vol. 5, pp. 621–628. Oxford University Press, London (1996) Google Scholar
  16. Giudici, P., Castelo, R.: Improving Markov chain Monte Carlo model search for data mining. Mach. Learn. 50, 127–158 (2003) MATHCrossRefGoogle Scholar
  17. Giudici, P., Green, P.J.: Decomposable graphical Gaussian model determination. Biometrika 86(4), 785–801 (1999) MATHCrossRefMathSciNetGoogle Scholar
  18. Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., West, M.: Experiments in stochastic computation for high-dimensional graphical models. Stat. Sci. 20(4), 388–400 (2005) MATHCrossRefMathSciNetGoogle Scholar
  19. Larner, M.: Mass and its relationship to physical measurements. Technical Report, Department of Mathematics, University of Queensland, Australia (1996) Google Scholar
  20. Lauritzen, S.L.: Graphical Models. Oxford University Press, London (1996) Google Scholar
  21. Letac, G., Massam, H.: Wishart distributions for decomposable graphs. Ann. Stat. 35, 1278–1323 (2007) MATHCrossRefMathSciNetGoogle Scholar
  22. Liang, F., Rui, P., Molina, G., Clyde, M., Berger, J.: Mixtures of g priors for Bayesian variable selection. J. Am. Stat. Assoc. 103, 410–423 (2008) MATHCrossRefGoogle Scholar
  23. Liechty, J.C., Liechty, M.W., Müller, P.: Bayesian correlation estimation. Biometrika 91(1), 1–14 (2004) MATHCrossRefMathSciNetGoogle Scholar
  24. Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Academic Press, San Diego (1979) MATHGoogle Scholar
  25. Muirhead, R.: Aspects of Multivariate Statistical Theory. Wiley, New York (1982) MATHCrossRefGoogle Scholar
  26. Roverato, A.: Cholesky decomposition of a hyper inverse Wishart matrix. Biometrika 87, 99–112 (2000) MATHCrossRefMathSciNetGoogle Scholar
  27. Roverato, A.: Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models. Scand. J. Stat. 29, 391–411 (2002) MATHCrossRefMathSciNetGoogle Scholar
  28. Sargent, D., Hodges, J., Carlin, B.: Structured Markov chain Monte Carlo. J. Comput. Graph. Stat. 9, 217–234 (2000) CrossRefMathSciNetGoogle Scholar
  29. Smith, M., Kohn, R.: Bayesian parsimonious covariance matrix estimation for longitudinal data. J. Am. Stat. Assoc. 87, 1141–1153 (2002) CrossRefMathSciNetGoogle Scholar
  30. Tierney, L.: Markov chains for exploring posterior distributions. Ann. Stat. 22(1), 1701–1728 (1994) MATHCrossRefMathSciNetGoogle Scholar
  31. Whittaker, J.: Graphical Models in Applied Mathematical Analysis. Wiley, New York (1990) Google Scholar
  32. Wong, F., Carter, C., Kohn, R.: Efficient estimation of covariance selection models. Biometrika 90, 809–830 (2003) CrossRefMathSciNetGoogle Scholar
  33. Wormald, N.: Counting labelled chordal graphs. Graphs Comb. 1, 193–200 (1985) MATHCrossRefMathSciNetGoogle Scholar
  34. Yang, R., Berger, J.: Estimation of a covariance matrix using the reference prior. Ann. Stat. 22, 1195–1211 (1994) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Helen Armstrong
    • 1
  • Christopher K. Carter
    • 2
  • Kin Foon Kevin Wong
    • 3
  • Robert Kohn
    • 2
  1. 1.School of Mathematics and StatisticsUniversity of New South WalesSydneyAustralia
  2. 2.Australian School of BusinessUniversity of New South WalesSydneyAustralia
  3. 3.Neuroscience Statistics Research LaboratoryMassachusetts General HospitalBostonUSA

Personalised recommendations