Skip to main content
SpringerLink
Log in

Multivariate mixtures of normals with unknown number of components

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

We present full Bayesian analysis of finite mixtures of multivariate normals with unknown number of components. We adopt reversible jump Markov chain Monte Carlo and we construct, in a manner similar to that of Richardson and Green (1997), split and merge moves that produce good mixing of the Markov chains. The split moves are constructed on the space of eigenvectors and eigenvalues of the current covariance matrix so that the proposed covariance matrices are positive definite. Our proposed methodology has applications in classification and discrimination as well as heterogeneity modelling. We test our algorithm with real and simulated data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bottolo L., Consonni G., Dellaportas P., and Lijoi A. 2003. Bayesian analysis of extreme values by mixture modeling. Extremes 6: 25–47.

    Article  MATH  MathSciNet  Google Scholar 

  • Brooks S.P., Giudici P., and Roberts G.O. 2003. Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions (with Discussion). Journal of the Royal Statistical Society, Series B 65: 3–56.

    Article  MATH  MathSciNet  Google Scholar 

  • Cappé O., Robert C.P., and Rydén T. 2001. Reversible jump MCMC converging to birth-and-death MCMC and more general continuous time samplers. Technical Report 65: 679–700.

  • Cappé O., Robert C.P., and Rydén T. 2003. Reversible jump, birth-and-death and more general continuous time Markov chain Monte Carlo samplers. Journal of the Royal Statistical Socisety, Series B 65: 679–700.

    Article  MATH  Google Scholar 

  • Celeux G., Hurn M., and Robert C.P. 2000. Computational and differential difficulties with mixture posterior distributions. Journal of the American Statistical Association 95: 957–70.

    Article  MATH  MathSciNet  Google Scholar 

  • Consonni G. and Veronese P. 1995. A Bayesian method for combining results from several binomial experiments. Journal of the American Statistical Association 90: 935–944.

    Article  MATH  MathSciNet  Google Scholar 

  • Dellaportas P. 1998. Bayesian classification of neolithic tools. Applied Statistics 47: 279–297.

    MATH  Google Scholar 

  • Diebolt J. and Robert C.P. 1994. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Socisety, Series B 56: 363–375.

    MATH  MathSciNet  Google Scholar 

  • Fernandez C. and Green P.J. 2002. Modelling spatially correlated data via mixtures: A Bayesian approach. Journal of the Royal Statistical Socisety, Series B 64: 805–826.

    Article  MATH  MathSciNet  Google Scholar 

  • Frühwirth-Schnatter S. 2001. Markov Chain Monte Carlo estimation of classical and dynamic switching and mixture models. Journal of the American Statistical Association 96: 194–209.

    Article  MATH  MathSciNet  Google Scholar 

  • Green P.J. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732.

    Article  MATH  MathSciNet  Google Scholar 

  • Geyer C. and Møller J. 1994. Simulation procedures and likelihood inference for spatial point processes, Scandinavian Journal of Statistics 21: 359–373.

    MATH  Google Scholar 

  • Green P. J. and Richardson S. 2001. Modelling heterogeneity with and without the Dirichlet process. Scandinavian Journal of Statistics 28: 355–376.

    Article  MATH  MathSciNet  Google Scholar 

  • Hartigan, J.A. (1990). Partition models. Communication in Statistics, Series A: Theory and Methods 19: 2745–2756.

    MathSciNet  Google Scholar 

  • Hurn M., Justel A., and Robert C.P. 2003. Estimating mixtures of regressions. Journal of Computational and Graphical Statistics 12: 1–25.

    Article  MathSciNet  Google Scholar 

  • Jasra A., Holmes C.C., and Stephens D.A. 2005. Markov chain Monte Carlo and the label switching problem in Bayesian mixture modelling. Statistical Science 20: 5067.

    Article  MathSciNet  Google Scholar 

  • Lavine M. and West M. 1992. A Bayesian method for classification and discrimination. The Canadian Journal of Statistics 20: 451–461.

    MATH  MathSciNet  Google Scholar 

  • Lindsay B. G. 1995. Mixture models: Theory, Geometry and Applications. Institute of Mathematical Statistics, Hayward.

    Google Scholar 

  • Liu J. S., Zhang J.L., Palumbo M.J., and Lawrence C.E. 2003. Bayesian Clustering with Variable and Transformation Selections. Bayesian Statistics 7 (eds J.M. Bernardo, Bayarri M.J., Berger J.O., Dawid A.P., Heckerman D., Smith A.F.M. and West M.) pp. 249–275, Oxford University press, UK.

  • Lopes H.F., Muller P., and Rosner G.L. 2003. Bayesian Meta-analysis for Longitudinal data models using multivariate mixture priors. Biometrics 59: 66–72.

    Article  MathSciNet  Google Scholar 

  • Magnus J.R. and Neudecker H. 1988. Matrix differential calculus with applications in statistics and econometrics. John Wiley and Sons, NY.

    MATH  Google Scholar 

  • McLachlan G.J. and Basford K. E. 1988. Mixture Models: Inference and Applications to Clustering. Marcel Dekker, New York.

    Google Scholar 

  • Møller J. and Waagepetersen R. 2003. Statistical Inference and Simulation for Spatial Point Processes Series. Monographs on Statistics and Applied Probability, Vol. 100, Chapman and Hall, UK.

  • Neal R. 1996. Sampling from multimodal distributions using tempered transitions. Statistics and Computing 4: 353–366.

    Article  Google Scholar 

  • Nobile A. and Green P.J. 2000. Bayesian analysis of factorial experiments by mixture modelling. Biometrika 87: 15–35.

    Article  MATH  MathSciNet  Google Scholar 

  • Richardson S. and Green P. J. 1997. On the Bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Socisety, Series B 59: 731–792.

    Article  MATH  MathSciNet  Google Scholar 

  • Robert C. P. 1996. Mixtures of distributions: Inference and estimation, Markov chain Monte Carlo in Practice (eds W.R. Gilks, S. Richardson and D.J. Spiegelhalter), pp. 441–464, Chapman and Hall, UK.

  • Robert C.P., Rydén T., and Titterington D.M. 2000. Bayesian inference in hidden Markov models through the reversible jump MArkov chain Monte CArlo method. Journal of the Royal Statistical Socisety, Series B 62: 57–76.

    Google Scholar 

  • Stephens M. 2000. Dealing with label switching in mixture models. Journal of the Royal Statistical Society, Series B 62: 795–809.

    Article  MATH  MathSciNet  Google Scholar 

  • Stephens M. 2000. Bayesian analysis of mixture models with an unknown number of components -an alternative to reversible jump methods. Journal of the Royal Statistical Society, Series B 62: 795–809.

    Article  MATH  MathSciNet  Google Scholar 

  • Titterington D.M., Smith A. F. M., and Makov U. E. 1985. Statistical Analysis of Finite Mixture Distributions, Wiley, New York.

    MATH  Google Scholar 

  • Tubb A., Parker A. J., and Nickless G. 1980. The Analysis of Romano-British pottery by atomic absorption spectrophotometry. Archaeometry, 22: 153–171.

    Google Scholar 

  • Zhang Z., Chan K.L., Wu Y., and Chen C. 2004. Learning a multivariate Gaussian mixture models with the reversible jump MCMC algorithm. Statistics and Computing 14: 343–355.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Athens University of Economics and Business, Greece

    Petros Dellaportas & Ioulia Papageorgiou

Authors
  1. Petros Dellaportas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioulia Papageorgiou
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dellaportas, P., Papageorgiou, I. Multivariate mixtures of normals with unknown number of components. Stat Comput 16, 57–68 (2006). https://doi.org/10.1007/s11222-006-5338-6

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-006-5338-6

Keywords

Search

Navigation