Skip to main content
SpringerLink
Log in

Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

The label switching problem is caused by the likelihood of a Bayesian mixture model being invariant to permutations of the labels. The permutation can change multiple times between Markov Chain Monte Carlo (MCMC) iterations making it difficult to infer component-specific parameters of the model. Various so-called ‘relabelling’ strategies exist with the goal to ‘undo’ the label switches that have occurred to enable estimation of functions that depend on component-specific parameters. Existing deterministic relabelling algorithms rely upon specifying a loss function, and relabelling by minimising its posterior expected loss. In this paper we develop probabilistic approaches to relabelling that allow for estimation and incorporation of the uncertainty in the relabelling process. Variants of the probabilistic relabelling algorithm are introduced and compared to existing deterministic relabelling algorithms. We demonstrate that the idea of probabilistic relabelling can be expressed in a rigorous framework based on the EM algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aitkin, M.: Likelihood and Bayesian analysis of mixtures. Stat. Model 1, 287–304 (2001)

    Article  Google Scholar 

  • Celeux, G., Diebolt, J.: The SEM algorithm: a probabilistic teacher algorithm derived from the em algorithm for the mixture problem. Comput. Stat. Q. 2, 73–82 (1985)

    Google Scholar 

  • Celeux, G., Hurn, M., Robert, C.P.: Computational and inferential difficulties with mixture posterior distributions. J. Am. Stat. Assoc. 95, 957–970 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  • Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Stat. Soc. Ser. B 39, 1–38 (1977)

    MATH  MathSciNet  Google Scholar 

  • Diebolt, J., Robert, C.P.: Estimation of finite mixture distributions through Bayesian sampling. J. R. Stat. Soc. Ser. B 56, 363–375 (1994)

    MATH  MathSciNet  Google Scholar 

  • Farrar, D.: Approaches to the label-switching problem of classification, based on partition-space relabeling and label-invariant visualization. Technical Report, Statistical Consulting Center and Department of Statistics, Virginia Polytechnic (2006)

  • Geweke, J.: Interpretation and inference in mixture models: simple MCMC works. Comput. Stat. Data Anal. 51, 3529–3550 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  • Hurn, M.A., Justel, A., Robert, C.P.: Estimating mixtures of regressions. J. Comput. Graph. Stat. 12, 55–79 (2003)

    Article  MathSciNet  Google Scholar 

  • Jasra, A.: Bayesian inference for mixture models via Monte Carlo. Ph.D. Thesis, Imperial College London (2005)

  • Jasra, A., Holmes, C.C., Stephens, D.A.: Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modelling. Stat. Sci. 20, 50–67 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  • Marin, J.M., Mengersen, K.L., Robert, C.P.: Bayesian Modelling and Inference on Mixtures of Distributions. Elsevier, Amsterdam (2005)

    Google Scholar 

  • McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2000)

    Book  MATH  Google Scholar 

  • Nobile, A., Fearnside, A.T.: Bayesian finite mixtures with an unknown number of components: the allocation sampler. Stat. Comput. 17(2), 147–162 (2007)

    Article  MathSciNet  Google Scholar 

  • Postman, M., Huchra, J.P., Geller, M.J.: Probes of large-scale structure in the Corona Borealis region. Astron. J. 92, 1238–1246 (1986)

    Article  Google Scholar 

  • Richardson, S., Green, P.J.: On Bayesian analysis of mixtures with an unknown number of components. J. R. Stat. Soc. Ser. B 59, 758–764 (1997). With discussion

    MathSciNet  Google Scholar 

  • Stephens, M.: Bayesian methods for mixtures of normal distributions. Ph.D. Thesis, University of Oxford (1997a)

  • Stephens, M.: Discussion of on Bayesian analysis of mixtures with an unknown number of components. J. R. Stat. Soc. Ser. B 59, 768–769 (1997b)

    Google Scholar 

  • Stephens, M.: Dealing with label-switching in mixture models. J. R. Stat. Soc. Ser. B 62, 795–809 (2000)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, Lancaster University, Lancaster, LA1 4YF, UK

    M. Sperrin & T. Jaki

  2. Institute of Mathematics and Computing Science, University of Groningen, Nijenborgh 9, 9747, AG Groningen, The Netherlands

    E. Wit

Authors
  1. M. Sperrin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. Jaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. E. Wit
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Sperrin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sperrin, M., Jaki, T. & Wit, E. Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models. Stat Comput 20, 357–366 (2010). https://doi.org/10.1007/s11222-009-9129-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-009-9129-8

Keywords

Search

Navigation