Abstract
We demonstrate how to perform direct simulation for discrete mixture models. The approach is based on directly calculating the posterior distribution using a set of recursions which are similar to those of the Forward-Backward algorithm. Our approach is more practicable than existing perfect simulation methods for mixtures. For example, we analyse 1096 observations from a 2 component Poisson mixture, and 240 observations under a 3 component Poisson mixture (with unknown mixture proportions and Poisson means in each case). Simulating samples of 10,000 perfect realisations took about 17 minutes and an hour respectively on a 900 MHz ultraSPARC computer. Our method can also be used to perform perfect simulation from Markov-dependent mixture models. A byproduct of our approach is that the evidence of our assumed models can be calculated, which enables different models to be compared.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baum L.E., Petrie, T., Soules, G., and Weiss, N. 1970. A maximisation technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics 41: 164–171.
Bernardo J.M. and Smith A.F.M. 1994. Bayesian Theory. Wiley, Chichester.
Brewer M.J. 2003. Discretisation for inference on normal mixture models. Statistics and Computing 13: 209–219.
Breyer L.A. and Roberts G.O. 2001. Catalytic perfect simulation. Methodology and Computing in Applied Probability 3: 161–177.
Carpenter J., Clifford P., and Fearnhead P. 1999. An improved particle filter for non-linear problems. IEE proceedings-Radar, Sonar and Navigation 146: 2–7.
Casella G., Mengerson K.L., Robert C.P., and Titterington D.M. 2002. Perfect samplers for mixtures of distributions. Journal of the Royal Statistical Society, Series B 64: 777–790.
Celeux G., Hurn M., and Robert, C.P. 2000. Computational and inferential difficulties with mixture posterior distributions. Journal of the American Statistical Association 95: 957–970.
Fearnhead P. 2001. Perfect simulation from population genetic models with selection. Theoretical Population Biology 59: 263–279.
Fearnhead P. and Clifford P. 2003. Online inference for hidden Markov models. Journal of the Royal Statistical Society, Series B 65: 887–899.
Fearnhead P. and Meligkotsidou L. 2004. Exact filtering for partially-observed continuous-time Markov models. Journal of the Royal Statistical Society, Series B 66: 771–789.
Green P. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732.
Green P.J. 2003. Trans-dimensional Markov chain Monte Carlo. In: P.J. Green, N.L. Hjort and S. Richardson (Eds.), Highly Structured Stochastic Systems, Oxford University Press.
Hasselblad V. 1969. Estimation of finite mixtures of distributions from the exponential family. Journal of the American Statistical Association 64: 1459–1471.
Hobert J.P., Robert C.P., and Titterington D.M. 1999. On perfect simulation for some mixtures of distributions. Statistics and Computing 9: 287–298.
Kendall W.S. and Thonnes E. 1999. Perfect simulation in stochastic geometry. Pattern Recognition 32: 1569–1586.
Lemdani M. and Pons O. 1995. Tests for genetic linkage and homogeneity. Biometrics 51: 1033–1041.
Leroux B.G. and Puterman M.L. 1992. Maximum-penalized-likelihood estimation for independent and Markov-dependent mixture models. Biometrics 48: 545–558.
McLachlan G.J. and Batsford K.E. 1988. Mixture Models: Inference and Applications to Clustering. Decker, New York.
Mira A., Moller J., and Roberts G.O. 2001. Perfect slice samplers. Journal of the Royal Statistical Society, Series B 63: 593–606.
Murdoch D.J. and Green P.J. 1998. Exact sampling from a continuous state space. Scandinavian Journal of Statistics 25: 483–502.
O’Neill P.D. and Roberts G.O. 1999. Bayesian inference for partially observed stochastic epidemics. Journal of the Royal Statistical Society, Series A 162: 121–129.
Press W.M., Teukolsky S.A., Vettering W.T., and Flannery B.D. 1992. Numerical Recipes, 2nd edition. Cambridge University Press.
Pritchard J.K., Stephens M., and Donnelly P. 2000. Inference of population structure using multilocus genotype data. Genetics 155: 945–959.
Propp J.G. and Wilson D.B. 1996. Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures and Algorithms 9: 223–252.
Richardson S. and Green P.J. 1997. On Bayesian analysis of mixtures with an unknown number of components. Journal of the Royal Statistical Society, Series B 59: 731–792.
Schilling W. 1947. A frequency distribution represented as the sum of two Poisson distributions. Journal of the American Statistical Association 42: 407–424.
Scott S.L. 2002. Bayesian methods for hidden Markov models: Recursive computing in the 21st century. Journal of the American Statistical Association 97: 337–351.
Shoukri M.M. and Lathrop G.M. 1993. Statistical testing of genetic linkage under heterogeneity. Biometrics 49: 151–161.
Stephens M. 2000. Bayesian analysis of mixture models with an unknown number of components—an alternative to reversible jump methods. Annals of Statistics 28: 40–74.
Titterington D.M., Smith A.F.M. and Makov U.E. 1985. Statistical Analysis of Finite Mixture Distributions. Wiley, New York.
Wakeley J. 1993. Substitution rate variation among sites in hypervariable region 1 of human mitochondrial DNA. Journal of Molecular Evolution 37: 613–623.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fearnhead, P. Direct simulation for discrete mixture distributions. Stat Comput 15, 125–133 (2005). https://doi.org/10.1007/s11222-005-6204-7
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11222-005-6204-7