Abstract
We consider the analysis of data under mixture models where the number of components in the mixture is unknown. We concentrate on mixture Dirichlet process models, and in particular we consider such models under conjugate priors. This conjugacy enables us to integrate out many of the parameters in the model, and to discretize the posterior distribution. Particle filters are particularly well suited to such discrete problems, and we propose the use of the particle filter of Fearnhead and Clifford for this problem. The performance of this particle filter, when analyzing both simulated and real data from a Gaussian mixture model, is uniformly better than the particle filter algorithm of Chen and Liu. In many situations it outperforms a Gibbs Sampler. We also show how models without the required amount of conjugacy can be efficiently analyzed by the same particle filter algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akashi H. and Kumamoto H. 1977. Random sampling approach to state estimation in switching environments. Automatica 13: 429-434.
Antoniak C.E. 1974. Mixture of Dirichlet processes with applications to Bayesian nonparametric problems. Annals of Statistics 2: 1152-1174.
Blackwell D. and MacQueen J.B. 1973. Ferguson distributions via Polya urn schemes. Annals of Statistics 1: 353-355.
Bush C.A. and MacEachern S.N. 1996. A semiparametric Bayesian model for randomised block design. Biometrika 83: 275-285.
Carpenter J., Clifford P. and Fearnhead P. 1999. An improved particle filter for non-linear problems. IEE Proceedings-Radar, Sonar and Navigation 146: 2-7.
Chen R. and Liu J. 2000. Mixture Kalman filters. Journal of the Royal Statistical Society, Series B 62: 493-508.
Chopin N. 2002. A sequential particle filter for static models. Biometrika 89: 539-551.
Crisan D and doucet A. 2000 Convergence of sequential Monte Carlo methods. Technical report: Signal Processing Group, Cambridge University available from http://www-sigproc.eng.cam.ac.uk/~ad2/arnaud_doucet.html.
Del Moral P. and Guionnet A. 1999. Central limit theorem for nonlinear filtering and interacting particle systems. The Annals of Applied Probability 9: 275-297.
Doucet A., de Freitas J.F.G., and Gordon N.J. (Eds.) 2001. Sequential Monte Carlo Methods in Practice. Springer-Verlag, New York.
Escobar M.D. 1994. Normal means with a Dirichlet process prior. Journal of the American Statistical Association 89: 268-277.
Escobar M.D. and West M. 1995. Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association 90: 577-588.
Fearnhead P. 2003.MCMC, sufficient statistics and particle filters. Journal of Computational and Graphical Statistics. Biometrika 89: 848-862.
Fearnhead P. and Clifford P. 2003. Online inference for well-log data. Journal of the Royal Statistical Society, Series B. 65: 887-899.
Ferguson T.S. 1973. A Bayesian analysis of some nonparametric problems. Annals of Statistics 1: 209-230.
Gilks W.R. and Berzuini C. 2001. Following a moving target-Monte Carlo inference for dynamic Bayesian models. Journal of the Royal Statistical Society, Series B 63: 127-146.
Gilks W.R., Richardson S., and Spiegelhalter D.J. 1996. Markov Chain Monte Carlo in Practice. Chapman and Hall, London.
Gordon N., Salmond D., and Smith A.F.M. 1993. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proceedings-F 140: 107-113.
Green P. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711-732.
Kong A., Liu J.S., and Wong W.H. 1994. Sequential imputations and Bayesian missing data problems. Journal of the American Statistical Association 93: 278-288.
Liu J.S. 1996a. Metropolised independent sampling with comparisons to rejection sampling and importance sampling. Statistics and Computing 6: 113-119.
Liu J.S. 1996b. Nonparametric hierarchical Bayes via sequential imputations. Annals of Statistics 24: 911-930.
Liu J.S. 2001. Monte Carlo Strategies in Scientific Computing. Springer, New York.
Liu J.S. and Chen R. 1998. Sequential Monte Carlo methods for dynamic systems. Journal of the American Statistical Association. 93: 1032-1044.
Liu J.S., Chen R., and Wong W.H. 1996. Rejection control and importance sampling.Technical report: Stanford University, Department of Statistics.
Liu J.S., Chen R., and Logvinenko T. 2001. A theoretical framework for sequential importance sampling with resampling. In: Doucet A., de Freitas N., and Gordon N. (Eds.), Sequential Monte Carlo Methods in Practice. Springer-Verlag, New York, pp. 225-246.
MacEachern S.N., Clyde M.A., and Liu J.S. 1999. Sequential importance sampling for nonparametric Bayes models: The next generation. Canadian Journal of Statistics 27: 251-267.
Mukhopadhyay S. and Gelfand A.E. 1997. Dirichlet process mixed generalized linear models. Journal of the American Statistical Association 92: 633-639.
Muller P., Erkanli A., and West M. 1996. Bayesian curve fitting using multivariate normal mixtures. Biometrika 83: 67-79.
Pitt M.K. and Shephard N. 1999. Filtering via simulation: Auxiliary particle filters. Journal of the American Statistical Association 94: 590-599.
Postman M., Huchra J.P., and Geller M.J. 1986. Probes of large-scale structure in the Corona Borealis region. The Astronomical Journal 92: 1238-1247.
Quintana F.A. 1998. Nonparametric Bayesian analysis for assessing homogeniety in k × lcontinguency table with fixed right margin totals. Journal of the American Statistical Association 93: 1140-1149.
Quintana F.A. and Newton M.A. 2000. Computational aspects of nonparametric Bayesian analysis with applications to the modelling of multiple binary sequences. Journal of Computational and Graphical Statistics 9: 711-737.
Richardson S. and Green P.J. 1997. On Bayesian analysis of mixtures with an unknown number of components. Journal of the Royal Statistical Society, Series B 59: 731-792.
Roeder K. 1990. Density estimation with confidence sets exemplified by superclusters and voids in galaxies. Journal of the American Statistical Association 85: 617-624.
Stephens M. 2000a. Bayesian analysis of mixture models with an unknown number of components-an alternative to reversible jump methods. Annals of Statistics 28: 40-74.
Stephens M. 2000b. Dealing with label-switching in mixture models. Journal of the Royal Statistical Society, Series B 62: 795-809.
Tugnait J.K. 1982. Detection and estimation for abruptly changing systems. Automatica 18: 607-615.
West M. 1992. Modelling with mixtures. In: Bernardo J.M., Berger J.O., Dawid A.P., and Smith A.F.M. (Eds.), Bayesian Statistics 4. Clarendon Press, London.
West M. and Harrison J. 1997. Bayesian Forecasting and Dynamic Models; 2nd ed. Springer-Verlag, New York.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fearnhead, P. Particle filters for mixture models with an unknown number of components. Statistics and Computing 14, 11–21 (2004). https://doi.org/10.1023/B:STCO.0000009418.04621.cd
Issue Date:
DOI: https://doi.org/10.1023/B:STCO.0000009418.04621.cd