Dirichlet Process Gaussian Mixture Models: Choice of the Base Distribution
- 512 Downloads
In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mixture models sidestep the problem of finding the “correct” number of mixture components by assuming infinitely many components. In this paper Dirichlet process mixture (DPM) models are cast as infinite mixture models and inference using Markov chain Monte Carlo is described. The specification of the priors on the model parameters is often guided by mathematical and practical convenience. The primary goal of this paper is to compare the choice of conjugate and non-conjugate base distributions on a particular class of DPM models which is widely used in applications, the Dirichlet process Gaussian mixture model (DPGMM). We compare computational efficiency and modeling performance of DPGMM defined using a conjugate and a conditionally conjugate base distribution. We show that better density models can result from using a wider class of priors with no or only a modest increase in computational effort.
KeywordsBayesian nonparametrics Dirichlet processes Gaussian mixtures
Unable to display preview. Download preview PDF.
- Aldous D. Exchangeability and Related Topics. Ecole d’´Eté de Probabilités de Saint-Flour XIII–1983, Berlin: Springer, 1985, pp.1-198.Google Scholar
- Pitman J. Combinatorial Stochastic Processes Ecole d’Eté de Probabilités de Saint-Flour XXXII – 2002, Lecture Notes in Mathematics, Vol. 1875, Springer, 2006.Google Scholar
- Sethuraman J, Tiwari R C. Convergence of Dirichlet Measures and the Interpretation of Their Parameter. Statistical Decision Theory and Related Topics, III, Gupta S S, Berger J O (eds.), London: Academic Press, Vol.2, 1982, pp.305-315.Google Scholar
- Neal R M. Bayesian mixture modeling. In Proc. the 11th International Workshop on Maximum Entropy and Bayesian Methods of Statistical Analysis, Seattle, USA, June, 1991, pp.197-211.Google Scholar
- Rasmussen C E. The infinite Gaussian mixture model. Advances in Neural Information Processing Systems, 2000, 12: 554-560.Google Scholar
- West M, Müller P, Escobar M D. Hierarchical Priors and Mixture Models with Applications in Regression and Density Estimation. Aspects of Uncertainty, Freeman P R, Smith A F M (eds.), John Wiley, 1994, pp.363-386.Google Scholar
- Neal R M. Markov chain sampling methods for Dirichlet process mixture models. Technical Report 4915, Department of Statistics, University of Toronto, 1998.Google Scholar
- Scott D W. Multivariate Density Estimation: Theory, Practice and Visualization, Wiley, 1992.Google Scholar
- Fisher R A. The use of multiple measurements in axonomic problems. Annals of Eugenics, 1936, 7: 179-188.Google Scholar
- Forina M, Armanino C, Castino M, Ubigli M. Multivariate data analysis as a discriminating method of the origin of wines. Vitis, 1986, 25(3): 189-201.Google Scholar