Statistics and Computing

, Volume 13, Issue 3, pp 209–219 | Cite as

Discretisation for inference on normal mixture models

  • Mark J. Brewer


The problem of inference in Bayesian Normal mixture models is known to be difficult. In particular, direct Bayesian inference (via quadrature) suffers from a combinatorial explosion in having to consider every possible partition of n observations into k mixture components, resulting in a computation time which is O(kn). This paper explores the use of discretised parameters and shows that for equal-variance mixture models, direct computation time can be reduced to O(Dknk), where relevant continuous parameters are each divided into D regions. As a consequence, direct inference is now possible on genuine data sets for small k, where the quality of approximation is determined by the level of discretisation. For large problems, where the computational complexity is still too great in O(Dknk) time, discretisation can provide a convergence diagnostic for a Markov chain Monte Carlo analysis.

normal mixture models discretisation of continuous parameters Bayesian inference 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Basford K.E., McLachlan G.M., and York M.G. 1997. Modelling the distribution of stamp paper thickness via finite normal mixtures: The 1872 Hidalgo stamp issue of Mexico revisited. Journal of Applied Statistics 24: 169–179.Google Scholar
  2. Bowmaker J.K., Jacobs G.H., Spiegelhalter D.J., and Mollon J.D. 1985. Two types of trichromatic squirrel monkey share a pigment in the red-green region. Vision Research 25: 1937–1946.Google Scholar
  3. Casella G., Mengersen K.L., Robert C.P., and Titterington D.M. 2000 Perfect slice samplers for mixtures of distributions. Journal of the Royal Statistical Society B, 64: 777–790.Google Scholar
  4. Dempster A.P., Laird N.M., and Rubin D. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39: 1–38.Google Scholar
  5. Diebolt J. and Robert C.P. 1994. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society B 56: 363–375.Google Scholar
  6. Hall P. and Wand M.P 1996. On the accuracy of Binned kernel density estimators. Journal of Multivariate Analysis 56: 165–184.Google Scholar
  7. Hobert J.P., Robert C.P., and Titterington D.M. 1999. On perfect simulation for some mixtures of distributions. Statistics and Computing 9: 287–298.Google Scholar
  8. Jain S. and Neal R.M. 2000. A split-merge Markov Chain Monte Carlo procedure for the Dirichlet process mixture model. Technical Report 2003, Department of Statistics, University of Toronto.Google Scholar
  9. Lehmann E.L. 1983. Theory of Point Estimation. Wiley, New YorkGoogle Scholar
  10. Lunn D.J, Thomas A., Best N., and Spiegelhalter D. 2000. WinBUGS?A Bayesian modelling framework: Concepts, structure and extensibility. Statistics and Computing 10: 325–337.Google Scholar
  11. McLachlan G.J. and Peel D. 2000. Finite Mixture Models. Wiley, New York.Google Scholar
  12. Naylor J.C. and Smith A.F.M. 1982. Applications of a method for the efficient computation of posterior distributions. Applied Statistics 31: 214–225.Google Scholar
  13. Reilly P.M. 1976. The numerical computation of posterior distributions in Bayesian statistical inference. Applied Statistics 25: 201–209.Google Scholar
  14. Robert C.P. 1996. Mixtures of distributions: Inference and estimation. In: Gilks W.R., Richardson S., and Spiegelhalter D.J. (Eds.), Markov Chain Monte Carlo in Practice. Chapman and Hall, London, pp. 441–464.Google Scholar
  15. Roeder K. 1990. Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. Journal of the American Statistical Association 85: 617–624Google Scholar
  16. Titterington D.M., Smith A.F.M., and Makov U.E. 1985. Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Mark J. Brewer
    • 1
  1. 1.Biomathematics and Statistics ScotlandThe Macaulay InstituteCraigiebuckler, AberdeenUK

Personalised recommendations