Summary
In this paper, we show how Gibbs sampling can provide a reliable approximation for Bayesian estimation of the parameters of a mixture distribution. Moreover, we deduce from the Bayesian approach an alternative derivation of maximum likelihood estimators in this setting, where standard nonin-formative approaches do not apply. Our method uses conjugate priors on each component of the mixture and is called Prior Feedback because the hyperparameters of these conjugate priors are iteratively replaced by the cor-responding posterior values until convergence is attained. We illustrate the appeal of this method through an astrophysical example, where the small sample size prohibits the use of standard maximum likelihood methods. A second example shows that Prior Feedback is also able to reject an unrealistic mixture model.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barbanis, B. and Woltjer, I. (1967). Orbits in spiral galaxies and the velocity dispersion of Population I stars.Astrophys J. 150, 461–468.
Bernardo, J. and Girón, J. (1988). A Bayesian analysis of simple mixture problems.Bayesian Statistics 3 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.), Oxford: University Press, 67–78, (with discussion).
Besag, J. and Green, P. (1993). Spatial statistics and Bayesian computation.J. Roy. Statist. Soc. B 55, 25–37.
Celeux, G. and Diebolt, J. (1986). L'algorithme SEM: un algorithme d'apprentissage probabiliste pour la reconnaissance de mélanges de densités.Revue Statist. Appl. 34, 35–52.
Dempster, A., Laird, N. and Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm.J. Roy. Statist. Soc. B 39, 1–38.
Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential distributions.Ann. Statist. 7, 269–281.
Diebolt, J. and Robert, C.P. (1994). Estimation of finite mixture distributions through Bayesian sampling.J. Roy. Statist. Soc. B (to appear).
Gelfand, A. and Smith, A. (1990). Sampling-based approaches to calculating marginal densities.J. Amer. Statist. Assoc. 85, 398–409.
Gelman, A. and Rubin, D. (1991). A single series from the Gibbs sampler provides a false sense of security.Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds). Oxford: University Press, 625–631.
Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images.IEEE Trans. Patt. Anal. Mach. Intel. 6, 721–740.
Geyer, C. (1991). Markov chain Monte Carlo maximum likelihood.Computer Sciences and Statistics: Proc. 23rd Symp. Interface (E. M. Keramidas, ed.. Fairfax Station: Interface Foundation, 156–163.
Geyer, C. (1992). Practical Markov chain Monte Carlo.Statist. Sci. 7, 473–482.
Gómez, A. E., Grenier, S., Jaschek, M. and Heck, A. (1981). The absolute magnitude of the Am stars.Astron. Astrophys 93, 155–159.
Gómez, A. E., Delhaye, J., Grenier, S., Jaschek, C., Arenou, F. and Jaschek, M. (1990). Local kinematic properties of Population I (B5-F5)-type stars and galactic disk evolution.Astron. Astrophys 236, 95–98.
Grenier, S., Jaschek, M., Gómez, A. E., Jaschek, C. and Heck, A. (1981). The absolute magnitude of the Ap stars.Astron. Astrophys 100, 24–27.
van Laarhoven, P. (1988).Theoretical and Computational Aspects of Simulated Annealing. Amsterdam: CWI Tract 51.
Lacey, C. G. (1984). The influence of massive gas clouds on stellar velocity dispersions in galactic disks.Monthly Not. Roy. Astron. Soc. 208, 687–707.
Liu, J., Wong, W. H. and Kong, A. (1991). Correlation structure and convergence rate of the Gibbs sampler.Tech. Rep. 299, University of Chicago.
Mengersen, K. L. and Robert, C. P. (1993). Testing for mixture via entropy distance and Gibbs sampling.Tech. Rep. Université de Rouen.
Qian, W. and Titterington, D. M. (1991). Estimation of parameters in hidden Markov models.Phil. Trans. Royal Soc. London A 337, 407–428.
Robert, C. P. (1992). Comments on Meng and Rubin's paper.Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.). Oxford: University Press, 321–323.
Robert, C. P. (1993). Prior Feedback: a Bayesian tool for maximum likelihood estimation.J. Computational Statistics, (to appear).
Schervish, M. and Carlin, B. (1991). On the convergence of successive substitution sampling.J. Computational and Graphical Statistics 1, 111–128.
Smith, A. and Roberts, G. (1993). Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods.J. Roy. Statist. Soc. B 55, 3–23.
Spitzer, L. J. and Schwarzchild, M. (1953). The possible influence of interstellar clouds on stellar velocities.Astrophys. J. 118, 106–112.
Tanner, M. A. (1991).Tools for Statistical Inference. New York: Springer-Verlag.
Tanner, M. and Wong, W. (1987). The calculation of posterior distributions by data augmentation.J. Amer. Statist. Assoc. 82, 528–550.
Tierney, L. (1991). Markov chains for exploring posterior distributions.Computer Sciences and Statistics: Proc. 23rd Symp. Interface (E. M. keramidas, ed.). Fairfax Station: Interface Foundation, 563–570.
Titterington, D., Smith, A. and Makov, U. (1985).Statistical Analysis of Finite Mixture Distributions. New York: Wiley.
West, J. (1992). Modelling with mixtures.Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.). Oxford: University Press, 503–524, (with discussion).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Robert, C.P., Soubiran, C. Estimation of a normal mixture model through Gibbs sampling and Prior Feedback. Test 2, 125–146 (1993). https://doi.org/10.1007/BF02562672
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02562672