Skip to main content

A comparison between the simulated annealing and the EM algorithms in normal mixture decompositions

Abstract

We compare the performances of the simulated annealing and the EM algorithms in problems of decomposition of normal mixtures according to the likelihood approach. In this case the likelihood function has multiple maxima and singularities, and we consider a suitable reformulation of the problem which yields an optimization problem having a global solution and at least a smaller number of spurious maxima. The results are compared considering some distance measures between the estimated distributions and the true ones. No overwhelming superiority of either method has been demonstrated, though in one of our cases simulated annealing achieved better results.

This is a preview of subscription content, access via your institution.

References

  1. Boyles, R. A. (1983) On the convergence of the EM algorithm.Journal of the Royal Statistical Society B,45, 47–50.

    Google Scholar 

  2. Burden, R. L. and Faires, J. D. (1985)Numerical Analysis, Prindle, Weber & Schmidt, Boston, MA.

    Google Scholar 

  3. Day, N. E. (1969) Estimating the components of a mixture of normal distributions.Biometrika,56, 463–474.

    Google Scholar 

  4. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maximum likelihood estimation from incomplete data via the EM algorithm.Journal of the Royal Statistical Society B,39, 1–38.

    Google Scholar 

  5. Fowlkes, E. B. (1979) Some methods for studying the mixture of two normal (lognormal) distributions.Journal of the American Statistical Association,74, 561–575.

    Google Scholar 

  6. Hathaway, R. J. (1985a) A constrained formulation of maximum likelihood estimation for normal mixture decompositions.Annals of Statistics,13, 795–800.

    Google Scholar 

  7. Hathaway, R. J. (1985b) A constrained EM algorithm for univariate normal mixture.Journal of Statistical Computing and Simulation.

  8. Hathaway, R. J. (1986) Another interpretation of the EM algorithm for mixture distributions.Statistics and Probability Letters,4, 53–56.

    Google Scholar 

  9. Ingrassia, S. (1991) Mixture decomposition via the simulated annealing algorithm.Applied Stochastic Models and Data Analysis,7, 317–325.

    Google Scholar 

  10. Johnson, D. S., Aragon, C. R., McGeoch, L. A. and Schevon, C. (1986)Optimization by Simulated Annealing: An Experimental Evaluation. List of Abstracts, Workshop on Statistical Physics in Engineering and Biology. Yorktown Heights, NY.

  11. Kiefer, N. M. (1978) Discrete parameter variation: efficient estimation of a switching regression model.Econometrica,46, 427–434.

    Google Scholar 

  12. Kirkpatrick, S., Gelatt, C. G. and Vecchi, M. P. (1983) Optimization by the simulated annealing.Science,220, 671–680.

    Google Scholar 

  13. van Laarhoven, P. J. M. and Aarts, E. H. L. (1988)Simulated Annealing: Theory and Applications, D. Reidel, Dordrecht.

    Google Scholar 

  14. Redner, R. A. and Walker, H. F. (1984) Mixture densities, maximum likelihood and the EM algorithm.SIAM Reviews,46, 195–239.

    Google Scholar 

  15. Titterington, D. M., Smith, A. F. M. and Makov, U. E. (1985)Statistical Analysis of Finite Mixture Distributions, Wiley, Chichester.

    Google Scholar 

  16. Vanderbilt, D. and Louie, S. G. (1984) A Monte Carlo simulated annealing approach to optimization over continuous variables.Journal of Computational Physics,56, 259–271.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ingrassia, S. A comparison between the simulated annealing and the EM algorithms in normal mixture decompositions. Stat Comput 2, 203–211 (1992). https://doi.org/10.1007/BF01889680

Download citation

Keywords

  • EM algorithm
  • mixture decomposition
  • simulated annealing