We compare the performances of the simulated annealing and the EM algorithms in problems of decomposition of normal mixtures according to the likelihood approach. In this case the likelihood function has multiple maxima and singularities, and we consider a suitable reformulation of the problem which yields an optimization problem having a global solution and at least a smaller number of spurious maxima. The results are compared considering some distance measures between the estimated distributions and the true ones. No overwhelming superiority of either method has been demonstrated, though in one of our cases simulated annealing achieved better results.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price includes VAT (USA)
Tax calculation will be finalised during checkout.
Boyles, R. A. (1983) On the convergence of the EM algorithm.Journal of the Royal Statistical Society B,45, 47–50.
Burden, R. L. and Faires, J. D. (1985)Numerical Analysis, Prindle, Weber & Schmidt, Boston, MA.
Day, N. E. (1969) Estimating the components of a mixture of normal distributions.Biometrika,56, 463–474.
Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maximum likelihood estimation from incomplete data via the EM algorithm.Journal of the Royal Statistical Society B,39, 1–38.
Fowlkes, E. B. (1979) Some methods for studying the mixture of two normal (lognormal) distributions.Journal of the American Statistical Association,74, 561–575.
Hathaway, R. J. (1985a) A constrained formulation of maximum likelihood estimation for normal mixture decompositions.Annals of Statistics,13, 795–800.
Hathaway, R. J. (1985b) A constrained EM algorithm for univariate normal mixture.Journal of Statistical Computing and Simulation.
Hathaway, R. J. (1986) Another interpretation of the EM algorithm for mixture distributions.Statistics and Probability Letters,4, 53–56.
Ingrassia, S. (1991) Mixture decomposition via the simulated annealing algorithm.Applied Stochastic Models and Data Analysis,7, 317–325.
Johnson, D. S., Aragon, C. R., McGeoch, L. A. and Schevon, C. (1986)Optimization by Simulated Annealing: An Experimental Evaluation. List of Abstracts, Workshop on Statistical Physics in Engineering and Biology. Yorktown Heights, NY.
Kiefer, N. M. (1978) Discrete parameter variation: efficient estimation of a switching regression model.Econometrica,46, 427–434.
Kirkpatrick, S., Gelatt, C. G. and Vecchi, M. P. (1983) Optimization by the simulated annealing.Science,220, 671–680.
van Laarhoven, P. J. M. and Aarts, E. H. L. (1988)Simulated Annealing: Theory and Applications, D. Reidel, Dordrecht.
Redner, R. A. and Walker, H. F. (1984) Mixture densities, maximum likelihood and the EM algorithm.SIAM Reviews,46, 195–239.
Titterington, D. M., Smith, A. F. M. and Makov, U. E. (1985)Statistical Analysis of Finite Mixture Distributions, Wiley, Chichester.
Vanderbilt, D. and Louie, S. G. (1984) A Monte Carlo simulated annealing approach to optimization over continuous variables.Journal of Computational Physics,56, 259–271.
About this article
Cite this article
Ingrassia, S. A comparison between the simulated annealing and the EM algorithms in normal mixture decompositions. Stat Comput 2, 203–211 (1992). https://doi.org/10.1007/BF01889680
- EM algorithm
- mixture decomposition
- simulated annealing