Conditional Maximum Likelihood Estimation in Gaussian Mixtures

  • George E. PolicelloII
Chapter
Part of the NATO Advanced study Institutes Series book series (ASIC, volume 79)

Summary

The paper gives a brief history of the estimation problem for mixtures of gaussian populations. The classical maximum likelihood approach is not appropriate since the likelihood function is unbounded at each sample point. However, this does not seem to cause serious problems when iterative methods are used on a computer. This phenomenon is partially explained by the conditional likelihood approach taken in this paper. In addition, the conditional likelihood approach leads to consistent, asymptotically normal and efficient estimators for the parameters of the mixture. The results of a Monte Carlo study are reported for the univariate case and these show that the procedures provide reasonable estimators for small sample sizes. The methods are then extended to mixtures of multivariate gaussian populations.

Key Words

Gaussian mixtures maximum likelihood estimation mixtures of distributions normal mixtures 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ahmad, M., Giri, N., Sinha, B.K. (1980). Estimation of the mixing proportion of two known distributions. University of Montreal, Department of Mathematics and Statistics, Report, DMS. No. 80–4.Google Scholar
  2. Behoodian, J. (1970). On a mixture of normal distributions. Biometrika, 57, 215–217.CrossRefGoogle Scholar
  3. Bhattacharya, C.G. (1967). A simple method of resolution of a distribution into gaussian components. Biometrics, 23, 115–135.CrossRefGoogle Scholar
  4. Blischke, W.R. (1963). Mixtures of distributions. Classical and Contagious Discrete Distributions, G.P. Patil, ed. Statistical Publishing Society, Calcutta. Pages 351–373.Google Scholar
  5. Boswell, M.T., Ord, J.K., Patil, G.P. (1979). Chance mechanisms underlying univariate distributions. Statistical Distributions in Ecological Work, J.K. Ord, G.P. Patil, C. Taillie, eds. International Co-operative Publishing House, Fairland, Maryland. Pages 3–156.Google Scholar
  6. Chang, W.C. (1974). Estimation in a mixture of two multivariate normal distributions. Biometrics, 30, 377 (Abstract).Google Scholar
  7. Cohen, A.C. (1967). Estimation in mixtures of two normal distributions. Technometrics, 9, 15–28.MathSciNetMATHCrossRefGoogle Scholar
  8. Day, N.E. (1969). Estimating the components of a mixture of normal distributions. Biometrika, 56, 463–474.MathSciNetMATHCrossRefGoogle Scholar
  9. Dempster, A., Laird, N. M., Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1–38 (with discussion).Google Scholar
  10. Dick, N.P., Bowden, D.C. (1973). Maximum likelihood estimation for mixtures of two normal distributions. Biometrics, 29, 781–790.CrossRefGoogle Scholar
  11. Fryer, J.G., Robertson, C.A. (1972). A comparison of some methods for estimating mixed normal distributions. Biometrika, 59, 639–648.MathSciNetMATHCrossRefGoogle Scholar
  12. Gindler, E.M. (1970). Claculation of normal ranges by methods used for resolution of overlapping gaussian distributions. Clinical Chemistry, 16, 124–128.Google Scholar
  13. Gottschalk, V.H. (1948). Symmetric bimodal frequency curves. Journal of the Franklin Institute 245–252Google Scholar
  14. Grannis, G.F., Lott, J.A. (1978). Technique for determining the probability of abnormality. Clinical Chemistry, 24, 640–651.Google Scholar
  15. Gridgeman, N.T. (1970). A comparison of two methods of analysis of mixtures of normal distributions. Technometrics, 12, 823–833.MATHCrossRefGoogle Scholar
  16. Harding, J.F. (1949). The use of probability paper for the graphical analysis of polymodal frequency distributions. Journal of Maine Biological Association U.K., 28, 141–153.CrossRefGoogle Scholar
  17. Hasselbland, V. (1969). Estimation of parameters for a mixture of normal distributions. Technometrics, 8, 431–444.CrossRefGoogle Scholar
  18. Hasselbland, V. (1969). Estimation of finite mixtures of distributions from the exponential family. Journal of the American Statistical Association, 64, 1459–1471.CrossRefGoogle Scholar
  19. Hosmer, D.W. Jr. (1973). A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample. Biometrics, 29, 761–770.CrossRefGoogle Scholar
  20. James. I. R. (1978). Estimation of the mixing proportion in a mixture of two normal distributions for simple, rapid measurements. Biometrics, 34, 265–275.MATHCrossRefGoogle Scholar
  21. John, S. (1970). On identifying the pouplation of origin of each observation in a mixture of observations from two normal populations. Technometrics, 12, 553–563.CrossRefGoogle Scholar
  22. Kiefer, N.M. (1978). Discrete parameter variation: efficient estimation of a switching regression model. Econometrica, 46, 427–434.MathSciNetMATHCrossRefGoogle Scholar
  23. Macdonald, P.D.M. (1969). Fortran pr6grams for statistical estimation of distribution mixtures: Some techniques for statistical analysis of length frequency data. Fisheries Research Board of Canady. Technical Report #129.Google Scholar
  24. Marsaglia, G., MacLaren, M.D., Bray, T.A. (1964). A fast procedure for generating normal random variables. Communications of the ACM, 7, 4–10.MATHCrossRefGoogle Scholar
  25. Pearson, Karl (1894). Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society, 185A, 71–110.MATHCrossRefGoogle Scholar
  26. Pearson, K. (1915). On the problem of sexing osteometric material. Biometrika, 40, 479–484.CrossRefGoogle Scholar
  27. Pearson, K., Lee, A. (1908-1909). On the generalized probable error in multiple normal correlation. Biometrika 6, 59–68Google Scholar
  28. Peters, B.C., Walker, H.F. (1978). Iterative procedure for obtaining maximum likelihood estimates of parameters for a mixture of normal distributions. SIAM Journal on Applied Mathematics, 35, 362–378.MathSciNetMATHCrossRefGoogle Scholar
  29. Policello, G.E. II (1980). A conditional maximum likelihood approach to estimation in gaussian mixtures. The Ohio State University, Department of Statistics, Technical Report #208, 1–42.Google Scholar
  30. Preston, E.J. (1953). A graphical method for analysis of statistical distributions into normal components. Biometrika, 40, 460–464.MathSciNetMATHGoogle Scholar
  31. Quandt, R.E., Ramsey, J.B. (1978). Estimating mixtures of normal distributions and switching regressions. Journal of the American Statistical Association, 73, 730–752. (With discussion.)MathSciNetMATHCrossRefGoogle Scholar
  32. Rao, C.R. (1948). The utilization of multiple measurements in problems of biological classification. Journal of the Royal Statistical Society, Series B, 10, 159–203.MATHGoogle Scholar
  33. Rayment, P.R. (1972). The identification problem for a mixture of observations from two normal populations. Technometrics, 14, 911–918.MathSciNetMATHCrossRefGoogle Scholar
  34. Robertson, C.A., Fryer, J.G. (1970). The bias and accuracy of moment estimators. Biometrika, 57, 57–65.MathSciNetMATHCrossRefGoogle Scholar
  35. Tan, W.Y., Chang, W.C. (1972). Some comparisons of the method of moments and the method of maximum likelihood in estimating parameters of a mixture of normal densities. Journal of the American Statistical Association, 67, 702–708.MATHCrossRefGoogle Scholar
  36. Taylor, B.J.R. (1965). The analysis of polymodal frequency distributions. Journal of Animal Ecology, 34, 445–452.CrossRefGoogle Scholar
  37. Tubbs, J.D., Coberly, W.A. (1976). An Empirical sensitivity study of mixture proportion estimators. Communications in Statistics - Theory and Methods, A-5, 1115–1125.CrossRefGoogle Scholar
  38. Wolfe, J.H. (1965). A computer program for the analysis of tyes. Technical Bulletin 65–15, U.S. Naval Personnel and Training Research Laboratory, San Diego (Defense Documentation Center AD 620–026 ).Google Scholar
  39. Wolfe, J.H. (1970). Pattern clustering by multivariate mixture analysis. Multivariate Behavioral Research, 5, 329–350.CrossRefGoogle Scholar
  40. Wolfe, J.H. (1971). A Monte Carlo study of the sampling distribution of the likelihood ratio for mixtures of multi- normal distributions. Technical Bulletin STB 72–2, U. S. Naval Personnel and Training Research Laboratory.Google Scholar

Copyright information

© D. Reidel Publishing Company 1981

Authors and Affiliations

  • George E. PolicelloII
    • 1
  1. 1.Department of StatisticsThe Ohio State UniversityUSA

Personalised recommendations