Ensemble Gaussian mixture models for probability density estimation
- 512 Downloads
Estimation of probability density functions (PDF) is a fundamental concept in statistics. This paper proposes an ensemble learning approach for density estimation using Gaussian mixture models (GMM). Ensemble learning is closely related to model averaging: While the standard model selection method determines the most suitable single GMM, the ensemble approach uses a subset of GMM which are combined in order to improve precision and stability of the estimated probability density function. The ensemble GMM is theoretically investigated and also numerical experiments were conducted to demonstrate benefits from the model. The results of these evaluations show promising results for classifications and the approximation of non-Gaussian PDF.
KeywordsEnsemble learning Density estimation Finite mixture models
The presented work was developed within the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG) and DFG project SCHW 623/4-2. The work of Martin Schels is supported by a scholarship of the Carl-Zeiss Foundation.
- Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Methodological) 39(1):1–38Google Scholar
- Dietterich TG (2000) Ensemble methods in machine learning. In: Kittler J and Roli F (eds) Proceedings of the international workshop on multiple classifier systems (MCS), vol 1857 of lecture notes in computer science (LNCS). Springer, pp 1–15Google Scholar
- Jones MC, Marron JS, Sheather SJ (1996) A brief survey of bandwidth selection for density estimation. J Am Stat Assoc 91:401–407Google Scholar
- Ormoneit D, Tresp V (1998) Averaging, maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates. IEEE Trans Neural Netw 9(4):639–650Google Scholar
- Rabiner L, Juang B-H (1993) Fundamentals of speech recognition. Prentice Hall, Englewood CliffsGoogle Scholar
- Shinozaki T, Kawahara T (2008) GMM and HMM training by aggregated EM algorithm with increased ensemble sizes for robust parameter estimation. In: Proceedings of the international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 4405–4408Google Scholar