Skip to main content
SpringerLink
Log in

Ensemble Gaussian mixture models for probability density estimation

  • Original Paper
  • Published:
Computational Statistics Aims and scope Submit manuscript

Abstract

Estimation of probability density functions (PDF) is a fundamental concept in statistics. This paper proposes an ensemble learning approach for density estimation using Gaussian mixture models (GMM). Ensemble learning is closely related to model averaging: While the standard model selection method determines the most suitable single GMM, the ensemble approach uses a subset of GMM which are combined in order to improve precision and stability of the estimated probability density function. The ensemble GMM is theoretically investigated and also numerical experiments were conducted to demonstrate benefits from the model. The results of these evaluations show promising results for classifications and the approximation of non-Gaussian PDF.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

    MATH  Google Scholar 

  • Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MathSciNet  MATH  Google Scholar 

  • Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Methodological) 39(1):1–38

    Google Scholar 

  • Dietterich TG (2000) Ensemble methods in machine learning. In: Kittler J and Roli F (eds) Proceedings of the international workshop on multiple classifier systems (MCS), vol 1857 of lecture notes in computer science (LNCS). Springer, pp 1–15

  • Freund Y, Schapire E (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman JH, Stuetzle W, Schroeder A (1984) Projection pursuit density estimation. J Am Stat Assoc 79:599–608

    Article  MathSciNet  Google Scholar 

  • Fukunaga K (1990) Introduction to statistical pattern recognition. Academic Press, New York

    MATH  Google Scholar 

  • Hastie T, Tibshirani R, Friedman JH (2001) The elements of statistical learning: data mining, inference, and prediction. Springer, Berlin

    MATH  Google Scholar 

  • Hwang JN, Lay SR, Lippman A (1994) Nonparametric multivariate density estimation: a comparative study. IEEE Trans Signal Process 42:2795–2810

    Article  Google Scholar 

  • Jones MC, Marron JS, Sheather SJ (1996) A brief survey of bandwidth selection for density estimation. J Am Stat Assoc 91:401–407

    Google Scholar 

  • Kim C, Kim S, Park M, Lee H (2006) A bias reducing technique in kernel distribution function estimation. Comput Stat 21:589–601

    Article  MathSciNet  MATH  Google Scholar 

  • Kraus J, Müssel C, Palm G, Kestler HA (2011) Multi-objective selection for collecting cluster alternatives. Comput Stat 26:341–353

    Article  Google Scholar 

  • Kuncheva LI (2004) Combining pattern classifiers: methods and algorithms. Wiley, London

    Book  MATH  Google Scholar 

  • Maiboroda R, Markovich N (2004) Estimation of heavy-tailed probability density function with application to web data. Comput Stat 19:569–592

    Article  MathSciNet  MATH  Google Scholar 

  • Ormoneit D, Tresp V (1998) Averaging, maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates. IEEE Trans Neural Netw 9(4):639–650

    Google Scholar 

  • Rabiner L, Juang B-H (1993) Fundamentals of speech recognition. Prentice Hall, Englewood Cliffs

    Google Scholar 

  • Ripley D (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Scott W (1992) Multivariate density estimation: theory, practice, and visualization. Wiley, New York

    Book  MATH  Google Scholar 

  • Shinozaki T, Kawahara T (2008) GMM and HMM training by aggregated EM algorithm with increased ensemble sizes for robust parameter estimation. In: Proceedings of the international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 4405–4408

  • Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, London

    MATH  Google Scholar 

Download references

Acknowledgments

The presented work was developed within the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG) and DFG project SCHW 623/4-2. The work of Martin Schels is supported by a scholarship of the Carl-Zeiss Foundation.

Author information

Authors and Affiliations

  1. Institute of Neural Information Processing, University of Ulm, 89069 , Ulm, Germany

    Michael Glodek, Martin Schels & Friedhelm Schwenker

Authors
  1. Michael Glodek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Schels
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Friedhelm Schwenker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Friedhelm Schwenker.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Glodek, M., Schels, M. & Schwenker, F. Ensemble Gaussian mixture models for probability density estimation. Comput Stat 28, 127–138 (2013). https://doi.org/10.1007/s00180-012-0374-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00180-012-0374-5

Keywords

Search

Navigation