Advertisement

Adaptive Seeding for Gaussian Mixture Models

  • Johannes Blömer
  • Kathrin Bujna
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9652)

Abstract

We present new initialization methods for the expectation-maximization algorithm for multivariate Gaussian mixture models. Our methods are adaptions of the well-known K-means++ initialization and the Gonzalez algorithm. Thereby we aim to close the gap between simple random, e.g. uniform, and complex methods, that crucially depend on the right choice of hyperparameters. Our extensive experiments indicate the usefulness of our methods compared to common techniques and methods, which e.g. apply the original K-means++ and Gonzalez directly, with respect to artificial as well as real-world data sets.

Keywords

Covariance Matrice Gaussian Mixture Model Hierarchical Agglomerative Cluster Noise Point Spherical Covariance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    GeoNames geographical database. http://www.geonames.org/
  2. 2.
    Achtert, E., Goldhofer, S., Kriegel, H.-P., Schubert, E., Zimek, A.: Evaluation of Clusterings - Metrics and Visual Support. http://elki.dbs.ifi.lmu.de/wiki/DataSets/MultiView
  3. 3.
    Arthur, V.: k-means++: The advantages of careful seeding. In: SODA 2007 (2007)Google Scholar
  4. 4.
    Asuncion: UCI machine learning repository (2007). http://www.ics.uci.edu/mlearn/MLRepository.html
  5. 5.
    Baudry, J.-P., Celeux, G.: EM for mixtures. Stat. Comput. 25(4), 713–726 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Biernacki, C.: Initializing EM using the properties of its trajectories in Gaussian mixtures. Stat. Comput. 14(3), 267–279 (2004)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Biernacki, C., Celeux, G., Govaert, G.: Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models. Comput. Stat. Data Anal. 41(3–4), 561–575 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bishop, C.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, Secaucus (2006)zbMATHGoogle Scholar
  9. 9.
  10. 10.
    Celeux, G., Govaert, G.: A classification EM algorithm for clustering and two stochastic versions. Comput. Stat. Data Anal. 14(3), 315–332 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Dasgupta, S.: Experiments with random projection. In: UAI 2000 (2000)Google Scholar
  12. 12.
    Dasgupta, S.: Learning mixtures of gaussians. In: FOCS 1999 (1999)Google Scholar
  13. 13.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Stat. Methodol. 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Färber, I., Günnemann, S., Kriegel, H., Kröger, P., Müller, E., Schubert, E., Seidl, T., Zimek, A.: On using class-labels in evaluation of clusterings. In: MultiClust 2010 (2010)Google Scholar
  15. 15.
    Fayyad, U., Reina, C., Bradley, P.S.: Initialization of iterative refinement clustering algorithms. In: KDD 1998 (1998)Google Scholar
  16. 16.
    Geusebroek, J.M., Burghouts, G.J., Smeulders, A.W.M.: The Amsterdam library of object images. Int. J. Comput. Vis. 6(1), 103–112 (2005)CrossRefGoogle Scholar
  17. 17.
    Gonzalez, T.F.: Clustering to minimize the maximum intercluster distance. Theor. Comput. Sci. 38, 293–306 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Kriegel, H.-P., Schubert, E., Zimek, A.: Evaluation of multiple clustering solutions. In: MultiClust 2010 (2010)Google Scholar
  19. 19.
    Krüger, A., Leutnant, V., Haeb-Umbach, R., Ackermann, M., Blömer, J.: On the initialization of dynamic models for speech features. In: Sprachkommunikation 2010 (2010)Google Scholar
  20. 20.
    Kwedlo, W.: A new random approach for initialization of the multiple restart EM algorithm for Gaussian model-based clustering. Pattern Anal. Appl. 18(4), 757–770 (2015)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Maitra, R.: Initializing partition-optimization algorithms. IEEE/ACM Trans. Comput. Biol. Bioinform. 6(1), 144–157 (2009)CrossRefGoogle Scholar
  22. 22.
    Maitra, R., Melnykov, V.: Simulating data to study performance of finite mixture modeling and clustering algorithms. J. Comput. Graph. Stat. 19(2), 354–376 (2010)MathSciNetCrossRefGoogle Scholar
  23. 23.
    McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. Wiley Series in Probability and Statistics, 2nd edn. Wiley-Interscience, New York (2008)CrossRefzbMATHGoogle Scholar
  24. 24.
    Meilă, M., Heckerman, D.: An experimental comparison of several clustering and initialization methods. In: UAI 1998. Morgan Kaufmann Inc., San Francisco (1998)Google Scholar
  25. 25.
    Melnykov, V., Melnykov, I.: Initializing the EM algorithm in Gaussian mixture models with an unknown number of components. Comput. Stat. Data Anal. 56, 1381–1395 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Thiesson, B.: Accelerated quantification of Bayesian networks with incomplete data. University of Aalborg (1995)Google Scholar
  27. 27.
    Verbeek, J.J., Vlassis, N., Kröse, B.: Efficient greedy learning of Gaussian mixture models. Neural Comput. 15(2), 469–485 (2003)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Paderborn UniversityPaderbornGermany

Personalised recommendations