Advertisement

Statistics and Computing

, Volume 17, Issue 3, pp 209–218 | Cite as

An online classification EM algorithm based on the mixture model

  • Allou SaméEmail author
  • Christophe Ambroise
  • Gérard Govaert
Article

Abstract

Mixture model-based clustering is widely used in many applications. In certain real-time applications the rapid increase of data size with time makes classical clustering algorithms too slow. An online clustering algorithm based on mixture models is presented in the context of a real-time flaw-diagnosis application for pressurized containers which uses data from acoustic emission signals. The proposed algorithm is a stochastic gradient algorithm derived from the classification version of the EM algorithm (CEM). It provides a model-based generalization of the well-known online k-means algorithm, able to handle non-spherical clusters. Using synthetic and real data sets, the proposed algorithm is compared with the batch CEM algorithm and the online EM algorithm. The three approaches generate comparable solutions in terms of the resulting partition when clusters are relatively well separated, but online algorithms become faster as the size of the available observations increases.

Keywords

Clustering Mixture model EM CEM Stochastic gradient Exponential family 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Biernacki, C., Celeux, G., Govaert, G.: Assessing a mixture model for clustering with the integrated completed likelihood. IEEE PAMI 22, 719–725 (2000) Google Scholar
  2. Bottou, L.: Une approche théorique de l’apprentissage connexioniste; applications à la reconnaissance de la parole. PhD thesis, Université d’Orsay (1991) Google Scholar
  3. Bottou, L., Bengio, Y.: Convergence properties of the K-means algorithm. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 585–592. MIT, Cambridge (1995) Google Scholar
  4. Bryant, P.: Large-samples results for optimization based clustering methods. J. Classif. 8, 31–44 (1991) zbMATHCrossRefGoogle Scholar
  5. Bryant, P., Williamson, J.A.: Assymptotic behaviour of classification maximum likelihood estimates. Biometrika 65, 273–281 (1978) zbMATHCrossRefGoogle Scholar
  6. Celeux, G., Govaert, G.: A classification EM algorithm for clustering and two stochastic versions. Comput. Stat. Data Anal. 14, 315–332 (1992) zbMATHCrossRefGoogle Scholar
  7. Celeux, G., Govaert, G.: Comparison of the mixture and the classification maximum likelihood in cluster analysis. J. Stat. Comput. Simul. 47, 127–146 (1993) CrossRefGoogle Scholar
  8. Celeux, G., Govaert, G.: Gaussian parsimonious clustering models. Pattern Recognit. 28, 781–793 (1995) CrossRefGoogle Scholar
  9. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. B 39, 1–38 (1977) zbMATHGoogle Scholar
  10. Liu, Z., Almhana, J., Choulakian, V., McGorman, R.: Online EM algorithm for mixture with application to Internet traffic modeling. Comput. Stat. Data Anal. 50, 1052–1071 (2006) CrossRefGoogle Scholar
  11. MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley Symposium on Mathematics, Statistics and Probability, vol. 1, pp. 281–298 (1967) Google Scholar
  12. Mariott, F.H.C.: Separating mixtures of normal distributions. Biometrics 31, 767–769 (1975) CrossRefGoogle Scholar
  13. Scott, A.J., Symons, M.J.: Clustering methods based on likelihood ratio criteria. Biometrics 27, 387–397 (1971) CrossRefGoogle Scholar
  14. Titterington, D.M.: Recursive parameter estimation using incomplete data. J. Roy. Stat. Soc. Ser. B 46, 257–267 (1984) zbMATHGoogle Scholar
  15. Wang, S., Zhao, Y.: Almost sure convergence of Titterington’s recursive estimator for mixture models. In: IEEE International Symposium on Information Theory, ISIT (2002) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Allou Samé
    • 1
    Email author
  • Christophe Ambroise
    • 2
  • Gérard Govaert
    • 2
  1. 1.Institut National de Recherche sur les Transports et leur SécuritéArcueilFrance
  2. 2.Laboratoire HEUDIASYCUniversité de Technologie de CompiègneCompiègne CedexFrance

Personalised recommendations