Skip to main content
SpringerLink
Log in

Recent Developments in Model-Based Clustering with Applications

  • Chapter
  • First Online:
Partitional Clustering Algorithms

Abstract

Model-based clustering is a popular technique relying on the notion of finite mixture models that proved to be efficient in modeling heterogeneity in data. The underlying idea is to model each data group by a particular mixture component. This relationship between mixed distributions and clusters forms an attractive interpretation of groups: each cluster is assumed to be a sample from the corresponding distribution. In practice, however, there are many issues that have to be accounted for by the researcher. The area of model-based clustering is very dynamic and rapidly developing, with many questions yet to be answered. In this paper, we review and discuss the latest developments in model-based clustering including semi-supervised clustering, non-parametric mixture modeling, choice of initialization strategies, merging mixture components for clustering, handling spurious solutions, and assessing variability of obtained partitions. We also demonstrate the utility of model-based clustering by considering several challenging applications to real-life problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Second international symposium on information theory, pp 267–281

    Google Scholar 

  2. Anh NK, Tam NT, Van Linh N (2013) Document clustering using Dirichlet process mixture model of von Mises-Fisher distributions. In: Proceedings of the fourth symposium on information and communication technology, New York, pp 131–138

    Google Scholar 

  3. Attias H (1999) Inferring parameters and structure of latent variable models by variational Bayes. In: Proceedings of the fifteenth conference on uncertainty in artificial intelligence

    Google Scholar 

  4. Azzalini A, Bowman AW (1990) A look at some data on the old faithful geyser. J R Stat Soc C 39:357–365

    MATH  Google Scholar 

  5. Azzalini A, Menardi G (2013) Package pdfCluster: cluster analysis via nonparametric density estimation. http://cran.r-project.org/web/packages/pdfCluster

  6. Azzalini A, Torelli N (2007) Clustering via nonparametric density estimation. Stat Comput 17:71–80

    MathSciNet  Google Scholar 

  7. Banerjee A, Dhillon IS, Ghosh J, Sra S (2005) Clustering on the unit hypersphere using von Mises-Fisher distributions. J Mach Learn Res 6:1345–1382

    MATH  MathSciNet  Google Scholar 

  8. Banfield JD, Raftery AE (1993) Model-based Gaussian and non-Gaussian clustering. Biometrics 49:803–821

    MATH  MathSciNet  Google Scholar 

  9. Bar-Hillel A, Hertz T, Shental N, Weinshall D (2003) Learning distance functions using equivalence relations. In: Proceedings of the twentieth international conference on machine learning, pp 11–18

    Google Scholar 

  10. Basso R, Lachos V, Cabral C, Ghosh P (2010) Robust mixture modeling based on scale mixtures of skew-normal distributions. Comput Stat Data Anal 54(12):2926–2941

    MATH  MathSciNet  Google Scholar 

  11. Basu S, Banerjee A, Mooney R (2002) Semi-supervised clustering by seeding. In: Proceedings of the 19th international conference on machine learning, pp 19–26

    Google Scholar 

  12. Basu S, Bilenko M, Mooney RJ (2004) A probabilistic framework for semi-supervised clustering. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining, pp 59–68

    Google Scholar 

  13. Baudry JP, Raftery A, Celeux G, Lo K, Gottardo R (2010) Combining mixture components for clustering. J Comput Graph Stat 19(2):332–353

    MathSciNet  Google Scholar 

  14. Benaglia T, Chauveau D, Hunter DR (2009) An EM-like algorithm for semi- and nonparametric estimation in multivariate mixtures. J Comput Graph Stat 18(2):505–526

    MathSciNet  Google Scholar 

  15. Benaglia T, Chauveau D, Hunter DR, S YD (2009) mixtools: an R package for analyzing mixture models. J Stat Softw 32(6):1–29

    Google Scholar 

  16. Benaglia T, Chauveau D, Hunter DR (2011) Bandwidth selection in an EM-like algorithm for nonparametric multivariate mixtures. In: Hunter D, Richards DSP, Rosenberger J (eds) Nonparametric statistics and mixture models, A Festschrift in honor of Thomas P Hettmansperger. World Scientific, Singapore, pp 15–27

    Google Scholar 

  17. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc 57:289–300

    MATH  MathSciNet  Google Scholar 

  18. Berlinet AF, Roland C (2012) Acceleration of the em algorithm: P-em versus epsilon algorithm. Comput Stat Data Anal 56(12):4122–4137

    MATH  MathSciNet  Google Scholar 

  19. Biernacki C, Celeux G, Gold EM (2000) Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans Pattern Anal Mach Intell 22:719–725

    Google Scholar 

  20. Biernacki C, Celeux G, Govaert G (2003) Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models. Comput Stat Data Anal 413:561–575

    MathSciNet  Google Scholar 

  21. Böhning D, Dietz E, Schaub R, Schlattmann P, Lindsay B (1994) The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family. Ann Inst Stat Math 46(2):373–388

    MATH  Google Scholar 

  22. Bouveyron C, Brunet C (2014) Model-based clustering of high-dimensional data: a review. Comput Stat Data Anal 71:52–78

    MathSciNet  Google Scholar 

  23. Bouveyron C, Girard S, Schmid C (2007) High-dimensional data clustering. Comput Stat Data Anal 52(1):502–519. http://lear.inrialpes.fr/pubs/2007/BGS07a

  24. Bridge M (2012) Locating the origins of wood resources: a review of dendroprovenancing. J Archaeol Sci 39(8):2828–2834

    Google Scholar 

  25. Butts CT, Handcock MS, Hunter DR (2014) Network: classes for relational data. Irvine. R package version 1.9.0, http://statnet.org/

  26. Cadez I, Heckerman D, Meek C, Smyth P, White S (2003) Model-based clustering and visualization of navigation patterns on a web site. Data Min Knowl Discov 7:399–424

    MathSciNet  Google Scholar 

  27. Campbell NA, Mahon RJ (1974) A multivariate study of variation in two species of rock crab of Genus Leptograsus. Aust J Zool 22:417–25

    Google Scholar 

  28. Celebi ME, Kingravi H, Vela PA (2013) A comparative study of efficient initialization methods for the K-means clustering algorithm. Expert Syst Appl 40(1):200–210

    Google Scholar 

  29. Celeux G, Govaert (1995) Gaussian parsimonious clustering models. Comput Stat Data Anal 28:781–93

    Google Scholar 

  30. Celeux C, Martin-Magniette ML, Maugis C, Raftery A (2011) Letter to the editor. J Am Stat Assoc 106:383

    MathSciNet  Google Scholar 

  31. Chandra S (1977) On the mixtures of probability distributions. Scand J Stat 4:105–112

    MATH  Google Scholar 

  32. Chen WC, Maitra R (2011) Model-based clustering of regression time series data via APECM – an AECM algorithm sung to an even faster beat. Stat Anal Data Min 4:567–578

    MathSciNet  Google Scholar 

  33. Chen J, Tan X, Zhang R (2008) Consistency of penalized MLE for normal mixtures in mean and variance. Stat Sin 18:443–465

    MATH  MathSciNet  Google Scholar 

  34. Ciuperca G, Ridolfi A, Idier J (2003) Penalized maximum likelihood estimator for normal mixtures. Scand J Stat 30(1):45–59

    MATH  MathSciNet  Google Scholar 

  35. Corduneanu A, Bishop CM (2001) Variational Bayesian model selection for mixture distributions. In: Proceedings eighth international conference on artificial intelligence and statistics, pp 27–34

    Google Scholar 

  36. Dean N, Raftery A, Scrucca L (2013) Package clustvarsel: variable selection for model-based clustering. http://cran.r-project.org/web/packages/clustvarsel

  37. Demiriz A, Bennett K, Embrechts MJ (1999) Semi-supervised clustering using genetic algorithms. In: Artificial neural networks in engineering (ANNIE-99). ASME Press, New York, pp 809–814

    Google Scholar 

  38. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood for incomplete data via the EM algorithm (with discussion). J R Stat Soc Ser B 39:1–38

    MATH  MathSciNet  Google Scholar 

  39. Dertinger JJ, Walker AV (2013) Ionic liquid matrix-enhanced secondary ion mass spectrometry: the role of proton transfer. J Am Soc Mass Spectrom 24:348–355

    Google Scholar 

  40. Dhillon IS, Modha DS (2001) Concept decompositions for large sparse text data using clustering. Mach Learn 42:143–175

    MATH  Google Scholar 

  41. Diebolt J, Robert C (1994) Estimation of finite mixture distributions by Bayesian sampling. J R Stat Soc Ser B 56:363–375

    MATH  MathSciNet  Google Scholar 

  42. Digalakis VV, Rtischev D, Neumeyer LG (1995) Speaker adaptation using constrained estimation of Gaussian mixtures. IEEE Trans Speech Audio Process 3(5):357–366

    Google Scholar 

  43. Dortet-Bernadet J, Wicker N (2008) Model-based clustering on the unit sphere with an illustration using gene expression profiles. Biostatistics 9(1):66–80

    MATH  Google Scholar 

  44. Efron B, Tibshirani R, d Storey J, Tusher V (2001) Empirical Bayes analysis of a microarray experiment. J Am Stat Assoc 96:1151–1160

    Google Scholar 

  45. Escobar MD, West M (1995) Bayesian density estimation and inference using mixtures. J Am Stat Assoc 90:577–588

    MATH  MathSciNet  Google Scholar 

  46. Esper J, Cook E, Schweingruber F (2002) Low-frequency signals in long tree-ring chronologies for reconstructing past temperature variability. Science 295(5563):2250–2253

    Google Scholar 

  47. Feng Z, McCulloch C (1996) Using bootstrap likelihood ratio in finite mixture models. J R Stat Soc B 58:609–617

    MATH  Google Scholar 

  48. Forgy E (1965) Cluster analysis of multivariate data: efficiency vs. interpretability of classifications. Biometrics 21:768–780

    Google Scholar 

  49. Fraley C (1998) Algorithms for model-based Gaussian hierarchical clustering. SIAM J Sci Comput 20:270–281

    MATH  MathSciNet  Google Scholar 

  50. Fraley C, Raftery AE (2002) Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc 97:611–631

    MATH  MathSciNet  Google Scholar 

  51. Fraley C, Raftery AE (2006) MCLUST version 3 for R: normal mixture modeling and model-based clustering. Technical Report 504, Department of Statistics, University of Washington, Seattle

    Google Scholar 

  52. Frühwirth-Schnatter S (2001) Markov Chain Monte Carlo estimation of classical and dynamic switching and mixture models. J Am Stat Assoc 96:194–209

    MATH  Google Scholar 

  53. Frühwirth-Schnatter S, Pyne S (2010) Bayesian inference for finite mixtures of univariate and multivariate skew-normal and skew-t distributions. Biostatistics 11:317–336

    Google Scholar 

  54. Gallegos MT, Ritter G (2009) Trimmed ML estimation of contaminated mixtures. Sankhya Ser A 71:164–220

    MATH  MathSciNet  Google Scholar 

  55. Garcia-Escudero L, Gordaliza A, Mayo-Iscar A (2013) A constrained robust proposal for mixture modeling avoiding spurious solutions. Adv Data Anal Classif 1–17. doi:10.1007/s11634-013-0153-3

    Google Scholar 

  56. Gopal S, Yang Y (2014) von Mises-Fisher clustering models. J Mach Learn Res 32:154–162

    Google Scholar 

  57. Gormley IC, Murphy TB (2010) A mixture of experts latent position cluster model for social network data. Stat Methodol 7:385–405

    MATH  MathSciNet  Google Scholar 

  58. Guo J, Levina E, Michailidis G, Zhu J (2010) Pairwise variable selection for high-dimensional model-based clustering. Biometrics 66:793–804

    MATH  MathSciNet  Google Scholar 

  59. Hall P, Ormerod JT, Wand MP (2011) Theory of Gaussian variational approximation for a Poisson mixed model. Stat Sin 21:369–389

    MATH  MathSciNet  Google Scholar 

  60. Hammer R, Hertz T, Hochstein S, Weinshall D (2007) Classification with positive and negative equivalence constraints: theory, computation and human experiments. In: Proceedings of the 2nd international conference on advances in brain, vision and artificial intelligence, Springer-Verlag Berlin, pp 264–276

    Google Scholar 

  61. Handcock MS, Raftery AE, Tantrum JM (2007) Model-based clustering for social networks. J R Stat Soc Ser A 170:301–354

    MathSciNet  Google Scholar 

  62. Haneca K, Wazny T, Van Acker J, Beeckman H (2005) Provenancing Baltic timber from art historical objects: success and limitations. J Archaeol Sci 32(2):261–271

    Google Scholar 

  63. Hartigan JA (1981) Consistency of single linkage for high-density clusters. J Am Stat Assoc 76:388–394

    MATH  MathSciNet  Google Scholar 

  64. Hathaway RJ (1985) A constrained formulation of maximum-likelihood estimation for normal mixture distributions. Stat Probab Lett 4:53–56

    MathSciNet  Google Scholar 

  65. Hennig C (2004) Breakdown points for maximum likelihood-estimators of location-scale mixtures. Ann Stat 32:1313–1340

    MATH  MathSciNet  Google Scholar 

  66. Hennig C (2010) Methods for merging Gaussian mixture components. Adv Data Anal Classif 4:3–34

    MathSciNet  Google Scholar 

  67. Hennig C, Coretto P (2008) The noise component in model-based cluster analysis. In: Preisach C, Burkhardt H, Schmidt-Thieme L, Decker R (eds) Data analysis, machine learning and applications, studies in classification, data analysis, and knowledge organization. Springer, Berlin, Heidelberg, pp 127–138

    Google Scholar 

  68. Hoff PD, Raftery AE, Handcock MS (2002) Latent space approaches to social network analysis. J Am Stat Assoc 97:460:1090–1098

    MATH  MathSciNet  Google Scholar 

  69. Holzmann H, Munk A, Gneiting T (2006) Identifiability of finite mixtures of elliptical distributions. Scand J Stat 33:753–763

    MATH  MathSciNet  Google Scholar 

  70. Huang JT, Hasegawa-Johnson M (2009) On semi-supervised learning of Gaussian mixture models for phonetic classification. In: NAACL HLT workshop on semi-supervised learning

    Google Scholar 

  71. Inbarani HH, Thangavel K (2009) Mining and analysis of clickstream patterns. In: Abraham A, Hassanien AE, Leon F de Carvalho A, Snášel V (eds) Foundations of computational, intelligence, vol 6. Studies in computational intelligence, vol 206. Springer, Berlin, Heidelberg, pp 3–27

    Google Scholar 

  72. Jasra A, Holmes CC, Stephens DA (2005) Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling. Stat Sci 20:50–67

    MATH  MathSciNet  Google Scholar 

  73. Jiao S, Zhang S (2008) The t-mixture model approach for detecting differentially expressed genes in microarrays. Funct Integr Genomics 8:181–186

    Google Scholar 

  74. Jolliffe IT, Jones B, Morgan BJT (1995) Identifying influential observations in hierarchical cluster analysis. J Appl Stat 22(1):61–80

    MathSciNet  Google Scholar 

  75. Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82:35–45

    Google Scholar 

  76. Kent J (1983) Identifiability of finite mixtures for directional data. Ann Stat 11(3):984–988

    MATH  MathSciNet  Google Scholar 

  77. Kiefer NM (1978) Discrete parameter variation: efficient estimation of a switching regression model. Econometrica 46:427–434

    MATH  MathSciNet  Google Scholar 

  78. Kim D, Seo B (2014) Assessment of the number of components in Gaussian mixture models in the presence of multiple local maximizers. J Multivar Anal 125:100–120

    MATH  MathSciNet  Google Scholar 

  79. Klein D, Kamvar SD, Manning C (2002) From instance-level constraints to space-level constraints: making the most of prior knowledge in data clustering. In: Proceedings of the nineteenth international conference on machine learning (ICML-2002), pp 307–314

    Google Scholar 

  80. Krivitsky PN, Handcock MS (2008) Fitting position latent cluster models for social networks with latentnet. J Stat Softw 24(5). http://statnetproject.org

  81. Krivitsky PN, Handcock MS (2009) latentnet: Latent position and cluster models for statistical networks. R package version 2.2-2. http://statnetproject.org

  82. Lauritzen SL (1996) Graphical models. Clarendon Press, Oxford

    Google Scholar 

  83. Law MHC, Topchy A, Jain AK (2005) Model-based clustering with probabilistic constraints. In: 2005 SIAM international conference on data mining, pp 641–645

    Google Scholar 

  84. Lee H, Li J (2012) Variable selection for clustering by separability based on ridgelines. J Comput Graph Stat 21:315–337

    MathSciNet  Google Scholar 

  85. Lee S, McLachlan G (2013) On mixtures of skew normal and skew t-distributions. Adv Data Anal Classif 7:241–266

    MATH  MathSciNet  Google Scholar 

  86. Li J, Zha H (2006) Two-way Poisson mixture models for simultaneous document classification and word clustering. Comput Stat Data Anal 50(1):163–180

    MATH  MathSciNet  Google Scholar 

  87. Li J, Ray S, Lindsay B (2007) A nonparametric statistical approach to clustering via mode identification. J Mach Learn Res 8:1687–1723

    MATH  MathSciNet  Google Scholar 

  88. Lin TI (2009) Maximum likelihood estimation for multivariate skew normal mixture models. J Multivar Anal 100:257–265

    MATH  Google Scholar 

  89. Lin TI, Lee JC, Yen SY (2007) Finite mixture modelling using the skew normal distribution. Stat Sin 17:909–927

    MATH  MathSciNet  Google Scholar 

  90. Liu B (2011) Web data mining: exploring hyperlinks, contents, and usage data, 2nd edn. Springer, New York

    Google Scholar 

  91. Liu C, Rubin DB (1994) The ECME algorithm: a simple extension of EM and ECM with faster monotone convergence. Biometrika 81:633–648

    MATH  MathSciNet  Google Scholar 

  92. Liu C, Rubin DB, Wu YN (1998) Parameter expansion to accelerate em: the PX-EM algorithm. Biometrika 85:755–770

    MATH  MathSciNet  Google Scholar 

  93. Lotsi A, Wit E (2013) High dimensional sparse Gaussian graphical mixture model. arXiv:13083381v3

    Google Scholar 

  94. Lu Z, Leen TK (2007) Penalized probabilistic clustering. Neural Comput 19:1528–1567

    MATH  MathSciNet  Google Scholar 

  95. MacEachern SN, Muller P (1998) Estimating mixtures of Dirichlet process models. J Comput Graph Stat 7:223–238

    Google Scholar 

  96. Maitra R (2009) Initializing partition-optimization algorithms. IEEE/ACM Trans Comput Biol Bioinform 6:144–157. http://doi.ieeecomputersociety.org/10.1109/TCBB.2007.70244

  97. Maitra R, Melnykov V (2010) Simulating data to study performance of finite mixture modeling and clustering algorithms. J Comput Graph Stat 19(2):354–376. doi:10.1198/ jcgs.2009.08054

    MathSciNet  Google Scholar 

  98. Mardia KV, Jupp PE (2000) Directional statistics. Wiley, New York

    MATH  Google Scholar 

  99. Markitsis A, Lai Y (2010) The t-mixture model approach for detecting differentially expressed genes in microarrays. Bioinformatics 26:640–646

    Google Scholar 

  100. Martinez-Uso A, Pla F, Sotoca J (2010) A semi-supervised Gaussian mixture model for image segmentation. In: International conference on pattern recognition, pp 2941–2944

    Google Scholar 

  101. Masseran N, Razali A, Ibrahim K, Latif M (2013) Fitting a mixture of von Mises-distributions in order to model data on wind direction in Peninsular Malaysia. Energy Convers Manag 72:94–102

    Google Scholar 

  102. Maugis C, Celeux G, Martin-Magniette ML (2009) Variable selection for clustering with Gaussian mixture models. Biometrics 65(3):701–709

    MATH  MathSciNet  Google Scholar 

  103. Maugis C, Celeux G, Martin-Magniette ML (2009) Variable selection in model-based clustering: a general variable role modeling. Comput Stat Data Anal 53(11):3872–3882

    MATH  MathSciNet  Google Scholar 

  104. Maugis-Rabusseau C, Martin-Magniette ML, Pelletier S (2012) Selvarclustmv: variable selection approach in model-based clustering allowing for missing values. J Soc Fr Stat 153(2):21–36

    MathSciNet  Google Scholar 

  105. McGrory C, Titterington D (2007) Variational approximations in Bayesian model selection for finite mixture distributions. Comput Stat Data Anal 51(11):5352–5367. doi:10.1016/j. csda.2006.07.020, http://www.sciencedirect.com/science/article/B6V8V-4KMYRPW-1/2/42 8635340ac2d823187a0c04164508c5. Advances in Mixture Models

  106. McLachlan G (1987) On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. Appl Stat 36:318–324

    Google Scholar 

  107. McLachlan GJ, Basford KE (1988) Mixture models: inference and applications to clustering. Marcel Dekker, New York

    MATH  Google Scholar 

  108. McLachlan G, Krishnan T (2008) The EM algorithm and extensions, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  109. McLachlan G, Peel D (2000) Finite mixture models. Wiley, New York

    MATH  Google Scholar 

  110. McLachlan G, Peel G, Basford K, Adams P (1999) Fitting of mixtures of normal and \(t\)-components. J Stat Softw 4:2

    Google Scholar 

  111. McLachlan G, Been R, Jones LT (2006) A simple implementation of a normal mixture approach to differential gene expression in multiclass microarrays. Bioinformatics 22:1608–1615

    Google Scholar 

  112. McNeil DR (1977) Interactive data analysis. Wiley, New York

    Google Scholar 

  113. Melnikov V, Litvinov V, Koppe V, Bobkov V (2008) Sims study of the processes in buffer solutions of bioorganic systems. Bull Russ Acad Sci Phys 72:929–933

    Google Scholar 

  114. Melnykov V (2012) Efficient estimation in model-based clustering of Gaussian regression time series. Stat Anal Data Min 5:95–99

    MathSciNet  Google Scholar 

  115. Melnykov V (2013) Challenges in model-based clustering. Wiley Interdiscip Rev Comput Stat 5:135–148

    Google Scholar 

  116. Melnykov V (2013) Finite mixture modelling in mass spectrometry analysis. J R Stat Soc Ser C 62:573–592

    MathSciNet  Google Scholar 

  117. Melnykov V (2013) On the distribution of posterior probabilities in finite mixture models with application in clustering. J Multivar Anal 122:175–189

    MATH  MathSciNet  Google Scholar 

  118. Melnykov V (2014) Merging mixture components for clustering through pairwise overlap. J Comput Graph Stat (tentatively accepted)

    Google Scholar 

  119. Melnykov V (2014) Model-based biclustering of clickstream data. Comput Stat Data Anal (under minor revision)

    Google Scholar 

  120. Melnykov V, Maitra R (2011) CARP: software for fishing out good clustering algorithms. J Mach Learn Res 12:69–73

    MATH  Google Scholar 

  121. Melnykov V, Melnykov I (2012) Initializing the EM algorithm in Gaussian mixture models with an unknown number of components. Comput Stat Data Anal 56:1381–1395

    MATH  MathSciNet  Google Scholar 

  122. Melnykov I, Melnykov V (2014) On k-means algorithm with the use of Mahalanobis distances. Stat Probab Lett 84:88–95

    MATH  MathSciNet  Google Scholar 

  123. Melnykov V, Michael S (2014) Finite mixture modeling of Gaussian regression time series with application to dendrochronology. J Classif (under review)

    Google Scholar 

  124. Melnykov V, Chen WC, Maitra R (2012) MixSim: an R package for simulating data to study performance of clustering algorithms. J Stat Softw 51:1–25

    Google Scholar 

  125. Meng XL, van Dyk D (1997) The EM algorithm - an old folk song sung to a fast new tune (with discussion). J R Stat Soc Ser B 59:511–567

    MATH  Google Scholar 

  126. Meng XL, Rubin DB (1993) Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80(2):267–278

    MATH  MathSciNet  Google Scholar 

  127. Michael S, Melnykov V (2014) Studying complexity of model-based clustering. Commun Stat Simul Comput (accepted)

    Google Scholar 

  128. Moore A (1998) Very fast EM-based mixture model clustering using multiresolution kd-trees. In: In advances in neural information processing systems 11. MIT Press, Cambridge, pp 543–549

    Google Scholar 

  129. Neal R (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9:249–265

    MathSciNet  Google Scholar 

  130. Neal RM, Hinton GE (1993) A new view of the EM algorithm that justifies incremental and other variants. In: Learning in graphical models. Kluwer, Dordrecht, pp 355–368

    Google Scholar 

  131. Newcomb S (1886) A generalized theory of the combination of observations so as to obtain the best result. Am J Math 8:343–366

    MATH  MathSciNet  Google Scholar 

  132. Neykov N, Filzmoser P, Dimova R, Neytchev P (2007) Robust fitting of mixtures using the trimmed likelihood estimator. Comput Stat Data Anal 17:299–308

    MathSciNet  Google Scholar 

  133. Nigam K, McCallum AK, Thrun S, Mitchell T (2000) Text classification from labeled and unlabeled documents using EM. Mach Learn 39:103–134

    MATH  Google Scholar 

  134. Ortega JM, Rheinboldt WC (1970) Iterative solutions of nonlinear equations in several variables. Academic, Princeton

    Google Scholar 

  135. Pan W, Shen X (2007) Penalized model-based clustering with application to variable selection. J Mach Learn Res 8:1145–1164

    MATH  Google Scholar 

  136. Pan W, Shen X, Jiang A, Hebbel R (2006) Semisupervised learning via penalized mixture model with application to microarray sample classification. Bioinformatics 22(19):2388–2395

    Google Scholar 

  137. Papastamoulis P, Iliopoulos G (2010) An artificial allocations based solution to the label switching problem in Bayesian analysis of mixtures of distributions. J Comput Graph Stat 19:313–331

    MathSciNet  Google Scholar 

  138. Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, Los Altos

    Google Scholar 

  139. Pearson K (1894) Contribution to the mathematical theory of evolution. Philos Trans R Soc 185:71–110

    MATH  Google Scholar 

  140. Peel D, McLachlan G (2000) Robust mixture modeling using the t distribution. Stat Comput 10:339–348

    Google Scholar 

  141. Peel D, Whiten W, McLachlan G (2001) Fitting mixtures of Kent distributions to aid in joint set identifications. J Am Stat Assoc 96:56–63

    MathSciNet  Google Scholar 

  142. Raftery AE, Dean N (2006) Variable selection for model-based clustering. J Am Stat Assoc 101:168–178

    MATH  MathSciNet  Google Scholar 

  143. Raftery AE, Niu X, Hoff PD, Yeung KY (2012) Fast inference for the latent space network model using a case-control approximate likelihood. J Comput Graph Stat 21(4):901–919

    MathSciNet  Google Scholar 

  144. Ray S, Cheng Y (2014) Package Modalclust: hierarchical modal clustering. http://cran.r-project.org/web/packages/Modalclust

  145. Ray S, Lindsay B (2005) The topography of multivariate normal mixtures. Ann Stat 33(5):2042–2065

    MATH  MathSciNet  Google Scholar 

  146. Richardson S, Green PJ (1997) On Bayesian analysis of mixtures with an unknown number of components (with discussion). J R Stat Soc Ser B 59:731–792

    MATH  MathSciNet  Google Scholar 

  147. Robin S, Bar-Hen A, Daudin JJ, Pierre L (2007) A semi-parametric approach for mixture models: application to local false discovery rate estimation. Comput Stat Data Anal 51:5483–5493

    MATH  MathSciNet  Google Scholar 

  148. Rodriguez CE, Walker SG (2014) Label switching in Bayesian mixture models: deterministic relabeling strategies. J Comput Graph Stat 23(1):25–45

    MathSciNet  Google Scholar 

  149. Saídaoui F (2010) Acceleration of the em algorithm via extrapolation methods: review, comparison and new methods. Comput Stat Data Anal 54(3):750–766

    Google Scholar 

  150. Salter-Townshend M, Murphy TB (2013) Variational Bayesian inference for the latent position cluster model for network data. Comput Stat Data Anal 57:661–671

    MathSciNet  Google Scholar 

  151. Sampson SF (1969) Crisis in a cloister. Ph.D. thesis, Department of Sociology, Cornell University, Ithaca

    Google Scholar 

  152. Schwarz G (1978) Estimating the dimensions of a model. Ann Stat 6:461–464

    MATH  Google Scholar 

  153. Seo B, Kim D (2012) Root selection in normal mixture models. Comput Stat Data Anal 56:2454–2470

    MATH  MathSciNet  Google Scholar 

  154. Shental N, Bar-Hillel A, Hertz T, Weinshall D (2003) Computing Gaussian mixture models with EM using equivalence constraints. In: Advances in NIPS, A Bradford Book, vol 15

    Google Scholar 

  155. Steiner P, Hudec M (2007) Classification of large data sets with mixture models via sufficient em. Comput Stat Data Anal 51:5416–5428

    MATH  MathSciNet  Google Scholar 

  156. Stuetzle W (2003) Estimating the cluster tree of a density by analyzing the minimal spanning tree of a sample. J Classif 20:25–47

    MATH  MathSciNet  Google Scholar 

  157. Stuetzle W, Nugent R (2010) A generalized single linkage method for estimating the cluster tree of a density. J Comput Graph Stat 19:397–418

    MathSciNet  Google Scholar 

  158. Tanabe A, Fukumizu K, Oba S, Takenouchi T, Ishii S (2007) Parameter estimation for von Mises-Fisher distributions. Comput Stat 22:145–157

    MATH  MathSciNet  Google Scholar 

  159. Teicher H (1963) Identifiability of finite mixtures. Ann Math Stat 34:1265–1269

    MATH  MathSciNet  Google Scholar 

  160. Vardi Y, Shepp LA, Kaufman LA (1985) A statistical model for positron emission tomography. J Am Stat Assoc 80:8–37

    MATH  MathSciNet  Google Scholar 

  161. Vicari D, Alfó M (2014) Model based clustering of customer choice data. Comput Stat Data Anal 71:3–13

    Google Scholar 

  162. Vu DQ, Hunter DR, Schweinberger M (2013) Model-based clustering of large networks. Ann Appl Stat 7:1010–1039

    MATH  MathSciNet  Google Scholar 

  163. Wagstaff K, Cardie C, Rogers S, Schroedl S (2001) Constrained K-means clustering with background knowledge. In: Proceedings of the eighteenth international conference on machine learning (ICML-2001), pp 577–584

    Google Scholar 

  164. Wang B, Titterington D (2006) Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model. Bayesian Anal 1(3):625–650

    MathSciNet  Google Scholar 

  165. Wang S, Zhu J (2008) Variable selection for model-based high-dimensional clustering and its application to microarray data. Biometrics 64:440–448

    MATH  MathSciNet  Google Scholar 

  166. Wang H, Zhang Q, Luo B, Wei S (2004) Robust mixture modelling using multivariate t-distribution with missing information. Pattern Recognit Lett 25:701–710

    Google Scholar 

  167. Wei GCG, Tanner MA (1990) A Monte Carlo implementation of the EM algorithm and the Poor Man’s data augmentation algorithms. J Am Stat Assoc 85(411):699–704

    Google Scholar 

  168. Wishart D (1969) Mode analysis: a generalization of nearest neighbor which reduces chaining effect. In: Cole AJ (ed) Numerical taxonomy. Academic, London, pp 282–311

    Google Scholar 

  169. Wolfe JH (1967) NORMIX: computational methods for estimating the parameters of multivariate normal mixture distributions. Technical bulletin USNPRA SRM 6

    Google Scholar 

  170. Xie B, Pan W, Shen X (2010) Penalized mixtures of factor analyzers with application to clustering high-dimensional microarray data. Bioinformatics 26:501–508

    Google Scholar 

  171. Xing EP, Ng AY, Jordan MI, Russell S (2003) Distance metric learning with application to clustering with side-information. In: Thrun S, Becker S, Obermayer K (eds) Advances in neural information processing systems, vol 15. MIT Press, Cambridge, pp 505–512

    Google Scholar 

  172. Yakowitz SJ, Spragins JD (1968) On the identifiability of finite mixtures. Ann Math Stat 39(1):209–214

    MATH  MathSciNet  Google Scholar 

  173. Ypma A, Heskes T (2002) Categorization of web pages and user clustering with mixtures of hidden Markov models. In: Proceedings of the international workshop on web knowledge discovery and data mining WEBKDD’02, Edmonton, pp 31–43

    Google Scholar 

  174. Yuan M, Lin Y (2007) Model selection and estimation in the Gaussian graphical model. Biometrika 94:19–35

    MATH  MathSciNet  Google Scholar 

  175. Zhou H, Pan W, X S (2009) Penalized model-based clustering with unconstrained covariance matrices. Electron J Stat 3:1473–1496

    Google Scholar 

  176. Zhu X (2005) Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Alabama, Tuscaloosa, AL, 35487, USA

    Volodymyr Melnykov & Semhar Michael

  2. Colorado State University–Pueblo, Pueblo, CO, 81001, USA

    Igor Melnykov

Authors
  1. Volodymyr Melnykov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Semhar Michael
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Igor Melnykov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Volodymyr Melnykov .

Editor information

Editors and Affiliations

  1. Computer Science dept., Louisiana State University Shreveport, Shreveport, Louisiana, USA

    M. Emre Celebi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Melnykov, V., Michael, S., Melnykov, I. (2015). Recent Developments in Model-Based Clustering with Applications. In: Celebi, M. (eds) Partitional Clustering Algorithms. Springer, Cham. https://doi.org/10.1007/978-3-319-09259-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-09259-1_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-09258-4

  • Online ISBN: 978-3-319-09259-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation