Advertisement

Statistical Methods & Applications

, Volume 24, Issue 4, pp 623–649 | Cite as

Cluster-weighted \(t\)-factor analyzers for robust model-based clustering and dimension reduction

  • Sanjeena SubediEmail author
  • Antonio Punzo
  • Salvatore Ingrassia
  • Paul D. McNicholas
Article

Abstract

Cluster-weighted models represent a convenient approach for model-based clustering, especially when the covariates contribute to defining the cluster-structure of the data. However, applicability may be limited when the number of covariates is high and performance may be affected by noise and outliers. To overcome these problems, common/uncommon \(t\)-factor analyzers for the covariates, and a \(t\)-distribution for the response variable, are here assumed in each mixture component. A family of twenty parsimonious variants of this model is also presented and the alternating expectation-conditional maximization algorithm, for maximum likelihood estimation of the parameters of all models in the family, is described. Artificial and real data show that these models have very good clustering performance and that the algorithm is able to recover the parameters very well.

Keywords

Cluster-weighted models Factor analyzers Multivariate \(t\)-distributions Parsimonious models 

Supplementary material

10260_2015_298_MOESM1_ESM.pdf (195 kb)
Supplementary material 1 (pdf 195 KB)

References

  1. Airoldi J, Hoffmann R (1984) Age variation in voles (Microtus californicus, M. ochrogaster) and its significance for systematic studies. Occasional papers of the Museum of Natural History 111, University of Kansas, Lawrence, KSGoogle Scholar
  2. Aitken AC (1926) On Bernoulli’s numerical solution of algebraic equations. In: Proceedings of the Royal Society of Edinburgh, vol 46, pp 289–305Google Scholar
  3. Andrews JL, McNicholas PD (2011) Extending mixtures of multivariate \(t\)-factor analyzers. Stat Comput 21(3):361–373MathSciNetCrossRefGoogle Scholar
  4. Baek J, McLachlan GJ (2011) Mixtures of common \(t\)-factor analyzers for clustering high-dimensional microarray data. Bioinformatics 27(9):1269–1276CrossRefGoogle Scholar
  5. Baek J, McLachlan GJ, Flack LK (2010) Mixtures of factor analyzers with common factor loadings: applications to the clustering and visualization of high-dimensional data. IEEE Trans Pattern Anal Mach Intell 32(7):1298–1309CrossRefGoogle Scholar
  6. Biernacki C, Celeux G, Govaert G (2000) Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans Pattern Anal Mach Intell 22(7):719–725CrossRefGoogle Scholar
  7. Böhning D, Dietz E, Schaub R, Schlattmann P, Lindsay B (1994) The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family. Ann Inst Stat Math 46(2):373–388zbMATHCrossRefGoogle Scholar
  8. Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Methodol) 39(1):1–38zbMATHMathSciNetGoogle Scholar
  9. DeSarbo WS, Cron WL (1988) A maximum likelihood methodology for clusterwise linear regression. J Classif 5(2):249–282zbMATHMathSciNetCrossRefGoogle Scholar
  10. Ehrlich I (1973) Participation in illegitimate activities: a theoretical and empirical investigation. J Polit Econ 81(3):521–565CrossRefGoogle Scholar
  11. Flury B (1997) A first course in multivariate statistics. Springer, New YorkzbMATHCrossRefGoogle Scholar
  12. Gershenfeld N (1997) Nonlinear inference and cluster-weighted modeling. Ann NY Acad Sci 808(1):18–24CrossRefGoogle Scholar
  13. Grün B, Leisch F (2008) FlexMix version 2: finite mixtures with concomitant variables and varying and constant parameters. J Stat Softw 28(4):1–35CrossRefGoogle Scholar
  14. Hennig C (2000) Identifiablity of models for clusterwise linear regression. J Classif 17(2):273–296zbMATHMathSciNetCrossRefGoogle Scholar
  15. Ingrassia S, Minotti SC, Vittadini G (2012) Local statistical modeling via the cluster-weighted approach with elliptical distributions. J Classif 29(3):363–401MathSciNetCrossRefGoogle Scholar
  16. Ingrassia S, Minotti SC, Punzo A (2014) Model-based clustering via linear cluster-weighted models. Comput Stat Data Anal 71:159–182MathSciNetCrossRefGoogle Scholar
  17. Ingrassia S, Punzo A, Vittadini G (2015) The generalized linear mixed cluster-weighted model. J Classif 32 (in press)Google Scholar
  18. Leisch F (2004) FlexMix: a general framework for finite mixture models and latent class regression in \({\sf R}\). J Stat Softw 11(8):1–18CrossRefGoogle Scholar
  19. Lindsay BG (1995) Mixture models: theory, geometry and applications. In: NSF-CBMS regional conference series in probability and statistics, vol 5. Institute of Mathematical Statistics, HaywardGoogle Scholar
  20. Lo Y (2008) A likelihood ratio test of a homoscedastic normal mixture against a heteroscedastic normal mixture. Stat Comput 18(3):233–240MathSciNetCrossRefGoogle Scholar
  21. McLachlan GJ (1987) On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. J R Stat Soc Ser C (Appl Stat) 36(3):318–324MathSciNetGoogle Scholar
  22. McLachlan GJ, Peel D (2000) Finite mixture models. Wiley, New YorkzbMATHCrossRefGoogle Scholar
  23. McLachlan GJ, Bean RW, Ben-Tovim Jones L (2007) Extension of the mixture of factor analyzers model to incorporate the multivariate \(t\)-distribution. Comput Stat Data Anal 51(11):5327–5338zbMATHMathSciNetCrossRefGoogle Scholar
  24. Meng XL, van Dyk D (1997) The EM algorithm—an old folk-song sung to a fast new tune. J R Stat Soc Ser B (Stat Methodol) 59(3):511–567zbMATHCrossRefGoogle Scholar
  25. Punzo A (2014) Flexible mixture modeling with the polynomial Gaussian cluster-weighted model. Stat Model 14(3):257–291MathSciNetCrossRefGoogle Scholar
  26. Punzo A, Ingrassia S (2015) Parsimonious generalized linear Gaussian cluster-weighted models. In: Morlini I, Minerva T, Palumbo F (eds) Advances in statistical models for data analysis, studies in classification, data analysis and knowledge organization. Springer, SwitzerlandGoogle Scholar
  27. Punzo A, Browne RP, McNicholas PD (2014) Hypothesis testing for parsimonious Gaussian mixture models. http://arxiv.org/abs/1405.0377
  28. R Core Team (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, ViennaGoogle Scholar
  29. Sakamoto Y, Ishiguro M, Kitagawa G (1983) Akaike information criterion statistics. Reidel, BostonGoogle Scholar
  30. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464zbMATHCrossRefGoogle Scholar
  31. Subedi S, Punzo A, Ingrassia S, McNicholas PD (2013) Clustering and classification via cluster-weighted factor analyzers. Adv Data Anal Classif 7(1):5–40zbMATHMathSciNetCrossRefGoogle Scholar
  32. Vandaele W (1987) Participation in illegitimate activities: Ehrlich revisited. Report, U.S. Department of Justice, National Institute of JusticeGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Sanjeena Subedi
    • 1
    Email author
  • Antonio Punzo
    • 2
  • Salvatore Ingrassia
    • 2
  • Paul D. McNicholas
    • 3
  1. 1.Department of Mathematics and StatisticsUniversity of GuelphGuelphCanada
  2. 2.Department of Economics and BusinessUniversity of CataniaCataniaItaly
  3. 3.Department of Mathematics and StatisticsMcMaster UniversityHamiltonCanada

Personalised recommendations