On the Use of the Sub-Gaussian \(\alpha \)-Stable Distribution in the Cluster-Weighted Model

  • Shaho Zarei
  • Adel Mohammadpour
  • Salvatore Ingrassia
  • Antonio Punzo
Research Paper
  • 7 Downloads

Abstract

The Gaussian cluster-weighted model (CWM) is a mixture of regression models with random covariates that allows for flexible clustering of a random vector composed of a response variable and some covariates. In each mixture component, a Gaussian distribution is adopted for both the covariates and the response given the covariates. To make the approach robust with respect to the presence of atypical observations, we propose to replace the Gaussian distribution with the sub-Gaussian \(\alpha \)-stable (SG\(\alpha \)S) distribution, an elliptical generalization of the Gaussian distribution having one additional parameter, \(\alpha \), governing the tails’ weight. The resulting SG\(\alpha \)S CWM is able to accommodate outliers and leverage points, concepts of primary importance in the robust regression analysis. Advantageously with respect to the t-distribution, the tails of the SG\(\alpha \)S distribution can be heavier, thus allowing robustness also with respect to gross atypical observations. A new algorithm, based on a combination of stochastic and conditional expectation maximizations, is used to obtain maximum likelihood estimates of the model parameters. Simulated and real data are used to illustrate and compare the proposal with CWMs based on Gaussian and t distributions.

Keywords

Cluster-weighted model Sub-Gaussian \(\alpha \)-stable Model-based clustering Mixture models Mixtures of regressions 

Mathematics Subject Classification

62H30 60E07 

Notes

Acknowledgements

The authors would like to thank the anonymous referees for their helpful comments and for careful reading that greatly improved the article.

References

  1. Aitkin M, Wilson GT (1980) Mixture models, outliers, and the EM algorithm. Technometrics 22(3):325–331CrossRefMATHGoogle Scholar
  2. Bagnato L, Punzo A (2013) Finite mixtures of unimodal beta and gamma densities and the \(k\)-bumps algorithm. Comput Stat 28(4):1571–1597MathSciNetCrossRefMATHGoogle Scholar
  3. Bagnato L, Punzo A, Zoia MG (2017) The multivariate leptokurtic-normal distribution and its application in model-based clustering. Can J Stat 45(1):95–119MathSciNetCrossRefGoogle Scholar
  4. Berta P, Ingrassia S, Punzo A, Vittadini G (2016) Multilevel cluster-weighted models for the evaluation of hospitals. Metron 74(3):275–292MathSciNetCrossRefMATHGoogle Scholar
  5. Biernacki C, Celeux G, Govaert G (2003) Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models. Comput Stat Data Anal 41(3):561–575MathSciNetCrossRefMATHGoogle Scholar
  6. Celeux G, Diebolt J (1985) The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Comput Stat 2(1):73–82Google Scholar
  7. Dang UJ, Punzo A, McNicholas PD, Ingrassia S, Browne RP (2017) Multivariate response and parsimony for Gaussian cluster-weighted models. J Classif 34(1):4–34MathSciNetCrossRefMATHGoogle Scholar
  8. Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Methodological) 39(1):1–38MathSciNetMATHGoogle Scholar
  9. DeSarbo WS, Cron WL (1988) A maximum likelihood methodology for clusterwise linear regression. J Classif 5(2):249–282MathSciNetCrossRefMATHGoogle Scholar
  10. Dunn JC (1974) Well-separated clusters and optimal fuzzy partitions. J Cybern 4(1):95–104MathSciNetCrossRefMATHGoogle Scholar
  11. Frühwirth-Schnatter S (2006) Finite mixture and Markov switching models. Springer, New YorkMATHGoogle Scholar
  12. Gershenfeld N (1997) Nonlinear inference and cluster-weighted modeling. Ann N Y Acad Sci 808(1):18–24CrossRefGoogle Scholar
  13. Gómez E, Gómez-Viilegas MA, Marin JM (1998) A multivariate generalization of the power exponential family of distributions. Commun Stat Theory Methods 27(3):589–600MathSciNetCrossRefMATHGoogle Scholar
  14. Harrison D, Rubinfeld DL (1978) Hedonic housing prices and the demand for clean air. J Environ Econ Manag 5(1):81–102CrossRefMATHGoogle Scholar
  15. Hennig C (2000) Identifiablity of models for clusterwise linear regression. J Classif 17(2):273–296MathSciNetCrossRefMATHGoogle Scholar
  16. Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218CrossRefMATHGoogle Scholar
  17. Ingrassia S, Minotti SC, Punzo A (2014) Model-based clustering via linear cluster-weighted models. Comput Stat Data Anal 71:159–182MathSciNetCrossRefGoogle Scholar
  18. Ingrassia S, Minotti SC, Vittadini G (2012) Local statistical modeling via the cluster-weighted approach with elliptical distributions. J Classif 29(3):363–401MathSciNetCrossRefMATHGoogle Scholar
  19. Ingrassia S, Punzo A (2016) Decision boundaries for mixtures of regressions. J Korean Stat Soc 45(2):295–306MathSciNetCrossRefMATHGoogle Scholar
  20. Ingrassia S, Punzo A, Vittadini G, Minotti SC (2015) The generalized linear mixed cluster-weighted model. J Classif 32(1):85–113MathSciNetCrossRefMATHGoogle Scholar
  21. Kring S, Rachev ST, Höchstötter M, Fabozzi FJ (2009) Estimation of \(\alpha \)-stable sub-Gaussian distributions for asset returns. In: Risk assessment: decisions in banking and finance. Springer/Physika, Heidelberg, pp 111–152Google Scholar
  22. Lange KL, Little RJA, Taylor JMG (1989) Robust statistical modeling using the \(t\)-distribution. J Am Stat Assoc 84(408):881–896MathSciNetGoogle Scholar
  23. Maruotti A, Punzo A (2017) Model-based time-varying clustering of multivariate longitudinal data with covariates and outliers. Comput Stat Data Anal 113:475–496MathSciNetCrossRefGoogle Scholar
  24. Mazza A, Punzo A (2018) Mixtures of multivariate contaminated normal regression models. Stat Pap.  https://doi.org/10.1007/s00362-017-0964-y
  25. Mazza A, Punzo A, Ingrassia S (2018). flexCWM: a flexible framework for cluster-weighted models. J Stat Softw 1–27Google Scholar
  26. Meng XL, Rubin DB (1993) Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80(2):267–278MathSciNetCrossRefMATHGoogle Scholar
  27. Nolan JP (1998) Parameterizations and modes of stable distributions. Stat Probab Lett 38(2):187–195MathSciNetCrossRefMATHGoogle Scholar
  28. Nolan JP (2013) Multivariate elliptically contoured stable distributions: theory and estimation. Comput Stat 28(5):2067–2089MathSciNetCrossRefMATHGoogle Scholar
  29. Nolan JP (2016) Stable distributions: models for heavy-tailed data. Birkhauser, Boston (Unfinished manuscript, Chapter 1 online at academic2.american.edujpnolan) Google Scholar
  30. Nolan JP, Ojeda-Revah D (2013) Linear and nonlinear regression with stable errors. J Econom 172(2):186–194MathSciNetCrossRefMATHGoogle Scholar
  31. Punzo A (2014) Flexible mixture modeling with the polynomial Gaussian cluster-weighted model. Stat Model 14(3):257–291MathSciNetCrossRefGoogle Scholar
  32. Punzo A, Bagnato L, Maruotti A (2018) Compound unimodal distributions for insurance losses. Insur Math Econ.  https://doi.org/10.1016/j.insmatheco.2017.10.007
  33. Punzo A, Browne RP, McNicholas PD (2016) Hypothesis testing for mixture model selection. J Stat Comput Simul 86(14):2797–2818MathSciNetCrossRefGoogle Scholar
  34. Punzo A, Ingrassia S (2013) On the use of the generalized linear exponential cluster-weighted model to asses local linear independence in bivariate data. QdS J Methodol Appl Stat 15:131–144Google Scholar
  35. Punzo A, Ingrassia S (2015) Parsimonious generalized linear Gaussian cluster-weighted models. In: Morlini I, Minerva T, Vichi M (eds) Advances in statistical models for data analysis, studies in classification, data analysis and knowledge organization. Springer International Publishing, Switzerland, pp 201–209Google Scholar
  36. Punzo A, Ingrassia S (2016) Clustering bivariate mixed-type data via the cluster-weighted model. Comput Stat 31(3):989–1013MathSciNetCrossRefMATHGoogle Scholar
  37. Punzo A, Maruotti A (2016) Clustering multivariate longitudinal observations: the contaminated Gaussian hidden Markov model. J Comput Graph Stat 25(4):1097–1116MathSciNetCrossRefGoogle Scholar
  38. Punzo A, Mazza A, McNicholas PD (2018) ContaminatedMixt: an \(\textsf{R}\) package for fitting parsimonious mixtures of multivariate contaminated normal distributions. J Stat Softw 1–25Google Scholar
  39. Punzo A, McNicholas PD (2016) Parsimonious mixtures of multivariate contaminated normal distributions. Biom J 58(6):1506–1537MathSciNetCrossRefMATHGoogle Scholar
  40. Punzo A, McNicholas PD (2017) Robust clustering in regression analysis via the contaminated Gaussian cluster-weighted model. J Classif 34(2):249–293MathSciNetCrossRefMATHGoogle Scholar
  41. Ritter G (2015) Robust cluster analysis and variable selection, Chapman & Hall/CRC Monographs on Statistics & Applied Probability, vol 137. CRC Press, Boca RatonGoogle Scholar
  42. Roche A (2011) EM algorithm and variants: an informal tutorial. arXiv:1105.1476
  43. Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65CrossRefMATHGoogle Scholar
  44. Samorodnitsky G, Taqqu MS (1994) Stable non-Gaussian random processes. Chapman and Hall, New YorkMATHGoogle Scholar
  45. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464MathSciNetCrossRefMATHGoogle Scholar
  46. Subedi S, Punzo A, Ingrassia S, McNicholas PD (2013) Clustering and classification via cluster-weighted factor analyzers. Adv Data Anal Classif 7(1):5–40MathSciNetCrossRefMATHGoogle Scholar
  47. Subedi S, Punzo A, Ingrassia S, McNicholas PD (2015) Cluster-weighted \(t\)-factor analyzers for robust model-based clustering and dimension reduction. Stat Methods Appl 24(4):623–649MathSciNetCrossRefMATHGoogle Scholar
  48. Teimouri M, Rezakhah S, Mohammdpour A (2017) Robust mixture modelling using sub-Gaussian stable distribution. arXiv:1701.06749
  49. Teimouri M, Rezakhah S, Mohammdpour A (2018) EM algorithm for symmetric stable mixture model. Commun Stat Simul Comput 47(2):582-604.  https://doi.org/10.1080/03610918.2017.1288244 MathSciNetCrossRefGoogle Scholar
  50. Tukey JW (1960) A survey of sampling from contaminated distributions. In: Olkin I (ed) Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling, Stanford Studies in Mathematics and Statistics, chapter 39. Stanford University Press, California, pp 448–485Google Scholar

Copyright information

© Shiraz University 2018

Authors and Affiliations

  1. 1.Department of Statistics, Faculty of Mathematics and Computer ScienceAmirkabir University of Technology (Tehran Polytechnic)TehranIran
  2. 2.Department of Economics and BusinessUniversity of CataniaCataniaItaly

Personalised recommendations