Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

On the Use of the Sub-Gaussian \(\alpha \)-Stable Distribution in the Cluster-Weighted Model

  • 36 Accesses

  • 2 Citations

Abstract

The Gaussian cluster-weighted model (CWM) is a mixture of regression models with random covariates that allows for flexible clustering of a random vector composed of a response variable and some covariates. In each mixture component, a Gaussian distribution is adopted for both the covariates and the response given the covariates. To make the approach robust with respect to the presence of atypical observations, we propose to replace the Gaussian distribution with the sub-Gaussian \(\alpha \)-stable (SG\(\alpha \)S) distribution, an elliptical generalization of the Gaussian distribution having one additional parameter, \(\alpha \), governing the tails’ weight. The resulting SG\(\alpha \)S CWM is able to accommodate outliers and leverage points, concepts of primary importance in the robust regression analysis. Advantageously with respect to the t-distribution, the tails of the SG\(\alpha \)S distribution can be heavier, thus allowing robustness also with respect to gross atypical observations. A new algorithm, based on a combination of stochastic and conditional expectation maximizations, is used to obtain maximum likelihood estimates of the model parameters. Simulated and real data are used to illustrate and compare the proposal with CWMs based on Gaussian and t distributions.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    Available at http://www.robustanalysis.com.

References

  1. Aitkin M, Wilson GT (1980) Mixture models, outliers, and the EM algorithm. Technometrics 22(3):325–331

  2. Bagnato L, Punzo A (2013) Finite mixtures of unimodal beta and gamma densities and the \(k\)-bumps algorithm. Comput Stat 28(4):1571–1597

  3. Bagnato L, Punzo A, Zoia MG (2017) The multivariate leptokurtic-normal distribution and its application in model-based clustering. Can J Stat 45(1):95–119

  4. Berta P, Ingrassia S, Punzo A, Vittadini G (2016) Multilevel cluster-weighted models for the evaluation of hospitals. Metron 74(3):275–292

  5. Biernacki C, Celeux G, Govaert G (2003) Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models. Comput Stat Data Anal 41(3):561–575

  6. Celeux G, Diebolt J (1985) The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Comput Stat 2(1):73–82

  7. Dang UJ, Punzo A, McNicholas PD, Ingrassia S, Browne RP (2017) Multivariate response and parsimony for Gaussian cluster-weighted models. J Classif 34(1):4–34

  8. Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Methodological) 39(1):1–38

  9. DeSarbo WS, Cron WL (1988) A maximum likelihood methodology for clusterwise linear regression. J Classif 5(2):249–282

  10. Dunn JC (1974) Well-separated clusters and optimal fuzzy partitions. J Cybern 4(1):95–104

  11. Frühwirth-Schnatter S (2006) Finite mixture and Markov switching models. Springer, New York

  12. Gershenfeld N (1997) Nonlinear inference and cluster-weighted modeling. Ann N Y Acad Sci 808(1):18–24

  13. Gómez E, Gómez-Viilegas MA, Marin JM (1998) A multivariate generalization of the power exponential family of distributions. Commun Stat Theory Methods 27(3):589–600

  14. Harrison D, Rubinfeld DL (1978) Hedonic housing prices and the demand for clean air. J Environ Econ Manag 5(1):81–102

  15. Hennig C (2000) Identifiablity of models for clusterwise linear regression. J Classif 17(2):273–296

  16. Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218

  17. Ingrassia S, Minotti SC, Punzo A (2014) Model-based clustering via linear cluster-weighted models. Comput Stat Data Anal 71:159–182

  18. Ingrassia S, Minotti SC, Vittadini G (2012) Local statistical modeling via the cluster-weighted approach with elliptical distributions. J Classif 29(3):363–401

  19. Ingrassia S, Punzo A (2016) Decision boundaries for mixtures of regressions. J Korean Stat Soc 45(2):295–306

  20. Ingrassia S, Punzo A, Vittadini G, Minotti SC (2015) The generalized linear mixed cluster-weighted model. J Classif 32(1):85–113

  21. Kring S, Rachev ST, Höchstötter M, Fabozzi FJ (2009) Estimation of \(\alpha \)-stable sub-Gaussian distributions for asset returns. In: Risk assessment: decisions in banking and finance. Springer/Physika, Heidelberg, pp 111–152

  22. Lange KL, Little RJA, Taylor JMG (1989) Robust statistical modeling using the \(t\)-distribution. J Am Stat Assoc 84(408):881–896

  23. Maruotti A, Punzo A (2017) Model-based time-varying clustering of multivariate longitudinal data with covariates and outliers. Comput Stat Data Anal 113:475–496

  24. Mazza A, Punzo A (2018) Mixtures of multivariate contaminated normal regression models. Stat Pap. https://doi.org/10.1007/s00362-017-0964-y

  25. Mazza A, Punzo A, Ingrassia S (2018). flexCWM: a flexible framework for cluster-weighted models. J Stat Softw 1–27

  26. Meng XL, Rubin DB (1993) Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80(2):267–278

  27. Nolan JP (1998) Parameterizations and modes of stable distributions. Stat Probab Lett 38(2):187–195

  28. Nolan JP (2013) Multivariate elliptically contoured stable distributions: theory and estimation. Comput Stat 28(5):2067–2089

  29. Nolan JP (2016) Stable distributions: models for heavy-tailed data. Birkhauser, Boston (Unfinished manuscript, Chapter 1 online at academic2.american.edujpnolan)

  30. Nolan JP, Ojeda-Revah D (2013) Linear and nonlinear regression with stable errors. J Econom 172(2):186–194

  31. Punzo A (2014) Flexible mixture modeling with the polynomial Gaussian cluster-weighted model. Stat Model 14(3):257–291

  32. Punzo A, Bagnato L, Maruotti A (2018) Compound unimodal distributions for insurance losses. Insur Math Econ. https://doi.org/10.1016/j.insmatheco.2017.10.007

  33. Punzo A, Browne RP, McNicholas PD (2016) Hypothesis testing for mixture model selection. J Stat Comput Simul 86(14):2797–2818

  34. Punzo A, Ingrassia S (2013) On the use of the generalized linear exponential cluster-weighted model to asses local linear independence in bivariate data. QdS J Methodol Appl Stat 15:131–144

  35. Punzo A, Ingrassia S (2015) Parsimonious generalized linear Gaussian cluster-weighted models. In: Morlini I, Minerva T, Vichi M (eds) Advances in statistical models for data analysis, studies in classification, data analysis and knowledge organization. Springer International Publishing, Switzerland, pp 201–209

  36. Punzo A, Ingrassia S (2016) Clustering bivariate mixed-type data via the cluster-weighted model. Comput Stat 31(3):989–1013

  37. Punzo A, Maruotti A (2016) Clustering multivariate longitudinal observations: the contaminated Gaussian hidden Markov model. J Comput Graph Stat 25(4):1097–1116

  38. Punzo A, Mazza A, McNicholas PD (2018) ContaminatedMixt: an \(\textsf{R}\) package for fitting parsimonious mixtures of multivariate contaminated normal distributions. J Stat Softw 1–25

  39. Punzo A, McNicholas PD (2016) Parsimonious mixtures of multivariate contaminated normal distributions. Biom J 58(6):1506–1537

  40. Punzo A, McNicholas PD (2017) Robust clustering in regression analysis via the contaminated Gaussian cluster-weighted model. J Classif 34(2):249–293

  41. Ritter G (2015) Robust cluster analysis and variable selection, Chapman & Hall/CRC Monographs on Statistics & Applied Probability, vol 137. CRC Press, Boca Raton

  42. Roche A (2011) EM algorithm and variants: an informal tutorial. arXiv:1105.1476

  43. Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65

  44. Samorodnitsky G, Taqqu MS (1994) Stable non-Gaussian random processes. Chapman and Hall, New York

  45. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464

  46. Subedi S, Punzo A, Ingrassia S, McNicholas PD (2013) Clustering and classification via cluster-weighted factor analyzers. Adv Data Anal Classif 7(1):5–40

  47. Subedi S, Punzo A, Ingrassia S, McNicholas PD (2015) Cluster-weighted \(t\)-factor analyzers for robust model-based clustering and dimension reduction. Stat Methods Appl 24(4):623–649

  48. Teimouri M, Rezakhah S, Mohammdpour A (2017) Robust mixture modelling using sub-Gaussian stable distribution. arXiv:1701.06749

  49. Teimouri M, Rezakhah S, Mohammdpour A (2018) EM algorithm for symmetric stable mixture model. Commun Stat Simul Comput 47(2):582-604. https://doi.org/10.1080/03610918.2017.1288244

  50. Tukey JW (1960) A survey of sampling from contaminated distributions. In: Olkin I (ed) Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling, Stanford Studies in Mathematics and Statistics, chapter 39. Stanford University Press, California, pp 448–485

Download references

Acknowledgements

The authors would like to thank the anonymous referees for their helpful comments and for careful reading that greatly improved the article.

Author information

Correspondence to Adel Mohammadpour.

Appendix

Appendix

Suppose \(\varvec{W} \sim S_{d}(\alpha ,\varvec{\Sigma },\varvec{\mu })\). Therefore, \(\varvec{W}\mathop {=}\limits ^d\varvec{\mu }+\sqrt{P}\varvec{Z},\) where \(\varvec{Z}\) is a d-variate zero-mean Gaussian random vector with variance–covariance matrix \(\varvec{\Sigma }\) and \(P\sim S(\frac{\alpha }{2},1,(\cos (\frac{\pi \alpha }{4}))^{\frac{2}{\alpha }},0)\) is a positive stable random variable. To compute \(E_{1}=E\Big (P^{-1}|\varvec{w},\alpha ,\varvec{\Sigma },\varvec{\mu }\Big )\), we should compute

$$\begin{aligned} f(p|\varvec{w})=\frac{f(\varvec{w},p)}{f(\varvec{w})}=\frac{f(p)f(\varvec{w}|p)}{\int _{0}^{\infty }f(p)f(\varvec{w}|p)\mathrm{d}p}. \end{aligned}$$

Since \(\varvec{W}|P=p\sim N(\varvec{\mu },p\varvec{\Sigma })\), we have

$$\begin{aligned} E_{1}=\frac{\displaystyle \int _{0}^{\infty }p^{-d/2-1}f_{P}(p|\alpha )\exp \left\{ \frac{-(\varvec{w}-\varvec{\mu })^{'}\varvec{\Sigma }^{-1}(\varvec{w}-\varvec{\mu })}{2p}\right\} {\mathrm{d}p}}{\displaystyle \int _{0}^{\infty }p^{-d/2}f_{P}(p|\alpha )\exp \left\{ \frac{-(\varvec{w}-\varvec{\mu })^{'}\varvec{\Sigma }^{-1}(\varvec{w}-\varvec{\mu })}{2p}\right\} {\mathrm{d}p}}, \end{aligned}$$

where \(f_{P}(.)\) is the density function of P. For approximating \( {E}_{1}\), we use a Monte Carlo method by generating M samples from P and calculating elements of the under integral. If \(p_{1},\ldots ,p_{M}\) is a random sample from P, then the approximate value of \({E}_{1}\) is

$$\begin{aligned} \frac{\sum _{i=1}^{M}p_{i}^{-d/2-1}f_{P}(p_{i}|\alpha )\exp \left\{ \frac{-(\varvec{w}-\varvec{\mu })^{'}\varvec{\Sigma }^{-1}(\varvec{w}-\varvec{\mu })}{2p_{i}}\right\} }{\sum _{i=1}^{M} {p_{i}}^{-d/2}f_{P}(p_{i}|\alpha )\exp \left\{ \frac{-(\varvec{w}-\varvec{\mu })^{'}\varvec{\Sigma }^{-1}(\varvec{w}-\varvec{\mu })}{2p_{i}}\right\} }. \end{aligned}$$

We take \(M=2000\) and update both \(e_{2ig}^{(t)}\) and \(e_{3ig}^{(t)}\), \(i=1,\ldots ,N\) and \(g=1,\ldots ,G\).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zarei, S., Mohammadpour, A., Ingrassia, S. et al. On the Use of the Sub-Gaussian \(\alpha \)-Stable Distribution in the Cluster-Weighted Model. Iran J Sci Technol Trans Sci 43, 1059–1069 (2019). https://doi.org/10.1007/s40995-018-0526-8

Download citation

Keywords

  • Cluster-weighted model
  • Sub-Gaussian \(\alpha \)-stable
  • Model-based clustering
  • Mixture models
  • Mixtures of regressions

Mathematics Subject Classification

  • 62H30
  • 60E07