Skip to main content
SpringerLink
Log in

Skew Multiple Scaled Mixtures of Normal Distributions with Flexible Tail Behavior and Their Application to Clustering

  • Special Issue: IFCS 2022
  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

The family of multiple scaled mixtures of multivariate normal (MSMN) distributions has been shown to be a powerful tool for modeling data that allow different marginal amounts of tail weight. An extension of the MSMN distribution is proposed through the incorporation of a vector of shape parameters, resulting in the skew multiple scaled mixtures of multivariate normal (SMSMN) distributions. The family of SMSMN distributions can express a variety of shapes by controlling different degrees of tailedness and versatile skewness in each dimension. Some characterizations and probabilistic properties of the SMSMN distributions are studied and an extension to finite mixtures thereof is also discussed. Based on a sort of selection mechanism, a feasible ECME algorithm is designed to compute the maximum likelihood estimates of model parameters. Numerical experiments on simulated data and three real data examples demonstrate the efficacy and usefulness of the proposed methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

Data and R codes for the implementation of our methodology are available at https://github.com/a-mahdavi/EM.mixSMSMN.git

References

  • Abanto-Valle, C. A., Bandyopadhyay, D., Lachos, V. H., & Enriquez, I. (2010). Robust Bayesian analysis of heavy-tailed stochastic volatility models using scale mixtures of normal distributions. Computational Statistics & Data Analysis, 54(12), 2883–2898.

    Article  MathSciNet  Google Scholar 

  • Andrews, D. F., & Mallows, C. L. (1974). Scale mixtures of normal distributions. Journal of the Royal Statistical Society: Series B (Methodological), 36(1), 99–102.

    Article  MathSciNet  Google Scholar 

  • Andrews, J. L., & McNicholas, P. D. (2012). Model-based clustering, classification, and discriminant analysis via mixtures of multivariate \(t\)-distributions: The \(t\) EIGEN family. Statistics and Computing, 22, 1021–1029.

    Article  MathSciNet  Google Scholar 

  • Arellano-Valle, R. B., & Azzalini, A. (2006). On the unification of families of skew-normal distributions. Scandinavian Journal of Statistics, 33(3), 561–574.

    Article  MathSciNet  Google Scholar 

  • Arellano-Valle, R. B., Branco, M. D., & Genton, M. G. (2006). A unified view on skewed distributions arising from selections. Canadian Journal of Statistics, 34(4), 581–601.

    Article  MathSciNet  Google Scholar 

  • Bagnato, L., Punzo, A., Bagnato, L., & Punzo, A. (2021). Unconstrained representation of orthogonal matrices with application to common principal components. Computational Statistics, 36(2), 1177–1195.

  • Banfield, J. D., & Raftery, A. E. (1993). Model-based Gaussian and non-Gaussian clustering. Biometrics, 803–821.

  • Barndorff-Nielsen, O. E. (1997). Normal inverse Gaussian distributions and stochastic volatility modelling. Scandinavian Journal of Statistics, 24(1), 1–13.

    Article  MathSciNet  Google Scholar 

  • Baudry, J. P., Raftery, A. E., Celeux, G., Lo, K., & Gottardo, R. (2010). Combining mixture components for clustering. Journal of Computational and Graphical Statistics, 19(2), 332–353.

    Article  MathSciNet  Google Scholar 

  • Bevilacqua, M., Caamaño-Carrillo, C., Arellano-Valle, R. B., & Morales-Oñate, V. (2021). Non-Gaussian geostatistical modeling using (skew) \(t\) processes. Scandinavian Journal of Statistics, 48(1), 212–245.

    Article  MathSciNet  Google Scholar 

  • Branco, M. D., & Dey, D. K. (2001). A general class of multivariate skew-elliptical distributions. Journal of Multivariate Analysis, 79(1), 99–113.

    Article  MathSciNet  Google Scholar 

  • Browne, R. P., & McNicholas, P. D. (2014). Orthogonal Stiefel manifold optimization for eigen-decomposed covariance parameter estimation in mixture models. Statistics and Computing, 24, 203–210.

    Article  MathSciNet  Google Scholar 

  • Browne, R. P., & McNicholas, P. D. (2014). Estimating common principal components in high dimensions. Advances in Data Analysis and Classification, 8, 217–226.

    Article  MathSciNet  Google Scholar 

  • Browne, R. P., McNicholas, P. D., & Sparling, M. D. (2011). Model-based learning using a mixture of mixtures of Gaussian and uniform distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 814–817.

    Article  Google Scholar 

  • Cabral, C. R. B., Lachos, V. H., & Prates, M. O. (2012). Multivariate mixture modeling using skew-normal independent distributions. Computational Statistics & Data Analysis, 56(1), 126–142.

    Article  MathSciNet  Google Scholar 

  • Celeux, G., & Govaert, G. (1995). Gaussian parsimonious clustering models. Pattern Recognition, 28(5), 781–793.

    Article  Google Scholar 

  • Charytanowicz, M., Niewczas, J., Kulczycki, P., Kowalski, P. A., Łukasik, S., & Żak, S. (2010). Complete gradient clustering algorithm for features analysis of x-ray images. In Information Technologies in Biomedicine: vol 2 (pp. 15–24). Springer Berlin Heidelberg.

  • Choy, S. T. B., & Smith, A. (1997). Hierarchical models with scale mixtures of normal distributions. Test, 6, 205–221.

    Article  Google Scholar 

  • Cuesta-Albertos, J. A., Gordaliza, A., & Matrán, C. (1997). Trimmed \(k\)-means: An attempt to robustify quantizers. The Annals of Statistics, 25(2), 553–576.

  • Dang, U. J., Punzo, A., McNicholas, P. D., Ingrassia, S., & Browne, R. P. (2017). Multivariate response and parsimony for Gaussian cluster-weighted models. Journal of Classification, 34, 4–34.

    Article  MathSciNet  Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1), 1–22.

    Article  MathSciNet  Google Scholar 

  • Ferreira, C. S., Bolfarine, H., & Lachos, V. H. (2020). Linear mixed models based on skew scale mixtures of normal distributions. Communications in Statistics-Simulation and Computation, 51(12), 7194–7214.

    Article  MathSciNet  Google Scholar 

  • Ferreira, C. S., Lachos, V. H., & Bolfarine, H. (2016). Likelihood-based inference for multivariate skew scale mixtures of normal distributions. AStA Advances in Statistical Analysis, 100, 421–441.

    Article  MathSciNet  Google Scholar 

  • Flury, B. N., & Gautschi, W. (1986). An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form. SIAM Journal on Scientific and Statistical Computing, 7(1), 169–184.

    Article  MathSciNet  Google Scholar 

  • Forbes, F., & Wraith, D. (2014). A new family of multivariate heavy-tailed distributions with variable marginal amounts of tailweight: Application to robust clustering. Statistics and Computing, 24(6), 971–984.

    Article  MathSciNet  Google Scholar 

  • Franczak, B. C., Tortora, C., Browne, R. P., & McNicholas, P. D. (2015). Unsupervised learning via mixtures of skewed distributions with hypercube contours. Pattern Recognition Letters, 58, 69–76.

    Article  Google Scholar 

  • Gallaugher, M. P., Tomarchio, S. D., McNicholas, P. D., & Punzo, A. (2022). Multivariate cluster weighted models using skewed distributions. Advances in Data Analysis and Classification, 1–32.

  • Garay, A. M., Lachos, V. H., Bolfarine, H., & Cabral, C. R. (2017). Linear censored regression models with scale mixtures of normal distributions. Statistical Papers, 58, 247–278.

    Article  MathSciNet  Google Scholar 

  • Hennig, C. (2015). What are the true clusters? Pattern Recognition Letters, 64, 53–62.

    Article  Google Scholar 

  • Hubert, L., & Arabie, P. (1985). Comparing partitions. Journal of Classification, 2, 193–218.

    Article  Google Scholar 

  • Kaufman, L., & Rousseeuw, P. J. (2008). Finding groups in data: An introduction to cluster analysis. John Wiley & Sons.

    Google Scholar 

  • Lee Lee, S. X., & McLachlan, G. J. (2013). Model-based clustering and classification with non-normal mixture distributions. Statistical Methods & Applications, 22(4), 427–454.

    Article  MathSciNet  Google Scholar 

  • Lee, S. X., & McLachlan, G. J. (2019). Scale mixture distribution (pp. 1–16). Wiley StatsRef: Statistics Reference Online.

  • Lichman, M. (2013). UCI machine learning repository Irvine, CA: University of California, School of Information and Computer Science. http://archive.ics.uci.edu/ml

  • Lin, T. I. (2009). Maximum likelihood estimation for multivariate skew normal mixture models. Journal of Multivariate Analysis, 100(2), 257–265.

    Article  MathSciNet  Google Scholar 

  • Lin, T. I. (2010). Robust mixture modeling using multivariate skew \(t\) distributions. Statistics and Computing, 20, 343–356.

    Article  MathSciNet  Google Scholar 

  • Lin, T. I. (2014). Learning from incomplete data via parameterized \(t\) mixture models through eigenvalue decomposition. Computational Statistics & Data Analysis, 71, 183–195.

    Article  MathSciNet  Google Scholar 

  • Lin, T. I., Lee, J. C., & Hsieh, W. J. (2007a). Robust mixture modeling using the skew \(t\) distribution. Statistics and Computing, 17, 81–92.

  • Lin, T. I., Lee, J. C., & Yen, S. Y. (2007b). Finite mixture modelling using the skew normal distribution. Statistica Sinica, 909–927.

  • Lin, T. I., & Wang, W. L. (2022). Multivariate linear mixed models with censored and nonignorable missing outcomes, with application to AIDS studies. Biometrical Journal, 64(7), 1325–1339.

    Article  MathSciNet  Google Scholar 

  • Lin, T. I., Wu, P. H., McLachlan, G. J., & Lee, S. X. (2015). A robust factor analysis model using the restricted skew-\(t\) distribution. Test, 24(3), 510–531.

    Article  MathSciNet  Google Scholar 

  • Liu, C., & Rubin, D. B. (1994). The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence. Biometrika, 81(4), 633–648.

    Article  MathSciNet  Google Scholar 

  • Liu, M., & Lin, T. I. (2015). Skew-normal factor analysis models with incomplete data. Journal of Applied Statistics, 42(4), 789–805.

    Article  MathSciNet  Google Scholar 

  • Mahdavi, A., Amirzadeh, V., Jamalizadeh, A., & Lin, T. I. (2021). Maximum likelihood estimation for scale-shape mixtures of flexible generalized skew normal distributions via selection representation. Computational Statistics, 36, 2201–2230.

    Article  MathSciNet  Google Scholar 

  • Maier, L. M., Anderson, D. E., De Jager, P. L., Wicker, L. S., & Hafler, D. A. (2007). Allelic variant in CTLA4 alters T cell phosphorylation patterns. Proceedings of the National Academy of Sciences, 104(47), 18607–18612.

    Article  Google Scholar 

  • Mazza, A., & Punzo, A. (2020). Mixtures of multivariate contaminated normal regression models. Statistical Papers, 61(2), 787–822.

    Article  MathSciNet  Google Scholar 

  • Meng, X. L., & Rubin, D. B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika, 80(2), 267–278.

    Article  MathSciNet  Google Scholar 

  • Pelleg, D. (2000). Extending \(K\)-means with efficient estimation of the number of clusters in ICML. In Proceedings of the 17th international conference on machine learning (pp. 277–281).

  • Prates, M. O., Lachos, V. H., & Cabral, C. R. B. (2013). mixsmsn: Fitting finite mixture of scale mixture of skew-normal distributions. Journal of Statistical Software, 54, 1–20.

    Article  Google Scholar 

  • Punzo, A., & Bagnato, L. (2020). Allometric analysis using the multivariate shifted exponential normal distribution. Biometrical Journal, 62(6), 1525–1543.

    Article  MathSciNet  Google Scholar 

  • Punzo, A., & Bagnato, L. (2021). The multivariate tail-inflated normal distribution and its application in finance. Journal of Statistical Computation and Simulation, 91(1), 1–36.

    Article  MathSciNet  Google Scholar 

  • Punzo, A., & Bagnato, L. (2022). Dimension-wise scaled normal mixtures with application to finance and biometry. Journal of Multivariate Analysis, 191, 105020.

    Article  MathSciNet  Google Scholar 

  • Punzo, A., & Bagnato, L. (2022). Multiple scaled symmetric distributions in allometric studies. The International Journal of Biostatistics, 18(1), 219–242.

    Article  Google Scholar 

  • Punzo, A., & McNicholas, P. D. (2016). Parsimonious mixtures of multivariate contaminated normal distributions. Biometrical Journal, 58(6), 1506–1537.

    Article  MathSciNet  Google Scholar 

  • Punzo, A., & McNicholas, P. D. (2017). Robust clustering in regression analysis via the contaminated Gaussian cluster-weighted model. Journal of Classification, 34, 249–293.

    Article  MathSciNet  Google Scholar 

  • Punzo, A., & Tortora, C. (2021). Multiple scaled contaminated normal distribution and its application in clustering. Statistical Modelling, 21(4), 332–358.

    Article  MathSciNet  Google Scholar 

  • R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria

  • Schumacher, F. L., Lachos, V. H., & Matos, L. A. (2021). Scale mixture of skew-normal linear mixed models with within-subject serial dependence. Statistics in Medicine, 40(7), 1790–1810.

    Article  MathSciNet  Google Scholar 

  • Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 461–464.

  • Tomarchio, S. D., Bagnato, L., & Punzo, A. (2022). Model-based clustering via new parsimonious mixtures of heavy-tailed distributions. AStA Advances in Statistical Analysis, 1–33.

  • Tortora, C., ElSherbiny, A., Browne, R. P., Franczak, B. C., McNicholas, P. D., & Amos, D. D. (2020). MixGHD: Model-based clustering, classification and discriminant analysis using the mixture of generalized hyperbolic distributions. R package version 2.3.4.

  • Tortora, C., Franczak, B. C., Browne, R. P., & McNicholas, P. D. (2019). A mixture of coalesced generalized hyperbolic distributions. Journal of Classification, 36, 26–57.

    Article  MathSciNet  Google Scholar 

  • Wang, W. L., Jamalizadeh, A., & Lin, T. I. (2020). Finite mixtures of multivariate scale-shape mixtures of skew-normal distributions. Statistical Papers, 61, 2643–2670.

    Article  MathSciNet  Google Scholar 

  • Wang, W. L., & Lin, T. I. (2022). Robust clustering via mixtures of \(t\) factor analyzers with incomplete data. Advances in Data Analysis and Classification, 16(3), 659–690.

    Article  MathSciNet  Google Scholar 

  • Wang, W. L., Liu, M., & Lin, T. I. (2017). Robust skew-\(t\) factor analysis models for handling missing data. Statistical Methods & Applications, 26, 649–672.

    Article  MathSciNet  Google Scholar 

  • Wraith, D., & Forbes, F. (2015). Location and scale mixtures of Gaussians with flexible tail behaviour: Properties, inference and application to multivariate clustering. Computational Statistics & Data Analysis, 90, 61–73.

    Article  MathSciNet  Google Scholar 

  • Young, D. S., & Hunter, D. R. (2010). Mixtures of regressions with predictor-dependent mixing proportions. Computational Statistics & Data Analysis, 54(10), 2253–2266.

    Article  MathSciNet  Google Scholar 

  • Zareifard, H., & Khaledi, M. J. (2013). Non-Gaussian modeling of spatial data using scale mixing of a unified skew Gaussian process. Journal of Multivariate Analysis, 114, 16–28.

    Article  MathSciNet  Google Scholar 

  • Zeller, C. B., Cabral, C. R., & Lachos, V. H. (2016). Robust mixture regression modeling based on scale mixtures of skew-normal distributions. Test, 25, 375–396.

    Article  MathSciNet  Google Scholar 

  • Zeller, C. B., Cabral, C. R. B., Lachos, V. H., & Benites, L. (2019). Finite mixture of regression models for censored data based on scale mixtures of normal distributions. Advances in Data Analysis and Classification, 13, 89–116.

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the Editor-in-Chief, the Guest Editor, and three anonymous referees for their valuable comments and constructive suggestions which greatly improved the content of this paper.

Funding

T.I. LIN is grateful for the financial support from the National Science and Technology Council of Taiwan under Grant no. NSTC 112-2118-M-005-004-MY3. A.F. Desmond acknowledges the support of NSERC Canada under Discovery Grant no. 401273.

Author information

Authors and Affiliations

  1. Department of Statistics, Vali-e-Asr University of Rafsanjan, Rafsanjan, Iran

    Abbas Mahdavi

  2. Department of Mathematics and Statistics, University of Guelph, Guelph, Ontario, N1G 2W1, Canada

    Anthony F. Desmond

  3. Department of Statistics, Faculty of Mathematics & Computer, Shahid Bahonar University of Kerman, Kerman, Iran

    Ahad Jamalizadeh

  4. Institute of Statistics, National Chung Hsing University, Taichung, 402, Taiwan

    Tsung-I Lin

  5. Department of Public Health, China Medical University, Taichung, 404, Taiwan

    Tsung-I Lin

Authors
  1. Abbas Mahdavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anthony F. Desmond
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahad Jamalizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tsung-I Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tsung-I Lin.

Ethics declarations

Ethical Approval

This research does not involve human or animal participants.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 92 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahdavi, A., Desmond, A.F., Jamalizadeh, A. et al. Skew Multiple Scaled Mixtures of Normal Distributions with Flexible Tail Behavior and Their Application to Clustering. J Classif (2024). https://doi.org/10.1007/s00357-024-09470-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00357-024-09470-6

Keywords

Search

Navigation