Finite mixtures of matrix normal distributions for classifying three-way data

Statistics and Computing volume 21, pages511–522(2011)Cite this article

Abstract

Matrix-variate distributions represent a natural way for modeling random matrices. Realizations from random matrices are generated by the simultaneous observation of variables in different situations or locations, and are commonly arranged in three-way data structures. Among the matrix-variate distributions, the matrix normal density plays the same pivotal role as the multivariate normal distribution in the family of multivariate distributions. In this work we define and explore finite mixtures of matrix normals. An EM algorithm for the model estimation is developed and some useful properties are demonstrated. We finally show that the proposed mixture model can be a powerful tool for classifying three-way data both in supervised and unsupervised problems. A simulation study and some real examples are presented.

This is a preview of subscription content, log in to check access.

References

  1. Banfield, J.D., Raftery, A.E.: Model-based Gaussian and non-Gaussian clustering. Biometrics 49, 803–821 (1993)

    MATH  Article  MathSciNet  Google Scholar 

  2. Basford, K.E., McLachlan, G.J.: The mixture method of clustering applied to three-way data. J. Classif. 2, 109–125 (1985)

    Article  Google Scholar 

  3. Biernacki, C., Celeux, G., Govaert, G.: Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans. PAMI 22, 719–725 (2000)

    Article  Google Scholar 

  4. Billard, L., Diday, E.: From the statistics of data to the statistics of knoweledge: symbolic data analysis. J. Am. Stat. Assoc. 98, 470–487 (2003)

    Article  MathSciNet  Google Scholar 

  5. Bouveyron, C., Girard, S., Schmid, C.: High-dimensional data clustering. Comput. Stat. Data Anal. 52, 502–519 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  6. Carroll, J.D., Arabie, P.: Multidimensional scaling. Ann. Rev. Psychol. 31, 607–649 (1980)

    Article  Google Scholar 

  7. Celeux, G., Govaert, G.: Gaussian parsimonious clustering models. Pattern Recogn. 28, 781–793 (1995)

    Article  Google Scholar 

  8. Chang, W.C.: On using principal components before separating a mixture of two multivariate normal distributions. Appl. Stat. 32, 267–275 (1983)

    MATH  Article  Google Scholar 

  9. Dawid, A.P.: Some matrix-variate distribution theory: notational considerations and a Bayesian application. Biometrika 68, 265–274 (1981)

    MATH  Article  MathSciNet  Google Scholar 

  10. De Wall, D.J.: Matrix-variate distributions. In: Knotz, S., Johnson, N.L. (eds.): Encyclopedia of Statistical Sciences, vol. 5, pp. 326–333. Wiley, New York (1988)

    Google Scholar 

  11. Dempster, N.M., Laird, A.P., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Stat. Soc. B 39, 1–38 (1977)

    MATH  MathSciNet  Google Scholar 

  12. Dutilleul, P.: The MLE algorithm for the matrix normal distribution. J. Stat. Comput. Simul. 64, 105–123 (1999)

    MATH  Article  Google Scholar 

  13. Fraley, C., Raftery, A.E.: MCLUST: Software for model-based cluster analysis. J. Classif. 16, 297–206 (1999)

    MATH  Article  Google Scholar 

  14. Fraley, C., Raftery, A.E.: Model-based clustering, discriminant analysis and density estimation. J. Am. Stat. Assoc. 97, 611–631 (2002a)

    MATH  Article  MathSciNet  Google Scholar 

  15. Fraley, C., Raftery, A.E.: MCLUST: Software for model-based clustering, discriminant analysis and density estimation, Technical Report No. 415, Department of Statistics, University of Washington (2002b)

  16. Fraley, C., Raftery, A.E.: Enhanced Software for model-based clustering, discriminant analysis and density estimation: MCLUST. J. Classif. 20, 263–286 (2003)

    MATH  Article  MathSciNet  Google Scholar 

  17. Ganesalingam, S., McLachlan, G.J.: A case study for two clustering methods based on maximum likelihood. Stat. Neerl. 33, 81–90 (1979)

    MATH  Article  MathSciNet  Google Scholar 

  18. Gordon, A.D., Vichi, M.: Partitions of partitions. J. Classif. 15, 265–285 (1998)

    MATH  Article  Google Scholar 

  19. Gupta, A.K., Nagar, D.K.: Matrix Variate Distributions. Chapman and Hall/CRC, London/Boca Raton (2000)

    Google Scholar 

  20. Hastie, T., Tibshirani, R.: Discriminant analysis by Gaussian mixtures. J. R. Stat. Soc. B 58, 155–176 (1996)

    MATH  MathSciNet  Google Scholar 

  21. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)

    Google Scholar 

  22. Hunt, L.A., Basford, K.E.: Fitting a Mixture Model to three-mode three-way data with categorical and continuous variables. J. Classif. 16, 283–296 (1999)

    MATH  Article  Google Scholar 

  23. Joe, H.: Generating random correlation matrices based on partial correlations. J. Multivar. Anal. 97, 2177–2189 (2006)

    MATH  Article  MathSciNet  Google Scholar 

  24. Jones, M.C., Sibson, R.: What is projection pursuit? (with discussion). J. R. Stat. Soc. A 150, 1–38 (1987)

    MATH  Article  MathSciNet  Google Scholar 

  25. McLachlan, G.J.: The classification and mixture maximum likelihood approaches to cluster analysis. In: Krishnaiah, P.R., Kanal, L.N. (eds.): Handbook of Statistics, vol. 2, pp. 199–208. North-Holland, Amsterdam (1982)

    Google Scholar 

  26. McLachlan, G.J.: Discriminant Analysis and Statistical Pattern Recognition. Wiley, New York (1992)

    Google Scholar 

  27. McLachlan, G.J., Basford, K.E.: Mixture Models: Inference and Application to Clustering. Dekker, New York (1988)

    Google Scholar 

  28. McLachlan, G.J., Peel, D.: Finite Mixture Models. Wiley, New York (2000)

    Google Scholar 

  29. McLachlan, G.J., Peel, D., Bean, R.W.: Modelling high-dimensional data by mixtures of factor analyzers. Comput. Stat. Data Anal. 41, 379–388 (2003)

    Article  MathSciNet  Google Scholar 

  30. Montanari, A., Viroli, C.: Heteroscedastic factor mixture analysis. Stat. Modell. Int. J. (2010, forthcoming)

  31. Mungomery, V.E., Shorter, R., Byth, D.E.: Genotype x environment interactions and environmental adaption. I. Pattern analysis—application to soya bean populations. Austr. J. Agric. Res. 25, 59–72 (1974)

    Article  Google Scholar 

  32. Nel, H.M.: On distributions and moments associated with matrix normal distributions. Mathematical Statistics Department Technical Report, 24, University of the Orange Free State, Bloemfontein, South Africa (1977)

  33. Rowe, B.R.: Multivariate Bayesian Statistics. Chapman and Hall/CRC, London/Boca Raton (2003)

    Google Scholar 

  34. Scott, D.W.: Multivariate Density Estimation. Wiley, New York (1992)

    Google Scholar 

  35. Vermunt, J.K.: Multilevel latent class models. Sociol. Method. 33, 213–239 (2003)

    Article  Google Scholar 

  36. Vermunt, J.K.: A hierarchical mixture model for clustering three-way data sets. Comput. Stat. Data Anal. 51, 5368–5376 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  37. Vichi, M.: One mode classification of a three-way data set. J. Classif. 16, 27–44 (1999)

    MATH  Article  Google Scholar 

  38. Vichi, M., Rocci, R., Kiers, A.L.: Simultaneous component and clustering models for three-way data: within and between approaches. J. Classif. 24, 71–98 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  39. Wolfe, J.H.: Pattern clustering by multivariate mixture analysis. Multivar. Behav. Res. 5, 329–350 (1970)

    Article  Google Scholar 

  40. Xie, X., Yan, S., Kwok, J.T., Huang, T.S.: Matrix-variate factor analysis and its applications. IEEE Trans. Neural Netw. 19, 1821–1826 (2008)

    Article  Google Scholar 

  41. Yung, Y.F.: Fitting mixtures in confirmatory factor-analysis models. Psychometrika 62, 297–330 (1997)

    MATH  Article  Google Scholar 

Download references

Author information

Affiliations

  1. Department of Statistics, University of Bologna, Bologna, Italy

    Cinzia Viroli

Authors
  1. Cinzia Viroli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cinzia Viroli.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Viroli, C. Finite mixtures of matrix normal distributions for classifying three-way data. Stat Comput 21, 511–522 (2011). https://doi.org/10.1007/s11222-010-9188-x

Download citation

Keywords

  • Model based clustering
  • Random matrix
  • Three-way data
  • EM-algorithm