Abstract
Within the mixture model-based clustering literature, parsimonious models with eigen-decomposed component covariance matrices have dominated for over a decade. Although originally introduced as a fourteen-member family of models, the current state-of-the-art is to utilize just ten of these models; the rationale for not using the other four models usually centers around parameter estimation difficulties. Following close examination of these four models, we find that two are actually easily implemented using existing algorithms but that two benefit from a novel approach. We present and implement algorithms that use an accelerated line search for optimization on the orthogonal Stiefel manifold. Furthermore, we show that the ‘extra’ models that these decompositions facilitate outperform the current state-of-the art when applied to two benchmark data sets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)
Anderson, E.: The irises of the Gaspé Peninsula. Bull. Am. Iris Soc. 59, 2–5 (1935)
Andrews, J.L., McNicholas, P.D.: Model-based clustering, classification, and discriminant analysis via mixtures of multivariate t-distributions. Stat. Comput. 22(5), 1021–1029 (2012)
Banfield, J.D., Raftery, A.E.: Model-based Gaussian and non-Gaussian clustering. Biometrics 49(3), 803–821 (1993)
Biernacki, C., Celeux, G., Govaert, G., Langrognet, F.: Model-based cluster analysis and discriminant analysis with the MIXMOD software. Comput. Stat. Data Anal. 51, 587–600 (2006)
Celeux, G., Govaert, G.: Gaussian parsimonious clustering models. Pattern Recognit. 28(5), 781–793 (1995)
Dasgupta, A., Raftery, A.E.: Detecting features in spatial point processes with clutter via model-based clustering. J. Am. Stat. Assoc. 93, 294–302 (1998)
Dean, N., Murphy, T.B., Downey, G.: Using unlabelled data to update classification rules with applications in food authenticity studies. J. R. Stat. Soc., Ser. C 55(1), 1–14 (2006)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc., Ser. B 39(1), 1–38 (1977)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annu. Eugen. 7(2), 179–188 (1936)
Flury, B.W., Gautschi, W.: An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form. SIAM J. Sci. Stat. Comput. 7(1), 169–184 (1986)
Forina, M., Armanino, C., Castino, M., Ubigli, M.: Multivariate data analysis as a discriminating method of the origin of wines. Vitis 25, 189–201 (1986)
Fraley, C., Raftery, A.E.: How many clusters? Which clustering methods? Answers via model-based cluster analysis. Comput. J. 41(8), 578–588 (1998)
Fraley, C., Raftery, A.E.: MCLUST: Software for model-based cluster analysis. J. Classif. 16, 297–306 (1999)
Fraley, C., Raftery, A.E.: Model-based clustering, discriminant analysis, and density estimation. J. Am. Stat. Assoc. 97(458), 611–631 (2002)
Hastie, T., Tibshirani, R.: Discriminant analysis by Gaussian mixtures. J. R. Stat. Soc., Ser. B 58(1), 155–176 (1996)
Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1985)
Hurley, C.: Clustering visualizations of multivariate data. J. Comput. Graph. Stat. 13(4), 788–806 (2004)
Keribin, C.: Estimation consistante de l’ordre de modèles de mélange. C. R. Acad. Sci., Sér. 1 Math. 326(2), 243–248 (1998)
Keribin, C.: Consistent estimation of the order of mixture models. Sankhya 62(1), 49–66 (2000)
Lebret, R., Iovleff, S., Langrognet, F.: Rmixmod: MIXture MODelling Package. R package version 1.1.1 (2012)
Murtagh, F., Raftery, A.E.: Fitting straight lines to point patterns. Pattern Recognit. 17(5), 479–483 (1984)
R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2011)
Raftery, A.E., Dean, N.: Variable selection for model-based clustering. J. Am. Stat. Assoc. 101(473), 168–178 (2006)
Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)
Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S-PLUS. Springer, Berlin (1999)
Acknowledgements
The authors gratefully acknowledge the helpful comments of three anonymous reviewers. This work was supported by a Collaborative Research and Development Grant from the Natural Sciences & Engineering Research Council of Canada; by a grant-in-aid from Compusense Inc.; by a Collaborative Research Grant from the Ontario Centres of Excellence; and by a University Research Chair from the University of Guelph. Equipment used in this work was purchased with support from the Canada Foundation for Innovation’s Leaders Opportunity Fund and from the Ontario Ministry for Research & Innovation’s Small Infrastructure Fund.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Browne, R.P., McNicholas, P.D. Orthogonal Stiefel manifold optimization for eigen-decomposed covariance parameter estimation in mixture models. Stat Comput 24, 203–210 (2014). https://doi.org/10.1007/s11222-012-9364-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-012-9364-2