Automatic Model Selection in Archetype Analysis
Archetype analysis involves the identification of representative objects from amongst a set of multivariate data such that the data can be expressed as a convex combination of these representative objects. Existing methods for archetype analysis assume a fixed number of archetypes a priori. Multiple runs of these methods for different choices of archetypes are required for model selection. Not only is this computationally infeasible for larger datasets, in heavy-noise settings model selection becomes cumbersome. In this paper, we present a novel extension to these existing methods with the specific focus of relaxing the need to provide a fixed number of archetypes beforehand. Our fast iterative optimization algorithm is devised to automatically select the right model using BIC scores and can easily be scaled to noisy, large datasets. These benefits are achieved by introducing a Group-Lasso component popular for sparse linear regression. The usefulness of the approach is demonstrated through simulations and on a real world application of document analysis for identifying topics.
KeywordsModel Selection Convex Hull Bayesian Information Criterion Solution Path Inverse Document Frequency
Unable to display preview. Download preview PDF.
- 3.Cutler, A., Breiman, L.: Archetypal analysis. Technometrics, 338–347 (1994)Google Scholar
- 4.Hastie, T., Taylor, J., Tibshirani, R., Walther, G.: Forward stagewise regression and the monotone Lasso. Electronic Journal of Statistics 1, 2007 (2006)Google Scholar
- 6.Lewis, D.D., Yang, Y., Rose, T.G., Li, F., Dietterich, G., Li, F.: RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research 5, 361–397 (2004)Google Scholar
- 8.Roth, V., Fischer, B.: The Group-Lasso for generalized linear models: uniqueness of solutions and efficient algorithms. In: ICML 2008, pp. 848–855. ACM (2008)Google Scholar
- 10.Xu, H., Caramanis, C., Sanghavi, S.: Robust PCA via outlier pursuit. In: Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems 23, pp. 2496–2504 (2010)Google Scholar
- 11.Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. Roy. Stat. Soc. B, 49–67 (2006)Google Scholar