A Comparative Study on Selection of Cluster Number and Local Subspace Dimension in the Mixture PCA Models
How to determine the number of clusters and the dimensions of local principal subspaces is an important and challenging problem in various applications. Based on a probabilistic model of local PCA, this problem can be solved by one of existing statistical model selection criteria in a two-phase procedure. However, such a two-phase procedure is too time-consuming especially when there is no prior knowledge. The BYY harmony learning has provided a promising mechanism to make automatic model selection in parallel with parameter learning. This paper investigates the BYY harmony learning with automatic model selection on a mixture PCA model in comparison with three typical model selection criteria: AIC, CAIC, and MDL. This comparative study is made by experiments for different model selection tasks on simulated data sets under different conditions. Experiments have shown that automatic model selection by the BYY harmony learning are not only as good as or even better than conventional methods in terms of performances, but also considerably supervisory in terms of much less computational cost.
KeywordsModel Selection Cluster Number Minimum Description Length Model Selection Criterion Parameter Learning
Unable to display preview. Download preview PDF.
- 1.Xu, L.: Beyond PCA Learning: From Linear to Nonlinear and From Global Representation to Local Representation. In: Proc. Intl. Conf. on Neural Information Processing (ICONIP 1994), Seoul, Korea, vol. 2, pp. 943–949 (1994)Google Scholar
- 2.Xu, L.: Theories for Unsupervised Learning: PCA and Its Nonlinear Extensions. In: Proc. of IEEE ICNN 1994, Orlando, Florida, vol. II, pp. 1252–1257 (1994)Google Scholar
- 3.Xu, L.: Bayesian-Kullback Coupled Ying-Yang Machines: Unified Leanings and New Results on Vector Quantization. In: Proc. Intl. Conf. on Neural Information Processing (ICONIP 1995), Beijing, China, pp. 977–988 (1995)Google Scholar
- 5.Xu, L.: Independent Component Analysis and Extensions with Noise and Time: A Bayesian Ying-Yang Learning Perspective. Neural Information Processing Letters and Reviews 1, 1–52 (2003)Google Scholar
- 11.Xu, L.: Bayesian Ying Yang Learning (I): A Unified Perspective for Statistical Modeling. In: Zhong, N., Liu, J. (eds.) Intelligent Technologies for Information Analysis, pp. 615–659. Springer, Heidelberg (2004)Google Scholar
- 12.Xu, L.: Bayesian Ying Yang Learning (II): A New Mechanism for Model Selection and Regularization. In: Zhong, N., Liu, J. (eds.) Intelligent Technologies for Information Analysis, pp. 661–706. Springer, Heidelberg (2004)Google Scholar
- 13.Roweis, S.: EM Algorithms for PCA and SPCA. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) Advances in Neural Information Processing Systems, vol. 10, pp. 626–632. The MIT Press, Cambridge (1998)Google Scholar
- 14.Sclove, S.L.: Some Aspects of Model-Selection Criteria. In: Bozdogan, H. (ed.) Proceedings of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach, vol. 2, pp. 37–67. Kluwer Academic Publishers, Dordrecht (1994)Google Scholar