Automatic Model Selection for Probabilistic PCA
The Mixture of Probabilistic Principal Components Analyzers (MPPCA) is a multivariate analysis technique which defines a Gaussian probabilistic model at each unit. The number of units and principal directions in each unit is not learned in the original approach. Variational Bayesian approaches have been proposed for this purpose, which rely on assumptions on the input distribution and/or approximations of certain statistics. Here we present a different way to solve this problem, where cross-validation is used to guide the search for an optimal model selection. This allows to learn the model architecture without the need of any assumptions other than those of the basic PPCA framework. Experimental results are presented, which show the probability density estimation capabilities of the proposal with high dimensional data.
KeywordsProbabilistic Principal Components Analysis (PPCA) dimensionality reduction cross-validation handwritten digit recognition
Unable to display preview. Download preview PDF.
- 1.Beal, M.J.: Software in Matlab. Available at: http://www.cse.buffalo.edu/faculty/mbeal/software.html
- 3.Burden, R.L., Faires, D.: Numerical Analysis. Brooks/Cole Publishing, Pacific Grove (2004)Google Scholar
- 5.Ghahramani, Z., Beal, M.J.: Variational Inference for Bayesian Mixtures of Factor Analysers. Advances in Neural Information Processing Systems 12, 449–455 (1999)Google Scholar
- 7.LeCun, Y., Cortes, C.: The MNIST Database of Handwritten Digits. In: Internet (November 2006), http://yann.lecun.com/exdb/mnist/
- 8.Oba, S., Sato, M., Ishii, S.: Prior Hyperparameters in Bayesian PCA. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 271–279. Springer, Heidelberg (2003)Google Scholar
- 10.VizieR service (March 29, 2004), Available at: http://vizier.cfa.harvard.edu/viz-bin/VizieR