Local Factor Analysis with Automatic Model Selection: A Comparative Study and Digits Recognition Application
A further investigation is made on an adaptive local factor analysis algorithm from Bayesian Ying-Yang (BYY) harmony learning, which makes parameter learning with automatic determination of both the component number and the factor number in each component. A comparative study has been conducted on simulated data sets and several real problem data sets. The algorithm has been compared with not only a recent approach called Incremental Mixture of Factor Analysers (IMoFA) but also the conventional two-stage implementation of maximum likelihood (ML) plus model selection, namely, using the EM algorithm for parameter learning on a series candidate models, and selecting one best candidate by AIC, CAIC, and BIC. Experiments have shown that IMoFA and ML-BIC outperform ML-AIC or ML-CAIC while the BYY harmony learning considerably outperforms IMoFA and ML-BIC. Furthermore, this BYY learning algorithm has been applied to the popular MNIST database for digits recognition with a promising performance.
KeywordsGaussian Mixture Model Minimum Description Length Handwritten Digit Digit Recognition Automatic Model Selection
Unable to display preview. Download preview PDF.
- 2.Kambhatla, N., Leen, T.K.: Fast non-linear dimension reduction. In: Advances in NIPS 6. In: Advances in NIPS, vol. 6. Morgan Kaufmann, San Francisco (1994)Google Scholar
- 3.Hinton, G.E., Revow, M., Dayan, P.: Recognizing handwritten digits using mixtures of Linear models. In: Advances in NIPS, vol. 7. MIT Press, Cambridge (1995)Google Scholar
- 10.Ghahramani, Z., Beal, M.: Variational inference for Bayesian mixture of factor analysers. In: Advances in NIPS, vol. 12, pp. 449–455 (2000)Google Scholar
- 11.Salah, A.A., Alpaydin, E.: Incremental Mixtures of Factor Analysers. In: Proc. 17th Intl Conf. on Pattern Recognition, vol. 1, pp. 276–279 (2004)Google Scholar
- 13.Xu, L.: Multisets Modeling Learning: An Unified Theory for Supervised and Unsupervised Learning. In: Proc. IEEE ICNN 1994, Orlando, Florida, June 26-July 2, vol. I, pp. 315–320 (1994) (invited talk)Google Scholar
- 16.Xu, L.: A Unified Perspective and New Results on RHT Computing, Mixture Based Learning, and Multi-learner Based Problem Solving. To appear in a special issue of Pattern Recognition (2006)Google Scholar
- 17.Xu, L.: Trends on Regularization and Model Selection in Statistical Learning: A Perspective from Bayesian Ying Yang Learning. In: Duch, W., Mandziuk, J., Zurada, J.M. (eds.) Challenges to Computational Intelligence. The Springers series - Studies in Computational Intelligence. Springer, Heidelberg (2006) (in press)Google Scholar