Using Kernel PCA for Initialisation of Variational Bayesian Nonlinear Blind Source Separation Method
The variational Bayesian nonlinear blind source separation method introduced by Lappalainen and Honkela in 2000 is initialised with linear principal component analysis (PCA). Because of the multilayer perceptron (MLP) network used to model the nonlinearity, the method is susceptible to local minima and therefore sensitive to the initialisation used. As the method is used for nonlinear separation, the linear initialisation may in some cases lead it astray. In this paper we study the use of kernel PCA (KPCA) in the initialisation. KPCA is a rather straightforward generalisation of linear PCA and it is much faster to compute than the variational Bayesian method. The experiments show that it can produce significantly better initialisations than linear PCA. Additionally, the model comparison methods provided by the variational Bayesian framework can be easily applied to compare different kernels.
KeywordsCost Function Independent Component Analysis Ensemble Learning Variational Bayesian Blind Signal Separation
Unable to display preview. Download preview PDF.
- 2.Jutten, C., Karhunen, J.: Advances in nonlinear blind source separation. In: Proc. of the 4th Int. Symp. on Independent Component Analysis and Blind Signal Separation (ICA 2003), pp. 245–256 (2003); Invited paper in the special session on nonlinear ICA and BSSGoogle Scholar
- 5.Lappalainen, H., Honkela, A.: Bayesian nonlinear independent component analysis by multi-layer perceptrons. In: Girolami, M. (ed.) Advances in Independent Component Analysis, pp. 93–121. Springer, Berlin (2000)Google Scholar
- 6.Valpola, H., Oja, E., Ilin, A., Honkela, A., Karhunen, J.: Nonlinear blind source separation by variational Bayesian learning. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E86-A(3), 532–541 (2003)Google Scholar
- 7.Hinton, G.E., van Camp, D.: Keeping neural networks simple by minimizing the description length of the weights. In: Proc. of the 6th Ann. ACM Conf. on Computational Learning Theory, Santa Cruz, CA, USA, pp. 5–13 (1993)Google Scholar
- 8.MacKay, D.J.C.: Developments in probabilistic modelling with neural networks – ensemble learning. In: Neural Networks: Artificial Intelligence and Industrial Applications. Proc. of the 3rd Annual Symposium on Neural Networks, pp. 191–198 (1995)Google Scholar
- 9.Valpola, H., Östman, T., Karhunen, J.: Nonlinear independent factor analysis by hierarchical models. In: Proc. 4th Int. Symp. on Independent Component Analysis and Blind Signal Separation (ICA2003), Nara, Japan, pp. 257–262 (2003)Google Scholar