Bayesian Non-Linear Independent Component Analysis by Multi-Layer Perceptrons
In this chapter, a non-linear extension to independent component analysis is developed. The non-linear mapping from source signals to observations is modelled by a multi-layer perceptron network and the distributions of source signals are modelled by mixture-of-Gaussians. The observations are assumed to be corrupted by Gaussian noise and therefore the method is more ade quately described as non-linear independent factor analysis. The non-linear mapping, the source distributions and the noise level are estimated from the data. Bayesian approach to learning avoids problems with overlearning which would otherwise be severe in unsupervised learning with flexible non-linear models.
KeywordsCost Function Hide Neuron Independent Component Analysis Posterior Variance Gaussian Source
Unable to display preview. Download preview PDF.
- 3.S. Hochreiter and J. Schmidhuber. LOCOCODE performs non-linear ICA without knowing the number of sources. Proceedings of the ICA ‘99, 149–154, 1999.Google Scholar
- 6.D.J.C. MacKay and M.N. Gibbs. Density networks. In J. Kay, editor, Proceedings of Society for General Microbiology Edinburgh Meeting, 1997.Google Scholar
- 7.J.-H. Oh and H.S. Seung. Learning generative models with the up-propagation algorithm. In M. I. Jordan, M. J. Kearns, S. A. Solla, editors, Advances in Neural Information Processing Systems 10, 605–611, MIT Press, Cambridge, MA, 1998Google Scholar