Bayesian Non-Linear Independent Component Analysis by Multi-Layer Perceptrons

  • Harri Lappalainen
  • Antti Honkela
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)


In this chapter, a non-linear extension to independent component analysis is developed. The non-linear mapping from source signals to observations is modelled by a multi-layer perceptron network and the distributions of source signals are modelled by mixture-of-Gaussians. The observations are assumed to be corrupted by Gaussian noise and therefore the method is more ade quately described as non-linear independent factor analysis. The non-linear mapping, the source distributions and the noise level are estimated from the data. Bayesian approach to learning avoids problems with overlearning which would otherwise be severe in unsupervised learning with flexible non-linear models.


Cost Function Hide Neuron Independent Component Analysis Posterior Variance Gaussian Source 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    H. Attias. Independent factor analysis. Neural Computation 11, (4): 803–851, 1999.CrossRefGoogle Scholar
  2. 2.
    C.M. Bishop, M. Svensén, C.K.I. Williams. GTM: The generative topographic mapping. Neural Computation 10, (1): 215–234, 1998.CrossRefGoogle Scholar
  3. 3.
    S. Hochreiter and J. Schmidhuber. LOCOCODE performs non-linear ICA without knowing the number of sources. Proceedings of the ICA ‘99, 149–154, 1999.Google Scholar
  4. 4.
    A. Hyvärinen and E. Oja. A fast fixed-point algorithm for independent component analysis. Neural Computation 9, (7): 1483–1492, 1997.CrossRefGoogle Scholar
  5. 5.
    T. Kohonen: Self-Organizing Maps. (Springer, New York, 1995).CrossRefGoogle Scholar
  6. 6.
    D.J.C. MacKay and M.N. Gibbs. Density networks. In J. Kay, editor, Proceedings of Society for General Microbiology Edinburgh Meeting, 1997.Google Scholar
  7. 7.
    J.-H. Oh and H.S. Seung. Learning generative models with the up-propagation algorithm. In M. I. Jordan, M. J. Kearns, S. A. Solla, editors, Advances in Neural Information Processing Systems 10, 605–611, MIT Press, Cambridge, MA, 1998Google Scholar

Copyright information

© Springer-Verlag London 2000

Authors and Affiliations

  • Harri Lappalainen
  • Antti Honkela

There are no affiliations available

Personalised recommendations