Post-nonlinear Independent Component Analysis by Variational Bayesian Learning

  • Alexander Ilin
  • Antti Honkela
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3195)

Abstract

Post-nonlinear (PNL) independent component analysis(ICA) is a generalisation of ICA where the observations are assumed to have been generated from independent sources by linear mixing followed by component-wise scalar nonlinearities. Most previous PNL ICA algorithms require the post-nonlinearities to be invertible functions. In this paper, we present a variational Bayesian approach to PNL ICA that also works for non-invertible post-nonlinearities. The method is based on a generative model with multi-layer perceptron (MLP) networks to model the post-nonlinearities. Preliminary results with a difficult artificial example are encouraging.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. J. Wiley, Chichester (2001)CrossRefGoogle Scholar
  2. 2.
    Hyvärinen, A., Pajunen, P.: Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks 12(3), 429–439 (1999)CrossRefGoogle Scholar
  3. 3.
    Taleb, A., Jutten, C.: Source separation in post-nonlinear mixtures. IEEE Trans. on Signal Processing 47(10), 2807–2820 (1999)CrossRefGoogle Scholar
  4. 4.
    Jutten, C., Karhunen, J.: Advances in nonlinear blind source separation. In: Proc. of the 4th Int. Symp. on Independent Component Analysis and Blind Signal Separation (ICA 2003), pp. 245–256 (2003) (invited paper in the special session on nonlinear ICA and BSS)Google Scholar
  5. 5.
    Ilin, A., Achard, S., Jutten, C.: Bayesian versus constrained structure approaches for source separation in post-nonlinear mixtures. In: Proc. International Joint Conference on Neural Networks, IJCNN 2004 (2004) (to appear)Google Scholar
  6. 6.
    Lappalainen, H., Honkela, A.: Bayesian nonlinear independent component analysis by multi-layer perceptrons. In: Girolami, M. (ed.) Advances in Independent Component Analysis, pp. 93–121. Springer, Berlin (2000)Google Scholar
  7. 7.
    Valpola, H., Oja, E., Ilin, A., Honkela, A., Karhunen, J.: Nonlinear blind source separation by variational Bayesian learning. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E86-A(3), 532–541 (2003)Google Scholar
  8. 8.
    Taleb, A., Jutten, C.: Batch algorithm for source separation in postnonlinear mixtures. In: Proc. Int. Workshop on Independent Component Analysis and Signal Separation (ICA 1999) (Aussois, France), pp. 155–160 (1999)Google Scholar
  9. 9.
    Hinton, G.E., van Camp, D.: Keeping neural networks simple by minimizing the description length of the weights. In: Proc. of the 6th Ann. ACM Conf. on Computational Learning Theory, Santa Cruz, CA, USA, pp. 5–13 (1993)Google Scholar
  10. 10.
    MacKay, D.J.C.: Developments in probabilistic modelling with neural networks – ensemble learning. In: Neural Networks: Artificial Intelligence and Industrial Applications. Proc. of the 3rd Annual Symposium on Neural Networks, pp. 191–198 (1995)Google Scholar
  11. 11.
    Lappalainen, H., Miskin, J.: Ensemble learning. In: Girolami, M. (ed.) Advances in Independent Component Analysis, pp. 75–92. Springer, Berlin (2000)Google Scholar
  12. 12.
    Attias, H.: Independent factor analysis. Neural Computation 11(4), 803–851 (1999)CrossRefGoogle Scholar
  13. 13.
    Lappalainen, H.: Ensemble learning for independent component analysis. In: Proc. Int. Workshop on Independent Component Analysis and Signal Separation (ICA 1999), Aussois, France, pp. 7–12 (1999)Google Scholar
  14. 14.
    Miskin, J., MacKay, D.J.C.: Ensemble learning for blind source separation. In: Roberts, S., Everson, R. (eds.) Independent Component Analysis: Principles and Practice, pp. 209–233. Cambridge University Press, Cambridge (2001)Google Scholar
  15. 15.
    Penny, W., Everson, R., Roberts, S.: ICA: model order selection and dynamic source models. In: Roberts, S., Everson, R. (eds.) Independent Component Analysis: Principles and Practice, pp. 299–314. Cambridge University Press, Cambridge (2001)Google Scholar
  16. 16.
    Valpola, H., Karhunen, J.: An unsupervised ensemble learning method for nonlinear dynamic state-space models. Neural Computation 14(11), 2647–2692 (2002)MATHCrossRefGoogle Scholar
  17. 17.
    Honkela, A.: Approximating nonlinear transformations of probability distributions for nonlinear independent component analysis. In: Proc. International Joint Conference on Neural Networks, IJCNN 2004 (2004) (to appear)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Alexander Ilin
    • 1
  • Antti Honkela
    • 1
  1. 1.Neural Networks Research CentreHelsinki University of TechnologyEspooFinland

Personalised recommendations