Neural Processing Letters

, Volume 10, Issue 1, pp 1–5 | Cite as

The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis

  • Aapo Hyvärinen
Article

Abstract

The author previously introduced a fast fixed-point algorithm for independent component analysis. The algorithm was derived from objective functions motivated by projection pursuit. In this paper, it is shown that the algorithm is closely connected to maximum likelihood estimation as well. The basic fixed-point algorithm maximizes the likelihood under the constraint of decorrelation, if the score function is used as the nonlinearity. Modifications of the algorithm maximize the likelihood without constraints.

independent component analysis blind source separation maximum likelihood 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amari, S.-I.: Neural learning in structured parameter spaces - natural riemannian gradient, In: Advances in Neural Information Processing 9 (Proc. NIPS*96), MIT Press, Cambridge, MA, 1997, pp. 127–133.Google Scholar
  2. 2.
    Bell, A. J. and Sejnowski, T. J.: An information-maximization approach to blind separation and blind deconvolution, Neural Computation 7 (1995), 1129–1159.Google Scholar
  3. 3.
    Cardoso, J. F.: Entropic contrasts for source separation, In: S. Haykin (ed.), Adaptive Unsupervised Learning, 1998.Google Scholar
  4. 4.
    Cichocki, A. and Unbehauen, R.: Robust neural networks with on-line learning for blind identification and blind separation of sources, IEEE Trans. on Circuits and Systems 43(11) (1996), 894–906.Google Scholar
  5. 5.
    Comon, P.: Independent component analysis - a new concept? Signal Processing 36 (1994), 287–314.Google Scholar
  6. 6.
    Huber, P. J.: Projection pursuit, Ann. Statistics 13(2) (1985), 435–475.Google Scholar
  7. 7.
    Hyvärinen, A.: A family of fixed-point algorithms for independent component analysis, In: Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP'97), Munich, Germany, 1997, pp. 3917–3920.Google Scholar
  8. 8.
    Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis, IEEE Trans. on Neural Networks, in press.Google Scholar
  9. 9.
    Hyvärinen, A. and Oja, E.: A fast fixed-point algorithm for independent component analysis, Neural Computation 9(7) (1997), 1483–1492.Google Scholar
  10. 10.
    Jutten, C. and Herault, J.: Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture, Signal Processing 24 (1991), 1–10.Google Scholar
  11. 11.
    Lee, T.-W., Girolami, M. and Sejnowski, T. J.: Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources, Neural Computation 11 (1999), 609–633.Google Scholar
  12. 12.
    Luenberger, D. G.: Optimization by Vector Space Methods, John Wiley & Sons, 1969.Google Scholar
  13. 13.
    Nadal, J.-P. and Parga, N.: Non-linear neurons in the low noise limit: a factorial code maximizes information transfer, Network 5 (1994), 565–581.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Aapo Hyvärinen
    • 1
  1. 1.Laboratory of Computer and Information ScienceHelsinki University of TechnologyHUTFinland, e-mail

Personalised recommendations