Newton-Like Methods for Nonparametric Independent Component Analysis

  • Hao Shen
  • Knut Hüper
  • Alexander J. Smola
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4232)


The performance of ICA algorithms significantly depends on the choice of the contrast function and the optimisation algorithm used in obtaining the demixing matrix. In this paper we focus on the standard linear nonparametric ICA problem from an optimisation point of view. It is well known that after a pre-whitening process, the problem can be solved via an optimisation approach on a suitable manifold. We propose an approximate Newton’s method on the unit sphere to solve the one-unit linear nonparametric ICA problem. The local convergence properties are discussed. The performance of the proposed algorithms is investigated by numerical experiments.


Independent Component Analysis Independent Component Analysis Blind Source Separation Contrast Function Independent Component Analysis Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boscolo, R., Pan, H., Roychowdhury, V.P.: Independent component analysis based on nonparametric density estimation. IEEE Transactions on Neural Networks 15(1), 55–65 (2004)CrossRefGoogle Scholar
  2. 2.
    Comon, P.: Independent component analysis, a new concept? Signal Processing 36(3), 287–314 (1994)MATHCrossRefGoogle Scholar
  3. 3.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley, New York (2001)CrossRefGoogle Scholar
  4. 4.
    Gretton, A., Bousquet, O., Smola, A.J., Schölkopf, B.: Measuring statistical dependence with Hilbert-S chmidt norms. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS (LNAI), vol. 3734, pp. 63–77. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Chen, A., Bickel, P.J.: Consistent independent component analysis and prewhitening. IEEE Transactions on Signal Processing 53(10), 3625–3632 (2005)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Hüper, K., Shen, H., Seghouane, A.-K.: Local convergence properties of Fast- ICA and some generalisations. In: Proceedings of the 31st IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2006), pp. V1009–V1012 (2006)Google Scholar
  7. 7.
    Shen, H., Hüper, K.: Newton-like methods for parallel independent component analysis. To appear in: LSP 2006, Maynooth, Ireland, September 6-8 (2006)Google Scholar
  8. 8.
    Shen, H., Hüper, K.: Local convergence analysis of FastICA. In: Rosca, J.P., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 893–900. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Shen, H., Hüper, K., Seghouane, A.-K.: Geometric optimisation and FastICA algorithms. To appear in: MTNS 2006, Kyoto, Japan, July 24-28 (2006)Google Scholar
  10. 10.
    Smith, S.: Optimization techniques on riemannian manifolds. Hamiltonian and gradientflows, algorithms and control, Fields Institute Communications 3, 113–136 (1994)Google Scholar
  11. 11.
    Cardoso, J.F.: Blind source separation: statistical principles. Proceedings of the IEEE (90), 2099–2026 (1998)Google Scholar
  12. 12.
    Miller, E.G., Fisher III, J.W.: ICA using spacings estimates of entropy. The Journal of Machine Learning Research 4(7-8), 1271–1295 (2004)MATHCrossRefGoogle Scholar
  13. 13.
    Amari, S., Cichocki, A., Yang, H.H.: A new learning algorithm for blind signal separation. In: Advances in Neural Information Processing Systems, vol. 8, pp. 757–763 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hao Shen
    • 1
    • 3
  • Knut Hüper
    • 1
    • 3
  • Alexander J. Smola
    • 2
    • 4
  1. 1.Systems Engineering and Complex Systems Research Program 
  2. 2.Statistical Machine Learning Research ProgramNational ICT AustraliaCanberraAustralia
  3. 3.Department of Information Engineering 
  4. 4.Computer Science Laboratory, Research School of Information Sciences and EngineeringThe Australian National UniversityCanberraAustralia

Personalised recommendations