Kernel-Based Nonlinear Independent Component Analysis

  • Kun Zhang
  • Laiwan Chan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4666)

Abstract

We propose the kernel-based nonlinear independent component analysis (ICA) method, which consists of two separate steps. First, we map the data to a high-dimensional feature space and perform dimension reduction to extract the effective subspace, which was achieved by kernel principal component analysis (PCA) and can be considered as a pre-processing step. Second, we need to adjust a linear transformation in this subspace to make the outputs as statistically independent as possible. In this way, nonlinear ICA, a complex nonlinear problem, is decomposed into two relatively standard procedures. Moreover, to overcome the ill-posedness in nonlinear ICA solutions, we utilize the minimal nonlinear distortion (MND) principle for regularization, in addition to the smoothness regularizer. The MND principle states that we would prefer the nonlinear ICA solution with the mixing system of minimal nonlinear distortion, since in practice the nonlinearity in the data generation procedure is usually not very strong.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Almeida, L.B.: MISEP - linear and nonlinear ICA based on mutual information. Journal of Machine Learning Research 4, 1297–1318 (2003)CrossRefGoogle Scholar
  2. 2.
    Amari, S., Cichocki, A., Yang, H.H.: A new learning algorithm for blind signal separation. In: Advances in Neural Information Processing Systems (1996)Google Scholar
  3. 3.
    Bach, F.R., Jordan, M.I.: Beyond independent components: trees and clusters. Journal of Machine Learning Research 4, 1205–1233 (2003)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Cardoso, J.F., Souloumiac, A.: Blind beamforming for non-Gaussian signals. IEE Proceeding-F 140(6), 362–370 (1993)Google Scholar
  5. 5.
    Harmeling, S., Ziehe, A., Kawanabe, M., Müller, K.R.: Kernel-based nonlinear blind source separation. Neural Computation 15, 1089–1124 (2003)MATHCrossRefGoogle Scholar
  6. 6.
    Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks 10(3), 626–634 (1999)CrossRefGoogle Scholar
  7. 7.
    Hyvärinen, A.: The fixed-point algorithm and maximum likelihood estimation for independent component analysis. Neural Processing Letters 10(1), 1–5 (1999)CrossRefGoogle Scholar
  8. 8.
    Hyvärinen, A., Pajunen, P.: Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks 12(3), 429–439 (1999)CrossRefGoogle Scholar
  9. 9.
    Jutten, C., Karhunen, J.: Advances in nonlinear blind source separation. In: Proc. ICA 2003, pp. 245–256 (2003) Invited paper in the special session on nonlinear ICA and BSSGoogle Scholar
  10. 10.
    Martinez, D., Bray, A.: Nonlinear blind source separation using kernels. IEEE Transaction on Neural Network 14(1), 228–235 (2003)CrossRefGoogle Scholar
  11. 11.
    Schölkopf, B., Smola, A., Muller, K.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  12. 12.
    Zhang, K., Chan, L.: Nonlinear independent component analysis with minimum nonlinear distortion. In: ICML 2007, Corvallis, OR, US, pp. 1127–1134 (2007)Google Scholar
  13. 13.
    Ziehe, A., Müller, K.R.: TDSEP − an efficient algorithm for blind separation using time structure. In: Proc. ICANN98, Skövde, Sweden, pp. 675–680 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Kun Zhang
    • 1
  • Laiwan Chan
    • 1
  1. 1.Department of Computer Science and Engineering, The Chinese University of Hong Kong, ShatinHong Kong

Personalised recommendations