Statistical Analysis of Sample-Size Effects in ICA

  • J. Michael Herrmann
  • Fabian J. Theis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4881)


Independent component analysis (ICA) solves the blind source separation problem by evaluating higher-order statistics, e.g. by estimating fourth-order moments. While estimation errors of the kurtosis can be shown to asymptotically decay with sample size according to a square-root law, they are subject to two further effects for finite samples. Firstly, errors in the estimation of kurtosis increase with the deviation from Gaussianity. Secondly, errors in kurtosis-based ICA algorithms increase when approaching the Gaussian case. These considerations allow us to derive a strict lower bound for the sample size to achieve a given separation quality, which we study analytically for a specific family of distributions and a particular algorithm (fastICA). We further provide results from simulations that support the relevance of the analytical results.


Independent Component Analysis Fisher Information Independent Component Analysis Blind Source Separation Recovery Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abramowitz, M., Stegun, I.A. (eds.): Handbook of mathematical functions with formulas, graphs, and mathematical tables. Dover, Mineola (1972)zbMATHGoogle Scholar
  2. 2.
    Amari, S., Cichocki, A., Yang, H.H.: A new learning algorithm for blind signal separation. Advances in Neural Information Processing Systems 8, 757–763 (1996)Google Scholar
  3. 3.
    Bai, J., Ng, S.: Tests for skewness, kurtosis, and normality for time series data. J. Business & Economic Statistics 23(1), 49–60 (2005)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Bethge, M.: Factorial coding of natural images: how effective are linear models in removing higher-order dependencies? J. Opt. Soc. Am. 23(6), 1253–1268 (2006)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Comon, P.: Independent component analysis - a new concept? Signal Processing 36, 287–314 (1994)zbMATHCrossRefGoogle Scholar
  6. 6.
    Dodel, S., Herrmann, J.M., Geisel, T.: Comparison of temporal and spatial ica in fmri data analysis. In: ICA 2000. Proc. Second Int. Workshop on Independent Component Analysis and Blind Signal Separation, vol. 13, pp. 543–547 (2000)Google Scholar
  7. 7.
    Farvardin, N., Modestino, J.: Optimum quantizer performance for a class of non-gaussian memoryless sources. IEEE Trans. Inf. Theory IT-30, 485–497 (1984)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent component analysis. John Wiley & Sons, Chichester (2001)Google Scholar
  9. 9.
    Hyvärinen, A., Oja, E.: A fast fixed-point algorithm for independent component analysis. Neural Computation 9, 1483–1492 (1997)CrossRefGoogle Scholar
  10. 10.
    Jutten, C., Hérault, J., Comon, P., Sorouchiary, E.: Blind separation of sources, parts I, II and III. Signal Processing 24, 1–29 (1991)zbMATHCrossRefGoogle Scholar
  11. 11.
    Koldovský, Z., Tichavský, P., Oja, E.: Efficient variant of algorithm fastICA for independent component analysis attaining the Cramér-Rao lower bound. IEEE Transactions on Neural Networks 17(5), 1265–1277 (2006)CrossRefGoogle Scholar
  12. 12.
    MacCallum, R.C., Browne, M.W., Sugawara, H.M.: Power analysis and determination of sample size for covariance structure models. Psychological Methods 1(2), 130–149 (1996)CrossRefGoogle Scholar
  13. 13.
    Molgedey, L., Schuster, G.: Separation of a mixture of independent signals using time delayed correlations. Physical Review Letters 72(23), 3634–3637 (1994)CrossRefGoogle Scholar
  14. 14.
    Regalia, P.A., Kofidis, E.: Monotonic convergence of fixed-point algorithms for ICA. IEEE Transactions on Neural Networks 14(4), 943–949 (2003)CrossRefGoogle Scholar
  15. 15.
    Theis, F.J.: A new concept for separability problems in blind source separation. Neural Computation 16, 1827–1850 (2004)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • J. Michael Herrmann
    • 1
    • 2
  • Fabian J. Theis
    • 1
    • 3
  1. 1.Bernstein Center for Computational Neuroscience Göttingen 
  2. 2.Göttingen University, Institute for Nonlinear Dynamics 
  3. 3.Max Planck Institute for Dynamics and Self-Organization, Bunsenstraße 10, 37073 GöttingenGermany

Personalised recommendations