Flexible Independent Component Analysis

  • Seungjin Choi
  • Andrzej Cichocki
  • Shun-Ichi Amari


This paper addresses an independent component analysis (ICA) learning algorithm with flexible nonlinearity, so named as flexible ICA, that is able to separate instantaneous mixtures of sub- and super-Gaussian source signals. In the framework of natural Riemannian gradient, we employ the parameterized generalized Gaussian density model for hypothesized source distributions. The nonlinear function in the flexible ICA algorithm is controlled by the Gaussian exponent according to the estimated kurtosis of demixing filter output. Computer simulation results and performance comparison with existing methods are presented.


Independent Component Analysis Independent Component Analysis Blind Source Separation Blind Signal Natural Gradient 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J.-F. Cardoso and A. Souloumiac, “Blind Beamforming for Non Gaussian Signals,”IEE Proceedings-F, vol. 140, no.6, 1993, pp. 362–370.Google Scholar
  2. 2.
    S. Choi, “Adaptive Blind Signal Separation for Multiuser Communications: An information-Theoretic Approach,” Journal of Electrical Engineering and Information science,, vol. 4, no.2, April 1999, pp. 249–256.Google Scholar
  3. 3.
    A. Bell and T. Sejnowski, “Learning the Higher-Order Structure of a Natural Sound,” Network: Computation in Neural Systems, vol. 7, 1996, pp. 261–266.CrossRefzbMATHGoogle Scholar
  4. 4.
    A. Bell and T. Sejnowski, “The Independent Components of Natural Scenes are Edge Filters,” Vision Research, vol. 37, no.23, 1997, pp. 3327–3338.CrossRefGoogle Scholar
  5. 5.
    S. Choi, A. Cichocki, and S. Amari, “Fetal Electrocardiogram Data Analysis Using Flexible Independent Component Analysis,” in The 4th Asia-Pacific Conference on Medical and Biological Engineering (APCMBE'99), Seoul, Korea, 1999.Google Scholar
  6. 6.
    S. Makeig, A. Bell, T.-P. Jung, and T. Sejnowski, “Blind Separation of Auditory Event-Related Brain Responses into Independent Components,” in Proc. of National Academy of Sciences, vol. 94, 1997, pp. 10979–10984.CrossRefGoogle Scholar
  7. 7.
    T.-P. Jung, C. Humphries, T. Lee, S. Makeig, M. McKeown, V. Iragui, and T. Sejnowski, “Extended ICA Removes Artifacts from Electroencephalographic Recordings,” in Advances in Neural Information Processing Systems, vol. 10, 1998, pp. 894–900.Google Scholar
  8. 8.
    M. Girolami, “Hierarchic Dichotomizing of Polychotomous Data–An ICA Based Data Mining Tool,” in First International Workshop on Independent Component Analysis and Signal Separation, 1999, pp. 197–201.Google Scholar
  9. 9.
    P. Comon, “Independent Component Analysis, A New Concept?,” Signal Processing, vol. 36, no.3, 1994, pp. 287–314.CrossRefzbMATHGoogle Scholar
  10. 10.
    S. Amari and J.-F. Cardoso, “Blind Source Separation: Semiparametric Statistical Approach,” IEEE Trans. Signal Processing, vol. 45, 1997, pp. 2692–2700.CrossRefGoogle Scholar
  11. 11.
    C. Jutten and J. Herault, “Blind Separation of Sources, Part I: An Adaptive Algorithm Based on Neuromimetic Architecture,” Signal Processing, vol. 24, 1991, pp. 1–10.CrossRefzbMATHGoogle Scholar
  12. 12.
    A. Cichocki, R. Unbehauen, and E. Rummert, “Robust Learning Algorithm for Blind Separation of Signals,” Electronics Letters, vol. 43, no.17, 1994, pp. 1386–1387.CrossRefGoogle Scholar
  13. 13.
    A. Cichocki and R. Unbehauen, “Robust Neural Networks with On-Line Learning for Blind Identification and Blind Separation of Sources,” IEEE Trans. Circuits and Systems-I: Fundamental Theory and Applications, vol. 43, 1996, pp. 894–906.CrossRefGoogle Scholar
  14. 14.
    A. Bell and T. Sejnowski, “An Information Maximisation Approach to Blind Separation and Blind Deconvolution,” Neural Computation, vol. 7, 1995, pp. 1129–1159.CrossRefGoogle Scholar
  15. 15.
    S. Amari, A. Cichocki, and H.H. Yang, “A New Learning Algorithm for Blind Signal Separation,” in Advances in Neural Information Processing Systems, D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo (Eds.), MIT press, vol. 8, 1996, pp. 757–763.Google Scholar
  16. 16.
    S. Amari and A. Cichocki, “Adaptive Blind Signal Processing– Neural Network Approaches,” in Proc. of IEEE, on Blind Identification and Estimation, Special Issue, vol. 86, no.10, Oct. 1998, pp. 2026–2048.Google Scholar
  17. 17.
    D.T. Pham, “Blind Separation of Instantaneous Mixtures of Sources Via an Independent Component Analysis,”IEEE Trans. Signal Processing, vol. 44, no.11, 1996, pp. 2768–2779.CrossRefGoogle Scholar
  18. 18.
    D.J.C. MacKay, “Maximum Likelihood and Covariant Algorithms for Independent Component Analysis,” Technical Report Draft 3.7, University of Cambridge, Cavendish Laboratory, 1996.Google Scholar
  19. 19.
    B. Pearlmutter and L. Parra, “Maximum Likelihood Blind Source Separation:AContext-Sensitive Generalization of ICA,” in Advances in Neural Information Processing Systems, M.C. Mozer, M.I. Jordan, and T. Petsche (Eds.), vol. 9, 1997, pp. 613–619.Google Scholar
  20. 20.
    J.-F. Cardoso, “Infomax and Maximum Likelihood for Source Separation,” IEEE Signal Processing Letters, vol. 4, no.4, Apr. 1997, pp. 112–114.CrossRefGoogle Scholar
  21. 21.
    J.-F. Cardoso and B.H. Laheld, “Equivariant Adaptive Source Separation,” IEEE Signal Processing Letters, vol. 44, no.12, Dec. 1996, pp. 3017–3030.CrossRefGoogle Scholar
  22. 22.
    J. Karhunen, “Neural Approaches to Independent Component Analysis, in European Symposium on Artificial Neural Networks, 1996, pp. 249–266.Google Scholar
  23. 23.
    E. Oja, “The Nonlinear PCA Learning Rule and Signal Separation–Mathematical Analysis,” Technical Report A26, Helsinki University of Technology, Laboratory of Computer and Information Science, 1995.Google Scholar
  24. 24.
    A. Hyvärinen and E. Oja, “A Fast Fixed-Point Algorithm for Independent Component Analysis,” Neural Computation, vol. 9, 1997, pp. 1483–1492.CrossRefGoogle Scholar
  25. 25.
    N. Delfosse and P. Loubaton, “Adaptive Blind Separation of Independent Sources: A Deflation Approach,” Signal Processing, vol. 45, 1995, pp. 59–83.CrossRefzbMATHGoogle Scholar
  26. 26.
    A. Cichocki, R. Thawonmas, and S. Amari, “Sequential Blind Signal Extraction in Order Specifed by Stochastic Properties,” Electronics Letters, vol. 33, no.1, 1997, pp. 64–65.CrossRefGoogle Scholar
  27. 27.
    S. Choi, R. Liu, and A. Cichocki, “A Spurious Equlibria-Free Learning Algorithm for the Blind Separation of Non-Zero Skewness Signals,” Neural Processing Letters, vol. 7, 1998, pp. 61–68.CrossRefGoogle Scholar
  28. 28.
    J.-P. Nadal and N. Parga, “Redundancy Reduction and Independent Component Analysis:Conditions on Cumulants and Adaptive Approaches,” Neural Computation, vol. 9, 1997, pp. 1421–1456.CrossRefGoogle Scholar
  29. 29.
    S. Choi and A. Cichocki, “A Linear Feedforward Neural Network with Lateral Feedback Connections for Blind Source Separation,” in IEEE Signal Processing Workshop on Higherorder Statistics, Banff, Canada, 1997, pp. 349–353.Google Scholar
  30. 30.
    M. Girolami and C. Fyfe, “Generalized Independent Component Analysis through Unsupervised Learning with Emergent Bussgang Properties,” in Proc. ICNN, 1997, pp. 1788–1791.Google Scholar
  31. 31.
    S.C. Douglas, A. Cichocki, and S. Amari, “Multichannel Blind Separation and Deconvolution of Sources with Arbitrary Distributions,” in Neural Networks for Signal Processing VII, J. Principe, L. Gile, N. Morgan, and E. Wilson (Eds.), 1997, pp. 436–445.Google Scholar
  32. 32.
    A. Cichocki, I. Sabala, S. Choi, B. Orsier, and R. Szupiluk, “Self-adaptive Independent Component Analysis for Sub-Gaussian and Super-Gaussian Mixtures with Unknown Number of Source Signals,” in International Symposium on Nonlinear Theory and Applications, 1997, pp. 731–734.Google Scholar
  33. 33.
    M. Girolami, “An Alternative Perspective on Adaptive Independent Component Analysis Algorithms,” Neural Computation, vol. 10, no.8, 1998, pp. 2103–2114.MathSciNetCrossRefGoogle Scholar
  34. 34.
    T.-W. Lee, M. Girolami, and T. Sejnowski, “Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources,” Neural Computation, vol. 11, no.2, 1999, pp. 609–633.CrossRefGoogle Scholar
  35. 35.
    S. Choi, A. Cichocki, and S. Amari, “Flexible Independent Component Analysis,” in Neural Networks for Signal Processing VIII, T. Constantinides, S.-Y. Kung, M. Niranjan, and E. Wilson (Eds.), 1998, pp. 83–92.Google Scholar
  36. 36.
    S. Amari, “Natural Gradient Works Efficiently in Learning,” Neural Computation, vol. 10, no.2, Feb. 1998, pp. 251–276.MathSciNetCrossRefGoogle Scholar
  37. 37.
    S. Amari, “Natural Gradient for Over-and Under-Complete Bases in ICA,” Neural Computation, vol. 11, no.8, Nov. 1999, pp. 1875–1883.MathSciNetCrossRefGoogle Scholar
  38. 38.
    A. Cichocki, I. Sabala, and S. Amari, “Intelligent Neural Networks for Blind Signal Separation with Unknown Number of Sources,” in Int. Symp. Engineering of Intelligent Systems, Tenerife, Spain, 1998, pp. 148–154.Google Scholar
  39. 39.
    I.S. Gradshteyn, I.M. Ryzhik, and A. Jeffrey, Table of Integrals, Series, and Products, Academic Press, 1994.Google Scholar
  40. 40.
    S. Amari, T.-P. Chen, and A. Cichocki, “Stability Analysis of Learning Algorithms for Blind Source Separation,” Neural Networks, vol. 10, no.8, 1997, pp. 1345–1351.CrossRefGoogle Scholar
  41. 41.
    J.-F. Cardoso, “On the Stability of Source Separation Algorithms,” in Neural Networks for Signal Processing VIII, T. Constantinides, S.-Y. Kung, M. Niranjan, and E. Wilson (Eds.), 1998, pp. 13–22.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Seungjin Choi
    • 1
  • Andrzej Cichocki
    • 2
    • 3
  • Shun-Ichi Amari
    • 2
  1. 1.Department of Electrical EngineeringChungbuk National UniversitySouth Korea
  2. 2.Brain-Style Information Systems Research GroupBrain Science InstituteRikenJapan
  3. 3.Warsaw University of TechnologyPoland

Personalised recommendations