Encyclopedia of Biometrics

2009 Edition
| Editors: Stan Z. Li, Anil Jain

Independent Component Analysis

  • Seungjin Choi
Reference work entry
DOI: https://doi.org/10.1007/978-0-387-73003-5_305


Blind source separation;  Independent factor analysis


Independent component analysis (ICA) is a statistical method, the goal of which is to decompose multivariate data into a linear sum of non-orthogonal basis vectors with coefficients (encoding variables, latent variables, hidden variables) being statistically independent. ICA generalizes a widely-used subspace analysis method such as principal component analysis (PCA) and factor analysis, allowing latent variables to be non-Gaussian and basis vectors to be non-orthogonal in general. Thus, ICA is a density estimation method where a linear model is learnt such that the probability distribution of the observed data is best captured, while factor analysis aims at best modeling the covariance structure of the observed data.


Linear latent variable model assumes that m-dimensional observed data \({{{\bf x}}}_{t} \in {\mathbb{R}}^{m}\)
This is a preview of subscription content, log in to check access.


  1. 1.
    Tipping, M.E., Bishop, C.M.: Mixtures of probabilistic principal component analyzers. Neural Comput. 11(2), 443–482 (1999)CrossRefGoogle Scholar
  2. 2.
    Lee, T.W.: Independent Component Analysis: Theory and Applications. Kluwer Academic: Boston (1998)Google Scholar
  3. 3.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley: New York (2001)Google Scholar
  4. 4.
    Cichocki, A., Amari, S.: Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. Wiley: West Sussex, England (2002)Google Scholar
  5. 5.
    Hyvärinen, A.: Survey on independent component analysis. Neural Computing Surveys 2, 94–128 (1999)Google Scholar
  6. 6.
    Choi, S., Cichocki, A., Park, H.M., Lee, S.Y.: Blind source separation and independent component analysis. A review. Neural Information Processing - Letters and Review 6(1), 1–57 (2005)Google Scholar
  7. 7.
    Barlow, H.B.: Unsupervised learning. Neural Computation 1, 295–311 (1989)CrossRefGoogle Scholar
  8. 8.
    Bell, A., Sejnowski, T.: An information maximisation approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159 (1995)CrossRefGoogle Scholar
  9. 9.
    Cardoso, J.F.: Source separation using higher-order moments. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (1989)Google Scholar
  10. 10.
    Tong, L., Soon, V.C., Huang, Y.F., Liu, R.: AMUSE: a new blind identification alogrithm. In: Proceedings of the IEEE International Symposium on Circuits and Systems, pp. 1784–1787 (1990)Google Scholar
  11. 11.
    Cardoso, J.F., Souloumiac, A.: Blind beamforming for non Gaussian signals. IEE Proceedings-F 140(6), 362–370 (1993)Google Scholar
  12. 12.
    Belouchrani, A., Abed-Merain, K., Cardoso, J.F., Moulines, E.: A blind source separation technique using second order statistics. IEEE Trans. Signal Processing 45, 434–444 (1997)CrossRefGoogle Scholar
  13. 13.
    Choi, S., Cichocki, A., Belouchrani, A.: Second order nonstationary source separation. J. VLST Signal Process Syst. Signal Image Video Technol. 32, 93–104 (2002)MATHGoogle Scholar
  14. 14.
    Choi, S., Cichocki, A., Belouchrani, A.: Blind separation of second-order nonstationary and temporally colored sources. In: Proceedings of IEEE Workshop on Statistical Signal Processing, pp. 444–447. Singapore (2001)Google Scholar
  15. 15.
    Choi, S., Cichocki, A.: Correlation matching approach to source sepration in the presence of spatially correlated noise. In: Proceedings of the IEEE International Symposium on Signal Processing and Applications. Kuala-Lumpur, Malaysia (2001)Google Scholar
  16. 16.
    Amari, S.: Natural gradient works efficiently in learning. Neural Comput. 10(2), 251–276 (1998)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)CrossRefGoogle Scholar
  18. 18.
    Choi, S., Cichocki, A., Amari, S.: Flexible independent component analysis. J. VLST Signal Process Syst. Signal Image Video Technol. 26(1/2), 25–38 (2000)MATHGoogle Scholar
  19. 19.
    Amari, S., Chen, T.P., Cichocki, A.: Stability analysis of learning algorithms for blind source separation. Neural Netw. 10(8), 1345–1351 (1997)CrossRefGoogle Scholar
  20. 20.
    Lewicki, M.S., Sejnowski, T.: Learning overcomplete representation. Neural Comput. 12(2), 337–365 (2000)CrossRefGoogle Scholar
  21. 21.
    Attias, H.: Independent factor analysis. Neural Comput. 11, 803–851 (1999)CrossRefGoogle Scholar
  22. 22.
    Welling, M., Weber, M.: A constrained EM algorithm for independent component analysis. Neural Comput. 13, 677–689 (2001)MATHCrossRefGoogle Scholar
  23. 23.
    Miskin, J.W., MacKay, D.J.C.: Ensemble learning for blind source separation. In: S. Roberts, R. Everson (eds.) Independent Component Analysis: Principles and Practice, pp. 209–233. Cambridge University Press (2001)Google Scholar
  24. 24.
    Bach, F., Jordan, M.I.: Kernel independent component analysis. J. Mach Learn Res. 3, 1–48 (2002)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Plumbley, M.D.: Algorithms for nonnegative independent component anlaysis. IEEE Trans. Neural Netw. 14(3), 534–543 (2003)CrossRefGoogle Scholar
  26. 26.
    Lee, B.I., Lee, J.S., Lee, D.S., Kang, W.J., Lee, J.J., Choi, S.: A clinical application of ensemble ICA to the quantification of myocardial blood flow in dynamic PET. J. VLST Signal Process Syst. Signal Image Video Technol. 49, 233–241 (2007)CrossRefGoogle Scholar
  27. 27.
    Matsuoka, K., Ohya, M., Kawamoto, M.: A neural net for blind separation of nonstationary signals. Neural Netw. 8(3), 411–419 (1995)CrossRefGoogle Scholar
  28. 28.
    Choi, S., Cichocki, A., Amari, S.: Equivariant nonstationary source separation. Neural Netw. 15(1), 121–130 (2002)CrossRefGoogle Scholar
  29. 29.
    Li, Y., Cichocki, A., Amari, S.: Blind estimation of channel parameters and source components for EEG signals: A sparse factorization approach. IEEE Trans. Neural Netw. 17(2), 419–431 (2006)CrossRefGoogle Scholar
  30. 30.
    Hyvärinen, A., Hoyer, P.: Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Comput. 12(7), 1705–1720 (2000)CrossRefGoogle Scholar
  31. 31.
    Bach, F.R., Jordan, M.I.: Beyond independent components: Trees and clusters. J Mach Learn Res. 4, 1205–1233 (2003)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Seungjin Choi
    • 1
  1. 1.Department of Computer SciencePohang University of Science and Technology Korea