Skip to main content
SpringerLink
Log in
Book cover

Handbook of Natural Computing pp 435–459Cite as

Independent Component Analysis

  • Reference work entry

Abstract

Independent component analysis (ICA) is a statistical method, the goal of which is to decompose multivariate data into a linear sum of non-orthogonal basis vectors with coefficients (encoding variables, latent variables, and hidden variables) being statistically independent. ICA generalizes widely used subspace analysis methods such as principal component analysis (PCA) and factor analysis, allowing latent variables to be non-Gaussian and basis vectors to be non-orthogonal in general. ICA is a density-estimation method where a linear model is learned such that the probability distribution of the observed data is best captured, while factor analysis aims at best modeling the covariance structure of the observed data. We begin with a fundamental theory and present various principles and algorithms for ICA.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   999.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   1,199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Amari S (1998) Natural gradient works efficiently in learning. Neural Comput 10(2):251–276

    Article  MathSciNet  Google Scholar 

  • Amari S, Cardoso JF (1997) Blind source separation: semiparametric statistical approach. IEEE Trans Signal Process 45:2692–2700

    Article  Google Scholar 

  • Amari S, Chen TP, Cichocki A (1997) Stability analysis of learning algorithms for blind source separation. Neural Networks 10(8):1345–1351

    Article  Google Scholar 

  • Amari S, Cichocki A (1998) Adaptive blind signal processing – neural network approaches. Proc IEEE (Special Issue on Blind Identification and Estimation) 86(10):2026–2048

    Google Scholar 

  • Amari S, Cichocki A, Yang HH (1996) A new learning algorithm for blind signal separation. In: Touretzky DS, Mozer MC, Hasselmo ME (eds) Advances in neural information processing systems (NIPS), vol 8. MIT Press, Cambridge, pp 757–763

    Google Scholar 

  • Attias H (1999) Independent factor analysis. Neural Comput 11:803–851

    Article  Google Scholar 

  • Bach F, Jordan MI (2002) Kernel independent component analysis. JMLR 3:1–48

    MathSciNet  Google Scholar 

  • Bach FR, Jordan MI (2003) Beyond independent components: trees and clusters. JMLR 4:1205–1233

    MathSciNet  Google Scholar 

  • Bell A, Sejnowski T (1995) An information maximisation approach to blind separation and blind deconvolution. Neural Comput 7:1129–1159

    Article  Google Scholar 

  • Belouchrani A, Abed‐Merain K, Cardoso JF, Moulines E (1997) A blind source separation technique using second order statistics. IEEE Trans Signal Process 45:434–444

    Article  Google Scholar 

  • Cardoso JF (1989) Source separation using higher‐order moments. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing (ICASSP), Paris, France, 23–26 May 1989

    Google Scholar 

  • Cardoso JF (1997) Infomax and maximum likelihood for source separation. IEEE Signal Process Lett 4(4):112–114

    Article  Google Scholar 

  • Cardoso JF, Laheld BH (1996) Equivariant adaptive source separation. IEEE Trans Signal Process 44(12):3017–3030

    Article  Google Scholar 

  • Cardoso JF, Souloumiac A (1993) Blind beamforming for non Gaussian signals. IEE Proc‐F 140(6):362–370

    Google Scholar 

  • Chang C, Ding Z, Yau SF, Chan FHY (2000) A matrix‐pencil approach to blind separation of colored nonstationary signals. IEEE Trans Signal Process 48(3):900–907

    Article  Google Scholar 

  • Choi S (2002) Adaptive differential decorrelation: a natural gradient algorithm. In: Proceedings of the international conference on artificial neural networks (ICANN), Madrid, Spain. Lecture notes in computer science, vol 2415. Springer, Berlin, pp 1168–1173

    Google Scholar 

  • Choi S (2003) Differential learning and random walk model. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing (ICASSP), IEEE, Hong Kong, pp 724–727

    Google Scholar 

  • Choi S, Cichocki A (2000a) Blind separation of nonstationary and temporally correlated sources from noisy mixtures. In: Proceedings of IEEE workshop on neural networks for signal processing, IEEE, Sydney, Australia. pp 405–414

    Google Scholar 

  • Choi S, Cichocki A (2000b) Blind separation of nonstationary sources in noisy mixtures. Electron Lett 36(9):848–849

    Article  Google Scholar 

  • Choi S, Cichocki A, Amari S (2000) Flexible independent component analysis. J VLSI Signal Process 26(1/2):25–38

    Article  MATH  Google Scholar 

  • Choi S, Cichocki A, Belouchrani A (2002) Second order nonstationary source separation. J VLSI Signal Process 32:93–104

    Article  MATH  Google Scholar 

  • Choi S, Cichocki A, Park HM, Lee SY (2005) Blind source separation and independent component analysis: a review. Neural Inf Process Lett Rev 6(1): 1–57

    Google Scholar 

  • Cichocki A, Amari S (2002) Adaptive blind signal and image processing: learning algorithms and applications. Wiley, Chichester

    Book  Google Scholar 

  • Cichocki A, Unbehauen R (1996) Robust neural networks with on‐line learning for blind identification and blind separation of sources. IEEE Trans Circ Syst Fund Theor Appl 43:894–906

    Article  Google Scholar 

  • Comon P (1994) Independent component analysis, a new concept? Signal Process 36(3):287–314

    Article  MATH  Google Scholar 

  • Fukunaga K (1990) An introduction to statistical pattern recognition. Academic, New York

    Google Scholar 

  • Golub GH, Loan CFV (1993) Matrix computations, 2nd edn. Johns Hopkins, Baltimore

    Google Scholar 

  • Haykin S (2000) Unsupervised adaptive filtering: blind source separation. Prentice‐Hall

    Google Scholar 

  • Hyvärinen A (1999) Survey on independent component analysis. Neural Comput Surv 2:94–128

    Google Scholar 

  • Hyvärinen A, Hoyer P (2000) Emergence of phase‐ and shift‐invariant features by decomposition of natural images into independent feature subspaces. Neural Comput 12(7):1705–1720

    Article  Google Scholar 

  • Hyvärinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, New York

    Book  Google Scholar 

  • Hyvärinen A, Oja E (1997) A fast fixed‐point algorithm for independent component analysis. Neural Comput 9:1483–1492

    Article  Google Scholar 

  • Jutten C, Herault J (1991) Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture. Signal Process 24:1–10

    Article  MATH  Google Scholar 

  • Karhunen J (1996) Neural approaches to independent component analysis and source separation. In: Proceedings of the European symposium on artificial neural networks (ESANN), Bruges, Belgium, pp 249–266

    Google Scholar 

  • Kim S, Choi S (2005) Independent arrays or independent time courses for gene expression data. In: Proceedings of the IEEE international symposium on circuits and systems (ISCAS), Kobe, Japan, 23–26 May 2005

    Google Scholar 

  • Kosko B (1986) Differential Hebbian learning. In: Proceedings of American Institute of Physics: neural networks for computing, Snowbird. American Institute of Physics, Woodbury, pp 277–282

    Google Scholar 

  • Lee TW (1998) Independent component analysis: theory and applications. Kluwer

    MATH  Google Scholar 

  • Lee TW, Girolami M, Sejnowski T (1999) Independent component analysis using an extended infomax algorithm for mixed sub‐Gaussian and super‐Gaussian sources. Neural Comput 11(2):609–633

    Article  Google Scholar 

  • Lewicki MS, Sejnowski T (2000) Learning overcomplete representation. Neural Comput 12(2):337–365

    Article  Google Scholar 

  • Li Y, Cichocki A, Amari S (2006) Blind estimation of channel parameters and source components for EEG signals: a sparse factorization approach. IEEE Trans Neural Networ 17(2):419–431

    Article  Google Scholar 

  • Liebermeister W (2002) Linear modes of gene expression determined by independent component analysis. Bioinformatics 18(1):51–60

    Article  Google Scholar 

  • MacKay DJC (1996) Maximum likelihood and covariant algorithms for independent component analysis. Technical Report Draft 3.7, University of Cambridge, Cavendish Laboratory

    Google Scholar 

  • Matsuoka K, Ohya M, Kawamoto M (1995) A neural net for blind separation of nonstationary signals. Neural Networks 8(3):411–419

    Article  Google Scholar 

  • Miskin JW, MacKay DJC (2001) Ensemble learning for blind source separation. In: Roberts S, Everson R (eds) Independent component analysis: principles and practice. Cambridge University Press, Cambridge, UK, pp 209–233

    Google Scholar 

  • Molgedey L, Schuster HG (1994) Separation of a mixture of independent signals using time delayed correlations. Phys Rev Lett 72:3634–3637

    Article  Google Scholar 

  • Oja E (1995) The nonlinear PCA learning rule and signal separation – mathematical analysis. Technical Report A26, Helsinki University of Technology, Laboratory of Computer and Information Science

    Google Scholar 

  • Pearlmutter B, Parra L (1997) Maximum likelihood blind source separation: a context‐sensitive generalization of ICA. In: Mozer MC, Jordan MI, Petsche T (eds) Advances in neural information processing systems (NIPS), vol 9. MIT Press, Cambridge, pp 613–619

    Google Scholar 

  • Pham DT (1996) Blind separation of instantaneous mixtures of sources via an independent component analysis. IEEE Trans Signal Process 44(11): 2768–2779

    Article  Google Scholar 

  • Plumbley MD (2003) Algorithms for nonnegative independent component analysis. IEEE Trans Neural Network 14(3):534–543

    Article  Google Scholar 

  • Stone JV (2004) Independent component analysis: a tutorial introduction. MIT Press, Cambridge

    Google Scholar 

  • Stone JV, Porrill J, Porter NR, Wilkinson IW (2002) Spatiotemporal independent component analysis of event‐related fMRI data using skewed probability density functions. NeuroImage 15(2):407–421

    Article  Google Scholar 

  • Tong L, Soon VC, Huang YF, Liu R (1990) AMUSE: a new blind identification algorithm. In: Proceedings of the IEEE international symposium on circuits and systems (ISCAS), IEEE, New Orleans, pp 1784–1787

    Google Scholar 

  • Welling M, Weber M (2001) A constrained EM algorithm for independent component analysis. Neural Comput 13:677–689

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Pohang University of Science and Technology, Pohang, South Korea

    Seungjin Choi

Authors
  1. Seungjin Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seungjin Choi .

Editor information

Editors and Affiliations

  1. LIACS, Leiden University, Leiden, The Netherlands

    Grzegorz Rozenberg

  2. Computer Science Department, University of Colorado, Boulder, USA

    Grzegorz Rozenberg

  3. LIACS, Leiden University, Leiden, The Netherlands

    Thomas Bäck

  4. LIACS, Leiden University, Leiden, The Netherlands

    Joost N. Kok

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this entry

Cite this entry

Choi, S. (2012). Independent Component Analysis. In: Rozenberg, G., Bäck, T., Kok, J.N. (eds) Handbook of Natural Computing. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-92910-9_13

Download citation

Publish with us

Policies and ethics

Search

Navigation