ICA and ICAMM Methods

Chapter
Part of the Springer Theses book series (Springer Theses, volume 4)

Abstract

Independent component analysis (ICA) aims to separate hidden sources from their observed linear mixtures without any prior knowledge. The only assumption about the sources is that they are mutually independent. Thus, the goal is blind source estimation; although it has been recently alleviated by incorporating prior knowledge about the sources into the ICA model in the so-called semi-blind source separation. This technique has been widely used in many fields of application such as telecommunications, bioengineering, and material testing.

Keywords

Mutual Information Independent Component Analysis Canonical Correlation Analysis Independent Component Analysis Leibler Divergence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    C. Jutten, J. Herault, Une solution neuromimétique au problème de séparation de sources. Traitement du Signal 5(6), 389–404 (1989)Google Scholar
  2. 2.
    C. Jutten, J. Herault, Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture. Signal Process. 24, 1–10 (1991)CrossRefMATHGoogle Scholar
  3. 3.
    C. Jutten, J. Herault, Blind separation of sources, part II: problems statement. Signal Process. 24, 11–20 (1991)CrossRefMATHGoogle Scholar
  4. 4.
    C. Jutten, J. Herault, Blind separation of sources, part III: stability analysis. Signal Process. 24, 21–29 (1991)CrossRefGoogle Scholar
  5. 5.
    A. Hyvärinen, J. Karhunen, E. Oja, Independent Component Analysis (Wiley, New York, 2001)CrossRefGoogle Scholar
  6. 6.
    C.W. Hesse, C.J. James, On semi-blind source separation using spatial constraints with applications in EEG Analysis. IEEE Trans. Biomed. Eng. 53(12-1), 2525–2534 (2006)CrossRefGoogle Scholar
  7. 7.
    J. Even, K. Sugimoto, An ICA approach to semi-blind identification of strictly proper systems based on interactor polynomial matrix. Int. J. Robust Nonlinear Control 17, 752–768 (2007)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Z. Ding, T. Ratnarajah, C.F.N. Cowan, HOS-based semi-blind spatial equalization for MIMO rayleigh fading channels. IEEE Trans. Signal Process. 56(1), 248–255 (2008)CrossRefMathSciNetGoogle Scholar
  9. 9.
    A. Cichocki, S. Amari, Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications (Wiley, New York, 2001)Google Scholar
  10. 10.
    T.W. Lee, Independent Component Analysis—Theory and Applications (Kluwer Academic Publishers, Boston, 1998)MATHGoogle Scholar
  11. 11.
    S. Roberts, R. Everson, Independent Component Analysis—Principles and Practice (Cambridge University Press, Cambridge, 2001)MATHGoogle Scholar
  12. 12.
    A. Cichocki, R. Zdunek, A.H. Phan, S. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation (Wiley, Hoboken, 2009)Google Scholar
  13. 13.
    P. Comon, C. Jutten (eds.), Handbook of Blind Source Separation Independent Component Analysis and Applications (Academic Press, Oxford, 2010)Google Scholar
  14. 14.
    M.S. Pedersen, J. Larsen, U. Kjems, L.C. Parra, A Survey of Convolutive Blind Source Separation Methods, ed. by J. Benesty, A. Huang. Multichannel Speech Processing Handbook, Chapter 51 (Springer, Berlin, 2007), pp. 1065–1084Google Scholar
  15. 15.
    H. Buchner, R. Aichner, W. Kellerman, TRINICON: a versatile framework for multichannel blind signal processing. in Proceedings of 29th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, pp. III-889–892, Montreal, Canada, 2004Google Scholar
  16. 16.
    W. Kellerman, H. Buchner, R. Aichner, Separating convolutive mixture with TRINICON. in Proceedings of 31st IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, pp. V-961–964, Toulouse, France, 2006Google Scholar
  17. 17.
    P. Comon, Independent component analysis—a new concept? Signal Process. 36(3), 287–314 (1994)CrossRefMATHGoogle Scholar
  18. 18.
    S. Amari, A. Cichocki, H. Yang, A new learning algorithm for blind signal separation, Advances in Neural Information Processing Systems, vol 8 (MIT Press, Cambridge, 1996), pp. 752–763Google Scholar
  19. 19.
    S. Amari, J.F. Cardoso, Blind source separation-semiparametric statistical approach. IEEE Trans. Signal Process. 45(11), 2692–2700 (1997)CrossRefGoogle Scholar
  20. 20.
    A. Hyvärinen, E. Oja, A fast fixed-point algorithm for independent component analysis. Neural Comput. 9(7), 1483–1492 (1998)CrossRefGoogle Scholar
  21. 21.
    D.T. Pham, P. Garrat, Blind separation of mixture of independent sources through a quasi-maximum likelihood approach. IEEE Trans. Signal Process. 45(7), 1712–1725 (1997)CrossRefMATHGoogle Scholar
  22. 22.
    A. Hyvarinen, Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999)CrossRefGoogle Scholar
  23. 23.
    T.W. Lee, M. Girolami, T.J. Sejnowski, Independent component analysis using an extended InfoMax algorithm for mixed sub-gaussian and super-gaussian sources. Neural Comput. 11(2), 417–441 (1999)CrossRefGoogle Scholar
  24. 24.
    S.I. Amari, T.P. Chen, A. Cichocki, Nonholonomic orthogonal learning algorithms for blind source separation. Neural Comput. 12, 1463–1484 (2000)CrossRefGoogle Scholar
  25. 25.
    J.F. Cardoso, Dependence, correlation and gaussianity in independent component analysis. J. Mach. Learn. Res. 4, 1177–1203 (2003)MathSciNetGoogle Scholar
  26. 26.
    A. Chen, P.J. Bickel, Consistent independent component analysis and prewhitening. IEEE Trans. Signal Process. 53(10), 3625–3632 (2005)CrossRefMathSciNetGoogle Scholar
  27. 27.
    W. Liu, D.P. Mandic, A. Cichocki, Blind source extraction based on a linear predictor. IET Signal Process. 1(1), 29–34 (2007)CrossRefGoogle Scholar
  28. 28.
    J.F. Cardoso, Blind signal separation: statistical principles. Proceedings of the IEEE. Special Issue on Blind Identification and Estimation, vol 9, pp. 2009–2025, 1998Google Scholar
  29. 29.
    A. Chen, P.J. Bickel, Efficient independent component analysis. Annals Stat. 34(6), 2825–2855 (2006)CrossRefMATHMathSciNetGoogle Scholar
  30. 30.
    F. Meinecke, A. Ziehe, M. Kawanabe, K.R. Müller, Resampling approach to estimate the stability of one-dimensional or multidimensional independent components. IEEE Trans. Biomed. Eng. 49(12), 1514–1525 (2002)CrossRefGoogle Scholar
  31. 31.
    J. Himberg, A. Hyvärinen, F. Esposito, Validating the independent components of neuroimaging time-series via clustering and visualization. Neuroimage 22(3), 1214–1222 (2004)CrossRefGoogle Scholar
  32. 32.
    A.J. Bell, T.J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159 (1995)CrossRefGoogle Scholar
  33. 33.
    J.F. Cardoso, A. Souloumiac, Blind beamforming for non gaussian signals. IEE Proc.-F 140(6), 362–370 (1993)Google Scholar
  34. 34.
    A. Ziehe, K.R. Müller, TDSEP- an efficient algorithm for blind separation using time structure. Proceedings of the 8th International Conference on Artificial Neural Networks, ICANN’98, Perspectives in Neural Computing, pp. 675–680, 1998Google Scholar
  35. 35.
    J.P. Nadal, N. Parga, Non linear neurons in the noise limit: a factorial code maximizes information transfer. Netw. Comput. Neural Syst. 5(3), 565–585 (1994)CrossRefMATHGoogle Scholar
  36. 36.
    J.F. Cardoso, InfoMax and maximum likelihood for blind source separation. IEEE Signal Process. Lett. 4(4), 112–114 (1997)CrossRefGoogle Scholar
  37. 37.
    S.I. Amari, Natural gradient works efficiently in learning. Neural Comput. 10, 251–276 (1998)CrossRefGoogle Scholar
  38. 38.
    J.F. Cardoso, B. Laheld, Equivariant adaptive source separation. IEEE Trans. Signal Process. 45(2), 434–444 (1996)Google Scholar
  39. 39.
    C. Nikias, A. Petropulu, Higher-order Spectral Analysis—A Nonlinear Signal Processing Framework (Prentice Hall, Englewood Cliffs, 1993)Google Scholar
  40. 40.
    J.F. Cardoso, P. Comon, Tensor-based independent component analysis. Proceedings of the Fifth European Signal Processing Conference, EUSIPCO 1990, pp. 673–676, 1990Google Scholar
  41. 41.
    J.F. Cardoso, A. Souloumiac, Jacobi angles for simultaneous diagonalization. SIAM J. Matrix Anal. Appl. 17(1), 161–164 (1996)CrossRefMATHMathSciNetGoogle Scholar
  42. 42.
    J.F. Cardoso, High-order contrasts for independent component analysis. Neural Comput. 11(1), 157–192 (1999)CrossRefMathSciNetGoogle Scholar
  43. 43.
    A. Ziehe, K.R. Muller, G. Nolte, B.M. Mackert, G. Curio, Artifact reduction in magnetoneurography based on time-delayed second order correlations. IEEE Trans. Biomed. Eng. 41, 75–87 (2000)CrossRefGoogle Scholar
  44. 44.
    A. Belouchrani, K. Abed-Meraim, J.F. Cardoso, E. Moulines, A blind source separation technique using second-order statistics. IEEE Trans. Signal Process. 45, 434–444 (1997)CrossRefGoogle Scholar
  45. 45.
    R. Boscolo, H. Pan, Independent component analysis based on nonparametric density estimation. IEEE Trans. Neural Netw. 15(1), 55–65 (2004)CrossRefGoogle Scholar
  46. 46.
    R. Boustany, J. Antoni, Blind extraction of a cyclostationary signal using reduced-rank cyclic regression—a unifying approach. Mech. Syst. Signal Process. 22, 520–541 (2008)CrossRefGoogle Scholar
  47. 47.
    J. Even, K. Sugimoto, An ICA approach to semi-blind identification of strictly proper systems based on interactor polynomial matrix. Int. J. Robust Nonlinear Control 17, 752–768 (2007)CrossRefMATHMathSciNetGoogle Scholar
  48. 48.
    F.R. Bach, M.I. Jordan, Kernel independent component analysis. J. Mach. Learn. Res. 3, 1–48 (2002)MathSciNetGoogle Scholar
  49. 49.
    T. Hastie, R. Tibshirani, Independent Component Analysis Through Product Density Estimation, Technical Report, Stanford University, 2002Google Scholar
  50. 50.
    E.G. Learned-Miller, J.W. Fisher, ICA using spacings estimates of entropy. J. Mach. Learn. Res. 4, 1271–1295 (2003)MathSciNetGoogle Scholar
  51. 51.
    A. Samarov, A. Tsybakov, Nonparametric independent component analysis. Bernoulli 10(4), 565–582 (2004)CrossRefMATHMathSciNetGoogle Scholar
  52. 52.
    B.W. Silverman, Density Estimation for Statistics and Data Analysis (Chapman and Hall, London, 1985)Google Scholar
  53. 53.
    R. Choudrey, S. Roberts, Variational mixture of bayesian independent component analysers. Neural Comput. 15(1), 213–252 (2002)CrossRefGoogle Scholar
  54. 54.
    M.E. Tipping, C.M. Bishop, Mixtures of probabilistic principal component analyzers. Neural Comput. 11(2), 443–482 (1999)CrossRefGoogle Scholar
  55. 55.
    Z. Ghahramani, M. Beal, Variational inference for Bayesian mixtures of factor analysers. Adv. Neural Inf. Process. Syst. 12, 449–445 (2000)Google Scholar
  56. 56.
    C. Archambeau, N. Delannay, M. Verleysen, Mixtures of robust probabilistic principal component analyzers. Neurocomputing 71(7–9), 1274–1282 (2008)CrossRefGoogle Scholar
  57. 57.
    M. Svensén, C.M. Bishop, Robust Bayesian mixture modelling. Neurocomputing 64, 235–252 (2005)CrossRefGoogle Scholar
  58. 58.
    T.W. Lee, M.S. Lewicki, T.J. Sejnowski, ICA mixture models for unsupervised classification of non-gaussian classes and automatic context switching in blind signal separation. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1078–1089 (2000)CrossRefGoogle Scholar
  59. 59.
    S. Roberts, W.D. Penny, Mixtures of independent component analyzers. in Proceedings of ICANN2001, Vienna, August 2001, pp. 527–534Google Scholar
  60. 60.
    J.A. Palmer, K. Kreutz-Delgado, S. Makeig, An Independent Component Analysis Mixture Model with Adaptive Source Densities, Technical Report, UCSD, 2006Google Scholar
  61. 61.
    K. Chan, T.W. Lee, T.J. Sejnowski, Variational learning of clusters of undercomplete nonsymmetric independent components. J. Mach. Learn. Res. 3, 99–114 (2002)MathSciNetGoogle Scholar
  62. 62.
    C.T. Lin, W.C. Cheng, S.F. Liang, An on-line ICA-mixture-model-based self-constructing fuzzy neural network. IEEE Trans. Circuits Syst. 52(1), 207–221 (2005)CrossRefMathSciNetGoogle Scholar
  63. 63.
    T. Yoshida, M. Sakagami, K. Yamazaki, T. Katura, M. Iwamoto, N. Tanaka, Extraction of neural activity from in vivo optical recordings using multiple independent component analysis. IEEJ Trans. Electron. Inf. Syst. 127(10), 1642–1650 (2007)Google Scholar
  64. 64.
    J.A. Palmer, S. Makeig, K. Kreutz-Delgado, B.D. Rao, Newton method for the ICA mixture model. Proceedings of the 33rd IEEE International Conference on Acoustics, Speech, and Signal, pp. 1805–1808, Las Vegas, USA, 2008Google Scholar
  65. 65.
    N.H. Mollah, M. Minami, S. Eguchi, Exploring latent structure of mixture ICA models by the minimum ß-Divergence method. Neural Comput. 18, 166–190 (2005)CrossRefGoogle Scholar
  66. 66.
    D. Erdogmus, J.C. Principe, From linear adaptive filtering to nonlinear information processing—the design and analysis of information processing systems. IEEE Signal Process. Mag. 23(6), 14–33 (2006)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Department of Communications, School of Telecommunication EngineeringPolytechnic University of ValenciaValenciaSpain

Personalised recommendations