Independent Subspace Analysis on Innovations

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3720)


Independent subspace analysis (ISA) that deals with multi-dimensional independent sources, is a generalization of independent component analysis (ICA). However, all known ISA algorithms may become ineffective when the sources possess temporal structure. The innovation process instead of the original mixtures has been proposed to solve ICA problems with temporal dependencies. Here we show that this strategy can be applied to ISA as well. We demonstrate the idea on a mixture of 3D processes and also on a mixture of facial pictures used as two-dimensional deterministic sources. ISA on innovations was able to find the original subspaces, while plain ISA was not.


Independent Component Analysis Innovation Process Independent Component Analysis Blind Source Separation Permutation Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Jutten, C., Herault, J.: Blind separation of sources: An adaptive algorithm based on neuromimetic architecture. Signal Proc. 24, 1–10 (1991)zbMATHCrossRefGoogle Scholar
  2. 2.
    Comon, P.: Independent component analysis, a new concept? Signal Proc. 36, 287–314 (1994)zbMATHCrossRefGoogle Scholar
  3. 3.
    Bell, A.J., Sejnowski, T.J.: An information maximisation approach to blind separation and blind deconvolution. Neural Comp. 7, 1129–1159 (1995)CrossRefGoogle Scholar
  4. 4.
    Bell, A.J., Sejnowski, T.J.: The independent components of natural scenes are edge filters. Vision Research 37, 3327–3338 (1997)CrossRefGoogle Scholar
  5. 5.
    Hyvärinen, A.: Sparse code shrinkage: Denoising of nongaussian data by maximum likelihood estimation. Neural Comp. 11, 1739–1768 (1999)CrossRefGoogle Scholar
  6. 6.
    Kiviluoto, K., Oja, E.: Independent component analysis for parallel financial time series. In: Proc. of ICONIP 1998, vol. 2, pp. 895–898 (1998)Google Scholar
  7. 7.
    Makeig, S., Bell, A.J., Jung, T.P., Sejnowski, T.J.: Independent component analysis of electroencephalographic data. In: Proc. of NIPS, vol. 8, pp. 145–151 (1996)Google Scholar
  8. 8.
    Vigário, R., Jousmaki, V., Hamalainen, M., Hari, R., Oja, E.: Independent component analysis for identification of artifacts in magnetoencephalographic recordings. In: Proc. of NIPS, vol. 10, pp. 229–235 (1997)Google Scholar
  9. 9.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley, New York (2001)CrossRefGoogle Scholar
  10. 10.
    Cardoso, J.: Multidimensional independent component analysis. In: Proc. of ICASSP 1998, Seattle, WA, p. 1941 (1998)Google Scholar
  11. 11.
    Hyvärinen, A., Hoyer, P.: Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Comp. 12, 1705–1720 (2000)CrossRefGoogle Scholar
  12. 12.
    Akaho, S., Kiuchi, Y., Umeyama, S.: MICA: Multimodal independent component analysis. In: Proc. of IJCNN, pp. 927–932 (1999)Google Scholar
  13. 13.
    Vollgraf, R., Obermayer, K.: Multi-dimensional ICA to separate correlated sources. In: Proc. of NIPS, vol. 14, pp. 993–1000 (2001)Google Scholar
  14. 14.
    Bach, F.R., Jordan, M.I.: Finding clusters in independent component analysis. In: Proc. of ICA 2003, pp. 891–896 (2003)Google Scholar
  15. 15.
    Póczos, B., Lőrincz, A.: Independent subspace analysis using k-nearest neighborhood distances. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 163–168. Springer, Heidelberg (2005)Google Scholar
  16. 16.
    Póczos, B., Lőrincz, A.: Independent subspace analysis using geodesic spanning trees. In: Proc. of ICML, Bonn, Germany (2005) (accepted)Google Scholar
  17. 17.
    Hyvärinen, A.: Independent component analysis for time-dependent stochastic processes. In: Proc. of ICANN, Skövde, Sweden, pp. 541–546 (1998)Google Scholar
  18. 18.
    Choi, S.: Acoustic source separation: Fundamental issues. In: Proc. of ICSP, Seoul, Korea, pp. 505–510 (1999)Google Scholar
  19. 19.
    Cheung, Y., Xu, L.: Dual multivariate auto-regressive modeling in state space for temporal signal separation. IEEE Trans. on Sys. Man. and Cyb. 33, 386–398Google Scholar
  20. 20.
    Yukich, J.E.: Probability Theory of Classical Euclidean Optimization Problems. In: Estublier, J. (ed.) SCM 1999. LNCS, vol. 1675. Springer, Heidelberg (1999)Google Scholar
  21. 21.
    Costa, J.A., Hero, A.O.: Manifold learning using k-nearest neighbor graphs. In: Proc. of ICASSP, Montreal, Canada (2004)Google Scholar
  22. 22.
    Amari, S., Cichocki, A., Yang, H.: A new learning algorithm for blind source separation. In: Proc. of NIPS, vol. 8, pp. 757–763 (1996)Google Scholar
  23. 23.
    Schneider, T., Neumaier, A.: Algorithm 808: ARfit - A Matlab package for the estimation of parameters and eigenmodes of multivariate autoregressive models. ACM Trans. Math. Softw. 27, 58–65 (2001)zbMATHCrossRefGoogle Scholar
  24. 24.
    Penny, W.D., Everson, R., Roberts, S.J.: Hidden Markov independent component analysis. In: Giroliami, M. (ed.) Advances in Independent Component Analysis, pp. 3–22. Springer, Heidelberg (2000)Google Scholar
  25. 25.
    Pearlmutter, B.A., Parra, L.C.: A context-sensitive generalization of ICA. In: Proc. of ICONIP 1996, Hong Kong, pp. 151–157 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  1. 1.Eötvös Loránd UniversityBudapestHungary

Personalised recommendations