Independent Process Analysis Without a Priori Dimensional Information

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4666)


Recently, several algorithms have been proposed for independent subspace analysis where hidden variables are i.i.d. processes. We show that these methods can be extended to certain AR, MA, ARMA and ARIMA tasks. Central to our paper is that we introduce a cascade of algorithms, which aims to solve these tasks without previous knowledge about the number and the dimensions of the hidden processes. Our claim is supported by numerical simulations. As an illustrative application where the dimensions of the hidden variables are unknown, we search for subspaces of facial components.


Mutual Information Hide Variable Independent Component Analysis Full Column Rank Separation Theorem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cardoso, J.F.: Multidimensional independent component analysis. In: Proc. of ICASSP, vol. 4, pp. 1941–1944 (1998)Google Scholar
  2. 2.
    Vollgraf, R., Obermayer, K.: Multi-dimensional ICA to separate correlated sources. In: Proc. of NIPS, vol. 14, pp. 993–1000. MIT Press, Cambridge (2001)Google Scholar
  3. 3.
    Stögbauer, H., Kraskov, A., Astakhov, S.A., Grassberger, P.: Least dependent component analysis based on mutual information. Phys. Rev. E, 70 (2004)Google Scholar
  4. 4.
    Bach, F.R., Jordan, M.I.: Beyond independent components: Trees and clusters. Journal of Machine Learning Research 4, 1205–1233 (2003)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Theis, F.J.: Blind signal separation into groups of dependent signals using joint block diagonalization. In: Proc. of ISCAS, pp. 5878–5881 (2005)Google Scholar
  6. 6.
    Hyvärinen, A., Köster, U.: FastISA: A fast fixed-point algorithm for independent subspace analysis. In: Proc. of ESANN, Evere, Belgium (2006)Google Scholar
  7. 7.
    Nolte, G., Meinecke, F.C., Ziehe, A., Müller, K.R.: Identifying interactions in mixed and noisy complex systems. Physical Review E, 73 (2006)Google Scholar
  8. 8.
    Póczos, B., Lőrincz, A.: Independent subspace analysis using geodesic spanning trees. In: Proc. of ICML, pp. 673–680. ACM Press, New York (2005)CrossRefGoogle Scholar
  9. 9.
    Hyvärinen, A.: Independent component analysis for time-dependent stochastic processes. In: Proc. of ICANN, pp. 541–546. Springer, Berlin (1998)Google Scholar
  10. 10.
    Póczos, B., Takács, B., Lőrincz, A.: Independent subspace analysis on innovations. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 698–706. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Choi, S., Cichocki, A., Park, H.-M., Lee, S.-Y.: Blind source separation and independent component analysis. Neural Inf. Proc. Lett. Rev. 6, 1–57 (2005)Google Scholar
  12. 12.
    Mills, T.C.: Time Series Techniques for Economists. Cambridge University Press, Cambridge (1990)Google Scholar
  13. 13.
    Theis, F.J.: Uniqueness of complex and multidimensional independent component analysis. Signal Processing 84(5), 951–956 (2004)CrossRefGoogle Scholar
  14. 14.
    Szabó, Z., Póczos, B., Lőrincz, A.: Separation theorem for \(\mathbb{K}\)-independent subspace analysis with sufficient conditions. Technical report (2006),
  15. 15.
    Theis, F.J.: Towards a general independent subspace analysis. In: Proc. of NIPS, vol. 19 (2007)Google Scholar
  16. 16.
    Bach, F.R., Jordan, M.I.: Finding clusters in Independent Component Analysis. In: Proc. of ICA 2003, pp. 891–896 (2003)Google Scholar
  17. 17.
    ó, Z.S., Póczos, B., Lőrincz, A.: Cross-entropy optimization for independent process analysis. In: Rosca, J., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 909–916. Springer, Heidelberg (2006)Google Scholar
  18. 18.
    Abed-Meraim, K., Belouchrani, A.: Algorithms for joint block diagonalization. In: Proc. of EUSIPCO, pp. 209–212 (2004)Google Scholar
  19. 19.
    Comon, P.: Independent Component Analysis, a new concept (Special issue on Higher-Order Statistics). Signal Processing, Elsevier 36(3), 287–314 (1994)zbMATHGoogle Scholar
  20. 20.
    Yu, S., Shi, J.: Multiclass spectral clustering. In: Proc. of ICCV (2003)Google Scholar
  21. 21.
    Rajagopal, R., Potter, L.C.: Multivariate MIMO FIR inverses. IEEE Transactions on Image Processing 12, 458–465 (2003)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Gorokhov, A., Loubaton, P.: Blind identification of MIMO-FIR systems: A generalized linear prediction approach. Signal Processing 73, 105–124 (1999)zbMATHCrossRefGoogle Scholar
  23. 23.
    Neumaier, A., Schneider, T.: Estimation of parameters and eigenmodes of multivariate autoregressive models. ACM Trans. on Math. Soft. 27(1), 27–57 (2001)zbMATHCrossRefGoogle Scholar
  24. 24.
    Bartlett, M., Movellan, J., Sejnowski, T.: Face recognition by independent component analysis. IEEE Trans. on Neural Networks 13(6), 1450–1464 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  1. 1.Department of Information Systems, Eötvös Loránd University, BudapestHungary

Personalised recommendations