Independent Process Analysis Without a Priori Dimensional Information

  • Barnabás Póczos
  • Zoltán Szabó
  • Melinda Kiszlinger
  • András Lőrincz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4666)

Abstract

Recently, several algorithms have been proposed for independent subspace analysis where hidden variables are i.i.d. processes. We show that these methods can be extended to certain AR, MA, ARMA and ARIMA tasks. Central to our paper is that we introduce a cascade of algorithms, which aims to solve these tasks without previous knowledge about the number and the dimensions of the hidden processes. Our claim is supported by numerical simulations. As an illustrative application where the dimensions of the hidden variables are unknown, we search for subspaces of facial components.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cardoso, J.F.: Multidimensional independent component analysis. In: Proc. of ICASSP, vol. 4, pp. 1941–1944 (1998)Google Scholar
  2. 2.
    Vollgraf, R., Obermayer, K.: Multi-dimensional ICA to separate correlated sources. In: Proc. of NIPS, vol. 14, pp. 993–1000. MIT Press, Cambridge (2001)Google Scholar
  3. 3.
    Stögbauer, H., Kraskov, A., Astakhov, S.A., Grassberger, P.: Least dependent component analysis based on mutual information. Phys. Rev. E, 70 (2004)Google Scholar
  4. 4.
    Bach, F.R., Jordan, M.I.: Beyond independent components: Trees and clusters. Journal of Machine Learning Research 4, 1205–1233 (2003)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Theis, F.J.: Blind signal separation into groups of dependent signals using joint block diagonalization. In: Proc. of ISCAS, pp. 5878–5881 (2005)Google Scholar
  6. 6.
    Hyvärinen, A., Köster, U.: FastISA: A fast fixed-point algorithm for independent subspace analysis. In: Proc. of ESANN, Evere, Belgium (2006)Google Scholar
  7. 7.
    Nolte, G., Meinecke, F.C., Ziehe, A., Müller, K.R.: Identifying interactions in mixed and noisy complex systems. Physical Review E, 73 (2006)Google Scholar
  8. 8.
    Póczos, B., Lőrincz, A.: Independent subspace analysis using geodesic spanning trees. In: Proc. of ICML, pp. 673–680. ACM Press, New York (2005)CrossRefGoogle Scholar
  9. 9.
    Hyvärinen, A.: Independent component analysis for time-dependent stochastic processes. In: Proc. of ICANN, pp. 541–546. Springer, Berlin (1998)Google Scholar
  10. 10.
    Póczos, B., Takács, B., Lőrincz, A.: Independent subspace analysis on innovations. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 698–706. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Choi, S., Cichocki, A., Park, H.-M., Lee, S.-Y.: Blind source separation and independent component analysis. Neural Inf. Proc. Lett. Rev. 6, 1–57 (2005)Google Scholar
  12. 12.
    Mills, T.C.: Time Series Techniques for Economists. Cambridge University Press, Cambridge (1990)Google Scholar
  13. 13.
    Theis, F.J.: Uniqueness of complex and multidimensional independent component analysis. Signal Processing 84(5), 951–956 (2004)CrossRefGoogle Scholar
  14. 14.
    Szabó, Z., Póczos, B., Lőrincz, A.: Separation theorem for \(\mathbb{K}\)-independent subspace analysis with sufficient conditions. Technical report (2006), http://arxiv.org/abs/math.ST/0608100
  15. 15.
    Theis, F.J.: Towards a general independent subspace analysis. In: Proc. of NIPS, vol. 19 (2007)Google Scholar
  16. 16.
    Bach, F.R., Jordan, M.I.: Finding clusters in Independent Component Analysis. In: Proc. of ICA 2003, pp. 891–896 (2003)Google Scholar
  17. 17.
    ó, Z.S., Póczos, B., Lőrincz, A.: Cross-entropy optimization for independent process analysis. In: Rosca, J., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 909–916. Springer, Heidelberg (2006)Google Scholar
  18. 18.
    Abed-Meraim, K., Belouchrani, A.: Algorithms for joint block diagonalization. In: Proc. of EUSIPCO, pp. 209–212 (2004)Google Scholar
  19. 19.
    Comon, P.: Independent Component Analysis, a new concept (Special issue on Higher-Order Statistics). Signal Processing, Elsevier 36(3), 287–314 (1994)MATHGoogle Scholar
  20. 20.
    Yu, S., Shi, J.: Multiclass spectral clustering. In: Proc. of ICCV (2003)Google Scholar
  21. 21.
    Rajagopal, R., Potter, L.C.: Multivariate MIMO FIR inverses. IEEE Transactions on Image Processing 12, 458–465 (2003)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Gorokhov, A., Loubaton, P.: Blind identification of MIMO-FIR systems: A generalized linear prediction approach. Signal Processing 73, 105–124 (1999)MATHCrossRefGoogle Scholar
  23. 23.
    Neumaier, A., Schneider, T.: Estimation of parameters and eigenmodes of multivariate autoregressive models. ACM Trans. on Math. Soft. 27(1), 27–57 (2001)MATHCrossRefGoogle Scholar
  24. 24.
    Bartlett, M., Movellan, J., Sejnowski, T.: Face recognition by independent component analysis. IEEE Trans. on Neural Networks 13(6), 1450–1464 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Barnabás Póczos
    • 1
  • Zoltán Szabó
    • 1
  • Melinda Kiszlinger
    • 1
  • András Lőrincz
    • 1
  1. 1.Department of Information Systems, Eötvös Loránd University, BudapestHungary

Personalised recommendations