Cross-Entropy Optimization for Independent Process Analysis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3889)


We treat the problem of searching for hidden multi-dimensional independent auto-regressive processes. First, we transform the problem to Independent Subspace Analysis (ISA). Our main contribution concerns ISA. We show that under certain conditions, ISA is equivalent to a combinatorial optimization problem. For the solution of this optimization we apply the cross-entropy method. Numerical simulations indicate that the cross-entropy method can provide considerable improvements over other state-of-the-art methods.


Independent Component Analysis Travelling Salesman Problem Independent Component Analysis Blind Source Separation Permutation Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Choi, S., Cichocki, A., Park, H.M., Lee, S.Y.: Blind Source Separation and Independent Component Analysis. Neural Inf. Proc. Lett. and Reviews (2005)Google Scholar
  2. 2.
    Cardoso, J.: Multidimensional Independent Component Analysis. In: ICASSP 1998, Seattle, WA (1998)Google Scholar
  3. 3.
    Akaho, S., Kiuchi, Y., Umeyama, S.: MICA: Multimodal Independent Component Analysis. In: IJCNN, pp. 927–932 (1999)Google Scholar
  4. 4.
    Póczos, B., Takács, B., Lőrincz, A.: Independent Subspace Analysis on Innovations. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 698–706. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Hyvärinen, A.: Independent Component Analysis for Time-dependent Stochastic Processes. In: ICANN 1998, pp. 541–546 (1998)Google Scholar
  6. 6.
    Vollgraf, R., Obermayer, K.: Multi-Dimensional ICA to Separate Correlated Sources. In: NIPS, vol. 14, pp. 993–1000 (2001)Google Scholar
  7. 7.
    Bach, F.R., Jordan, M.I.: Finding Clusters in Independent Component Analysis. In: ICA 2003, pp. 891–896 (2003)Google Scholar
  8. 8.
    Póczos, B., Lőrincz, A.: Independent Subspace Analysis Using k-Nearest Neighborhood Distances. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 163–168. Springer, Heidelberg (2005)Google Scholar
  9. 9.
    Póczos, B., Lőrincz, A.: Independent Subspace Analysis Using Geodesic Spanning Trees. In: ICML, pp. 673–680 (2005)Google Scholar
  10. 10.
    Theis, F.J.: Blind Signal Separation into Groups of Dependent Signals Using Joint Block Diagonalization. In: Proc. ISCAS 2005, Kobe, Japan, pp. 5878–5881 (2005)Google Scholar
  11. 11.
    Van Hulle, M.M.: Edgeworth Approximation of Multivariate Differential Entropy. Neural Comput. 17, 1903–1910 (2005)zbMATHCrossRefGoogle Scholar
  12. 12.
    Cheung, Y., Xu, L.: Dual Multivariate Auto-Regressive Modeling in State Space for Temporal Signal Separation. IEEE Tr. on Syst. Man Cyb. B 33, 386–398 (2003)Google Scholar
  13. 13.
    Theis, F.J.: Uniqueness of Complex andMultidimensional Independent Component Analysis. Signal Proc. 84, 951–956 (2004)zbMATHCrossRefGoogle Scholar
  14. 14.
    Szabó, Z., Póczos, B., Lőrincz, A.: Separation Theorem for Independent Subspace Analysis. Technical report, Eötvös Loránd University, Budapest (2005),
  15. 15.
    Yukich, J.E.: Probability Theory of Classical Euclidean Optimization Problems. Lecture Notes in Math., vol. 1675. Springer, Berlin (1998)zbMATHGoogle Scholar
  16. 16.
    Costa, J.A., Hero, A.O.: Manifold Learning Using k-Nearest Neighbor Graphs. In: ICASSP, Montreal, Canada (2004)Google Scholar
  17. 17.
    Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method. Springer, Heidelberg (2004)zbMATHGoogle Scholar
  18. 18.
    Ekman, P.: Emotion in the Human Face. Cambridge Univ. Press, New York (1982)Google Scholar
  19. 19.
    Póczos, B., Lőrincz, A.: Non-combinatorial Estimation of Independent Autoregressive Sources (2005) (submitted)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  1. 1.Department of Information SystemsEötvös Loránd UniversityBudapestHungary
  2. 2.Research Group on Intelligent Information SystemsHungarian Academy of Sciences 

Personalised recommendations