Information-Theoretic Model Selection for Independent Components

  • Claudia Plant
  • Fabian J. Theis
  • Anke Meyer-Baese
  • Christian Böhm
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6365)


Independent Component Analysis (ICA) is an essential building block for data analysis in many applications. Selecting the truly meaningful components from the result of an ICA algorithm, or comparing the results of different algorithms, however, are non-trivial problems. We introduce a very general technique for evaluating ICA results rooted in information-theoretic model selection. The basic idea is to exploit the natural link between non-Gaussianity and data compression: The better the data transformation represented by one or several ICs improves the effectiveness of data compression, the higher is the relevance of the ICs. In an extensive experimental evaluation we demonstrate that our novel information-theoretic measure robustly selects the most interesting components from data without requiring any assumptions or thresholds.


Probability Density Function Independent Component Analysis Gaussian Mixture Model Data Compression Reconstruction Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barron, A.R., Rissanen, J., Yu, B.: The minimum description length principle in coding and modeling. IEEE Transactions on Information Theory 44(6), 2743 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Pelleg, D., Moore, A.: X-means: Extending K-means with efficient estimation of the number of clusters. In: ICML Conference, pp. 727–734 (2000)Google Scholar
  3. 3.
    Lee, T.C.M.: Regression spline smoothing using the minimum description length principle. Statistics and Probability Letters 48, 71–82 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Böhm, C., Faloutsos, C., Plant, C.: Outlier-robust clustering using independent components. In: SIGMOD Conference, pp. 185–198 (2008)Google Scholar
  5. 5.
    Lee, T.W., Lewicki, M.S.: The generalized gaussian mixture model using ICA. In: ICA Conference (2000)Google Scholar
  6. 6.
    Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks 10(3), 626–634 (1999)CrossRefGoogle Scholar
  7. 7.
    Wismüller, A., Lange, O., Dersch, D.R., Leinsinger, G.L., Hahn, K., Pütz, B., Auer, D.: Cluster analysis of biomedical image time-series. Int. J. Comput. Vision 46(2), 103–128 (2002)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Claudia Plant
    • 1
  • Fabian J. Theis
    • 2
  • Anke Meyer-Baese
    • 1
  • Christian Böhm
    • 3
  1. 1.Florida State University 
  2. 2.Helmholtz Zentrum München 
  3. 3.University of Munich 

Personalised recommendations