Contrast Functions for Independent Subspace Analysis

  • Jason A. Palmer
  • Scott Makeig
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7191)


We consider the Independent Subspace Analysis problem from the point of view of contrast functions, showing that contrast functions are able to partially solve the ISA problem. That is, basic ICA can solve the ISA problem up to within-subspace separation/analysis. We define sub- and super-Gaussian subspaces and extend to ISA a previous result on freedom of ICA from local optima. We also consider new types of dependent densities that satisfy or violate the entropy power inequality (EPI) condition.


Mutual Information Fisher Information Independent Component Analysis Projection Pursuit Contrast Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pham, D.T.: Mutual information approach to blind separation of stationary sources. IEEE Trans. Information Theory 48(7), 1935–1946 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Cardoso, J.-F.: Infomax and maximum likelihood for source separation. IEEE Letters on Signal Processing 4(4), 112–114 (1997)CrossRefGoogle Scholar
  3. 3.
    Cardoso, J.-F.: Multidimensional independent component analysis. In: Proceedings of the IEEE International Conference on Acoustics and Signal Processing (ICASSP 1998), Seattle, WA, pp. 1941–1944 (1998)Google Scholar
  4. 4.
    Hyvärinen, A., Hoyer, P.O.: Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Computation 12, 1705–1720 (2000)CrossRefGoogle Scholar
  5. 5.
    Kim, T., Eltoft, T., Lee, T.-W.: Independent Vector Analysis: An Extension of ICA to Multivariate Components. In: Rosca, J., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 165–172. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Comon, P.: Independent component analysis: a new concept? Signal Processing 36(3), 287–314 (1994)CrossRefzbMATHGoogle Scholar
  7. 7.
    Huber, P.J.: Projection pursuit. The Annals of Statistics 13(2), 435–475 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Pham, D.T.: Contrast functions for blind separation and deconvolution of sources. Tech. Rep., Laboratoire de Mod’elisation et Calcul, CNRS, IMAG (2001)Google Scholar
  9. 9.
    Theis, F.J.: Blind signal separation into groups of dependent signals using joint block diagonalization. In: ISCAS (6), pp. 5878–5881 (2005)Google Scholar
  10. 10.
    Theis, F.J., Kawanabe, M.: Uniqueness of Non-Gaussian Subspace Analysis. In: Rosca, J.P., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) ICA 2006. LNCS, vol. 3889, pp. 917–925. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Theis, F.J.: Colored subspace analysis: Dimension reduction based on a signal’s autocorrelation structure. IEEE Trans. on Circuits and Systems 57-I(7), 1463–1474 (2010)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Gutch, H.W., Theis, F.J.: Independent Subspace Analysis is Unique, Given Irreducibility. In: Davies, M.E., James, C.J., Abdallah, S.A., Plumbley, M.D. (eds.) ICA 2007. LNCS, vol. 4666, pp. 49–56. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  13. 13.
    Castella, M., Comon, P.: Blind Separation of Instantaneous Mixtures of Dependent Sources. In: Davies, M.E., James, C.J., Abdallah, S.A., Plumbley, M.D. (eds.) ICA 2007. LNCS, vol. 4666, pp. 9–16. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Szabó, Z., Póczos, B., Lőrincz, A.: Undercomplete Blind Subspace Deconvolution via Linear Prediction. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 740–747. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Dembo, A., Cover, T.M., Thomas, J.A.: Information theoretic inequalities. IEEE Transactions on Information Theory 37(6), 1501–1518 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Palmer, J.A., Kreutz-Delgado, K., Makeig, S.: Strong Sub- and Super-Gaussianity. In: Vigneron, V., Zarzoso, V., Moreau, E., Gribonval, R., Vincent, E. (eds.) LVA/ICA 2010. LNCS, vol. 6365, pp. 303–310. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Palmer, J.A., Kreutz-Delgado, K., Rao, B.D., Makeig, S.: Modeling and Estimation of Dependent Subspaces with Non-Radially Symmetric and Skewed Densities. In: Davies, M.E., James, C.J., Abdallah, S.A., Plumbley, M.D. (eds.) ICA 2007. LNCS, vol. 4666, pp. 97–104. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jason A. Palmer
    • 1
  • Scott Makeig
    • 1
  1. 1.Swartz Center for Computational NeuroscienceUniversity of California San DiegoLa JollaUSA

Personalised recommendations