Dimensionality Reduction Based on ICA for Regression Problems

  • Nojun Kwak
  • Chunghoon Kim
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4131)


In manipulating data such as in supervised learning, we often extract new features from the original features for the purpose of reducing the dimensions of feature space and achieving better performance. In this paper, we show how standard algorithms for independent component analysis (ICA) can be applied to extract features for regression problems. The advantage is that general ICA algorithms become available to a task of feature extraction for regression problems by maximizing the joint mutual information between target variable and new features. Using the new features, we can greatly reduce the dimension of feature space without degrading the regression performance.


Feature Extraction Mutual Information Independent Component Analysis Learning Rule Regression Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Joliffe, I.T.: Principal Component Analysis. Springer, Heidelberg (1986)Google Scholar
  2. 2.
    Bell, A.J., Sejnowski, T.J.: An information-maximization approach to blind separation and blind deconvolution. Neural Computation 7(6) (June 1995)Google Scholar
  3. 3.
    Lee, T.-W., Girolami, M., Sejnowski, T.J.: Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources. Neural Computation 11(2) (February 1999)Google Scholar
  4. 4.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, London (1990)MATHGoogle Scholar
  5. 5.
    Kwak, N., Choi, C.-H.: Feature extraction based on ICA for binary classification problems. IEEE Trans. on Knowledge and Data Engineering 15(6), 1374–1388 (2003)CrossRefGoogle Scholar
  6. 6.

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Nojun Kwak
    • 1
  • Chunghoon Kim
    • 2
  1. 1.Samsung ElectronicsSuwon, Gyeonggi-DoKorea
  2. 2.School of Electrical Engineering and Computer ScienceSeoul National UniversitySeoulKorea

Personalised recommendations