Feature Extraction for Regression Problems and an Example Application for Pose Estimation of a Face

  • Nojun Kwak
  • Sang-Il Choi
  • Chong-Ho Choi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5112)


In this paper, we propose a new feature extraction method for regression problems. It is a modified version of linear discriminant analysis (LDA) which is a very successful feature extraction method for classification problems. In the proposed method, the between class and the within class scatter matrices in LDA are modified so that they fit in regression problems. The samples with small differences in the target values are used to constitute the within class scatter matrix while the ones with large differences in the target values are used for the between class scatter matrix. We have applied the proposed method in estimating the head pose and compared the performance with the conventional feature extraction methods.


Regression Feature extraction Dimensionality reduction LDA 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cios, K.J., Pedrycz, W., Swiniarski, R.W.: Data Mining Methods for Knowledge Discovery, ch. 9. Kluwer Academic Publishers, Dordrecht (1998)Google Scholar
  2. 2.
    Joliffe, I.T.: Principal Component Analysis. Springer, Heidelberg (1986)Google Scholar
  3. 3.
    Bell, A.J., Sejnowski, T.J.: An Information-Maximization Approach to Blind Separation and Blind Deconvolution. Neural Computation 7, 1129–1159 (1995)CrossRefGoogle Scholar
  4. 4.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, New York (1990)MATHGoogle Scholar
  5. 5.
    Weisberg, S.: Applied Linear Regression, ch. 3, 2nd edn., p. 324. John Wiley, New York (1985)Google Scholar
  6. 6.
    Loog, M.: Supervised Dimensionality Reduction and Contextual Pattern Recognition in Medical Image Processing, ch. 3. Ponsen & Looijen, Wageningen, The Netherlands (2004)Google Scholar
  7. 7.
    Li, K.C.: Sliced Inverse Regression for Dimension Reduction (with discussioin). J. the American Statistical Association. 86, 316–342 (1991)MATHCrossRefGoogle Scholar
  8. 8.
    Li, K.C.: On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein’s lemma. J. the American Statistical Assiciation. 87, 1025–1039 (1992)MATHCrossRefGoogle Scholar
  9. 9.
    Kwak, N., Kim, C.: Dimensionality reduction based on ICA for regression problems. In: Proc. Int’l Conf. on Artificial Neural Networks (IJCNN), pp. 1–10 (2006)Google Scholar
  10. 10.
    Sim, T., Baker, S., Bsat, M.: The CMU Pose, Illumination, and Expression Database. IEEE Trans. Pattern Analysis and Machine Intelligence 25, 1615–1618 (2003)CrossRefGoogle Scholar
  11. 11.
    Georghiades, A.S., Belhumeur, P.N.: Frome Few to Many: Illumination Cone Models for Face Recognition Under Variable Lighting and Pose. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 643–660 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Nojun Kwak
    • 1
  • Sang-Il Choi
    • 2
  • Chong-Ho Choi
    • 2
  1. 1.Division of Electrical & Computer EngineeringAjou UniversitySuwonKorea
  2. 2.School of Electrical Engineering and Computer ScienceSeoul National UniversitySeoulKorea

Personalised recommendations