A Fast Fixed-Point Algorithm for Two-Class Discriminative Feature Extraction

  • Zhirong Yang
  • Jorma Laaksonen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4132)


We propose a fast fixed-point algorithm to improve the Relevant Component Analysis (RCA) in two-class cases. Using an objective function that maximizes the predictive information, our method is able to extract more than one discriminative component of data for two-class problems, which cannot be accomplished by classical Fisher’s discriminant analysis. After prewhitening the data, we apply Newton’s optimization method which automatically chooses the learning rate in the iterative training of each component. The convergence of the iterative learning is quadratic, i.e. much faster than the linear optimization by gradient methods. Empirical tests presented in the paper show that feature extraction using the new method resembles RCA for low-dimensional ionosphere data and significantly outperforms the latter in efficiency for high-dimensional facial image data.


Linear Discriminant Analysis Discriminative Feature Ionosphere Data Linear Dimensionality Reduction Discriminative Component 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blake, C.L., Newman, D.J., Hettich, S., Merz, C.J.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
  2. 2.
    Duin, R.P.W., Loog, M.: Linear dimensionality reduction via a heteroscedastic extension of lda: the chernoff criterion. IEEE Transaction on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)CrossRefGoogle Scholar
  3. 3.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals of Eugenics 7 (1963)Google Scholar
  4. 4.
    Howland, P., Park, H.: Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(8), 995–1006 (2005)CrossRefGoogle Scholar
  5. 5.
    Peltonen, J., Kaski, S.: Discriminative components of data. IEEETransactions on Neural Networks 16(1), 68–83 (2005)CrossRefGoogle Scholar
  6. 6.
    Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face recognition algorithms. IEEE Trans. Pattern Analysis andMachine Intelligence 22, 1090–1104 (2000)CrossRefGoogle Scholar
  7. 7.
    Xu, Y., Yang, J.Y., Jin, Z.: Theory analysis on FSLDA and ULDA. Pattern Recognition 36(12), 3031–3033 (2003)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Zhirong Yang
    • 1
  • Jorma Laaksonen
    • 1
  1. 1.Laboratory of Computer and Information ScienceHelsinki University of TechnologyHUT, EspooFinland

Personalised recommendations