Discriminant Subspace Learning Based on Support Vectors Machines

  • Nikolaos Pitelis
  • Anastasios Tefas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7376)


A new method for dimensionality reduction and feature extraction based on Support Vector Machines and minimization of the within-class data dispersion is proposed. An iterative procedure is proposed that successively applies Support Vector Machines on perpendicular subspaces using the deflation transformation in such a way that the within-class variance is minimized. The proposed approach is proved to be a successive SVM using deflation kernels. The normal vectors of the successive hyperplanes contain discriminant information and they can be used as projection vectors for feature extraction and dimensionality reduction of the data. Experiments on various datasets are conducted in order to highlight the superior performance of the proposed algorithm.


Support Vector Machine Feature Extraction Linear Discriminant Analysis Scatter Matrix Discriminant Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Cover, T.M., Hart, P.E.: Nearest Neighbor Pattern Classification. IEEE Transactions In Information Theory, 21–26 (1967)Google Scholar
  2. 2.
    Duda, O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley (2001)Google Scholar
  3. 3.
    Scholkopf, B., Smola, A.: Learning with Kernels. MIT, Cambridge (2002)Google Scholar
  4. 4.
    Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer (2002)Google Scholar
  5. 5.
    Pearson, K.: On Lines and Planes of Closest Fit to Systems of Points in Space. Philosophical Magazine 2, 559–572 (1901)Google Scholar
  6. 6.
    Hotelling, H.: Analysis of a Complex of Statistical Variables into Principal Components. Journal of Educational Psychology 24, 417–441 (1933)CrossRefGoogle Scholar
  7. 7.
    Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a Kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  8. 8.
    Alpaydin, E.: Introduction to Machine Learning. MIT Press (2004)Google Scholar
  9. 9.
    Juwei, L., Plataniotis, K.N., Venetsanopoulos, A.N.: Face recognition using Kernel direct discriminant analysis algorithms. IEEE Transactions on Neural Networks 14, 117–126 (2003)CrossRefGoogle Scholar
  10. 10.
    Kocsor, A., Kovács, K., Szepesvári, C.: Margin Maximizing Discriminant Analysis. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 227–238. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Tefas, A., Kotropoulos, C., Pitas, I.: Using Support Vector Machines to Enhance the Performance of Elastic Graph Matching for Frontal Face Authentication. IEEE Transactions on Pattern Analalysis and Machine Intelligence 23(7), 735–746 (2001)CrossRefGoogle Scholar
  12. 12.
    Zafeiriou, S., Tefas, A., Pitas, I.: Minimum Class Variance Support Vector Machines. IEEE Transactions on Image Processing 16(10), 2551–2564 (2007)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Kung, S.Y., Diamantaras, K.I.: Neural networks for extracting unsymmetric principal components. In: Neural Networks for Signal Processing, pp. 50–59. IEEE Press, New York (1991)CrossRefGoogle Scholar
  14. 14.
    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. In: Data Minining Knowledge Discovery, vol. 2, pp. 121–167 (1998)Google Scholar
  15. 15.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  16. 16.
    Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, New York (1987)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Nikolaos Pitelis
    • 1
  • Anastasios Tefas
    • 2
  1. 1.School of Electronic Engineering and Computer ScienceQueen Mary, University of LondonUK
  2. 2.Department of InformaticsAristotle University of ThessalonikiGreece

Personalised recommendations