SVM-Based Feature Selection by Direct Objective Minimisation

  • Julia Neumann
  • Christoph Schnörr
  • Gabriele Steidl
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3175)


We propose various novel embedded approaches for (simultaneous) feature selection and classification within a general optimisation framework. In particular, we include linear and nonlinear SVMs. We apply difference of convex functions programming to solve our problems and present results for artificial and real-world data.


Support Vector Machine Feature Selection Neural Information Processing System Feature Selection Approach Feature Selection Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ben-Tal, A., Zibulevsky, M.: Penalty/barrier multiplier methods for convex programming problems. SIAM Journal on Optimization 7(2), 347–366 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Bennett, K.P., Mangasarian, O.L.: Robust linear programming discrimination of two linearly inseparable sets. Optimization Methods and Software 1, 23–34 (1992)CrossRefGoogle Scholar
  3. 3.
    Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
  4. 4.
    Bradley, P.S., Mangasarian, O.L.: Feature selection via concave minimization and support vector machines. In: Proceedings of the 15th International Conference on Machine Learning, San Francisco, CA, USA, pp. 82–90. Morgan Kaufmann, San Francisco (1998)Google Scholar
  5. 5.
    Cristianini, N., Shawe-Taylor, J., Elisseeff, A., Kandola, J.: On kernel-target alignment. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14, pp. 367–373. MIT Press, Cambridge (2002)Google Scholar
  6. 6.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. Journal of Machine Learning Research 3, 1157–1182 (2003)zbMATHCrossRefGoogle Scholar
  7. 7.
    Pham Dinh, T., Hoai An, L.T.: A d.c. optimization algorithm for solving the trust-region subproblem. SIAM Journal on Optimization 8(2), 476–505 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)zbMATHGoogle Scholar
  9. 9.
    Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  10. 10.
    Weston, J., Elisseeff, A., Schölkopf, B., Tipping, M.: Use of the zero-norm with linear models and kernel methods. Journal of Machine Learning Research 3, 1439–1461 (2003)zbMATHCrossRefGoogle Scholar
  11. 11.
    Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., Vapnik, V.: Feature selection for SVMs. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems 13, pp. 668–674. MIT Press, Cambridge (2001)Google Scholar
  12. 12.
    Zhu, J., Rosset, S., Hastie, T., Tibshirani, R.: 1-norm support vector machines. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems 16, MIT Press, Cambridge (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Julia Neumann
    • 1
  • Christoph Schnörr
    • 1
  • Gabriele Steidl
    • 1
  1. 1.Dept. of Mathematics and Computer ScienceUniversity of MannheimMannheimGermany

Personalised recommendations