Analytic Feature Selection for Support Vector Machines

  • Carly Stambaugh
  • Hui Yang
  • Felix Breuer
Conference paper

DOI: 10.1007/978-3-642-39712-7_17

Volume 7988 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Stambaugh C., Yang H., Breuer F. (2013) Analytic Feature Selection for Support Vector Machines. In: Perner P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2013. Lecture Notes in Computer Science, vol 7988. Springer, Berlin, Heidelberg

Abstract

Support vector machines (SVMs) rely on the inherent geometry of a data set to classify training data. Because of this, we believe SVMs are an excellent candidate to guide the development of an analytic feature selection algorithm, as opposed to the more commonly used heuristic methods. We propose a filter-based feature selection algorithm based on the inherent geometry of a feature set. Through observation, we identified six geometric properties that differ between optimal and suboptimal feature sets, and have statistically significant correlations to classifier performance. Our algorithm is based on logistic and linear regression models using these six geometric properties as predictor variables. The proposed algorithm achieves excellent results on high dimensional text data sets, with features that can be organized into a handful of feature types; for example, unigrams, bigrams or semantic structural features. We believe this algorithm is a novel and effective approach to solving the feature selection problem for linear SVMs.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Carly Stambaugh
    • 1
  • Hui Yang
    • 2
  • Felix Breuer
    • 1
  1. 1.Department of MathematicsSan Francisco State UniversitySan FranciscoUSA
  2. 2.Department of Computer ScienceSan Francisco State UniversitySan FranciscoUSA