Abstract
An algorithm for selecting features in the classification learning problem is considered. The algorithm is based on a modification of the standard criterion used in the support vector machine method. The new criterion adds to the standard criterion a penalty function that depends on the selected features. The solution of the problem is reduced to finding the minimax of a convex-concave function. As a result, the initial set of features is decomposed into three classes—unconditionally selected, weighted selected, and eliminated features.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
V. N. Vapnik and A. Ya. Chervonenkis, Theory of Pattern Classification (Nauka, Moscow, 1974) [in Russian].
V. N. Vapnik, Estimation of Dependences Based on Empirical Data (Nauka, Moscow, 1979; Springer, New York, 2006).
V. N. Vapnik, The Nature of Statistical Learning Theory (Springer, New York, 1995).
C. Cortes and V. Vapnik, “Support Vector Networks,” Machine Learning 20(3), 273–297 (1995).
C. J. C. Burges, “A Tutorial on Support Vector Machines for Pattern Recognition,” Knowledge Discovery and Data Mining 2(4), 121–167 (1998).
K. P. Bennet and C. Campbell, “Support Vector Machines: Hype or Hallelujah?” SIGKDD Explorations 2(2), 1–13 (2000).
M. A. Aizerman, E. M. Braverman, and L. I. Rozonoer, The Potential Function Method in the Theory of Machine Learning (Nauka, Moscow, 1970) [in Russian].
L. Molina, L. Belanche, and A. Nebot, “Feature Selection Algorithms: A Survey and Experimental Evaluation,” ICDM, 306–313 (2002).
Y. Goncharov, L. Muchnik, and L. Shvartser, “Simultaneous Feature Selection and Margin Maximization Using Saddle Point Approach,” DIMACS Techn. Rep., no. 2004-08 (2004).
R. T. Rockafellar, Convex Analysis (Princeton Univ. Press, Princeton, 1970; Mir, Moscow, 1973).
K. Arrow, L. Hurwicz, and H. Udzawa, Studies in Linear and Non-linear Programming (Stanford Univ. Press, Stanford 1958; Inostrannaya Literatura, Moscow, 1962).
A. S. Antipin, “Controllable Proximal Differential Systems for Solving Saddle Problems,” Differ. Uravn. 28, 1846–1861 (1992).
E. G. Gol’shtein and N. V. Tret’yakov, Modified Lagrange Functions (Nauka, Moscow, 1989) [in Russian].
G. M. Korpelevich, “Extragradient Method for Finding Saddle Points and Solving Other Problems,” Ekon. Mat. Metody 12, 747–756 (1976).
A. S. Antipin, “Gradient-type Method for Finding Saddle Points of Lagrangian Functions,” Ekon. Mat. Metody 13, 560–565 (1977).
A. S. Antipin, “From Optima to Equilibria,” Proc. of the Institute of Systems Analysis: Dynamics of Non-Homogeneous Systems (Moscow, 2000), Vol. 3, pp. 35–64.
Y. Censor, “Computational Acceleration of Projection Algorithms for the Linear Best Approximation Problem,” Techn. Rep. Dep. Math., Univ. Haifa, Israel, May (2005).
H. H. Bauschke and J. M. Borwein, “Dykstra’s Alternating Projection Algorithm for Two Sets,” J. Approximat. Theory 79, 418–443 (1994).
N. Gaffke and R. Mathar, “A Cyclic Projection Algorithm via Duality,” Metrika 36(1), 29–54 (1989).
T. Joachims, “Optimizing Search Engines Using Clickthrough Data,” ACM SIGKDD Conf. on Knowledge Discovery and Data Mining (KDD), 133–142 (2002).
T. Joachims, “Estimating the Generalization Performance of an SVM Efficiently,” Proc. of the 17th Int. Conf. on Machine Learning (ICML-00), pp. 431–438 (2000).
A. V. Anghelescu and I. B. Muchnik, “Optimization of SVM in a Space of Two Parameters: Weak Margin and Intercept,” DIMACS Working Group on Monitoring Message Streams, May (2003).
Y. Goncharov, I. Muchnik, and L. Shvartser, “Saddle Point Feature Selection in SVM Regression,” DIMACS Techn. Rep, no. 2007-08 (2007).
Author information
Authors and Affiliations
Corresponding author
Additional information
Original Russian Text Yu.V. Goncharov, I.B. Muchnik, L.V. Shvartser @, 2008, published in Zhurnal Vychislitel’noi Matematiki i Matematicheskoi Fiziki, 2008, Vol. 48, No. 7, pp. 1318–1336.
Rights and permissions
About this article
Cite this article
Goncharov, Y.V., Muchnik, I.B. & Shvartser, L.V. Feature selection algorithm in classification learning using support vector machines. Comput. Math. and Math. Phys. 48, 1243–1260 (2008). https://doi.org/10.1134/S0965542508070154
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S0965542508070154