Advertisement

Coefficient Structure of Kernel Perceptrons and Support Vector Reduction

  • Daniel García
  • Ana González
  • José R. Dorronsoro
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4527)

Abstract

Support Vector Machines (SVMs) with few support vectors are quite desirable, as they have a fast application to new, unseen patterns. In this work we shall study the coefficient structure of the dual representation of SVMs constructed for nonlinearly separable problems through kernel perceptron training. We shall relate them with the margin of their support vectors (SVs) and also with the number of iterations in which these SVs take part. These considerations will lead to a remove–and–retrain procedure for building SVMs with a small number of SVs where both suitably small and large coefficient SVs will be taken out from the training sample. Besides providing a significant SV reduction, our method’s computational cost is comparable to that of a single SVM training.

Keywords

Support Vector Machine Support Vector Machine Learn Research Convex Minimization Problem Quadratic Penalty 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bennett, K., Bredensteiner, E.: Geometry in learning. In: Gorini, C., Hart, E., Meyer, W., Phillips, T. (eds.) Geometry at Work, Mathematical Association of America, Washington D.C (1997)Google Scholar
  2. 2.
    Burges, C.J.C.: Simplified support vector decision rules. In: Saitta, L. (ed.) Proc. 13th International Conference on Machine Learning, pp. 71–77. Morgan Kaufmann, San Francisco (1996)Google Scholar
  3. 3.
    Chang, C., Lin, C.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/LIBSVM
  4. 4.
    Dekel, O., Shalev-Shwartz, S., Singer, Y.: The Forgetron: A Kernel-Based Perceptron on a Fixed Budget. Advances in Neural Processing Systems 18, 259–266 (2005)Google Scholar
  5. 5.
    Franc, V., Hlavac, V.: An iterative algorithm learning the maximal margin classier. Pattern Recognition 36, 1985–1996 (2003)CrossRefzbMATHGoogle Scholar
  6. 6.
    García, D., González, A., Dorronsoro, J.R.: Convex Perceptrons. In: Corchado, E.S., Yin, H., Botti, V., Fyfe, C. (eds.) IDEAL 2006. LNCS, vol. 4224, pp. 578–585. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Joachims, T.: Making Large-Scale Support Vector Machine Learning Practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel methods, pp. 169–184. MIT Press, Cambridge (1999)Google Scholar
  8. 8.
    Lee, Y., Mangasarian, O.L.: RSVM: reduced support vector machines. In: CD Proceedings of the First SIAM International Conference on Data Mining, Chicago (2001)Google Scholar
  9. 9.
    Keerthi, S., Chapelle, O., de Coste, D.: Building support vector machines with reduced complexity. Journal of Machine Learning Research 7, 1493–1515 (2006)Google Scholar
  10. 10.
    Parrado-Hernéndez, E., Mora-Jiménez, I., Arenas-García, J., Figueiras-Vidal, A.R., Navia-Vázquez, A.: Growing support vector classifiers with controlled complexity. Pattern Recognition 36, 1479–1488 (2003)CrossRefGoogle Scholar
  11. 11.
    Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel methods, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  12. 12.
    Schlesinger, M., Hlavac, V.: Ten Lectures on Statistical and Structural Pattern Recognition. Kluwer Academic Publishers, Dordrecht (2002)zbMATHGoogle Scholar
  13. 13.
    Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2001)Google Scholar
  14. 14.
    UCI-benchmark repository of machine learning data sets. University of California Irvine, http://www.ics.uci.edu
  15. 15.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Berlin (1995)zbMATHGoogle Scholar
  16. 16.
    Wu, M., Schölkopf, B., Bakir, G.: A Direct Method for Building Sparse Kernel Learning Algorithms. Journal of Machine Learning Research 7, 603–624 (2006)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Daniel García
    • 1
  • Ana González
    • 1
  • José R. Dorronsoro
    • 1
  1. 1.Dpto. de Ingeniería Informática and Instituto de Ingeniería del Conocimiento, Universidad Autónoma de Madrid, 28049 MadridSpain

Personalised recommendations