A Simple Maximum Gain Algorithm for Support Vector Regression

  • Álvaro Barbero
  • José R. Dorronsoro
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5517)


Shevade’s et al. Modification 2 is one of the most widely used algorithms to build Support Vector Regression (SVR) models. It selects as a size 2 working set the index pair giving the maximum KKT violation and combines it with the updating heuristics of Smola and Schölkopf enforcing at each training iteration a \(\alpha_i \alpha^*_i =0\) condition. In this work we shall present an alternative, much simpler procedure that selects the updating indices as those giving a maximum gain in the SVR dual function. While we do not try to enforce the \(\alpha_i \alpha^*_i =0\) condition, we show that it will hold at each iteration provided it does so at the starting multipliers. We will numerically show that the proposed procedure requires essentially the same number of iterations than Modification 2 having thus the same time performance while being much simpler to code.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smola, A.J., Schölkopf, B.: A tutorial on support vector regression. Statistics and Computing 14, 199–222 (2004)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)CrossRefzbMATHGoogle Scholar
  3. 3.
    Hsu, C.-W., Chang, C.-C., Lin C.-J.: A practical guide to support vector classification,
  4. 4.
    Shevade, S.K., Keerthi, S.S., Bhattacharyya, C., Murthy, K.R.K.: Improvements to the smo algorithm for svm regression. IEEE Transactions on Neural Networks 11, 1188–1193 (2000)CrossRefzbMATHGoogle Scholar
  5. 5.
    Chang, C.-C., Lin C.-J.: Libsvm regression dataset repository,
  6. 6.
    Glasmachers, T., Igel, C.: Second order smo improves svm online and active learning. Neural Computation 20(2), 374–382 (2008)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Álvaro Barbero
    • 1
  • José R. Dorronsoro
    • 1
  1. 1.Dpto. de Ingeniería Informática and Instituto de Ingeniería del ConocimientoUniversidad Autónoma de MadridMadridSpain

Personalised recommendations