Training Support Vector Machines with Multiple Equality Constraints

  • Wolf Kienzle
  • Bernhard Schölkopf
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3720)


In this paper we present a primal-dual decomposition algorithm for support vector machine training. As with existing methods that use very small working sets (such as Sequential Minimal Optimization (SMO), Successive Over-Relaxation (SOR) or the Kernel Adatron (KA)), our method scales well, is straightforward to implement, and does not require an external QP solver. Unlike SMO, SOR and KA, the method is applicable to a large number of SVM formulations regardless of the number of equality constraints involved. The effectiveness of our algorithm is demonstrated on a more difficult SVM variant in this respect, namely semi-parametric support vector regression.


  1. 1.
    Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)zbMATHGoogle Scholar
  2. 2.
    Chang, C.-C., Lin, C.-J.: Training ν-support vector classifiers: Theory and algorithms. Neural Computation 13(9), 2119–2147 (2001)zbMATHCrossRefGoogle Scholar
  3. 3.
    Chang, C.-C., Lin, C.-J.: LIBSVM – a library for support vector machines, version 2.71 (August 2004),
  4. 4.
    Chen, P.-H., Lin, C.-J., Schölkopf, B.: A tutorial on ν-support vector machines. Applied Stochastic Models in Business and Industry (2005) (to appear)Google Scholar
  5. 5.
    Collobert, R., Bengio, S.: SVMTorch: Support vector machines for large-scale regression problems. Journal of Machine Learning Research 1, 143–160 (2001)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Dong, J., Krzyzak, A., Suen, C.Y.: Fast svm training algorithm with decomposition on very large data sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(4), 603–618 (2005)CrossRefGoogle Scholar
  7. 7.
    Friess, T., Cristianini, N., Campbell, C.: The kernel adatron algorithm: a fast and simple learning procedure for support vector machine. In: International Conference on Machine Learning, pp. 188–196 (1998)Google Scholar
  8. 8.
    Joachims, T.: Making large-scale SVM learning practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods, pp. 42–56. MIT Press, Cambridge (1999)Google Scholar
  9. 9.
    Mangasarian, O.L., Musicant, D.R.: Successive overrelaxation for support vector machines. IEEE Transactions on Neural Networks 10, 1032–1037 (1999)CrossRefGoogle Scholar
  10. 10.
    Osuna, E., Freund, R., Girosi, F.: Support vector machines: Training and applications. Technical Report AIM-1602 (1997)Google Scholar
  11. 11.
    Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods, pp. 185–208. MIT Press, Cambridge (1998)Google Scholar
  12. 12.
    Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  13. 13.
    Smola, A.J., Friess, T., Schölkopf, B.: Semiparametric support vector and linear programming machines. In: Advances in Neural Information Processing Systems 11, pp. 585–591 (1998)Google Scholar
  14. 14.
    Vanderbei, R.J.: LOQO: An interior point code for quadratic programming. Technical Report SOR 94-15, Princeton University, NJ (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Wolf Kienzle
    • 1
  • Bernhard Schölkopf
    • 1
  1. 1.Empirical Inference DepartmentMax-Planck-Institute for Biological CyberneticsTübingenGermany

Personalised recommendations