Abstract
We describe a method for solving large-scale semiparametric support vector machines (SVMs) for regression problems. Most of the approaches proposed to date for large-scale SVMs cannot accommodate the multiple equality constraints that appear in semiparametric problems. Our approach uses a decomposition framework, with a primal-dual algorithm to find an approximate saddle point for the min-max formulation of each subproblem. We compare our method with algorithms previously proposed for semiparametric SVMs, and show that it scales well as the number of training examples grows.
Chapter PDF
References
Smola, A.J., Frieß, T.T., Schölkopf, B.: Semiparametric support vector and linear programming machines. In: Advances in Neural Information Processing Systems 11, pp. 585–591. MIT Press, Cambridge (1999)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (April 2009) version 2.89, http://www.csie.ntu.edu.tw/~cjlin/libsvm
Joachims, T.: Making large-scale support vector machine learning practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 169–184. MIT Press, Cambridge (1999)
Serafini, T., Zanghirati, G., Zanni, L.: Gradient projection methods for large quadratic programs and applications in training support vector machines. Optimization Methods and Software 20(2–3), 353–378 (2004)
Serafini, T., Zanni, L.: On the working set selection in gradient projection-based decomposition techniques for support vector machines. Optimization Methods and Software 20, 583–596 (2005)
Keerthi, S.S., Gilbert, E.G.: Convergence of a generalized smo algorithm for svm classifier design. Machine Learning 46(1-3), 351–360 (2002)
Dai, Y.H., Fletcher, R.: New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds. Mathematical Programming, Series A 106, 403–421 (2006)
Kienzle, W., Schölkopf, B.: Training support vector machines with multiple equality constraints. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 182–193. Springer, Heidelberg (2005)
Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: COLT 1992: Proceedings of the fifth annual workshop on Computational learning theory, pp. 144–152. ACM, New York (1992)
Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)
Tseng, P., Yun, S.: A coordinate gradient descent method for linearly constrained smooth optimization and support vector machines training. Published online in Computational Optimization and Applications (October 2008)
Lin, C.J.: Linear convergence of a decomposition method for support vector machines. Technical report, Department of Computer Science and Information Engineering, National Taiwan University (2001)
Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)
Gafni, E.M., Bertsekas, D.P.: Two-metric projection methods for constrained optimization. SIAM Journal on Control and Optimization 22, 936–964 (1984)
Vigotti, M.A., Rossi, G., Bisanti, L., Zanobetti, A., Schwartz, J.: Short term effects of urban air pollution on respiratory health in Milan, Italy, 1980-1989. Journal of Epidemiology Community Health 50, s71–s75 (1996)
Shi, W., Wahba, G., Wright, S.J., Lee, K., Klein, R., Klein, B.: LASSO-Patternsearch algorithm with application to opthalmology data. Statistics and its Interface 1, 137–153 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, S., Wright, S.J. (2009). Decomposition Algorithms for Training Large-Scale Semiparametric Support Vector Machines. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2009. Lecture Notes in Computer Science(), vol 5782. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04174-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-04174-7_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04173-0
Online ISBN: 978-3-642-04174-7
eBook Packages: Computer ScienceComputer Science (R0)