Abstract
Least–Squares Support Vector Machines (LS–SVMs) have been a successful alternative model for classification and regression Support Vector Machines (SVMs), and used in a wide range of applications. In spite of this, only a limited effort has been realized to design efficient algorithms for the training of this class of models, in clear contrast to the vast amount of contributions of this kind in the field of classic SVMs. In this work we propose to combine the popular Sequential Minimal Optimization (SMO) method with a momentum strategy that manages to reduce the number of iterations required for convergence, while requiring little additional computational effort per iteration, especially in those situations where the standard SMO algorithm for LS–SVMs fails to obtain fast solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barbero, Á., Dorronsoro, J.: Momentum Sequential Minimal Optimization: an Accelerated Method for Support Vector Machine Training. Submitted to IJCNN
Barbero, Á., López, J., Dorronsoro, J.R.: Cycle-breaking Acceleration of SVM Training. Neurocomputing 72(7-9), 1398–1406 (2009)
Beck, A., Teboulle, M.: A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems. SIAM J. Imaging Sciences 2(1), 183–202 (2009)
Chang, C.C., Lin, C.J.: LIBSVM Regression Datasets Repository, http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/regression.html , datasets available at http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/regression.html
Chu, W., Ong, C., Keerthi, S.: An Improved Conjugate Gradient Scheme to the Solution of Least Squares SVM. Neural Networks 16(2), 498–501 (2005)
De Brabanter, K., Karsmakers, P., Ojeda, F., Alzate, C., Brabanter, J.D., Pelckmans, K., Moor, B.D., Vandewalle, J., Suykens, J.: LS-SVMlab Toolbox User’s Guide version 1.7. Tech. Rep. 10-146, Katholieke Universiteit Leuven (2010), software available at http://www.esat.kuleuven.be/sista/lssvmlab
Fan, R.-E., Chen, P.-H., Lin, C.-J.: Working Set Selection using Second Order Information for Training Support Vector Machines. Journal of Machine Learning Research 6, 1889–1918 (2005)
Keerthi, S., Shevade, S.: SMO Algorithm for Least-Squares SVM Formulations. Neural Computation 15(2), 487–507 (2003)
López, J., Suykens, J.: First and Second Order SMO Algorithms for LS-SVM classifiers. Neural Processing Letters 33(1), 33–44 (2011)
Platt, J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Advances in Kernel Methods: Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)
Rätsch, G.: Benchmark Repository (2000), datasets available at http://ftp.tuebingen.mpg.de/pub/fml/raetsch-lab/benchmarks/
Suykens, J., Lukas, L., Van Dooren, P., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machine Classifiers: a Large Scale Algorithm. In: ECCTD 1999: Proceeding of the European Conference on Circuit Theory and Design, Stresa, Italy, pp. 839–842 (1999)
Suykens, J., Vandewalle, J.: Least Squares Support Vector Machine Classifiers. Neural Processing Letters 9(3), 293–300 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
López, J., Barbero, Á., Dorronsoro, J.R. (2011). Momentum Acceleration of Least–Squares Support Vector Machines. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2011. ICANN 2011. Lecture Notes in Computer Science, vol 6792. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21738-8_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-21738-8_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21737-1
Online ISBN: 978-3-642-21738-8
eBook Packages: Computer ScienceComputer Science (R0)