Abstract
This paper presents a strategy to optimize the learning phase of the Support Vector Machines algorithm (SVM). The SVM algorithm is widely used in solving different tasks like classification, regression, density estimation and clustering problems. However, the algorithm presents important disadvantages when learning large scale problems. Training a SVM involves finding the solution of a quadratic optimization problem (QP), which is very resource consuming. What is more, during the learning step, the best working set must be selected, which is a hard to perform task. In this work, we combine a heuristic approach, which selects the best working set data, with a projected conjugate gradient method, which is a fast and easy to implement algorithm that solves the quadratic programming problem involved in the SVM algorithm. We compare the performances of the optimization strategies using some well-known benchmark databases.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Alur, R., Courcoubetis, C., Halbwachs, N., Henzinger, T.A., Ho, P.H., Nicollin, X., Olivero, A., Sifakis, J., Yovine, S.: The algorithmic anal- ysis of hybrid systems. Theoretical Computer Science 138(1), 3–34 (1995)
Ariel, G.G., Neil, H.G.: A comparison of different initialization strategies to reduce the training time of support vector machines. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 613–618. Springer, Heidelberg (2005)
Asuncion, A., Newman, D.J.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Bishop, C.M.: Pattern recognition and machine learning. Springer, New York (2006)
Edgar, O., Freund, R., Girosi, F.: Support vector machines: Training and applications (1997)
Edgar, O., Freund, R., Girosi, F.: Training support vector machines: an application to face detection. In: 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Proceedings, pp. 130–136 (1997)
Goonatilake, S., Khebbal, S.: Intelligent hybrid systems. John Wiley, Inc., New York (1994)
Poulard, H., Stéve, D.: Barycentric correction procedure: A fast method of learning threshold unit (1995)
Poulard. H. and Stéve. D.: A convergence theorem for barycentric correction procedure. Technical Report 95180, LAAS-CNRS (1995)
Kecman, V.: Learning and Soft Computing, 1st edn. The MIT Press, Cambridge (2001)
Linda, K.: Solving the quadratic programming problem arising in support vector classification. In: Advances in kernel methods: support vector learning, pp. 147–167. MIT Press, Cambridge (1999)
Miguel, G.M., Neil, H.G., Andr, T.: Quadratic optimization fine tuning for the learning phase of svm (2005)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Bernhard SchlkopfChristopher, J.C., Smola, B.J. (eds.) Advances in Kernel Methods: Support Vector Learning. The MIT Press, Cambridge (1998)
Fletcher, R.: Practical methods of optimization, 2nd edn. Wiley- Interscience, Chichester (1987)
Tong, W., Alan, E., David, G.: A fast projected conjugate gradient algorithm for training support vector machines. Contemporary mathematics: theory and applications, 245–263 (2003)
Vladimir, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, Heidelberg (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gamboa, A.G., Gress, N.H., Mendoza, M.G., Vargas, J.M. (2009). Support Vector Optimization through Hybrids: Heuristics and Math Approach. In: Aguirre, A.H., Borja, R.M., Garciá, C.A.R. (eds) MICAI 2009: Advances in Artificial Intelligence. MICAI 2009. Lecture Notes in Computer Science(), vol 5845. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05258-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-05258-3_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05257-6
Online ISBN: 978-3-642-05258-3
eBook Packages: Computer ScienceComputer Science (R0)