Abstract
Support Vector Machines (SVMs) have become a well succeed algorithm due to the good performance it achieves on different learning problems. However, to perform well the SVM formulation requires adjustments on its model. Avoiding the trial and error procedure, the automatic SVM parameter selection is a way to deal with this. The automatic parameter selection is commonly considered an optimization problem whose goal is to find suitable configuration of parameters which attends some learning problem.
In the current work, we propose a study of the combination of Meta-learning (ML) with Particle Swarm Optimization (PSO) algorithms to optimize the SVM model, seeking for combinations of parameters which maximize the success rate of SVM. ML is used to recommend SVM parameters, to a given input problem, based on well-succeeded parameters adopted in previous similar problems. In this combination, initial solutions provided by ML are possibly located in good regions in the search space. Hence, using a reduced number of candidate search points, in the search process, to find an adequate solution, would be less expensive.
In our work, we implemented five benchmarks PSO approaches applied to select two SVM parameters for classification. The experiments consist in comparing the performance of the search algorithms using a traditional random initialization and using ML suggestions as initial population. This research analysed the influence of meta-learning on convergence of the optimization algorithms, verifying that the combination of PSO techniques with ML obtained solutions with higher quality on a set of 40 classification problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Soares, C., Brazdil, P., Kuba, P.: A meta-learning approach to select the kernel width in support vector regression (4), 195–209 (2000)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press (2000)
Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing multiple parameters for support vector machines. Machine Learning, 131–159 (2002)
Cristianini, N., Campbell, C., Shawe-Taylor, J.: Dynamically adapting kernels in support vector machines. In: NIPS, pp. 204–210 (1998)
Ali, S., Smith-Miles, K.: On optimal degree selection for polynomial kernel with support vector machines: Theoretical and empirical investigations. KES Journal 1(1), 1–18 (2007)
Narzisi, G.: An Experimental Multi-Objective Study of the SVM Model Selection problem
Lessmann, S., Stahlbock, R., Crone, S.: Genetic algorithms for support vector machine model selection. In: International Joint Conference on Neural Networks, pp. 3063–3069 (2006)
Friedrichs, F., Igel, C.: Evolutionary tuning of multiple svm parameters. Neurocomputing (2005)
Lorena, A., de Carvalho, A.: Evolutionary tuning of svm parameter values in multiclass problems. Neurocomputing, 16–18
Keerthi, S.S.: Efficient tuning of svm hyperparameters using radius/margin bound and iterative algorithms. IEEE Transactions on Neural Networks (2002)
Kuba, P., Brazdil, P., Soares, C., Woznica, A.: Exploiting sampling and meta-learning for parameter setting support vector machines. In: Proceedings of the IBERAMIA, pp. 217–225 (2001)
Cawley, G.: Model selection for support vector machines via adaptive step-size tabu search. In: Proceedings of the International Conference on Artificial Neural Networks and Genetic Algorithms, vol. 28(8), pp. 434–437 (2001)
de Souza, B., de Carvalho, A., Ishii, R.: Multiclass svm model selection using particle swarm optimization. In: Sixth International Conference on Hybrid Intelligent Systems (2006)
Glasmachers, T., Igel, C.: Gradient-based adaptation of general gaussian kernels. Neural Comput. (2005)
Ali, S., Smith-Miles, K.A.: A meta-learning approach to automatic kernel selection for support vector machines. Neurocomputing, 173–186 (2006)
Soares, C., Brazdil, P.: Selecting parameters of svm using meta-learning and kernel matrix-based meta-features. In: SAC (2006)
Ali, S., Smith, K.A.: Matching svm kernel’s suitability to data characteristics using tree by fuzzy c-means clustering. In: Third International Conference on Hybrid Intelligent Systems (2010)
Gomes, T., Prudencio, R.B.C., Soares, C., Rossi, A., Carvalho, A.: Combining meta-learning and search techniques to svm parameter selection. In: Brazilian Symposium on Neural Networks, pp. 79–84 (2010)
Keerthi, S.S., Lin, C.-J.: Asymptotic behaviors of support vector machines with gaussian kernel. In: Imielinski, T., Korth, H. (eds.) Mobile Computing, pp. 153–181. Kluwer Academic Publishers (1996)
Hsu, C.-W., Chang, C.-C., Lin, C.-J.: Scikit-Learn: A Toolkit of Machine Learning in Python, http://scikit-learn.org
Engelbrecht, A.P.: Fundamentals of Computational Swarm Intelligence. John Wiley & Sons (2005)
Sutton, A.M., Whitley, D., Lunacek, M., Howe, A.: PSO and multi-funnel landscapes: how cooperation might limit exploration. In: GECCO 2006: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, July 8-12, vol. 1, pp. 75–82. ACM Press, Seattle (2006)
WEKA, The University of Waikato, http://www.cs.waikato.ac.nz/ml/weka/
UCI Machine Learning Repository, http://archive.ics.uci.edu/ml/
Smith-Miles, K. A.: Cross-Disciplinary Perspectives on Meta-Learning for Algorithm Selection (2005)
Prudêncio, R.B.C., Ludermir, T.B.: Selective generation of training examples in active meta-learning. International Journal of Hybrid Intelligent Systems (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
de Miranda, P.B.C., Prudêncio, R.B.C., de Carvalho, A.C.P.L.F., Soares, C. (2012). An Experimental Study of the Combination of Meta-Learning with Particle Swarm Algorithms for SVM Parameter Selection. In: Murgante, B., et al. Computational Science and Its Applications – ICCSA 2012. ICCSA 2012. Lecture Notes in Computer Science, vol 7335. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31137-6_43
Download citation
DOI: https://doi.org/10.1007/978-3-642-31137-6_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31136-9
Online ISBN: 978-3-642-31137-6
eBook Packages: Computer ScienceComputer Science (R0)