Abstract
Hyper-parameter tuning for support vector machines has been widely studied in the past decade. A variety of metaheuristics, such as Genetic Algorithms and Particle Swarm Optimization have been considered to accomplish this task. Notably, exhaustive strategies such as Grid Search or Random Search continue to be implemented for hyper-parameter tuning and have recently shown results comparable to sophisticated metaheuristics. The main reason for the success of exhaustive techniques is due to the fact that only two or three parameters need to be adjusted when working with support vector machines. In this chapter, we analyze two Estimation Distribution Algorithms, the Univariate Marginal Distribution Algorithm and the Boltzmann Univariate Marginal Distribution Algorithm, to verify if these algorithms preserve the effectiveness of Random Search and at the same time make more efficient the process of finding the optimal hyper-parameters without increasing the complexity of Random Search.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
References
N. Christianini and J. Shawe-Taylor. An Introduction to SVM and other Kernel Based Methods. Cambridge, U.K.: Cambridge University Press, 2000.
V. Vapnik, Statistical Learning Theory, New York: John Wiley and Sons, 1998.
M. Kanevski, V. Timonin and A. Pozdnukhov, Machine Learning for Spatial Environmental Data: theory, applications and software, CRC Press, 2009.
O. Chapelle, V. Vapnik, O. Bousquet and S. Mukherjee, Choosing Multiple Parameters for Support Vector Machines Machine Learning, vol. 46 (1-3), pp. 131-159, 2002.
F. Friedrichs and C. Igel. Evolutionary tuning of multiple SVM parameters. Neurocomputing, vol 64 pp. 107-117, 2005.
C. Gold, A. Holub y P. Sollich. Bayesian approach to feature selection and parameter tuning for support vector machine classifiers. Neural Networks, vol 18 (5) pp. 693-701, 2005.
C.-L. Huang and W. Chieh-Jen. A GA-based feature selection and parameters optimization for support vector machines. Expert System with Applications, vol 31(2) pp. 231-240, 2006.
S.-W. Lin, K.-C. Ying y S.-C. L. Z.-J. Chen. Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines. Expert System with Applications, vol 35(4) pp. 1817-1824, 2008.
J. Bergstra and Y. Bengio. Random Search for Hyper-Parameter Optimization. Journal of Machine Learning Research, 13(1) pp. 281-305, 2012.
R. Mantovani, A. Rossi, J. Vanschoren and B. d.-C. A. Bischl. Effectiveness of Random Search in SVM hyper-parameter tuning. The International Joint Conference on Neural Networks (IJCNN), 2015.
A. Shigeo, Support Vector Machines for Pattern Classification, New York: Springer, 2010.
N. Deng, Y. Tian and C. Zhang, Support Vector Machines, Boca Raton: CRC Press, 2013.
M. Hauschild and M. Pelikan. An introduction and survey of estimation of distribution algorithms. Swarm and Evolutionary Computation, vol 1(3) pp. 111-128, 2011.
D. Simon, Evolutionary Optimization Algorithms: Biologically Inspired and Population-Based Approaches to Computer Intelligence, Hoboken: John Wiley and Sons, 2013.
S. I. Valdez, A. Hernández and S. Botello. A Boltzmann based estimation of distribution algorithm. Information Sciences, vol 236 pp. 126-137, 2013.
Acknowledgments
Luis Carlos Padierna and Alfonso Rojas wish to acknowledge the financial support of the Consejo Nacional de Ciencia y Tecnología (CONACYT grants 375524 and CATEDRAS-2598). The authors also thank Dr. Ivann Valdez from the Center of Research in Mathematics for his assistance and sharing his BUMDA-code.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Padierna, L.C., Carpio, M., Rojas, A., Puga, H., Baltazar, R., Fraire, H. (2017). Hyper-Parameter Tuning for Support Vector Machines by Estimation of Distribution Algorithms. In: Melin, P., Castillo, O., Kacprzyk, J. (eds) Nature-Inspired Design of Hybrid Intelligent Systems. Studies in Computational Intelligence, vol 667. Springer, Cham. https://doi.org/10.1007/978-3-319-47054-2_53
Download citation
DOI: https://doi.org/10.1007/978-3-319-47054-2_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47053-5
Online ISBN: 978-3-319-47054-2
eBook Packages: EngineeringEngineering (R0)