Bio-inspired Metaheuristics for Hyper-parameter Tuning of Support Vector Machine Classifiers
Support Vector Machines (SVMs) are machine learning models with many diverse applications. The performance of these models depends on a set of assigned hyper-parameters. The task of hyper-parameter tuning has been performed by metaheuristics methods and recent studies have shown that the effectiveness of these methods is statistically equivalent. In this work we compare four bio-inspired metaheuristics (Bat Algorithm, Firefly Algorithm, Particle Swarm Optimization Algorithm and Social Emotional Optimization Algorithm) to test the hypothesis that the efficiency among these differs while the effectiveness remains. Experimental results on several classification problems indicate that there exist bio-inspired algorithms with higher efficiency, in terms of the required number of SVM evaluations to find optimal hyper-parameters. Based on these results the Bat Algorithm is recommended for SVM hyper-parameter tuning.
KeywordsBio-inspired metaheuristics Hyper-parameter optimization Support vector machines Pattern classification
This work was partially supported by the National Council of Science and Technology (CONACYT) of Mexico [grant numbers: 375524 (Luis C. Padierna), CATEDRAS-2598 (A. Rojas), 416761 (Adán Godínez)].
- 1.V.N. Vapnik, Statistical Learning Theory (New York, 1998)Google Scholar
- 2.M. Heydari, M. Teimouri, Z. Heshmati, S.M. Alavinia, Comparison of various classification algorithms in the diagnosis of type 2 diabetes in Iran. Int. J. Diab. Dev. Countries (2015)Google Scholar
- 4.V.N. Vapnik, An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999)Google Scholar
- 5.L.C. Padierna, A. Rojas, Hyper-parameter tuning for support vector machines by estimation of distribution algorithms, pp. 787–800 (2017)Google Scholar
- 7.X. Yang, A new metaheuristic bat-inspired algorithm, pp. 1–10 (2010)Google Scholar
- 8.X. Yang, L. Press, Nature-Inspired Metaheuristic Algorithms Second Edition Google Scholar
- 9.J. Kennedy, R. Eberhart, Particle swarm optimization, pp. 1942–1948 (1995)Google Scholar
- 10.Z. Cui, Y. Xu, J. Zeng, Social emotional optimization algorithm with random emotional selection strategy. Theory New Appl. Swarm Intell (2012)Google Scholar
- 11.C.Z. Naiyang Deng, Y. Tian, Support Vector Machines (CRC Press, 2013)Google Scholar
- 12.J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis (Cambridge, 2004)Google Scholar
- 13.R. Poli, J. Kennedy, T. Blackwell, Particle swarm optimization an overview (2007)Google Scholar