Training Support Vector Machines with an Heterogeneous Particle Swarm Optimizer
Support vector machines are classification algorithms that have been successfully applied to problems in many different areas. Recently, evolutionary algorithms have been used to train support vector machines, which proved particularly useful in some multi-objective formulations and when indefinite kernels are used. In this paper, we propose a new heterogeneous particle swarm optimization algorithm, called scouting predator-prey optimizer, specially adapted for the training of support vector machines. We compare our algorithm with two other evolutionary approaches, using both positive definite and indefinite kernels, on a large set of benchmark problems. The experimental results confirm that the evolutionary algorithms can be competitive with the classic methods and even superior when using indefinite kernels. The scouting predator-prey optimizer can train support vector machines with similar or better classification accuracy than the other evolutionary algorithms, while requiring significantly less computational resources.
Keywordsparticle swarm optimization heterogeneous particle swarms support vector machines non PSD kernels
Unable to display preview. Download preview PDF.
- 2.Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000)Google Scholar
- 4.Lin, H.T., Lin, C.J.: A study on sigmoid kernel for svm and the training of non-psd kernels by smo-type methods. Technical report, National Taiwan University, Taipei, Department of Computer Science and Information Engineering (2003)Google Scholar
- 5.Ong, C.S., Mary, X., Canu, S., Smola, A.J.: Learning with non-positive kernels. In: Proceedings of the Twenty-First International Conference on Machine Learning, ICML 2004, ACM, New York (2004)Google Scholar
- 7.Mierswa, I.: Evolutionary learning with kernels: a generic solution for large margin problems. In: GECCO 2006: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 1553–1560. ACM, New York (2006)Google Scholar
- 10.Paquet, U., Engelbrecht, A.: Training support vector machines with particle swarms. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1593–1598 (2003)Google Scholar
- 11.Stoean, R., Preuss, M., Stoean, C., Dumitrescu, D.: Concerning the potential of evolutionary support vector machines. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 1436–1443 (2007)Google Scholar
- 14.Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
- 16.Poli, R.: Analysis of the publications on the applications of particle swarm optimisation. J. Artif. Evol. App., 4:1–4:10 (January 2008)Google Scholar
- 18.Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: Yale: Rapid prototyping for complex data mining tasks. In: Proceedings of the 12th International Conference on Knowledge Discovery and Data Mining, KDD 2006 (2006)Google Scholar