Abstract
In this paper, we investigate the use of an heterogeneous particle swarm optimizer, the scouting predator-prey optimizer, to train support vector machines with non positive definite kernels, including distance substitution based kernels. These kernels can arise in practical applications, resulting in multi-modal optimization problems where traditional algorithms can struggle to find the global optimum. We compare the scouting predator-prey algorithm with the previous best evolutionary approach to this problem and a standard quadratic programming based algorithm, on a large set of benchmark problems, using various non positive definite kernels. The use of cooperating scout particles allows the proposed algorithm to be more efficient than the other evolutionary approach, which is based on an evolution strategy. Both are shown to perform better than the standard algorithm in several dataset/kernel instances, a result that underlines the usefulness of evolutionary training algorithms for support vector machines.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge Univ. Press, Cambridge (2004)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000)
Haasdonk, B., Bahlmann, C.: Learning with distance substitution kernels. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 220–227. Springer, Heidelberg (2004)
Haasdonk, B.: Feature space interpretation of svms with indefinite kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 482–492 (2005)
Lin, H.T., Lin, C.J.: A study on sigmoid kernel for svm and the training of non-psd kernels by smo-type methods. Technical report, National Taiwan University, Taipei, Department of Computer Science and Information Engineering (2003)
Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)
Rüping, S.: mySVM-Manual. University of Dortmund, Lehrstuhl Informatik 8 (2000)
Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines (2001)
Ong, C.S., Mary, X., Canu, S., Smola, A.J.: Learning with non-positive kernels. In: Proceedings of the Twenty-First International Conference on Machine learning, ICML 2004. ACM, New York (2004)
Mierswa, I., Morik, K.: About the non-convex optimization problem induced by non-positive semidefinite kernel learning. Advances in Data Analysis and Classification 2, 241–258 (2008)
Samanta, B., Nataraj, C.: Application of particle swarm optimization and proximal support vector machines for fault detection. Swarm Intelligence 3, 303–325 (2009)
Gilsberts, A., Metta, G., Rothkrantz, L.: Evolutionary optimization of least-squares support vector machines. In: Data Mining. Annals of Information Systems, vol. 8, pp. 277–297. Springer, US (2010)
Paquet, U., Engelbrecht, A.: Training support vector machines with particle swarms. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1593–1598 (2003)
Stoean, R., Preuss, M., Stoean, C., Dumitrescu, D.: Concerning the potential of evolutionary support vector machines. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 1436–1443 (2007)
Mierswa, I.: Evolutionary learning with kernels: a generic solution for large margin problems. In: GECCO 2006: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 1553–1560. ACM, New York (2006)
Mierswa, I.: Controlling overfitting with multi-objective support vector machines. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1830–1837. ACM, New York (2007)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)
Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization. Swarm Intelligence 1, 33–57 (2007)
Poli, R.: Analysis of the publications on the applications of particle swarm optimisation. J. Artif. Evol. App. 2008, 4:1–4:10 (2008)
Silva, A., Neves, A., Gonçalves, T.: An heterogeneous particle swarm optimizer with predator and scout particles. In: Kamel, M., Karray, F., Hagras, H. (eds.) AIS 2012. LNCS, vol. 7326, pp. 200–208. Springer, Heidelberg (2012)
Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: Yale: Rapid prototyping for complex data mining tasks. In: Proceedings of the 12th International Conference on Knowledge Discovery and Data Mining, KDD 2006 (2006)
Silva, A., Gonçalves, T.: Training support vector machines with an heterogeneous particle swarm optimizer. In: Tomassini, M., Antonioni, A., Daolio, F., Buesser, P. (eds.) ICANNGA 2013. LNCS, vol. 7824, pp. 100–109. Springer, Heidelberg (2013)
Meyer, D., Leisch, F., Hornik, K.: Benchmarking support vector machines. In: SFB Adaptive Information Systems and Modelling in Economics and Management Science, vol. 78. WU Vienna University of Economics and Business, Vienna (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Silva, A., Gonçalves, T. (2014). Using a Scouting Predator-Prey Optimizer to Train Support Vector Machines with non PSD Kernels. In: Terrazas, G., Otero, F., Masegosa, A. (eds) Nature Inspired Cooperative Strategies for Optimization (NICSO 2013). Studies in Computational Intelligence, vol 512. Springer, Cham. https://doi.org/10.1007/978-3-319-01692-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-01692-4_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-01691-7
Online ISBN: 978-3-319-01692-4
eBook Packages: EngineeringEngineering (R0)