Training Support Vector Machines with an Heterogeneous Particle Swarm Optimizer

  • Arlindo Silva
  • Teresa Gonçalves
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7824)


Support vector machines are classification algorithms that have been successfully applied to problems in many different areas. Recently, evolutionary algorithms have been used to train support vector machines, which proved particularly useful in some multi-objective formulations and when indefinite kernels are used. In this paper, we propose a new heterogeneous particle swarm optimization algorithm, called scouting predator-prey optimizer, specially adapted for the training of support vector machines. We compare our algorithm with two other evolutionary approaches, using both positive definite and indefinite kernels, on a large set of benchmark problems. The experimental results confirm that the evolutionary algorithms can be competitive with the classic methods and even superior when using indefinite kernels. The scouting predator-prey optimizer can train support vector machines with similar or better classification accuracy than the other evolutionary algorithms, while requiring significantly less computational resources.


particle swarm optimization heterogeneous particle swarms support vector machines non PSD kernels 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge Univ. Press, Cambridge (2004)CrossRefGoogle Scholar
  2. 2.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000)Google Scholar
  3. 3.
    Mierswa, I., Morik, K.: About the non-convex optimization problem induced by non-positive semidefinite kernel learning. Advances in Data Analysis and Classification 2, 241–258 (2008)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Lin, H.T., Lin, C.J.: A study on sigmoid kernel for svm and the training of non-psd kernels by smo-type methods. Technical report, National Taiwan University, Taipei, Department of Computer Science and Information Engineering (2003)Google Scholar
  5. 5.
    Ong, C.S., Mary, X., Canu, S., Smola, A.J.: Learning with non-positive kernels. In: Proceedings of the Twenty-First International Conference on Machine Learning, ICML 2004, ACM, New York (2004)Google Scholar
  6. 6.
    Mierswa, I.: Controlling overfitting with multi-objective support vector machines. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1830–1837. ACM, New York (2007)CrossRefGoogle Scholar
  7. 7.
    Mierswa, I.: Evolutionary learning with kernels: a generic solution for large margin problems. In: GECCO 2006: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 1553–1560. ACM, New York (2006)Google Scholar
  8. 8.
    Samanta, B., Nataraj, C.: Application of particle swarm optimization and proximal support vector machines for fault detection. Swarm Intelligence 3, 303–325 (2009)CrossRefGoogle Scholar
  9. 9.
    Gilsberts, A., Metta, G., Rothkrantz, L.: Evolutionary optimization of least-squares support vector machines. In: Data Mining. Annals of Information Systems, vol. 8, pp. 277–297. Springer, US (2010)CrossRefGoogle Scholar
  10. 10.
    Paquet, U., Engelbrecht, A.: Training support vector machines with particle swarms. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1593–1598 (2003)Google Scholar
  11. 11.
    Stoean, R., Preuss, M., Stoean, C., Dumitrescu, D.: Concerning the potential of evolutionary support vector machines. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 1436–1443 (2007)Google Scholar
  12. 12.
    Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)CrossRefGoogle Scholar
  13. 13.
    Haasdonk, B.: Feature space interpretation of svms with indefinite kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 482–492 (2005)CrossRefGoogle Scholar
  14. 14.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
  15. 15.
    Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization. Swarm Intelligence 1, 33–57 (2007)CrossRefGoogle Scholar
  16. 16.
    Poli, R.: Analysis of the publications on the applications of particle swarm optimisation. J. Artif. Evol. App., 4:1–4:10 (January 2008)Google Scholar
  17. 17.
    Silva, A., Neves, A., Gonçalves, T.: An Heterogeneous Particle Swarm Optimizer with Predator and Scout Particles. In: Kamel, M., Karray, F., Hagras, H. (eds.) AIS 2012. LNCS, vol. 7326, pp. 200–208. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: Yale: Rapid prototyping for complex data mining tasks. In: Proceedings of the 12th International Conference on Knowledge Discovery and Data Mining, KDD 2006 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Arlindo Silva
    • 1
  • Teresa Gonçalves
    • 2
  1. 1.Escola Superior de Tecnologia do Instituto Politécnico de Castelo BrancoPortugal
  2. 2.Universidade de ÉvoraPortugal

Personalised recommendations