Using a Scouting Predator-Prey Optimizer to Train Support Vector Machines with non PSD Kernels

Part of the Studies in Computational Intelligence book series (SCI, volume 512)

Abstract

In this paper, we investigate the use of an heterogeneous particle swarm optimizer, the scouting predator-prey optimizer, to train support vector machines with non positive definite kernels, including distance substitution based kernels. These kernels can arise in practical applications, resulting in multi-modal optimization problems where traditional algorithms can struggle to find the global optimum. We compare the scouting predator-prey algorithm with the previous best evolutionary approach to this problem and a standard quadratic programming based algorithm, on a large set of benchmark problems, using various non positive definite kernels. The use of cooperating scout particles allows the proposed algorithm to be more efficient than the other evolutionary approach, which is based on an evolution strategy. Both are shown to perform better than the standard algorithm in several dataset/kernel instances, a result that underlines the usefulness of evolutionary training algorithms for support vector machines.

Keywords

particle swarm optimization heterogeneous particle swarms support vector machines non PSD kernels 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge Univ. Press, Cambridge (2004)CrossRefGoogle Scholar
  2. 2.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000)Google Scholar
  3. 3.
    Haasdonk, B., Bahlmann, C.: Learning with distance substitution kernels. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 220–227. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Haasdonk, B.: Feature space interpretation of svms with indefinite kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 482–492 (2005)CrossRefGoogle Scholar
  5. 5.
    Lin, H.T., Lin, C.J.: A study on sigmoid kernel for svm and the training of non-psd kernels by smo-type methods. Technical report, National Taiwan University, Taipei, Department of Computer Science and Information Engineering (2003)Google Scholar
  6. 6.
    Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)CrossRefGoogle Scholar
  7. 7.
    Rüping, S.: mySVM-Manual. University of Dortmund, Lehrstuhl Informatik 8 (2000)Google Scholar
  8. 8.
    Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines (2001)Google Scholar
  9. 9.
    Ong, C.S., Mary, X., Canu, S., Smola, A.J.: Learning with non-positive kernels. In: Proceedings of the Twenty-First International Conference on Machine learning, ICML 2004. ACM, New York (2004)Google Scholar
  10. 10.
    Mierswa, I., Morik, K.: About the non-convex optimization problem induced by non-positive semidefinite kernel learning. Advances in Data Analysis and Classification 2, 241–258 (2008)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Samanta, B., Nataraj, C.: Application of particle swarm optimization and proximal support vector machines for fault detection. Swarm Intelligence 3, 303–325 (2009)CrossRefGoogle Scholar
  12. 12.
    Gilsberts, A., Metta, G., Rothkrantz, L.: Evolutionary optimization of least-squares support vector machines. In: Data Mining. Annals of Information Systems, vol. 8, pp. 277–297. Springer, US (2010)CrossRefGoogle Scholar
  13. 13.
    Paquet, U., Engelbrecht, A.: Training support vector machines with particle swarms. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1593–1598 (2003)Google Scholar
  14. 14.
    Stoean, R., Preuss, M., Stoean, C., Dumitrescu, D.: Concerning the potential of evolutionary support vector machines. In: IEEE Congress on Evolutionary Computation, CEC 2007, pp. 1436–1443 (2007)Google Scholar
  15. 15.
    Mierswa, I.: Evolutionary learning with kernels: a generic solution for large margin problems. In: GECCO 2006: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 1553–1560. ACM, New York (2006)Google Scholar
  16. 16.
    Mierswa, I.: Controlling overfitting with multi-objective support vector machines. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1830–1837. ACM, New York (2007)CrossRefGoogle Scholar
  17. 17.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)Google Scholar
  18. 18.
    Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization. Swarm Intelligence 1, 33–57 (2007)CrossRefGoogle Scholar
  19. 19.
    Poli, R.: Analysis of the publications on the applications of particle swarm optimisation. J. Artif. Evol. App. 2008, 4:1–4:10 (2008)Google Scholar
  20. 20.
    Silva, A., Neves, A., Gonçalves, T.: An heterogeneous particle swarm optimizer with predator and scout particles. In: Kamel, M., Karray, F., Hagras, H. (eds.) AIS 2012. LNCS, vol. 7326, pp. 200–208. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  21. 21.
    Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: Yale: Rapid prototyping for complex data mining tasks. In: Proceedings of the 12th International Conference on Knowledge Discovery and Data Mining, KDD 2006 (2006)Google Scholar
  22. 22.
    Silva, A., Gonçalves, T.: Training support vector machines with an heterogeneous particle swarm optimizer. In: Tomassini, M., Antonioni, A., Daolio, F., Buesser, P. (eds.) ICANNGA 2013. LNCS, vol. 7824, pp. 100–109. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  23. 23.
    Meyer, D., Leisch, F., Hornik, K.: Benchmarking support vector machines. In: SFB Adaptive Information Systems and Modelling in Economics and Management Science, vol. 78. WU Vienna University of Economics and Business, Vienna (2002)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Escola Superior de Tecnologia do Instituto Politécnico de Castelo BrancoCastelo BrancoPortugal
  2. 2.Universidade de ÉvoraEvoraPortugal

Personalised recommendations