Evolutionary Training of SVM for Multiple Category Classification Problems with Self-adaptive Parameters

  • Ángel Kuri-Morales
  • Iván Mejía-Guevara
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4140)

Abstract

We describe a methodology to train Support Vector Machines (SVM) where the regularization parameter (C) is determined automatically via an efficient Genetic Algorithm in order to solve multiple category classification problems. We call the kind of SVMs where C is determined automatically from the application of a GA a “Genetic SVM” or GSVM. In order to test the performance of our GSVM, we solved a representative set of problems by applying one-versus-one majority voting and one-versus-all winner-takes-all strategies. In all of these the algorithm displayed very good performance. The relevance of the problem, the algorithm, the experiments and the results obtained are discussed.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Jordaan, E.M., Smits, G.F.: Estimation of the regularization parameter for support vector regression. In: Honolulu, I. (ed.) Proc. of World Conference on Computational Intelligence, Hawaii, pp. 2785–2791 (2002)Google Scholar
  2. 2.
    Schmitdt, M., Grish, H.: Speaker identification via support vector classifiers. In: Proc. of International Conference on Acoustics, Speech and Signal Processing, pp. 105–108 (1996)Google Scholar
  3. 3.
    Drucker, H., Wu, D., Vapnik, V.: Support vector machine for spam categorization. Trans. on Neural Networks 10, 1048-1054 (1999)Google Scholar
  4. 4.
    Vapnik, V., Golowich, S., Smola, A.: Support vector method for function approximation, regression, estimation and signal processing. Adv. Neural Inform. Process. Syst. 9, 281–287 (1996)Google Scholar
  5. 5.
    Haykin, S.: Neural Networks. A comprehensive foundation, 2nd edn. Prentice Hall, New Jersey (1999)MATHGoogle Scholar
  6. 6.
    Mercer, J.: Functions of positive and negative type, and their connection with the theory of integral equations. Transactions of the London Philosophical Society (A) 209, 415–446 (1909)CrossRefGoogle Scholar
  7. 7.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, UK (2004)Google Scholar
  8. 8.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, UK (2000)Google Scholar
  9. 9.
    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2(2), 121–167 (1998)CrossRefGoogle Scholar
  10. 10.
    Haykin, S.: op. cit., pp. 318–350Google Scholar
  11. 11.
    Haykin, S.: op. cit., pp. 326–329Google Scholar
  12. 12.
    Mitchell, M.: An Introduction to Genetic Algorithms. MIT Press, Cambridge (1996)Google Scholar
  13. 13.
    Burges, C.J.C.: op. cit., pp. 121–167Google Scholar
  14. 14.
    Cristianini, N.: op. cit., pp. 93–124Google Scholar
  15. 15.
    Kuri, A., Gutiérrez, J.: Penalty Functions Methods for Constrained Optimisation with Genetic Algorithms. In: Coello Coello, C.A., de Albornoz, Á., Sucar, L.E., Battistutti, O.C. (eds.) MICAI 2002. LNCS, vol. 2313, pp. 108–117. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  16. 16.
    Kuri, A., Gutiérrez, J.: op. cit., pp.109–111Google Scholar
  17. 17.
    Fung, G., Mangasarian, O.L.: Multicategory proximal support vector machine classifiers. Data Mining Institute Technical Report, 1–6 (2001)Google Scholar
  18. 18.
    Allwein, E.L., Shapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2000)CrossRefGoogle Scholar
  19. 19.
    Bojanov, B., Hakopian, H., Sahakian, B.: Spline Functions and Multivariate Interpolations. Springer, Heidelberg (1993)MATHGoogle Scholar
  20. 20.
    Kuri, A., Mejía, I.: Evolutionary Training of SVM for Classification Problems with Self- Adaptive Parameters. In: Gelbukh, A., Monroy, R. (eds.) Advances in Artificial Intelligence Theory, pp. 207–216. IPN (2005)Google Scholar
  21. 21.
    Kuri Morales, A.: A Methodology for the Statistical Characterization of Genetic Algorithms. In: Proceedings of the Mexican International Congress on Artificial Intelligence, pp. 79–88. Springer, Heidelberg (2002)Google Scholar
  22. 22.
    Jordaan, E.M., Smits, G.F.: Estimation of the regularization parameter for support vector regression. In: Honolulu, I. (ed.) Proc. of World Conference on Computational Intelligence, Hawaii, pp. 2785–2791 (2002)MATHGoogle Scholar
  23. 23.
    Osuna, E., Freund, R., Girosi, F.: Improved training algorithm for support vector machines. In: Proc. IEEE Neural Networks in Signal Processing 1997 (1997)Google Scholar
  24. 24.
    Mangasarian, O.L., Support Vector Machine Classification via Parameterless Robust Linear Programming, Data Mining Institute Technical Report 03-01, (2003)Google Scholar
  25. 25.
    Haykin, S., op. cit., pp. 293-294Google Scholar
  26. 26.
    Coello Coello, C.A., de Albornoz, Á., Sucar, L.E., Battistutti, O.C. (eds.): MICAI 2002. LNCS (LNAI), vol. 2313. Springer, Heidelberg (2002)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ángel Kuri-Morales
    • 1
  • Iván Mejía-Guevara
    • 2
  1. 1.Departamento de ComputaciónInstituto Tecnológico Autónomo de MèxicoMéxico
  2. 2.Posgrado en Ciencia e Ingeniería de la ComputaciónUniversidad Nacional Autónoma de México, IIMASMéxico

Personalised recommendations