An Evolutionary Approach to Automatic Kernel Construction

  • Tom Howley
  • Michael G. Madden
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4132)


Kernel-based learning presents a unified approach to machine learning problems such as classification and regression. The selection of a kernel and associated parameters is a critical step in the application of any kernel-based method to a problem. This paper presents a data-driven evolutionary approach for constructing kernels, named KTree. An application of KTree to the Support Vector Machine (SVM) classifier is described. Experiments on a synthetic dataset are used to determine the best evolutionary strategy, e.g. what fitness function to use for kernel evaluation. The performance of an SVM based on KTree is compared with that of standard kernel SVMs on a synthetic dataset and on a number of real-world datasets. KTree is shown to outperform or match the best performance of all the standard kernels tested.


Support Vector Machine Radial Basis Function Synthetic Dataset Kernel Matrix Radial Basis Function Kernel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. Journal of Machine Learning Research 2 (2002)Google Scholar
  2. 2.
    Howley, T., Madden, M.G.: The Genetic Kernel Support Vector Machine: Description and Evaluation. Artificial Intelligence Review 24 (2005)Google Scholar
  3. 3.
    Scholkopf, B.: Statistical Learning and Kernel Methods. Technical Report MSR-TR-2000- 23, Microsoft Research, Microsoft Corporation (2000)Google Scholar
  4. 4.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)Google Scholar
  5. 5.
    Bahlmann, C., Haasdonk, B., Burkhardt, H.: On-line Handwriting Recognition with Support Vector Machines - A Kernel Approach. In: Proc. of the 8th Intl. Workshop on Frontiers in Handwriting Recognition (2002)Google Scholar
  6. 6.
    Lin, H., Lin, C.: A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods. Technical report, Dept. of Computer Science and Information Engineering, National Taiwan University (2003)Google Scholar
  7. 7.
    Cristianini, N., Shawe-Taylor, J.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar
  8. 8.
    Mangasarian, O., Musicant, D.: Lagrangian Support Vector Machines. Journal of Machine Learning Research 1 (2001)Google Scholar
  9. 9.
    Luke, S., Spector, L.: A Comparison of Crossover and Mutation in Genetic Programming. In: Genetic Programming: Proc. of the 2nd Annual Conference. Morgan Kaufmann, San Francisco (1997)Google Scholar
  10. 10.
    Newman, D., Hettich, S., Blake, C., Merz, C.: UCI Repository of machine learning databases (1998)Google Scholar
  11. 11.
    Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics 1, 80–83 (1945)CrossRefGoogle Scholar
  12. 12.
    Frolich, H., Chapelle, O., Scholkopf, B.: Feature Selection for SVMs by Means of Genetic Algorithms. In: Proc. of the Intl. IEEE Conference on Tools with AI, pp. 142–148 (2003)Google Scholar
  13. 13.
    Runarsson, T., Sigurdsson, S.: Asynchronous Parallel Evolutionary Model Selection for Support Vector Machines. Neural Information Processing - Letters and Reviews 3 (2004)Google Scholar
  14. 14.
    Friedrichs, F., Igel, C.: Evolutionary Tuning of Multiple SVM Parameters. In: Proc. of the 12th European Symposium on Artificial Neural Network, pp. 519–524 (2004)Google Scholar
  15. 15.
    Lessmann, S., Stahlbock, R., Crone, S.: Genetically constructed kernels for support vector machines. In: Proc. of German Operations Research, GOR (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Tom Howley
    • 1
  • Michael G. Madden
    • 1
  1. 1.National University of IrelandGalway

Personalised recommendations