Advertisement

Tuning and Evolving Support Vector Machine Models

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 659)

Abstract

Support vector machines (SVMs) are a well-established classifier, already applied in a variety of pattern recognition tasks. However, they suffer from several drawbacks—selecting their appropriate hyper-parameter values (the SVM model) along with the training sets being the most important. In this paper, we study the influence of applying various kernel functions in SVMs. We verify not only the classification performance of the classifier, but also the number of selected support vectors and the training time for each kernel. Also, we perform the qualitative analysis of the retrieved support vectors using an artificially generated dataset. Finally, we show how to optimize the SVM models using a genetic algorithm. An extensive experimental study revealed that evolved SVM models provide high-quality classification and are retrieved in much shorter time compared with the trial-and-error approaches.

Keywords

Support vector machine Kernel function Hyper-parameters Genetic algorithm Classification 

Notes

Acknowledgements

This research was supported by the Polish National Centre for Research and Development under the Innomed Research and Development Grant No. POIR.01.02.00-00-0030/15, and by the Institute of Informatics (Silesian University of Technology) research grant no. BKM-507/RAU2/2016.

References

  1. 1.
    Alamdar, F., Ghane, S., Amiri, A.: On-line twin independent support vector machines. Neurocomputing 186, 8–21 (2016)CrossRefGoogle Scholar
  2. 2.
    Ali, S., Smith-Miles, K.A.: A meta-learning approach to automatic kernel selection for support vector machines. Neurocomputing 70(1–3), 173–186 (2006)CrossRefGoogle Scholar
  3. 3.
    Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: COLT 1992, Pittsburgh, PA, USA, pp. 144–152 (1992)Google Scholar
  4. 4.
    Chou, J.S., Cheng, M.Y., Wu, Y.W., Pham, A.D.: Optimizing parameters of support vector machine using fast messy genetic algorithm for dispute classification. Expert Syst. Appl. 41(8), 3955–3964 (2014)CrossRefGoogle Scholar
  5. 5.
    Devos, O., Downey, G., Duponchel, L.: Simultaneous data pre-processing and SVM classification model selection based on a parallel genetic algorithm applied to spectroscopic data of olive oils. Food Chem. 148, 124–130 (2014)CrossRefGoogle Scholar
  6. 6.
    Elangovan, M., Sugumaran, V., Ramachandran, K., Ravikumar, S.: Effect of SVM kernels on classification of vibration signals of a single point cutting tool. Expert Syst. Appl. 38(12), 15202–15207 (2011)CrossRefGoogle Scholar
  7. 7.
    Friedrichs, F., Igel, C.: Evolutionary tuning of multiple SVM parameters. Neurocomputing 64, 107–117 (2005)CrossRefGoogle Scholar
  8. 8.
    Jebara, T.: Multi-task feature and kernel selection for SVMs. In: ICML 2004, Banff, Alberta, Canada, pp. 55–63 (2004)Google Scholar
  9. 9.
    Kapp, M.N., Sabourin, R., Maupin, P.: A dynamic model selection strategy for support vector machine classifiers. Appl. Soft Comput. 12(8), 2550–2565 (2012)CrossRefGoogle Scholar
  10. 10.
    Kawulok, M., Nalepa, J., Dudzik, W.: An alternating genetic algorithm for selecting SVM model and training set. In: Carrasco-Ochoa, J., et al. (eds.) Pattern Recognition. LNCS, vol. 10267, pp. 94–104. Springer, Cham (2017)Google Scholar
  11. 11.
    Khemchandani, R., Jayadeva, Chandra, S.: Optimal kernel selection in twin support vector machines. Optim. Lett. 3(1), 77–88 (2009)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Lebrun, G., Charrier, C., Lezoray, O., Cardot, H.: Tabu search model selection for SVM. Int. J. Neural Syst. 18(01), 19–31 (2008)CrossRefGoogle Scholar
  13. 13.
    Lessmann, S., Stahlbock, R., Crone, S.F.: Genetic algorithms for support vector machine model selection. In: IEEE IJCNN 2006, Vancouver, Canada, pp. 3063–3069 (2006)Google Scholar
  14. 14.
    Luxburg, U.v., Bousquet, O., Schülkopf, B.: A compression approach to support vector model selection. J. Mach. Learn. Res. 5, 293–323 (2004)Google Scholar
  15. 15.
    Nalepa, J.: Genetic and memetic algorithms for selection of training sets for support vector machines. Ph.D. thesis, Silesian University of Technology, Poland (2016)Google Scholar
  16. 16.
    Nalepa, J., Czech, Z.J.: New selection schemes in a memetic algorithm for the vehicle routing problem with time windows. In: Tomassini, M., Antonioni, A., Daolio, F., Buesser, P. (eds.) Adaptive and Natural Computing Algorithms. LNCS, vol. 7824, pp. 396–405. Springer, Heidelberg (2013)Google Scholar
  17. 17.
    Nalepa, J., Kawulok, M.: Adaptive memetic algorithm enhanced with data geometry analysis to select training data for SVMs. Neurocomputing 185, 113–132 (2016)CrossRefGoogle Scholar
  18. 18.
    Nalepa, J., Siminski, K., Kawulok, M.: Towards parameter-less support vector machines. In: ACPR 2015, Kuala Lumpur, Malaysia, pp. 211–215 (2015)Google Scholar
  19. 19.
    Phung, S., Bouzerdoum, A., Chai, D.: Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 148–154 (2005)CrossRefGoogle Scholar
  20. 20.
    Ripepi, G., Clematis, A., DAgostino, D.: A hybrid parallel implementation of model selection for SVMs. In: PDP 2015, Turku, Finland, pp. 145–149 (2015)Google Scholar
  21. 21.
    Rodriguez-Lujan, I., Cruz, C.S., Huerta, R.: Hierarchical linear support vector machine. Pattern Recogn. 45(12), 4414–4427 (2012)CrossRefMATHGoogle Scholar
  22. 22.
    Simiński, K.: Neuro-fuzzy system based kernel for classification with support vector machines. In: Gruca, A., Czachórski, T., Kozielski, S. (eds.) Man-Machine Interactions 3. AISC, vol. 242, pp. 415–422. Springer, Switzerland (2014)Google Scholar
  23. 23.
    Sullivan, K.M., Luke, S.: Evolving kernels for support vector machine classification. In: GECCO 2007, London, England, pp. 1702–1707 (2007)Google Scholar
  24. 24.
    Tang, Y., Guo, W., Gao, J.: Efficient model selection for support vector machine with Gaussian kernel function. In: CIDM 2009, Nashville, TN, USA, pp. 40–45 (2009)Google Scholar
  25. 25.
    Wang, D., Qiao, H., Zhang, B., Wang, M.: Online support vector machine based on convex hull vertices selection. IEEE Trans. Neural Netw. Learn. Syst. 24(4), 593–609 (2013)CrossRefGoogle Scholar
  26. 26.
    Zhang, X., Song, Q.: A multi-label learning based kernel automatic recommendation method for support vector machine. PLoS One 10, e0120455 (2015)CrossRefGoogle Scholar
  27. 27.
    Zhou, X., Xu, J.: A SVM model selection method based on hybrid genetic algorithm and empirical error minimization criterion. In: Wang, H., Shen, Y., Huang, T., Zeng, Z. (eds.) The Sixth International Symposium on Neural Networks. AISC, vol. 56, pp. 245–253. Springer, Heidelberg (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Jakub Nalepa
    • 1
    • 2
  • Michal Kawulok
    • 1
    • 2
  • Wojciech Dudzik
    • 2
  1. 1.Silesian University of TechnologyGliwicePoland
  2. 2.Future ProcessingGliwicePoland

Personalised recommendations