Skip to main content
SpringerLink
Log in

Parameter investigation of support vector machine classifier with kernel functions

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Support vector machine (SVM) is one of the well-known learning algorithms for classification and regression problems. SVM parameters such as kernel parameters and penalty parameter have a great influence on the complexity and performance of predicting models. Hence, the model selection in SVM involves the penalty parameter and kernel parameters. However, these parameters are usually selected and used as a black box, without understanding the internal details. In this paper, the behavior of the SVM classifier is analyzed when these parameters take different values. This analysis consists of illustrative examples, visualization, and mathematical and geometrical interpretations with the aim of providing the basics of kernel functions with SVM and to show how it works to serve as a comprehensive source for researchers who are interested in this field. This paper starts by highlighting the definition and underlying principles of SVM in details. Moreover, different kernel functions are introduced and the impact of each parameter in these kernel functions is explained from different perspectives.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. Some books and articles use the parameter \(\gamma =\frac{1}{2\sigma ^2}\).

  2. The minimum distance was zero as mentioned before; but, with \(C=0\), the penalty parameter is completely neglected. Therefore, we consider that the minimum distance is 0.1.

References

  1. Melki G, Kecman V, Ventura S, Cano A (2018) OLLAWV: online learning algorithm using worst-violators. Appl Soft Comput 66:384–393

    Article  Google Scholar 

  2. Melki G, Cano A, Ventura S (2018) MIRSVM: multi-instance support vector machine with bag representatives. Pattern Recognit 79:228–241

    Article  Google Scholar 

  3. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  4. Byvatov E, Schneider G (2002) Support vector machine applications in bioinformatics. Appl Bioinform 2(2):67–77

    Google Scholar 

  5. Vatsa M, Singh R, Noore A (2005) Improving biometric recognition accuracy and robustness using DWT and SVM watermarking. IEICE Electron. Express 2(12):362–367

    Article  Google Scholar 

  6. Moulin L, Da Silva AA, El-Sharkawi M, Marks RJ (2004) Support vector machines for transient stability analysis of large-scale power systems. IEEE Trans Power Syst 19(2):818–825

    Article  Google Scholar 

  7. Doucet JP, Barbault F, Xia H, Panaye A, Fan B (2007) Nonlinear SVM approaches to QSPR/QSAR studies and drug design. Curr Comput Aided Drug Des 3(4):263–289

    Article  Google Scholar 

  8. Wang L (2005) Support vector machines: theory and applications, vol 177. Springer, Berlin

    Book  MATH  Google Scholar 

  9. Melki G, Cano A, Kecman V, Ventura S (2017) Multi-target support vector regression via correlation regressor chains. Inf Sci 415:53–69

    Article  MathSciNet  Google Scholar 

  10. Friedrichs F, Igel C (2005) Evolutionary tuning of multiple SVM parameters. Neurocomputing 64:107–117

    Article  Google Scholar 

  11. Tharwat A, Hassanien AE (2018) Chaotic antlion algorithm for parameter optimization of support vector machine. Appl Intell 48(3):670–686

    Article  Google Scholar 

  12. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159

    Article  MATH  Google Scholar 

  13. Wu CH, Tzeng GH, Lin RH (2009) A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36(3):4725–4735

    Article  Google Scholar 

  14. Subasi A (2013) Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders. Comput Biol Med 43(5):576–586

    Article  Google Scholar 

  15. Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824

    Article  Google Scholar 

  16. Zhang X, Chen X, He Z (2010) An ACO-based algorithm for parameter optimization of support vector machines. Expert Syst Appl 37(9):6618–6628

    Article  Google Scholar 

  17. Tharwat A, Hassanien AE, Elnaghi BE (2017) A BA-based algorithm for parameter optimization of support vector machine. Pattern Recognit Lett 93:13–22

    Article  Google Scholar 

  18. Ali S, Smith K (2003) Automatic parameter selection for polynomial kernel. In: Proceedings of IEEE international conference on information reuse and integration (IRI 2003), Lens, France, October 27–29, IEEE, pp 243–249

  19. Abu-Mostafa YS, Magdon-Ismail M, Lin HT (2012) Learning from data, vol 4. AML Book, New York

    Google Scholar 

  20. Tharwat A (2016) Linear vs. quadratic discriminant analysis classifier: a tutorial. Int J Appl Pattern Recognit 3(2):145–180

    Article  Google Scholar 

  21. Scholköpf B, Smola AJ (2001) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

    Google Scholar 

  22. Nilsson NJ (1965) Learning machines: foundations of trainable pattern-classifying systems, 1st edn. McGraw-Hill, New York

    MATH  Google Scholar 

  23. Schölkopf B, Burges CJ (1999) Advances in kernel methods: support vector learning. MIT Press, Cambridge

    MATH  Google Scholar 

  24. Ben-Hur A, Weston J (2010) A user’s guide to support vector machines. In: Carugo O, Eisenhaber F (eds) Data mining techniques for the life sciences. Methods in molecular biology (Methods and protocols), vol 609. Humana Press

  25. Kecman V (2001) Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. MIT Press, Cambridge

    MATH  Google Scholar 

  26. Guyon I, Boser B, Vapnik V (1993) Automatic capacity tuning of very large VC-dimension classifiers. In: Hanson SJ, Cowan JD, Giles CL (eds) Advances in neural information processing systems 5. Morgan-Kaufmann

  27. Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222

    Article  MathSciNet  Google Scholar 

  28. Keerthi SS, Lin CJ (2003) Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Comput 15(7):1667–1689

    Article  MATH  Google Scholar 

  29. Ben-Hur A, Ong CS, Sonnenburg S, Schölkopf B, Rätsch G (2008) Support vector machines and kernels for computational biology. PLoS Comput Biol 4(10):e1000173

    Article  Google Scholar 

  30. Lin CJ (2001) Formulations of support vector machines: a note from an optimization point of view. Neural Comput 13(2):307–317

    Article  MATH  Google Scholar 

  31. Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167

    Article  Google Scholar 

  32. Smits GF, Jordaan EM (2002) Improved SVM regression using mixtures of kernels. In: Proceedings of the 2002 international joint conference on neural networks (IJCNN’02), vol 3. IEEE, pp 2785–2790

  33. Chen F, Tang B, Song T, Li L (2014) Multi-fault diagnosis study on roller bearing based on multi-kernel support vector machine with chaotic particle swarm optimization. Measurement 47:576–590

    Article  Google Scholar 

  34. Cortes C, Mohri M, Rostamizadeh A (2012) Algorithms for learning kernels based on centered alignment. J Mach Learn Res 13(Mar):795–828

    MathSciNet  MATH  Google Scholar 

  35. Tsuda K, Uda S, Kin T, Asai K (2004) Minimizing the cross validation error to mix kernel matrices of heterogeneous biological data. Neural Process Lett 19(1):63–72

    Article  Google Scholar 

  36. Yeh CY, Huang CW, Lee SJ (2011) A multiple-kernel support vector regression approach for stock market price forecasting. Expert Syst Appl 38(3):2177–2186

    Article  Google Scholar 

  37. Devos O, Ruckebusch C, Durand A, Duponchel L, Huvenne JP (2009) Support vector machines (SVM) in near infrared (NIR) spectroscopy: focus on parameters optimization and model interpretation. Chemom Intell Lab Syst 96(1):27–33

    Article  Google Scholar 

  38. Arana-Daniel N, Gallegos AA, López-Franco C, Alanís AY, Morales J, López-Franco A (2016) Support vector machines trained with evolutionary algorithms employing kernel adatron for large scale classification of protein structures. Evolut Bioinform Online 12:285

    Google Scholar 

  39. Melgani F, Bazi Y (2008) Classification of electrocardiogram signals with support vector machines and particle swarm optimization. IEEE Trans Inf Technol Biomed 12(5):667–677

    Article  Google Scholar 

  40. Tharwat A, Moemen YS, Hassanien AE (2017) Classification of toxicity effects of biotransformed hepatic drugs using whale optimized support vector machines. J Biomed Inform 68:132–149

    Article  Google Scholar 

  41. Taie SA, Ghonaim W (2017) CSO-based algorithm with support vector machine for brain tumor’s disease diagnosis. In: Proceedings of IEEE international conference on pervasive computing and communications workshops (PerCom Workshops), IEEE, pp 183–187

  42. Yuan SF, Chu FL (2007) Fault diagnostics based on particle swarm optimisation and support vector machines. Mech Syst Signal Process 21(4):1787–1798

    Article  Google Scholar 

  43. Tang X, Zhuang L, Cai J, Li C (2010) Multi-fault classification based on support vector machine trained by chaos particle swarm optimization. Knowl Based Syst 23(5):486–490

    Article  Google Scholar 

  44. Li X, Zhang X, Li C, Zhang L et al (2013) Rolling element bearing fault detection using support vector machine with improved ant colony optimization. Measurement 46(8):2726–2734

    Article  Google Scholar 

  45. Can H, Jianchun X, Ruide Z, Juelong L, Qiliang Y, Liqiang X (2013) A new model for software defect prediction using particle swarm optimization and support vector machine. In: 25th conference Chinese control and decision conference (CCDC), IEEE, pp 4106–4110

  46. Wei J, Jian-qi Z, Xiang Z (2011) Face recognition method based on support vector machine and particle swarm optimization. Expert Syst Appl 38(4):4390–4393

    Article  Google Scholar 

  47. Xiao T, Ren D, Lei S, Zhang J, Liu X (2014) Based on grid-search and PSO parameter optimization for support vector machine. In: Proceedings of the 11th world congress on intelligent control and automation (WCICA), IEEE, pp 1529–1533

  48. Abdulameer MH, Abdullah S, Huda SN, Othman ZA (2014) Support vector machine based on adaptive acceleration particle swarm optimization. Sci World J 2014:1–8

    Google Scholar 

  49. Li L, Yan Shi D, Xu J (2013) Color image segmentation based-on SVM using mixed features and combined kernel. In: International conference on intelligent science and big data engineering, Springer, pp 401–409

  50. Kharrat A, Abid M (2014) Toward efficient segmentation of brain tumors based on support vector machine classifier through optimized RBF kernel parameters and optimal texture features. Int J Cognit Inf Nat Intell (IJCINI) 8(2):15–33

    Article  Google Scholar 

  51. Ye Z, Ma L, Wang M, Chen H, Zhao W (2015) Texture image classification based on support vector machine and bat algorithm. In: Proceedings of IEEE 8th international conference on Intelligent data acquisition and advanced computing systems: technology and applications (IDAACS), vol 1., IEEE, pp 309–314

  52. Blake CL (1998) UCI Repository of machine learning databases. Irvine, University of California. http://www.ics.uci.edu/~mlearn/MLRepository.html

  53. Tharwat A (2016) Principal component analysis—a tutorial. Int J Appl Pattern Recognit 3(3):197–240

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Engineering, Frankfurt University of Applied Sciences, Frankfurt am Main, Germany

    Alaa Tharwat

Authors
  1. Alaa Tharwat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alaa Tharwat.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tharwat, A. Parameter investigation of support vector machine classifier with kernel functions. Knowl Inf Syst 61, 1269–1302 (2019). https://doi.org/10.1007/s10115-019-01335-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-019-01335-4

Keywords

Search

Navigation