Advertisement

Parameters optimization of support vector machines for imbalanced data using social ski driver algorithm

  • Alaa TharwatEmail author
  • Thomas Gabel
Original Article
  • 303 Downloads

Abstract

The parameters of support vector machines (SVMs) such as kernel parameters and the penalty parameter have a great influence on the accuracy and complexity of the classification models. In the past, different evolutionary optimization algorithms were employed for optimizing SVMs; in this paper, we propose a social ski-driver (SSD) optimization algorithm which is inspired from different evolutionary optimization algorithms for optimizing the parameters of SVMs, with the aim of improving the classification performance. To cope with the problem of imbalanced data which is one of the challenging problems for building robust classification models, the proposed algorithm (SSD-SVM) was enhanced to deal with imbalanced data. In this study, eight standard imbalanced datasets were used for testing our proposed algorithm. For verification, the results of the SSD-SVM algorithm are compared with grid search, which is a conventional method of searching parameter values, and particle swarm optimization (PSO). The experimental results show that the SSD-SVM algorithm is capable of finding near-optimal values of SVMs parameters. The results also demonstrated high classification performance compared to the PSO algorithm.

Keywords

Optimization algorithms Support vector machine (SVM) Parameter optimization Imbalanced data 

Notes

Compliance with ethical standards

Conflict of Interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  2. 2.
    Wang L (2005) Support vector machines: theory and applications, vol 177. Springer, BerlinCrossRefzbMATHGoogle Scholar
  3. 3.
    Wang Y, Wang Y, Tan T (2004) Combining fingerprint and voiceprint biometrics for identity verification: an experimental comparison. In: Biometric authentication, pp 289–294Google Scholar
  4. 4.
    Bouzerdoum M, Mellit A, Pavan AM (2013) A hybrid model (SARIMA-SVM) for short-term power forecasting of a small-scale grid-connected photovoltaic plant. Sol Energy 98:226–235CrossRefGoogle Scholar
  5. 5.
    Tharwat A, Moemen YS, Hassanien AE (2017) Classification of toxicity effects of biotransformed hepatic drugs using whale optimized support vector machines. J Biomed Inform 68:132–149CrossRefGoogle Scholar
  6. 6.
    Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824CrossRefGoogle Scholar
  7. 7.
    Zhang X, Chen X, He Z (2010) An aco-based algorithm for parameter optimization of support vector machines. Expert Syst Appl 37(9):6618–6628CrossRefGoogle Scholar
  8. 8.
    Yamany W, Tharwat A, Hassanin M F, Gaber T, Hassanien AE, Kim TH (2015) A new multi-layer perceptrons trainer based on ant lion optimization algorithm. In: Fourth international conference on information science and industrial applications (ISI). IEEE, pp 40–45Google Scholar
  9. 9.
    Poli R, Kennedy J, Blackwell T (2007) Particle swarm optimization. Swarm Intell 1(1):33–57CrossRefGoogle Scholar
  10. 10.
    Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61CrossRefGoogle Scholar
  11. 11.
    Mirjalili S (2016) Sca: a sine cosine algorithm for solving optimization problems. Knowl-Based Syst 96:120–133CrossRefGoogle Scholar
  12. 12.
    He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRefGoogle Scholar
  13. 13.
    Friedrichs F, Igel C (2005) Evolutionary tuning of multiple SVM parameters. Neurocomputing 64:107–117CrossRefGoogle Scholar
  14. 14.
    LaValle SM, Branicky MS, Lindemann SR (2004) On the relationship between classical grid search and probabilistic roadmaps. Int J Robot Res 23(7–8):673–692CrossRefGoogle Scholar
  15. 15.
    Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159CrossRefzbMATHGoogle Scholar
  16. 16.
    Subasi A (2013) Classification of EMG signals using pso optimized SVM for diagnosis of neuromuscular disorders. Comput Biol Med 43(5):576–586CrossRefGoogle Scholar
  17. 17.
    Wu CH, Tzeng GH, Lin RH (2009) A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36(3):4725–4735CrossRefGoogle Scholar
  18. 18.
    Tharwat A, Hassanien AE, Elnaghi BE (2017) A ba-based algorithm for parameter optimization of support vector machine. Pattern Recognit Lett 93:13–22CrossRefGoogle Scholar
  19. 19.
    Tharwat A, Gabel T, Hassanien AE (2017) Parameter optimization of support vector machine using dragonfly algorithm. In: International conference on advanced intelligent systems and informatics. Springer, pp 309–319Google Scholar
  20. 20.
    Aydin I, Karakose M, Akin E (2011) A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Appl Soft Comput 11(1):120–129CrossRefGoogle Scholar
  21. 21.
    Rojas-Domínguez A, Padierna LC, Valadez JMC, Puga-Soberanes HJ, Fraire HJ (2018) Optimal hyper-parameter tuning of svm classifiers with application to medical diagnosis. IEEE Access 6:7164–7176CrossRefGoogle Scholar
  22. 22.
    Tharwat A, Hassanien AE (2018) Chaotic antlion algorithm for parameter optimization of support vector machine. Appl Intell 48:670–686CrossRefGoogle Scholar
  23. 23.
    Kecman V (2001) Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. MIT Press, CambridgezbMATHGoogle Scholar
  24. 24.
    Tharwat A (2019) Parameter investigation of support vector machine classifier with kernel functions. Knowl Inf Syst 1–34.  https://doi.org/10.1007/s10115-019-01335-4
  25. 25.
    Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167CrossRefGoogle Scholar
  26. 26.
    Sun Y, Kamel MS, Wong AK, Wang Y (2007) Cost-sensitive boosting for classification of imbalanced data. Pattern Recognit 40(12):3358–3378CrossRefzbMATHGoogle Scholar
  27. 27.
    Sun Y, Wong AK, Kamel MS (2009) Classification of imbalanced data: a review. Int J Pattern Recognit Artif Intell 23(04):687–719CrossRefGoogle Scholar
  28. 28.
    Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357CrossRefzbMATHGoogle Scholar
  29. 29.
    Tharwat A (2018) Classification assessment methods. Appl Comput Inform.  https://doi.org/10.1016/j.aci.2018.08.003
  30. 30.
    Huang CL, Wang CJ (2006) A GA-based feature selection and parameters optimizationfor support vector machines. Expert Syst Appl 31(2):231–240CrossRefGoogle Scholar
  31. 31.
    Moore G, Bergeron C, Bennett KP (2011) Model selection for primal SVM. Mach Learn 85(1–2):175MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Zhang Y, Zhang P (2015) Machine training and parameter settings with social emotional optimization algorithm for support vector machine. Pattern Recognit Lett 54:36–42CrossRefGoogle Scholar
  33. 33.
    Tharwat A, Gaber T, Hassanien AE, Elnaghi BE (2017) Particle swarm optimization: a tutorial. In: Handbook of research on machine learning innovations and trends. IGI Global, pp 614–635Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Faculty of Computer Science and EngineeringFrankfurt University of Applied SciencesFrankfurt am MainGermany

Personalised recommendations