Advertisement

A parsimonious SVM model selection criterion for classification of real-world data sets via an adaptive population-based algorithm

  • Omid Naghash Almasi
  • Mohammad Hassan Khooban
Original Article

Abstract

This paper proposes and optimizes a two-term cost function consisting of a sparseness term and a generalized v-fold cross-validation term by a new adaptive particle swarm optimization (APSO). APSO updates its parameters adaptively based on a dynamic feedback from the success rate of the each particle’s personal best. Since the proposed cost function is based on the choosing fewer numbers of support vectors, the complexity of SVM model is decreased while the accuracy remains in an acceptable range. Therefore, the testing time decreases and makes SVM more applicable for practical applications in real data sets. A comparative study on data sets of UCI database is performed between the proposed cost function and conventional cost function to demonstrate the effectiveness of the proposed cost function.

Keywords

Parameter selection Model complexity Support vector machines Adaptive particle swarm optimization Classification Real-world data sets 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

  1. 1.
    Vapnik VN (1998) Statistical learning theory. Wiley, New YorkzbMATHGoogle Scholar
  2. 2.
    Almasi ON, Rouhani M (2016) Fast and de-noise support vector machine training method based on fuzzy clustering method for large real world datasets. Turk J Electr Eng Comput 241:219–233CrossRefGoogle Scholar
  3. 3.
    Peng X, Wang Y (2009) A geometric method for model selection in support vector machine. Expert Syst Appl 36:5745–5749CrossRefGoogle Scholar
  4. 4.
    Wang S, Meng B (2011) Parameter selection algorithm for support vector machine. Environ Sci Conf Proc 11:538–544CrossRefGoogle Scholar
  5. 5.
    Chapelle O, Vapnik VN, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 461:131–159CrossRefzbMATHGoogle Scholar
  6. 6.
    Jaakkola T, Haussler D (1999) Probabilistic kernel regression models. Artif Int Stat 126:1–4Google Scholar
  7. 7.
    Opper M, Winther O (2000) Gaussian processes and SVM: mean field and leave-one-out estimator. In: Smola A, Bartlett P, Scholkopf B, Schuurmans D (eds) Advances in large margin classifiers. MIT Press, Cambridge, MAGoogle Scholar
  8. 8.
    Vapnik V, Chapelle O (2000) Bounds on error expectation for support vector machines. Neural Comput 12(9):2013–2016CrossRefGoogle Scholar
  9. 9.
    Keerthi SS (2002) Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Trans Neural Netw 135:1225–1229CrossRefGoogle Scholar
  10. 10.
    Sun J, Zheng C, Li X, Zhou Y (2010) Analysis of the distance between two classes for tuning SVM hyperparameters. IEEE Trans Neural Netw 212:305–318Google Scholar
  11. 11.
    Guo XC, Yang JH, Wu CG, Wang CY, Liang YC (2008) A novel LS-SVMs hyper-parameter selection based on particle swarm optimization. Neurocomputing 71:3211–3215CrossRefGoogle Scholar
  12. 12.
    Glasmachers T, Igel C (2005) Gradient-based adaptation of general Gaussian kernels. Neural Comput 1710:2099–2105MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Lin KM, Lin CJ (2003) A study on reduced support vector machines. IEEE Trans Neural Netw 146:1449–1459Google Scholar
  14. 14.
    Wang S, Meng B (2010) PSO algorithm for support vector machine. In: Electronic commerce and security conference, pp 377–380Google Scholar
  15. 15.
    Lei P,  Lou Y (2010) Parameter selection of support vector machine using an improved PSO algorithm. In: Intelligent human–machine systems and cybernetics conference, pp 196–199Google Scholar
  16. 16.
    Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 354:1817–1824CrossRefGoogle Scholar
  17. 17.
    Zhang W, Niu P (2011) LS-SVM based on chaotic particle swarm optimization with simulated annealing and application. In: Intelligent control and information processing, 2011 2nd international conference, vol 2, pp 931–935Google Scholar
  18. 18.
    Blondin J, Saad A (2010) Metaheuristic techniques for support vector machine model selection. In: Hybrid intelligent systems, 2010 10th international conference, pp 197–200Google Scholar
  19. 19.
    Almasi ON, Akhtarshenas E, Rouhani M (2014) An efficient model selection for SVM in real-world datasets using BGA and RGA. Neural Netw World 24(5):501CrossRefGoogle Scholar
  20. 20.
    Lihu A, Holban S (2012) Real-valued genetic algorithms with disagreements. Stud Comp Intell 4(4):317–325Google Scholar
  21. 21.
    Cervantes J, Garcia-Lamont F, Rodriguez L, Lopez A, Castilla JR, Trueba A (2017) PSO-based method for SVM classification on skewed data sets. Neurocomputing 228:187–197CrossRefGoogle Scholar
  22. 22.
    Williams P, Li S, Feng J, Wu S (2007) A geometrical method to improve performance of the support vector machine. IEEE Trans Neural Netw 183:942–947CrossRefGoogle Scholar
  23. 23.
    An S, Liu W, Venkatesh S (2007) Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression. Pattern Recognit 408:2154–2162CrossRefzbMATHGoogle Scholar
  24. 24.
    Huang CM, Lee YJ, Lin DK, Huang SY (2007) Model selection for support vector machines via uniform design. Comput Stat Data Anal 521:335–346MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Almasi ON, Rouhani M (2016) A new fuzzy membership assignment and model selection approach based on dynamic class centers for fuzzy SVM family using the firefly algorithm. Turk J Electr Eng Comput 243:1797–1814CrossRefGoogle Scholar
  26. 26.
    Almasi BN, Almasi ON, Kavousi M, Sharifinia A (2013) Computer-aided diagnosis of diabetes using least square support vector machine. J Adv Computer Sci Technol 22:68–76Google Scholar
  27. 27.
    Craven P, Wahba G (1978) Smoothing noisy data with spline functions. Numer Math 314:377–403CrossRefzbMATHGoogle Scholar
  28. 28.
    Efron B (1986) How biased is the apparent error rate of a prediction rule? J Am Stat Assoc 81394:461–470MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Li KC (1987) Asymptotic optimality for Cp, CL, cross-validation and generalized cross-validation: discrete index set. Ann Stat 15(3):958–975CrossRefzbMATHGoogle Scholar
  30. 30.
    Cao Y, Golubev Y (2006) On oracle inequalities related to smoothing splines. Math Methods Stat 154:398–414MathSciNetGoogle Scholar
  31. 31.
    Kennedy J, Eberhart RC (2001) Swarm intelligence. Academic Press, USAGoogle Scholar
  32. 32.
    Beyer HG, Schwefel HP (2002) Evolution strategies: a comprehensive introduction. Nat Comput 352:2002MathSciNetzbMATHGoogle Scholar
  33. 33.
    Yuan X, Wang L, Yuan Y (2008) Application of enhanced PSO approach to optimal scheduling of hydro system. Energy Convers Manag 49:2966–2972CrossRefGoogle Scholar
  34. 34.
    Taherkhani M, Safabakhsh R (2016) A novel stability-based adaptive inertia weight for particle swarm optimization. Appl Soft Comput 31:281–295CrossRefGoogle Scholar
  35. 35.
    Chauhan P, Deep K, Pant M (2013) Novel inertia weight strategies for particle swarm optimization. Memet Comput 5:229–251CrossRefGoogle Scholar
  36. 36.
    Yang X, Yuan J, Yuan J, Mao H (2007) A modified particle swarm optimizer with dynamic adaptation. Appl Math Comput 189:1205–1213MathSciNetzbMATHGoogle Scholar
  37. 37.
    Schwefel HPP (1993) Evolution and optimum seeking: the sixth generation. John Wiley & Sons, IncGoogle Scholar
  38. 38.
    Almasi ON, Naghedi AA, Tadayoni E, Zare A (2014) Optimal design of T-S fuzzy controller for a nonlinear system using a new adaptive particle swarm optimization algorithm. J Adv Comput Sci Technol 31:37–47Google Scholar
  39. 39.
    Wang Y, Li B, Weise T, Wang J, Yuan B, Tian Q (2011) Self-adaptive learning based particle swarm optimization. Inf Sci 181:4515–4538MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Keerthi SS, Lin CJ (2003) Asymptotic behavior of support vector machines with gaussian kernel. Neural Comput 157:1667–1689CrossRefzbMATHGoogle Scholar
  41. 41.
    Bordes A, Ertekin S, Weston J, Bottou L (2005) Fast kernel classifiers with online and active learning. J Mach Learn Res 6:1579–1619MathSciNetzbMATHGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  • Omid Naghash Almasi
    • 1
  • Mohammad Hassan Khooban
    • 2
  1. 1.Young Researchers and Elite Club, Mashhad BranchIslamic Azad UniversityMashhadIran
  2. 2.Department of Electrical EngineeringShiraz University of TechnologyShirazIran

Personalised recommendations