Advertisement

Artificial Intelligence Review

, Volume 50, Issue 2, pp 261–281 | Cite as

A multi-objective genetic algorithm for simultaneous model and feature selection for support vector machines

  • Amal Bouraoui
  • Salma Jamoussi
  • Yassine BenAyed
Article

Abstract

The Support Vector Machines (SVM) constitute a very powerful technique for pattern classification problems. However, its efficiency in practice depends highly on the selection of the kernel function type and relevant parameter values. Selecting relevant features is another factor that can also impact the performance of SVM. The identification of the best set of parameters values for a classification model such as SVM is considered as an optimization problem. Thus, in this paper, we aim to simultaneously optimize SVMs parameters and feature subset using different kernel functions. We cast this problem as a multi-objective optimization problem, where the classification accuracy, the number of support vectors, the margin and the number of selected features define our objective functions. To solve this optimization problem, a method based on multi-objective genetic algorithm NSGA-II is suggested. A multi-criteria selection operator for our NSGA-II is also introduced. The proposed method is tested on some benchmark data-sets. The experimental results show the efficiency of the proposed method where features were reduced and the classification accuracy has been improved.

Keywords

Parameter selection Kernel function setting Feature selection Multi-objective genetic algorithm NSGA-II Support vector machines (SVMs) 

References

  1. Andy, Fernando M, Halim K, Sanjaya G (2015) Optimization features using GA-SVM approach. Int J Sci Res 4(9):193–197Google Scholar
  2. Balakrishnan S, Narayanaswamy R (2009) Feature selection using fcbf in type ii diabetes databases. In: International conference on IT to celebrate S Charmonman’s 72nd Birthday, ThailandGoogle Scholar
  3. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1):131–159CrossRefMATHGoogle Scholar
  4. Chen Z, Lin T, Tang N, Xia X (2016) A parallel genetic algorithm based feature selection and parameter optimization for support vector machines. Sci Program 2016:2739621Google Scholar
  5. Chung K, Kao W, Sun C, Lin C (2003) Radius margin bounds for support vector machines with RBF kernel. Neural Comput 15(11):2643–2681CrossRefMATHGoogle Scholar
  6. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197CrossRefGoogle Scholar
  7. Deb K, Beyer H (1999) Self-adaptive genetic algorithms with simulated binary crossover. Complex Syst 9:431–454Google Scholar
  8. Deb K, Tiwari S (2008) Omni-optimizer: a generic evolutionary algorithm for single and multiobjective optimization. Eur J Oper Res 185(3):1062–1087CrossRefMATHGoogle Scholar
  9. Frohlich H, Chapelle O (2003) Feature selection for support vector machines by means of genetic algorithms. In: Proceedings of the 15th IEEE international conference on tools with artificial intelligence, Sacramento, CA, USA, pp 142–148Google Scholar
  10. Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46(1–3):389–422CrossRefMATHGoogle Scholar
  11. Guyon I, Gunn S, Nikravesh M, Zadeh L (2006) Feature extraction, foundations and applications. Springer, BerlinCrossRefMATHGoogle Scholar
  12. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATHGoogle Scholar
  13. Hsu C, Lin C (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425CrossRefGoogle Scholar
  14. Huang C, Wang C (2006) A ga-based feature selection and parameters optimization for support vector machine. Expert Syst Appl 31(2):231–240CrossRefGoogle Scholar
  15. Keerthi S, Lin C (2003) Asymptotic behaviors of support vector machines with gaussian kernel. Neural Comput 15:1667–1689CrossRefMATHGoogle Scholar
  16. Kotsia I, Pitas I (2007) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 16(1):172–187MathSciNetCrossRefGoogle Scholar
  17. Lee M (2009) Using support vector machine with a hybrid feature selection method to the stock trend prediction. Expert Syst Appl 36:10896–10904CrossRefGoogle Scholar
  18. Liang X, Liu F (2002) Choosing multiple parameters for svm based on genetic algorithm. In: 6th international conference on signal processing, vol 1, pp 117–119Google Scholar
  19. Liao P, Zhang X, Li K, Fu Y, Wang M, Wang S (2015) Parameter optimization for support vector machine based on nested genetic algorithms. J Autom Control Eng 3(6):507–511CrossRefGoogle Scholar
  20. Lin K-C, Huang Y-H, Hung J-C, Lin Y-T (2015) Feature selection and parameter optimization of support vector machines based on modified cat swarm optimization. Int J Distrib Sens Netw 2015:365869Google Scholar
  21. Lin S, Ying K, Chen S, Lee Z (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824CrossRefGoogle Scholar
  22. Liu H, Wang Y, Lu X (2005a) A method to choose kernel function and its parameters for support vector machines. Proc IEEE Int Conf Mach Learn Cybern 7:4277–4280Google Scholar
  23. Liu S, Jia C, Ma H (2005b) A new weighted support vector machine with ga-based parameter selection. In: Proceedings of 2005 international conference on machine learning and cybernetics, vol 7, pp 4351–4355Google Scholar
  24. Ma J, Nguyen M, Rajapakse J (2009) Gene classification using codon usage and support vector machines. ACM Trans Comput Biol Bioinf 6(1):134–143CrossRefGoogle Scholar
  25. Mao K (2004) Feature subset selection for support vector machines through discriminative function pruning analysis. IEEE Trans Syst Man Cybern 34(1):60–67CrossRefGoogle Scholar
  26. Melgani F, Bazi Y (2008) Classification of electrocardiogram signals with support vector machines and swarm optimization. IEEE Trans Inf Technol Biomed 12(5):667–677CrossRefGoogle Scholar
  27. Pardo M, Sberveglieri G (2005) Classification of electronic nose data with support vector machines. Sens Actuators B Chem 107:730–737Google Scholar
  28. Plat J (1999) Fast training of support vector machine using sequential minimal optimization. In: Smola A (ed) Advances in kernel methods? Support vector learning. MIT Press, CambridgeGoogle Scholar
  29. Quang A, Zhang Q, Li X (2002) Evolving support vector machine parameters. In: Proceedings of 2002 international conference on machine learning and cybernetics, pp 548–551Google Scholar
  30. Raymer M, Punch W, Goodman E, Kuhn L, Jain A (2000) Dimensionality reduction using genetic algorithms. IEEE Trans Evol Comput 4(2):164–171CrossRefGoogle Scholar
  31. Rinaldia F, Sciandronea M (2010) Feature selection combining linear support vector machines and concave optimization. Optim Methods Softw 25(1):117–128MathSciNetCrossRefGoogle Scholar
  32. Rojas S, Fernandez-Reyes D (2005) Adapting multiple kernel parameters for support vector machines using genetic algorithms. In: The 2005 IEEE Congress on Evolutionary Computation, vol 1, pp 626–631Google Scholar
  33. Thadani K, Jayaraman V, Sundararajan V (2006) Evolutionary selection of kernels in support vector machines. In: Proceedings of IEEE international conference on advanced computing and communications, pp 19–24Google Scholar
  34. Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkCrossRefMATHGoogle Scholar
  35. Wu CH, Tzeng GH, Lin RH (2009) A Novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36:4725–4735CrossRefGoogle Scholar
  36. Xue B, Zhang M, Browne W-N, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626Google Scholar
  37. Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst 13(2):44–49CrossRefGoogle Scholar
  38. Yu L, Chen H, Wang S, Lai K (2009) Evolving least squares support vector machines for stock market trend mining. IEEE Trans Evol Comput 13(1):87–102CrossRefGoogle Scholar
  39. Yuan FC (2012) Parameters optimization using genetic algorithms in support vector regression for sales volume forecasting. Appl Math 3:1480–1486CrossRefGoogle Scholar
  40. Yuxia H, Hongtao Z (2012) Chaos optimization method of SVM parameters selection for chaotic time series forecasting. Phys Proc 25(2012):588–594 2012 international conference on solid state devices and materials scienceCrossRefGoogle Scholar
  41. Zhang G (2000) Neural network for classification: a survey. IEEE Trans Syst Man Cybern Part C Appl Rev 30(4):451–462CrossRefGoogle Scholar
  42. Zhao M, Fu C, Ji L, Tang K, Zhou M (2011) Feature selection and parameter optimization for support vector machines: a new approach based on genetic algorithm with feature chromosomes. Expert Syst Appl 38(5):5197–5204CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  • Amal Bouraoui
    • 1
  • Salma Jamoussi
    • 1
  • Yassine BenAyed
    • 1
  1. 1.Multimedia, InfoRmation systems and Advanced Computing Laboratory MIRACL-Sfax UniversitySfaxTunisia

Personalised recommendations