Advertisement

SVM Hyper-parameters optimization using quantized multi-PSO in dynamic environment

  • Dhruba Jyoti KalitaEmail author
  • Shailendra Singh
Methodologies and Application
  • 12 Downloads

Abstract

Support vector machine (SVM) is considered as one of the most powerful classifiers. They are parameterized models build upon the support vectors extracted during the training phase. One of the crucial tasks in the modeling of SVM is to select optimal values for its hyper-parameters, because the effectiveness and efficiency of SVM depend upon these parameters. This task of selecting optimal values for the SVM hyper-parameters is often called as the SVM model selection problem. Till now a lot of methods have been proposed to deal with this SVM model selection problem, but most of these methods consider the model selection problem in static environment only, where the knowledge about a problem does not change over time. In this paper we have proposed a framework to deal with SVM model selection problem in dynamic environment. In dynamic environment, knowledge about a problem changes over time due to which static optimum values for yper-parameters may degrade the performance of the classifier. For this there should be one efficient mechanism which can re-evaluate the optimal values of hyper-parameters when the knowledge about a problem changes. Our proposed framework uses multi-swarm-based optimization with exclusion and anti-convergence theory to select the optimal values for the SVM hyper-parameters in dynamic environment. The experiments performed using the proposed framework have shown better results in comparison with other techniques like traditional gird search, first grid search, PSO, chained PSO and dynamic model selection in terms of effectiveness and efficiency.

Keywords

Support vector machine Dynamic environment Model selection problem Multi-swarm optimization Exclusion Anti-convergence 

Notes

Compliance with the ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human participants and/or animals

The chapters do not contain any studies with human participants or animals performed by any of the authors

Informed consent

No human participants have been involved in the study

References

  1. Ayat N, Cheriet M, Suen C (2005) Automatic model selection for the optimization of SVM kernels. Pattern Recognit 38(10):1733–1745CrossRefGoogle Scholar
  2. Bansal S (2014) Optimal Golomb ruler sequence generation for FWM crosstalk elimination: soft computing versus conventional approaches. Appl Soft Comput 22:443–457CrossRefGoogle Scholar
  3. Bansal S, Gupta N, Singh AK (2017a) Nature–inspired metaheuristic algorithms to find near–OGR sequences for WDM channel allocation and their performance comparison. Open Math 15(1):520–547MathSciNetCrossRefzbMATHGoogle Scholar
  4. Bansal S, Singh AK, Gupta N (2017b) Optimal Golomb ruler sequences generation for optical WDM systems: a novel parallel hybrid multi-objective bat algorithm. J Inst Eng (India) Ser B 98(1):43–64CrossRefGoogle Scholar
  5. Blackwell T (2005) Particle swarms and population diversity. Soft Comput 9(11):793–802CrossRefzbMATHGoogle Scholar
  6. Blackwell TM, Bentley P (2002) Don’t push me! Collision avoiding swarms. In: Proceedings of congress on evolutionary computation, 2002, pp 1691–1696Google Scholar
  7. Blackwell TM, Bentley P (2002) Dynamic search with charged swarms. In: Langdon WB et al (eds) Proceedings of genetic and computation conference, pp 19–26Google Scholar
  8. Blackwell T, Branke J (2004) Multi-swarm optimization in dynamic environments. In: Raidl GR et al (eds) Applications of evolutionary computing. Series Lecture Notes in Computer Science, vol 3005. Springer, BerlinGoogle Scholar
  9. Blackwell T, Branke J (2006a) Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans Evol Comput 10(4):460–463CrossRefGoogle Scholar
  10. Blackwell T, Branke J (2006b) Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans Evol Comput 10(4):462–471CrossRefGoogle Scholar
  11. Bottou L, Lin C-J (2007) Support vector machine solvers. Large Scale kernel Mach 3:301–320Google Scholar
  12. Change CC, Lin CJ (2005) LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm/
  13. Chapelle O, Vapnik V (1999) Model selection for support vector machines. In: Advances in neural information processing systems, pp 230–236Google Scholar
  14. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159CrossRefzbMATHGoogle Scholar
  15. Chatelain C, Adam S, Lecourtier Y, Heutte L, Paquet T (2007) Multiobjective optimization for SVM model selection. In: Proceedings of the 9th international conference on documentation analysis and recognition, 2007, pp 427–431Google Scholar
  16. Chunhong Z, Licheng J (2004) Automatic parameters selection for SVM based on GA. In: Proceedings of the 5th world congress on intelligent control and automation, 2004, pp 1869–1872Google Scholar
  17. Cohen G, Hilario M, Geissbuhler A (2004) Model selection for support vector classifiers via genetic algorithms. An application to medical decision support. In: Proceedings of the 5th international symposium on biological and medical data analysis, 2004, pp 200–211Google Scholar
  18. Critianini N, Shawe- Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  19. de Auza BF, de Carvalho ACPLF, Calvo R, Ishii RP (2006) Multiclass SVM model selection using particle swarm optimization. In: Proceedings of the 6th international conference on hybrid intelligence systems, 2006, pp 31–34Google Scholar
  20. Hsu C-W, Lin C-J (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13:415–425CrossRefGoogle Scholar
  21. Hu X, Eberhart R (2002) Adaptive particle swarm optimization: detection and response to dynamic systems. In: Proceedings of Congress on Evolutionary Computation, 2002, pp 1666–1670Google Scholar
  22. Huang C-M, Lee Y-J, Lin DK, Huang S-Y (2007) Model selection for support vector machines via uniform design. Comput Stat Data Anal 52(1):335–346MathSciNetCrossRefzbMATHGoogle Scholar
  23. Janson S, Middendorf M (2004) A hierarchical particle swarm optimizer for dynamic optimization problems. In: Raidle GR (ed) Applications of evolutionary computing. Series Lecture Notes in Computer Science, vol 3005. Springer, Berlin, pp 513–524Google Scholar
  24. Jiang M, Yuan X (2007) Construction and application of PSO–SVM model for personal credit scoring. In: Proceedings of the international conference on computational science. Lecture Notes in Computer Science, 2007, pp 158–161Google Scholar
  25. Junfei L, Baolei Z (2014) Online learning algorithm of direct support vector machine for regression based on Cholesky factorization. In: 2014 international conference on information science, electronics and electrical engineering, vol 3. IEEEGoogle Scholar
  26. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference neural network, pp 1942–1948Google Scholar
  27. Kennedy J, Mendes R (2002) Population structure and particle swarm performance. In: Proceedings of congress on evolutionary computation, 2002, pp 1671–1676Google Scholar
  28. Marcelo NK, Sabourin R, Maupin P (2012) A dynamic model selection strategy for support vector machine classifiers. Appl Soft Comput 12(2012):2550–2565Google Scholar
  29. Parsopoulos KE, Vrahatis MN (2004) On the computational of all global minimizers through particle swarm optimization. IEEE Trans Evol Comput 8(3):211–224CrossRefGoogle Scholar
  30. Riedrichs F, Igel C (2004) Evolutionary tuning of multiple SVM parameters. In: Proceedings of the 12th European symposium on artificial neural networks, 2004, pp 519–524Google Scholar
  31. Schoeman IL, Engelbrecht AP (2005) A parallel vector-based particle swarm optimizer. In: Ribeiro B, Albrecht RF, Dobnikar A, Pearson DW, Steele NC (eds) Adaptive and natural computing algorithms. Springer, Vienna, pp 268–271CrossRefGoogle Scholar
  32. Suttorp T, Igel C (2006) Multi-objective optimization of support vector machines. In: Jin Y (ed) Multi-objective machine learning. Studies in computational intelligence, vol 16. Springer, Berlin, Heidelberg, pp 199–220CrossRefGoogle Scholar
  33. Vapnik VN (1995) The nature of statistical learning theory, vol 5. Springer, New YorkCrossRefzbMATHGoogle Scholar
  34. Vapnik VN (1998) Statistical learning theory. Wiley, New York, p 1998zbMATHGoogle Scholar
  35. Zhou X, Zhang X, Wang B (2016) Online support vector machine: a survey. In: Kim J, Geem Z (eds) Harmony search algorithm. Advances in intelligent systems and computing, vol 382. Springer, Berlin, pp 269–278CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Gaya College of EngineeringGayaIndia
  2. 2.National Institute of Technical Teachers’ Training and ResearchBhopalIndia

Personalised recommendations