Abstract
This paper presents a new approach for pruning dataset features (i.e., feature selection) based on genetic algorithms (GAs) and sparse least squares support vector machines (LSSVM) for classification tasks. LSSVM is a modified version of standard Support Vector Machine (SVM), which is in general faster to train than SVM since the training process of SVM requires the solution of a quadratic programming problem while the LSSVM demands only the solution of a linear equation system. GAs are applied to solve optimization problems without the assumption of linearity, differentiability, continuity or convexity of the objective function. There are some works where GAs and LSSVM work together, however, mostly to find the LSSVM kernel and/or classifier parameters. Nevertheless, our new proposal combines LSSVM and GAs for achieving sparse models, in which each support vector has just a few features in a feature selection sense. The idea behind our proposal is to remove non-relevant features from the patterns by using GAs. Removing a pattern has less impact than removing a feature since the training dataset has in general more patterns than features. On the basis of the results, our proposal leaves non-relevant features out of the set of features and still maintains or even improves the classification accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Karush-Kuhn-Tucker systems.
- 2.
References
Alencar, A.S., Neto, A.R.R., Gomes, J.P.P.: A new pruning method for extreme learning machines via genetic algorithms. Appl. Soft Comput. 44, 101–107 (2016)
Avci, E.: An expert system based on wavelet neural network-adaptive norm entropy for scale invariant texture classification. Expert Syst. Appl. 32(3), 919–926 (2007)
Avci, E.: Comparison of wavelet families for texture classification by using wavelet packet entropy adaptive network based fuzzy inference system. Appl. Soft Comput. 8(1), 225–231 (2008)
Barber, D.: The use of principal components in the quantitative analysis of gamma camera dynamic studies. Phys. Med. Biol. 25(2), 283 (1980)
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. Technical report, Yale University New Haven United States (1997)
Çalişir, D., Dogantekin, E.: A new intelligent hepatitis diagnosis system: PCA-LSSVM. Expert Syst. Appl. 38(8), 10705–10708 (2011)
Du, Q., Fowler, J.E.: Hyperspectral image compression using JPEG2000 and principal component analysis. IEEE Geosci. Remote Sens. Lett. 4(2), 201–205 (2007)
Geebelen, D., Suykens, J.A., Vandewalle, J.: Reducing the number of support vectors of SVM classifiers using the smoothed separable case approximation. IEEE Trans. Neural Netw. Learn. Syst. 23(4), 682–688 (2012)
Morchid, M., Dufour, R., Bousquet, P.M., Linares, G., Torres-Moreno, J.M.: Feature selection using principal component analysis for massive retweet detection. Pattern Recogn. Lett. 49, 33–39 (2014)
Mustafa, M., Sulaiman, M., Shareef, H., Khalid, S.A.: Reactive power tracing in pool-based power system utilising the hybrid genetic algorithm and least squares support vector machine. IET Gener. Transm. Distrib. 6(2), 133–141 (2012)
Peres, R.T., Pedreira, C.E.: Generalized risk zone: selecting observations for classification. IEEE Trans. Pattern Anal. Mach. Intell. 31(7), 1331–1337 (2009)
Silva, D.A., Rocha Neto, A.R.: A genetic algorithms-based LSSVM classifier for fixed-size set of support vectors. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2015. LNCS, vol. 9095, pp. 127–141. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19222-2_11
Silva, D.A., Silva, J.P., Neto, A.R.R.: Novel approaches using evolutionary computation for sparse least square support vector machines. Neurocomputing 168, 908–916 (2015)
Silva, J.P., Neto, A.R.D.R.: Sparse least squares support vector machines via genetic algorithms. In: 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence (BRICS-CCI & CBIC), pp. 248–253. IEEE (2013)
Smith, L.I.: A tutorial on principal components analysis. University of Otago, Technical report (2002)
Steinwart, I.: Sparseness of support vector machines. J. Mach. Learn. Res. 4(Nov), 1071–1105 (2003)
Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)
Thomaz, C.E., Giraldi, G.A.: A new ranking method for principal components analysis and its application to face image analysis. Image Vis. Comput. 28(6), 902–913 (2010)
Vapnik, V.: Statistical Learning Theory, vol. 3. Wiley, New York (1998)
Yu, L., Chen, H., Wang, S., Lai, K.K.: Evolving least squares support vector machines for stock market trend mining. IEEE Trans. Evol. Comput. 13(1), 87–102 (2009)
Zhao, W., Krishnaswamy, A., Chellappa, R., Swets, D.L., Weng, J.: Discriminant analysis of principal components for face recognition. In: Wechsler, H., Phillips, P.J., Bruce, V., Soulié, F.F., Huang, T.S. (eds.) Face Recognition, pp. 73–85. Springer, Heidelberg (1998). https://doi.org/10.1007/978-3-642-72201-1_4
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Machado Araújo, P.H., Rocha Neto, A.R. (2019). Sparse Least Squares Support Vector Machines Based on Genetic Algorithms: A Feature Selection Approach. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science(), vol 11507. Springer, Cham. https://doi.org/10.1007/978-3-030-20518-8_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-20518-8_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20517-1
Online ISBN: 978-3-030-20518-8
eBook Packages: Computer ScienceComputer Science (R0)