Abstract
In this paper, we describe a novel model selection approach for a SVM. Each model can be composed by a feature selection method and a pre-processing method besides the classifier. Our approach is based on a multi-objective evolutionary algorithm and on the bias-variance definition. This strategy allows us to explore the hyperparameters space and to select the solutions with the best bias-variance trade-off. The proposed method is evaluated using a number of benchmark data sets for classification tasks. Experimental results show that it is possible to obtain models with an acceptable generalization performance using the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aydin, I., Karakose, M., Akin, E.: A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Appl. Soft Comput. 11(1), 120–129 (2011)
Chatelain, C., Adam, S., Lecourtier, Y., Heutte, L., Paquet, T.: Multi-objective optimization for svm model selection. In: Ninth ICDAR, vol. 1, pp. 427–431 (2007)
Chatelain, C., Adam, S., Lecourtier, Y., Heutte, L., Paquet, T.: A multi-model selection framework for unknown and/or evolutive misclassification cost problems. Pattern Recogn. 43(3), 815–823 (2010)
Coello Coello, C.A., Lamont, G.B., Veldhuizen, D.A.V.: Evolutionary Algorithms for Solving Multi-Objective Problems. In: Genetic and Evolutionary Computation. Springer, US (2007)
Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii. In: Deb, K., Rudolph, G., Lutton, E., Merelo, J.J., Schoenauer, M., Schwefel, H.-P., Yao, X. (eds.) PPSN VI. LNCS, vol. 1917, pp. 849–858. Springer, Heidelberg (2000)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Escalante, H.J., Montes, M., Sucar, L.E.: Particle swarm model selection. J. Mach. Learn. Res. 10, 405–440 (2009)
Friedman, J.H.: On bias, variance, 0/1loss, and the curse-of-dimensionality. Data Min. Knowl. Discov. 1, 55–77 (1997)
Ji, Y., Sun, S.: Feature selection for ensembles using non-dominated sorting in genetic algorithms. In: 2010 Sixth International Conference on Natural Computation (ICNC), vol. 2, pp. 888–891 (August 2010)
Kohavi, R., Wolpert, D.: Bias plus variance decomposition for zero-one loss functions. In: Proceedings of the 13th ICML, pp. 275–283. Morgan-Kaufmann Publishers (1996)
Kong, E., Dietterich, T.: Error-correcting output coding corrects bias and variance. In: Proceedings of the 12th ICML, pp. 313–321. Morgan-Kaufmann Publishers (1995)
Li, W., Liu, L., Gong, W.: Multi-objective uniform design as a svm model selection tool for face recognition. Expert Syst. Appl. 38(6), 6689–6695 (2011)
Rätsch, G., Onoda, T., Müller, K.R.: Soft margins for adaboost. Mach. Learn. 42, 287–320 (2001)
Saffari, A., Guyon, I.: Quick start guide for clop. Tech. rep. (May 2006), http://ymer.org/research/files/clop/QuickStartV1.0.pdf
Suttorp, T., Igel, C.: Multi-objective optimization of support vector machines. In: Jin, Y. (ed.) Multi-Objective Machine Learning. SCI, vol. 16, pp. 199–220. Springer, Heidelberg (2006)
Webb, G.I.: Multiboosting: A technique for combining boosting and wagging. Mach. Learn. 40, 159–196 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rosales-Pérez, A., Escalante, H.J., Gonzalez, J.A., Reyes-Garcia, C.A., Coello Coello, C.A. (2013). Bias and Variance Multi-objective Optimization for Support Vector Machines Model Selection. In: Sanches, J.M., Micó, L., Cardoso, J.S. (eds) Pattern Recognition and Image Analysis. IbPRIA 2013. Lecture Notes in Computer Science, vol 7887. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38628-2_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-38628-2_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38627-5
Online ISBN: 978-3-642-38628-2
eBook Packages: Computer ScienceComputer Science (R0)