Abstract
In this article, model selection for support vector machines is viewed as a multi-objective optimization problem, where model complexity and training accuracy define two conflicting objectives. Different optimization criteria are evaluated: Split modified radius margin bounds, which allow for comparing existing model selection criteria, and the training error in conjunction with the number of support vectors for designing sparse solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abbass, H.A.: An evolutionary artificial neural networks approach for breast cancer diagnosis. Artificial Intelligence in Medicine 25, 265–281 (2002)
Abbass, H.A.: Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Computation 15, 2705–2726 (2003)
Jin, Y., Okabe, T., Sendhoff, B.: Neural network regularization and ensembling using multi-objective evolutionary algorithms. In: Congress on Evolutionary Computation (CEC2004), pp. 1–8. IEEE Press, Los Alamitos (2004)
Wiegand, S., Igel, C., Handmann, U.: Evolutionary multi-objective optimization of neural networks for face detection. International Journal of Computational Intelligence and Applications 4, 237–253 (2004)
Coello Coello, C.A., Van Veldhuizen, D.A., Lamont, G.B.: Evolutionary Algorithms for Solving Multi-Objective Problems. Kluwer Academic Publishers, Dordrecht (2002)
Deb, K.: Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, Chichester (2001)
Sawaragi, Y., Nakayama, H., Tanino, T.: Theory of Multiobjective Optimization. Mathematics in Science and Engineering, vol. 176. Academic Press, London (1985)
Chung, K.M., Kao, W.C., Sun, C.L., Lin, C.J.: Radius margin bounds for support vector machines with RBF kernel. Neural Computation 15, 2643–2681 (2003)
Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 182–197 (2002)
Beyer, H.G.: The Theory of Evolution Strategies. Springer, Heidelberg (2001)
Beyer, H.G., Schwefel, H.P.: Evolution strategies: A comprehensive introduction. Natural Computing 1, 3–52 (2002)
Bäck, T.: An overview of parameter control methods by self-adaptation in evolutionary algorithms. Fundamenta Informaticae 35, 51–66 (1998)
Igel, C., Toussaint, M.: Neutrality and self-adaptation. Natural Computing 2, 117–132 (2003)
Laumanns, M., Rudolph, G., Schwefel, H.P.: Mutation control and convergence in evolutionary multi-objective optimization. In: Matousek, R., Osmera, P. (eds.) Proceedings of the 7th International Mendel Conference on Soft Computing (MENDEL 2001), pp. 24–29. University of Technology, Brno (2001)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000)
Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995)
Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing multiple parameters for support vector machines. Machine Learning 46, 131–159 (2002)
Gold, C., Sollich, P.: Model selection for support vector machine classification. Neurocomputing 55, 221–249 (2003)
Keerthi, S.S.: Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transactions on Neural Networks 13, 1225–1229 (2002)
Friedrichs, F., Igel, C.: Evolutionary tuning of multiple SVM parameters. In: Verleysen, M. (ed.) 12th European Symposium on Artificial Neural Networks (ESANN 2004), pp. 519–524. d-side publications, Evere (2004)
Runarsson, T.P., Sigurdsson, S.: Asynchronous parallel evolutionary model selection for support vector machines. Neural Information Processing – Letters and Reviews 3, 59–68 (2004)
Fröhlich, H., Chapelle, O., Schölkopf, B.: Feature selection for support vector machines by means of genetic algorithms. In: 15th IEEE International Conference on Tools with AI (ICTAI 2003), pp. 142–148. IEEE Computer Society, Los Alamitos (2003)
Eads, D.R., Hill, D., Davis, S., Perkins, S.J., Ma, J., Porter, R.B., Theiler, J.P.: Genetic algorithms and support vector machines for time series classification. In: Bosacchi, B., Fogel, D.B., Bezdek, J.C. (eds.) Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation V. Proceedings of the SPIE, vol. 4787, pp. 74–85 (2002)
Jong, K., Marchiori, E., van der Vaart, A.: Analysis of proteomic pattern data for cancer detection. In: Raidl, G.R., Cagnoni, S., Branke, J., Corne, D.W., Drechsler, R., Jin, Y., Johnson, C.G., Machado, P., Marchiori, E., Rothlauf, F., Smith, G.D., Squillero, G. (eds.) EvoWorkshops 2004. LNCS, vol. 3005, pp. 41–51. Springer, Heidelberg (2004)
Miller, M.T., Jerebko, A.K., Malley, J.D., Summers, R.M.: Feature selection for computer-aided polyp detection using genetic algorithms. In: Clough, A.V., Amini, A.A. (eds.) Medical Imaging 2003: Physiology and Function: Methods, Systems, and Applications. Proceedings of the SPIE, vol. 5031, pp. 102–110 (2003)
Schölkopf, B., Burges, C.J.C., Vapnik, V.: Extracting support data for a given task. In: Fayyad, U.M., Uthurusamy, R. (eds.) Proceedings of the First International Conference on Knowledge Discovery & Data Mining, pp. 252–257. AAAI Press, Menlo Park (1995)
Duan, K., Keerthi, S.S., Poo, A.: Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing 51, 41–59 (2003)
Blake, C., Merz, C.: UCI repository of machine learning databases (1998)
Rätsch, G., Onoda, T., Müller, K.R.: Soft margins for adaboost. Machine Learning 42, 287–320 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Igel, C. (2005). Multi-objective Model Selection for Support Vector Machines. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds) Evolutionary Multi-Criterion Optimization. EMO 2005. Lecture Notes in Computer Science, vol 3410. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-31880-4_37
Download citation
DOI: https://doi.org/10.1007/978-3-540-31880-4_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24983-2
Online ISBN: 978-3-540-31880-4
eBook Packages: Computer ScienceComputer Science (R0)