Advertisement

Tuning L1-SVM Hyperparameters with Modified Radius Margin Bounds and Simulated Annealing

  • Javier Acevedo
  • Saturnino Maldonado
  • Philip Siegmann
  • Sergio Lafuente
  • Pedro Gil
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4507)

Abstract

In the design of support vector machines an important step is to select the optimal hyperparameters. One of the most used estimators of the performance is the Radius-Margin bound. Some modifications of this bound have been made to adapt it to soft margin problems, giving a convex optimization problem for the L2 soft margin formulation. However, it is still interesting to consider the L1 case due to the reduction in the support vector number. There have been some proposals to adapt the Radius-Margin bound to the L1 case, but the use of gradient descent to test them is not possible in some of them because these bounds are not differentiable. In this work we propose to use simulated annealing as a method to find the optimal hyperparameters when the bounds are not differentiable, have multiple local minima or the kernel is not differentiable with respect to its hyperparameters.

Keywords

Support Vector Machine Simulated Annealing Gradient Descent Soft Margin Multiple Local Minimum 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Vapnik, N.V.: The Natureof Statistical Learning Theory (1 edn. 1998). Springer, Heidelberg (2000)Google Scholar
  2. 2.
    Joachims, T.: Estimating the generalization performance of a SVM efficiently. In: Langley, P. (ed.) Proc. of ICML-00, pp. 431–438. Morgan Kaufmann, San Francisco (2000), citeseer.ist.psu.edu/article/joachims99estimating.html Google Scholar
  3. 3.
    Vapnik, V., Chapelle, O.: Bounds on error expectation for support vector machines. Neural Computation 12(9), 2013–2036 (2000)CrossRefGoogle Scholar
  4. 4.
    Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing multiple parameters for support vector machines. Machine Learning 46(1), 131–159 (2002)zbMATHCrossRefGoogle Scholar
  5. 5.
    Duan, K., Sathiya, S., Poo, A.: Evaluation of simple performance measures for tuning the svm hyperparameters. Neurocomputing 51, 41–59 (2003)CrossRefGoogle Scholar
  6. 6.
    Chung, K.-M., Kao, W.-C., Sun, C.-L., Wang, L.-L., Lin, C.-J.: Radius margin bounds for support vector machines with the rbf kernel. Neural Computation 15, 2643–2681 (2003)zbMATHCrossRefGoogle Scholar
  7. 7.
    Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)Google Scholar
  8. 8.
    Kirkpatrick, S., Gellat, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, New York (2004)Google Scholar
  10. 10.
    Ingber, A.L.: Adaptive simulated annealing (asa): Lessons learned. Control and Cybernetics 25(1), 33–54 (1996)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Javier Acevedo
    • 1
  • Saturnino Maldonado
    • 1
  • Philip Siegmann
    • 1
  • Sergio Lafuente
    • 1
  • Pedro Gil
    • 1
  1. 1.University of Alcala, Teoría de la señal, Alcala de HenaresSpain

Personalised recommendations