A New Algorithm for Training SVMs Using Approximate Minimal Enclosing Balls

  • Emanuele Frandi
  • Maria Grazia Gasparo
  • Stefano Lodi
  • Ricardo Ñanculef
  • Claudio Sartori
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6419)

Abstract

It has been shown that many kernel methods can be equivalently formulated as minimal enclosing ball (MEB) problems in a certain feature space. Exploiting this reduction, efficient algorithms to scale up Support Vector Machines (SVMs) and other kernel methods have been introduced under the name of Core Vector Machines (CVMs). In this paper, we study a new algorithm to train SVMs based on an instance of the Frank-Wolfe optimization method recently proposed to approximate the solution of the MEB problem. We show that, specialized to SVM training, this algorithm can scale better than CVMs at the price of a slightly lower accuracy.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bădoiu, M., Clarkson, K.: Smaller core-sets for balls. In: Proceedings of the SODA 2003, pp. 801–802. SIAM, Philadelphia (2003)Google Scholar
  2. 2.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2010)Google Scholar
  3. 3.
    Clarkson, K.: Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. In: Proceedings of SODA 2008, pp. 922–931. SIAM, Philadelphia (2008)Google Scholar
  4. 4.
    Fan, R.-E., Chen, P.-H., Lin, C.-J.: Working set selection using second order information for training support vector machines. Journal of Machine Learning Research 6, 1889–1918 (2005)MathSciNetMATHGoogle Scholar
  5. 5.
    Fine, S., Scheinberg, K.: Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research 2, 243–264 (2002)MATHGoogle Scholar
  6. 6.
    Fung, G., Mangasarian, O.: Finite newton method for lagrangian support vector machine classification. Neurocomputing 55(1-2), 39–55 (2003)CrossRefGoogle Scholar
  7. 7.
    Hettich, S., Bay, S.: The UCI KDD Archive (2010), http://kdd.ics.uci.edu
  8. 8.
    Joachims, T.: Making large-scale support vector machine learning practical, pp. 169–184. MIT Press, Cambridge (1999)Google Scholar
  9. 9.
    Kressel, U.: Pairwise classification and support vector machines. In: Advances in Kernel Methods: Support Vector Learning, pp. 255–268. MIT Press, Cambridge (1999)Google Scholar
  10. 10.
    Kumar, K., Bhattacharya, C., Hariharan, R.: A randomized algorithm for large scale support vector learning. In: Advances in Neural Information Processing Systems, vol. 20, pp. 793–800. MIT Press, Cambridge (2008)Google Scholar
  11. 11.
    Lee, Y.-J., Huang, S.: Reduced support vector machines: A statistical theory. IEEE Transactions on Neural Networks 18(1), 1–13 (2007)CrossRefGoogle Scholar
  12. 12.
    Pavlov, D., Mao, J., Dom, B.: An improved training algorithm for support vector machines. In: Proceedings of the 15th International Conference on Pattern Recognition, vol. 2, pp. 2219–2222. IEEE, Los Alamitos (2000)Google Scholar
  13. 13.
    Platt, J.: Fast training of support vector machines using sequential minimal optimization, pp. 185–208 (1999)Google Scholar
  14. 14.
    Scheinberg, K.: An efficient implementation of an active set method for SVMs. Journal of Machine Learning Research 7, 2237–2257 (2006)MathSciNetMATHGoogle Scholar
  15. 15.
    Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)Google Scholar
  16. 16.
    Tsang, I., Kocsor, A., Kwok, J.: Simpler core vector machines with enclosing balls. In: ICML 2007, pp. 911–918. ACM, New York (2007)Google Scholar
  17. 17.
    Tsang, I., Kocsor, A., Kwok, J.: LibCVM Toolkit (2009)Google Scholar
  18. 18.
    Tsang, I., Kwok, J., Cheung, P.-M.: Core vector machines: Fast SVM training on very large data sets. J. of Machine Learning Research 6, 363–392 (2005)MathSciNetMATHGoogle Scholar
  19. 19.
    Tsang, I., Kwok, J., Zurada, J.: Generalized core vector machines. IEEE Transactions on Neural Networks 17(5), 1126–1140 (2006)CrossRefGoogle Scholar
  20. 20.
    Yildirim, E.A.: Two algorithms for the minimum enclosing ball problem. SIAM Journal on Optimization 19(3), 1368–1391 (2008)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Emanuele Frandi
    • 4
  • Maria Grazia Gasparo
    • 3
  • Stefano Lodi
    • 1
  • Ricardo Ñanculef
    • 2
  • Claudio Sartori
    • 1
  1. 1.Dept. of Electronics, Computer Science and SystemsUniversity of BolognaItaly
  2. 2.Dept. of InformaticsFederico Santa María UniversityChile
  3. 3.Dept. of Energetics Sergio SteccoUniversity of FlorenceItaly
  4. 4.Dept. of Mathematics Ulisse DiniUniversity of FlorenceItaly

Personalised recommendations