Abstract
SVM training is usually discussed under two different algorithmic points of view. The first one is provided by decomposition methods such as SMO and SVMLight while the second one encompasses geometric methods that try to solve a Nearest Point Problem (NPP), the Gilbert–Schlesinger–Kozinec (GSK) and Mitchell–Demyanov–Malozemov (MDM) algorithms being the most representative ones. In this work we will show that, indeed, both approaches are essentially coincident. More precisely, we will show that a slight modification of SMO in which at each iteration both updating multipliers correspond to patterns in the same class solves NPP and, moreover, that this modification coincides with an extended MDM algorithm. Besides this, we also propose a new way to apply the MDM algorithm for NPP problems over reduced convex hulls.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Schölkopf, B., Smola, A.: Learning with kernels support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge (2002)
Platt, J.: Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods - Support Vector Machines, 185–208 (1999)
Joachims, T.: Making large-scale support vector machine learning practical. Advances in Kernel Methods - Support Vector Machines, 169–184 (1999)
Keerthi, S., Shevade, S., Bhattacharyya, C., Murthy, K.: Improvements to platt’s smo algorithm for SVM classifier design. Neural Computation 13(3), 637–649 (2001)
Fan, R., Chen, P.H., Lin, C.: Working set selection using second order information for training support vector machines. Journal of Machine Learning Research 6, 1889–1918 (2005)
Bennett, K., Bredensteiner, E.: Duality and geometry in svm classifiers. In: Proc. 17th Int. Conf. Machine Learning, pp. 57–64 (2000)
Keerthi, S., Shevade, S., Bhattacharyya, C., Murthy, K.: A fast iterative nearest point algorithm for support vector machine classifier design. IEEE Transactions on Neural Networks 11(1), 124–136 (2000)
Franc, V.: Simple solvers for large quadratic programming tasks. In: Kropatsch, W., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 75–84. Springer, Heidelberg (2005)
Gilbert, E.: Minimizing the quadratic form on a convex set. SIAM J. Contr. 4, 61–79 (1966)
Franc, V., Hlavác̆, V.: An iterative algorithm learning the maximal margin classiffier. Pattern Recognition 36, 1985–1996 (2003)
Mitchell, B., Dem’yanov, V., Malozemov, V.: Finding the point of a polyhedron closest to the origin. SIAM J. Contr. 12, 19–26 (1974)
Shawe-Taylor, J., Cristianini, N.: On the generalisation of soft margin algorithms. IEEE Transactions on Information Theory 48(10), 2711–2735 (2002)
Glasmachers, T., Igel, C.: Second order smo improves svm online and active learning. Neural Computation 20(2), 374–382 (2008)
Hush, D., Scovel, C.: Polynomial-time decomposition algorithms for support vector machines. Machine Learning 51(1), 51–71 (2003)
Chang, C.C., Lin, C.J.: Training ν-support vector classifiers: Theory and algorithms. Neural Computation 13(9), 2119–2147 (2001)
Tao, Q., Wu, G.W., Wang, J.: A general soft method for learning svm classifiers with l1-norm penalty. Pattern Recogn. 41(3), 939–948 (2008)
Rätsch, G.: Benchmark repository, ida.first.fraunhofer.de/projects/bench/benchmarks.htm
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
López, J., Barbero, Á., Dorronsoro, J.R. (2008). On the Equivalence of the SMO and MDM Algorithms for SVM Training. In: Daelemans, W., Goethals, B., Morik, K. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2008. Lecture Notes in Computer Science(), vol 5211. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87479-9_37
Download citation
DOI: https://doi.org/10.1007/978-3-540-87479-9_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87478-2
Online ISBN: 978-3-540-87479-9
eBook Packages: Computer ScienceComputer Science (R0)