Abstract
Support Vector Machines (SVM) is widely considered to be the best algorithm for text classification because it is based on a well-founded theory (SRM): in the separable case it provides the best result possible for a given set of separation functions, and therefore it does not require tuning. In this paper we scrutinize these suppositions, and encounter some paradoxes.
In a large-scale experiment it is shown that even in the separable case SVM’s extension to non-separable data may give a better result by minimizing the confidence interval of the risk. However, the use of this extension necessitates the tuning of the complexity constant.
Furthermore, the use of SVM for optimizing precision and recall through the F function necessitates the tuning of the threshold found by SVM. But the tuned classifier does not generalize well. Furthermore, a more precise definition is given to the notion of training errors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ayat, N.E., Cheriet, M., Suen, C.Y.: Kmod-a two parameter svm kernel for pattern recognition. In: ICPR, pp. 30331–30334 (2002)
Basu, A., Watters, C., Shepherd, M.: Support vector machines for text categorization. In: HICSS 2003: Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS 2003) - Track 4, Washington, DC, USA, p. 103. 3. IEEE Computer Society, Los Alamitos (2003)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)
Cummins, R., O’Riordan, C.: Evolved term-weighting schemes in Information Retrieval: an analysis of the solution space. Artificial Intelligence Review, 35–47 (November 2007)
Dumais, S., Platt, J., Heckerman, D., Sahami, M.: Inductive learning algorithms and representations for text categorization. In: CIKM 1998: Proceedings of the seventh international conference on Information and knowledge management, pp. 148–155. ACM Press, New York (1998)
Eitrich, T., Lang, B.: Efficient optimization of support vector machine learning parameters for unbalanced datasets. J. Comput. Appl. Math. 196(2), 425–436 (2006)
Huang, J.: Face recognition using component-based svm classification and morphable models. In: SVM, pp. 334–341 (2002)
Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998)
Koster, C.H.A., Beney, J.G.: On the importance of parameter tuning in text classification. In: Virbitskaite, I., Voronkov, A. (eds.) PSI 2006. LNCS, vol. 4378, pp. 270–283. Springer, Heidelberg (2007)
Krier, M., Zaccà, F.: Automatic categorisation applications at the european patent office. World Patent Information 24, 187–196 (2002)
Lauser, B., Hotho, A.: Automatic multi-label subject indexing in a multilingual environment. In: Koch, T., Sølvberg, I.T. (eds.) ECDL 2003. LNCS, vol. 2769, pp. 140–151. Springer, Heidelberg (2003)
Li, Y., Bontcheva, K., Cunningham, H.: Svm based learning system for information extraction. In: Proceedings of Sheffield Machine Learning Workshop. LNCS. Springer, Heidelberg (2005)
Li, Y., Bontcheva, K., Cunningham, H.: Using Uneven Margins SVM and Perceptron for Information Extraction. In: Proceedings of Ninth Conference on Computational Natural Language Learning, CoNLL 2005 (2005)
Lukianitsa, A.A., Zhdanov, F.M., Zaitsev, F.S.: Analyses of iter operation mode using the support vector machine technique for plasma discharge classification. Plasma Physics and Controlled Fusion 50(6), 065013, 14 p. (2008)
Platt, J.: Sequential minimal optimization: A fast algorithm for training support vector machines. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge (1998)
Rifkin, R., Mukherjee, S., Tamayo, P., Ramaswamy, S., Yeang, C.h., Angelo, M., Reich, M., Poggio, T., Eric, S.L., Golub, T.R., Mesirov., J.P.: An analytical method for multiclass molecular cancer classification. SIAM Review 45, 706–723 (2003)
Sebastiani, F.: Classification of text, automatic. In: The Encyclopedia of Language and Linguistics, pp. 457–463. Elsevier Science Publishers, Amsterdam (2006)
Thorsten, J.: Making large-scale svm learning practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods – Support Vector Learning, ch. 11, pp. 41–56. MIT Press, Cambridge (1999)
Vapnik, V.: The nature of Statistical Learning Theory, 2nd edn. Springer, New York (2000)
Yue, Y., Finley, T.: A support vector method for optimizing average precision. In: Proceedings of SIGIR 2007 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Beney, J., Koster, C.H.A. (2010). SVM Paradoxes. In: Pnueli, A., Virbitskaite, I., Voronkov, A. (eds) Perspectives of Systems Informatics. PSI 2009. Lecture Notes in Computer Science, vol 5947. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11486-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-11486-1_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-11485-4
Online ISBN: 978-3-642-11486-1
eBook Packages: Computer ScienceComputer Science (R0)