Abstract
The standard form for dealing with multi-class classification problems when bi-classifiers are used is to consider a two-phase (decomposition, reconstruction) training scheme. The most popular decomposition procedures are pairwise coupling (one versus one, 1-v-1), which considers a learning machine for each Pair of classes, and the one-versus-all scheme (one versus all, 1-v-r), which takes into consideration each class versus the remaining classes. In this article a 1-v-1 tri-class Support Vector Machine (SVM) is presented. The expansion of the architecture of this machine into three categories specifically addresses the decomposition problem of how to prevent the loss of information which occurs in the usual 1-v-1 training procedure. The proposed machine, by means of a third class, allows all the information to be incorporated into the remaining training patterns when a multi-class problem is considered in the form of a 1-v-1 decomposition. Three general structures are presented where each improves some features from the precedent structure. In order to deal with multi-classification problems, it is demonstrated that the final machine proposed allows ordinal regression as a form of decomposition procedure. Examples and experimental results are presented which illustrate the performance of the new tri-class SV machine.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- 1-v-1:
-
one versus one; all versus all; pairwise coupling
- 1-v-r:
-
one versus the rest; one versus all
- s.t.:
-
subject to
- SV:
-
Support Vector
- SVM:
-
Support Vector Machine
References
Anguita D., Ridella S. and Sterpi D.: A New Method for Multiclass Support Vector Machines. In: Proceedings of the IEEE IJCNN2004. Budapest (Hungary), 2004.
Angulo, C.: Learning with Kernel Machines into a Multi-Class Environment. Doctoral thesis, Technical University of Catalonia. In Spanish, 2001.
C. Angulo A. Català (2000) ArticleTitleA Multi-class Support Vector Machine Lecture Notes in Computer Science 1810 55–64
Angulo, C. and Català, A.: Ordinal regression with K-SVCR machines. In: J. Mira and A. Prieto (eds.), Proceedings of IWANN 2001, Part I, Vol. 2084 of Lecture Notes in Computer Science. pp. 661–668, 2001.
Angulo, C. and González, L.: 1-v-1 tri-class SV machine. In: Proceedings of the 11th European Symposium on Artificial Neural Networks. Bruges (Belgium), pp. 355–360, 2003.
C. Angulo X. Parra A. Català (2003) ArticleTitleK-SVCR. A support vector machine for multi-class classification Neurocomputing 55 IssueID1–2 57–77
Blake C. and Merz C. UCI Repository of Machine Learning Databases, 1998.
S. Canu Y. Grandvalet A. Rakotomamonjy (2003) SVM and Kernel Methods Matlab Toolbox. Perception Systèmes et Information INSA de Rouen Rouen, France
Cristianini, N. and Shawe-Taylor, J.: An Introduction to Support Vector Machines and other Kernel-based Learning Methods. Cambridge University press, 2000.
González, L.: Discriminative analysis using kernel vector machines support. The similarity kernel function. Doctoral thesis, University of Seville. In Spanish, 2002.
González, L., Angulo, C., Velasco, F., and Vílchez, M.: Máquina ℓ-SVCR con salidas probabilísticas (ℓ-SVCR machine with probabilistic outputs). Inteligencia Artificial. Revista Iberoamericana de IA (17) (2002) 72–82. In Spanish.
Hebrich, R.: Learning Kernel Classifiers. Theory and Algorithms. The MIT Press, 2002.
R. Herbrich T. Graepel K. Obermayer (2000) Advances in Large Margin Classifiers MIT Press Cambridge, MA 115–132
Hsu, C.-W., Chang, C.-C. and Lin, C.-J.: A practical guide to support vector classification. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, 2003.
C.-W. Hsu C.-J. Lin (2002) ArticleTitleA Comparison of methods for multiclass support vector machine IEEE Transactions on Neural Networks 13 IssueID2 415–425 Occurrence Handle1951653
U. Kressel (1999) Pairwise classification and support vector machine B. Schölkopf C. Burgues A. Smola (Eds) Advances in Kernel Methods: Support Vector Learning MIT Press Cambridge, MA 255–268
Mayoraz, E. and Alpaydin, E.: Support vector machines for multi-class classification. In: J. Mira and J. V. Sánchez-Andrés (eds.), Proceedings of IWANN 1999, Part II, Vol. 1607 of Lecture Notes in Computer Science, 1999.
Platt, J., Cristianini, N. and Shawe-Taylor, J.: Large margin DAGs for multiclass classification. Neural Information Processing Systems, 12 (2000).
R. Rifkin A. Klautau (2004) ArticleTitleIn defense of one-vs-all classification Journal of Machine Learning Research 5 101–141
Shashua, A. and Levin, A.: Taxonomy of large margin principle algorithms for ordinal regression problems. Neural Information Processing Systems, 16 (2002).
Vapnik, V.: Statistical Learning Theory. John Wiley & Sons, Inc., 1998.
Vert, J.-P., Tsuda, K. and Schölkopf, B.: Kernel Methods in Computational Biology, Chapt. A Primer on Kernel Methods, pp. 35–70. The MIT Press, 2004.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Angulo, C., Ruiz, F.J., González, L. et al. Multi-Classification by Using Tri-Class SVM. Neural Process Lett 23, 89–101 (2006). https://doi.org/10.1007/s11063-005-3500-3
Issue Date:
DOI: https://doi.org/10.1007/s11063-005-3500-3