Neural Processing Letters

, Volume 23, Issue 1, pp 89–101 | Cite as

Multi-Classification by Using Tri-Class SVM

  • Cecilio AnguloEmail author
  • Francisco J. Ruiz
  • Luis González
  • Juan Antonio Ortega


The standard form for dealing with multi-class classification problems when bi-classifiers are used is to consider a two-phase (decomposition, reconstruction) training scheme. The most popular decomposition procedures are pairwise coupling (one versus one, 1-v-1), which considers a learning machine for each Pair of classes, and the one-versus-all scheme (one versus all, 1-v-r), which takes into consideration each class versus the remaining classes. In this article a 1-v-1 tri-class Support Vector Machine (SVM) is presented. The expansion of the architecture of this machine into three categories specifically addresses the decomposition problem of how to prevent the loss of information which occurs in the usual 1-v-1 training procedure. The proposed machine, by means of a third class, allows all the information to be incorporated into the remaining training patterns when a multi-class problem is considered in the form of a 1-v-1 decomposition. Three general structures are presented where each improves some features from the precedent structure. In order to deal with multi-classification problems, it is demonstrated that the final machine proposed allows ordinal regression as a form of decomposition procedure. Examples and experimental results are presented which illustrate the performance of the new tri-class SV machine.


bi-classifier multi-classification ordinal regression Support Vector Machine 



one versus one; all versus all; pairwise coupling


one versus the rest; one versus all


subject to


Support Vector


Support Vector Machine


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anguita D., Ridella S. and Sterpi D.: A New Method for Multiclass Support Vector Machines. In: Proceedings of the IEEE IJCNN2004. Budapest (Hungary), 2004.Google Scholar
  2. 2.
    Angulo, C.: Learning with Kernel Machines into a Multi-Class Environment. Doctoral thesis, Technical University of Catalonia. In Spanish, 2001.Google Scholar
  3. 3.
    Angulo, C., Català, A. 2000A Multi-class Support Vector MachineLecture Notes in Computer Science18105564Google Scholar
  4. 4.
    Angulo, C. and Català, A.: Ordinal regression with K-SVCR machines. In: J. Mira and A. Prieto (eds.), Proceedings of IWANN 2001, Part I, Vol. 2084 of Lecture Notes in Computer Science. pp. 661–668, 2001.Google Scholar
  5. 5.
    Angulo, C. and González, L.: 1-v-1 tri-class SV machine. In: Proceedings of the 11th European Symposium on Artificial Neural Networks. Bruges (Belgium), pp. 355–360, 2003.Google Scholar
  6. 6.
    Angulo, C., Parra, X., Català, A. 2003K-SVCR. A support vector machine for multi-class classificationNeurocomputing555777Google Scholar
  7. 7.
    Blake C. and Merz C. UCI Repository of Machine Learning Databases, 1998.Google Scholar
  8. 8.
    Canu, S., Grandvalet, Y., Rakotomamonjy, A. 2003SVM and Kernel Methods Matlab Toolbox. Perception Systèmes et InformationINSA de RouenRouen, FranceGoogle Scholar
  9. 9.
    Cristianini, N. and Shawe-Taylor, J.: An Introduction to Support Vector Machines and other Kernel-based Learning Methods. Cambridge University press, 2000.Google Scholar
  10. 10.
    González, L.: Discriminative analysis using kernel vector machines support. The similarity kernel function. Doctoral thesis, University of Seville. In Spanish, 2002.Google Scholar
  11. 11.
    González, L., Angulo, C., Velasco, F., and Vílchez, M.: Máquina ℓ-SVCR con salidas probabilísticas (ℓ-SVCR machine with probabilistic outputs). Inteligencia Artificial. Revista Iberoamericana de IA (17) (2002) 72–82. In Spanish.Google Scholar
  12. 12.
    Hebrich, R.: Learning Kernel Classifiers. Theory and Algorithms. The MIT Press, 2002.Google Scholar
  13. 13.
    Herbrich, R., Graepel, T., Obermayer, K. 2000Advances in Large Margin ClassifiersMIT PressCambridge, MA115132Chapt. Large Margin Rank Boundaries for Ordinal RegressionGoogle Scholar
  14. 14.
    Hsu, C.-W., Chang, C.-C. and Lin, C.-J.: A practical guide to support vector classification. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, 2003.Google Scholar
  15. 15.
    Hsu, C.-W., Lin, C.-J. 2002A Comparison of methods for multiclass support vector machineIEEE Transactions on Neural Networks13415425MathSciNetGoogle Scholar
  16. 16.
    Kressel, U. 1999Pairwise classification and support vector machineSchölkopf, B.Burgues, C.Smola, A. eds. Advances in Kernel Methods: Support Vector LearningMIT PressCambridge, MA255268Google Scholar
  17. 17.
    Mayoraz, E. and Alpaydin, E.: Support vector machines for multi-class classification. In: J. Mira and J. V. Sánchez-Andrés (eds.), Proceedings of IWANN 1999, Part II, Vol. 1607 of Lecture Notes in Computer Science, 1999.Google Scholar
  18. 18.
    Platt, J., Cristianini, N. and Shawe-Taylor, J.: Large margin DAGs for multiclass classification. Neural Information Processing Systems, 12 (2000).Google Scholar
  19. 19.
    Rifkin, R., Klautau, A. 2004In defense of one-vs-all classificationJournal of Machine Learning Research5101141Google Scholar
  20. 20.
    Shashua, A. and Levin, A.: Taxonomy of large margin principle algorithms for ordinal regression problems. Neural Information Processing Systems, 16 (2002).Google Scholar
  21. 21.
    Vapnik, V.: Statistical Learning Theory. John Wiley & Sons, Inc., 1998.Google Scholar
  22. 22.
    Vert, J.-P., Tsuda, K. and Schölkopf, B.: Kernel Methods in Computational Biology, Chapt. A Primer on Kernel Methods, pp. 35–70. The MIT Press, 2004.Google Scholar

Copyright information

© Springer 2006

Authors and Affiliations

  • Cecilio Angulo
    • 1
    Email author
  • Francisco J. Ruiz
    • 1
  • Luis González
    • 2
  • Juan Antonio Ortega
    • 3
  1. 1.Grup de Recerca en Enginyeria del ConeixementUniversitat Politècnica de CatalunyaVilanova i la GeltrúSpain
  2. 2.Departamento de Economía Aplicada IUniversidad de SevillaSevillaSpain
  3. 3.Escuela Técnica Superior de Ingeniería InformáticaUniversidad de SevillaSevillaSpain

Personalised recommendations