Abstract
Support vector machines (SVMs) are originally designed for binary classifications. As for multi-classifications, they are usually converted into binary ones. In the conventional multi-classifiable algorithms, One-against-One algorithm is a very power method. However, there exists a middle unclassifiable region. In order to overcome this drawback, a novel method called Nesting Algorithm is presented in this paper. Our ideas are as follows: firstly, construct the optimal hyperplanes based on One-against-One approach. Secondly, if there exist data points in the middle unclassifiable region, select them to construct the optimal hyperplanes with the same hyperparameters. Thirdly, repeat the second step until there are no data points in the unclassifiable region or the region is disappeared. In this paper, we also prove the validity of the proposed algorithm for unclassifiable region and give the computational complexity analysis of the method. In order to examine the training accuracy and the generalization performance of the proposed algorithm, One-against-One algorithm, fuzzy least square support vector machine (FLS-SVM) and the proposed algorithm are applied to five UCI datasets. The results show that the training accuracy of the proposed algorithm is higher than the others, and its generalization performance is also comparable with them.
Similar content being viewed by others
References
Abe S (2003) Analysis of multiclass support vector machines. In: Proceeding of international conference on computational intelligence for modelling control and automation, Vienna, Austria, pp 385–396
Angulo C, Parra X, Català A (2003) K-SVCR a support vector machine for multi-class classification. Neurocomputing 55:57–77
Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Haussler D (ed) Proceeding 5th Annu, ACM workshop comput learning theory, pp 144–152
Bredensteniner EJ, Bennett KP (1999) Multicategory classification by support vector machines. Comput Optim Appl 12:53–79
Fürnkranz J (2002) Round robin classification. J Mach Learn Res 2:721–747
Hao ZF et al (2005) Twi-map support vector machine for multi-classification problems. In: Proceeding of ISNN 2005 conference, lecture notes in computer science 3496. Chongqing, PT1, pp 869–874
Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Tran Neural Netw 13:415–425
Inoue T, Abe S (2001) Fuzzy support vector machines for pattern classification. In Proceedings of international joint conference on neural networks (IJCNN‘01) 2:1449–1454
KreBel UHG (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJ Smola AJ (eds) Advances in Kernel methods: support vector learning. The MIT Press, Cambridge, pp 255–268
Liu B, Hao ZF, Yang XW (2006) Binary tree support vector machine based on kernel fisher discriminant for multi-classification. In: Proceeding of ISNN 2006 conference, lecture notes in computer science (in press)
Murphy PM, Aha DW (1992) UCI repository of machine learning database
Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Solla SA, Leen TK, Müller KR (eds) Advances in neural information processing systems 12. The MIT Press, Cambridge, pp 547–553
Rifkin R, Klautau A (2004) In defense of One-Vs-All classification. J Mach Learn Res 5:101–141
Saitoh S (1988) Theory of reproducing kernels and its applications. Longman, Harlow
Suykens JAK (2000) Least squares support vector machine for classification and nonlinear modeling. Neural Netw World 10(1–2): 29–47
Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Netw 16:785–792
Vapnik VN (1995) The nature of statistical learning theory. Springer, London
Vapnik VN (1998) Statistical learning theory. Wiley, New York
Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, B., Hao, Z. & Yang, X. Nesting Algorithm for Multi-Classification Problems. Soft Comput 11, 383–389 (2007). https://doi.org/10.1007/s00500-006-0093-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-006-0093-3