Nesting Algorithm for Multi-Classification Problems
- 97 Downloads
Support vector machines (SVMs) are originally designed for binary classifications. As for multi-classifications, they are usually converted into binary ones. In the conventional multi-classifiable algorithms, One-against-One algorithm is a very power method. However, there exists a middle unclassifiable region. In order to overcome this drawback, a novel method called Nesting Algorithm is presented in this paper. Our ideas are as follows: firstly, construct the optimal hyperplanes based on One-against-One approach. Secondly, if there exist data points in the middle unclassifiable region, select them to construct the optimal hyperplanes with the same hyperparameters. Thirdly, repeat the second step until there are no data points in the unclassifiable region or the region is disappeared. In this paper, we also prove the validity of the proposed algorithm for unclassifiable region and give the computational complexity analysis of the method. In order to examine the training accuracy and the generalization performance of the proposed algorithm, One-against-One algorithm, fuzzy least square support vector machine (FLS-SVM) and the proposed algorithm are applied to five UCI datasets. The results show that the training accuracy of the proposed algorithm is higher than the others, and its generalization performance is also comparable with them.
KeywordsSupport vector machines Least squares support vector machine One-against-One algorithm FLS-SVM Nesting algorithm
Unable to display preview. Download preview PDF.
- 1.Abe S (2003) Analysis of multiclass support vector machines. In: Proceeding of international conference on computational intelligence for modelling control and automation, Vienna, Austria, pp 385–396Google Scholar
- 3.Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Haussler D (ed) Proceeding 5th Annu, ACM workshop comput learning theory, pp 144–152Google Scholar
- 6.Hao ZF et al (2005) Twi-map support vector machine for multi-classification problems. In: Proceeding of ISNN 2005 conference, lecture notes in computer science 3496. Chongqing, PT1, pp 869–874Google Scholar
- 8.Inoue T, Abe S (2001) Fuzzy support vector machines for pattern classification. In Proceedings of international joint conference on neural networks (IJCNN‘01) 2:1449–1454Google Scholar
- 9.KreBel UHG (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJ Smola AJ (eds) Advances in Kernel methods: support vector learning. The MIT Press, Cambridge, pp 255–268Google Scholar
- 10.Liu B, Hao ZF, Yang XW (2006) Binary tree support vector machine based on kernel fisher discriminant for multi-classification. In: Proceeding of ISNN 2006 conference, lecture notes in computer science (in press)Google Scholar
- 11.Murphy PM, Aha DW (1992) UCI repository of machine learning databaseGoogle Scholar
- 12.Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Solla SA, Leen TK, Müller KR (eds) Advances in neural information processing systems 12. The MIT Press, Cambridge, pp 547–553Google Scholar
- 13.Rifkin R, Klautau A (2004) In defense of One-Vs-All classification. J Mach Learn Res 5:101–141Google Scholar