Abstract
Twin support vector machine (TSVM) is a novel machine learning algorithm, which aims at finding two nonparallel planes for each class. In order to do so, one needs to resolve a pair of smaller-sized quadratic programming problems rather than a single large one. Classical TSVM is proposed for the binary classification problem. However, multi-class classification problem is often met in our real world. For this problem, a new multi-class classification algorithm, called Twin-KSVC, is proposed in this paper. It takes the advantages of both TSVM and K-SVCR (support vector classification-regression machine for k-class classification) and evaluates all the training points into a “1-versus-1-versus-rest” structure, so it generates ternary outputs { −1, 0, +1}. As all the samples are utilized in constructing the classification hyper-plane, our proposed algorithm yields higher classification accuracy in comparison with other two algorithms. Experimental results on eleven benchmark datasets demonstrate the feasibility and validity of our proposed algorithm.
Similar content being viewed by others
Notes
http://archive.ics. uci. edu/ml/datasets.html.
References
Vapnik V. The nature of statistical learning theory. New York: Springer; 1995.
Ripley BD. Pattern recognition and neural networks. Cambridge, Cambridge University Press; 1996.
Jayadeva K, Khemchandani R, Chandra S. Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell. 2007;29(5):905–10.
Khemchandani R, Jayadeva, Chandra S. Optimal kernel selection in twin support vector machines. Optim Lett. 2009;3(1):77–88.
Ghorai S, Mukherjee A, Dutta PK. (2009) Nonparallel plane proximal classifier. Signal Process. 89(4):510–22.
Fung G, mangasarian OL. Multicategory proximal support vector machine classifiers. Mach Learn. 2005;59:77–97.
Peng XJ. A new twin support vector machine clssifier and its geometric algorithms. Inf Sci. 2010;180(20):3863–75.
Xu Y, Wang L, Zhong P. A rough margin-based ν-twin support vector machine. Neural Comput Appl. 2012;21(6):1307–17.
Kumar MA, Gopal M. Least squares twin support vector machines for pattern classification. Expert Syst Appl. 2009;36(4):7535–43.
Peng XJ. TSVR: an efficient twin support vector machine for regression. Neural Netw. 2010;23(3):365–72.
Xu Y, Wang L. A weighted twin support vector regression. Knowl-Based Syst. 2012;33:92–101.
Lin C, Wang S. Fuzzy support vector machines. IEEE Trans. Neural Netw. 2002;13(2):464–71.
Jayadeva K, Khemchandani R, Chandra S. Fuzzy multi-category proximal support vector classification via generalized eigenvalues. Soft Comput. 2007;11(7):679–685.
Khemchandani R, Jayadeva, Chandra S. Fuzzy twin support vector machines for pattern classification. Mathematical programming and game theory for decision making. Singapore: World Scientific; 2008. p. 131–42.
Arun Kumar M, Gopal M. Application of smoothing technique on twin support vector machines. Pattern Recognit Lett. 2008; 29(13):1842–48.
Chen X, Yang J, Liang J, Ye Q. Smooth twin support vector regression. Neural Comput Appl. 2012;21(3):505–13.
Ghorai S, Hossain SJ, Dutta PK, Mukherjee A. Newtons method for nonparallel plane proximal classifier with unity norm hyperplanes. Signal Process. 2010; 90(1):93–104.
Hastie TJ, Tibshirani RJ. Classification by pairwise coupling. In: Jordan MI, Kearns MJ, Solla SA, editors. Advances in neural information processing systems 10. Cambridge MA: MIT Press; 1998. p. 507–13.
Krebel U. Pairwise classification and support vector machines. In: Scholkopf B, Burges CJC, Smola AJ, editors. Advances in Kernel methods: support vector learning. Cambridge, MA: MIT Press; 1999. p. 255–68.
Hsu C, Lin C. A comparision of methods for multiclass support vector machine. IEEE Trans. Neural Netw. 2002;13:415–25.
Subirats JL, Jerez JM, Gomez I, Franco L. Multiclass pattern recognition extension for the new C-Mantec constructive neural network algorithm. Cogn Comput. 2010;2(4):285–90.
Thomas GD, Ghulum B. Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res. 1995;2:263–86.
Nagase A, Kawanaka H, Shoaib Bhuiyan Md, Oguri K. Multi-class identification of drivers cognitive distraction with error-correcting output coding (ECOC) method. In: Proceedings of the 12th international IEEE conference on intelligent transportation systems; 2009. p. 96–101.
Zhou ZH, Liu XY. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans Knowl Data Eng. 2006;18:63–77.
Angulo C, Parra X, Catal A. K-SVCR: a support vector machine for multi-class classification. Neurocomputing. 2003;55:57–77.
Cong H, Yang C, Pu X. Efficient speaker recognition based on multi-class twin support vector machines and GMMs, robotics, automation and mechatronics, 2008 IEEE conference on 21–24 Sept. 2008. p. 348–52.
Zhong P, Fukshima M. A new multi-class support vector algorithm. Optim Methods Softw. 2006;3:359–72.
Platt J. Fast training of support vector machine using sequential minimal optimization. In:Advances in kernel methods-support vector learning . Cambridge: MIT Press; 1999. p. 185–208.
Yang K, Zhao L. A new optimizing parameter approach of LSSVM multiclass classification model. Neural Comput Appl. 2012;21(5):945–55.
He H, Edwardo AG. Learning from imbalanced data. IEEE Trans Knowl Data Eng. 2009;21(9):1263–84.
Acknowledgments
The authors thank all anonymous referees for helpful comments that have leaded to improvement of the paper. This work was supported by National Natural Science Foundation of China (Grant No. 61153003, 11171346).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xu, Y., Guo, R. & Wang, L. A Twin Multi-Class Classification Support Vector Machine. Cogn Comput 5, 580–588 (2013). https://doi.org/10.1007/s12559-012-9179-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-012-9179-7