Abstract
Twin support vector machine (TSVM) is regarded as a milestone in the development of the powerful SVM. It finds two nonparallel planes by resolving a pair of smaller-sized quadratic programming problems rather than a single large one, which makes the learning speed of TSVM approximately four times faster than that of the standard SVM. However, the empirical risk minimization principle is implemented in the TSVM, so it easily leads to the over-fitting problem and reduces the prediction accuracy of the classifier. ν-TSVM, as a variant of TSVM, also implements the empirical risk minimization principle. To enhance the generalization ability of the classifier, we propose an improved ν-TSVM by introducing a regularization term into the objective function, so there are two parts in the objective function, one of which is to maximize the margin between the two parallel hyper-planes, and the other one is to minimize the training errors of two classes of samples. Therefore the structural risk minimization principle is implemented in our improved ν-TSVM. Numerical experiments on one artificial dataset and nine benchmark datasets show that our improved ν-TSVM yields better generalization performance than SVM, ν-SVM, and ν-TSVM. Moreover, numerical experiments with different proportions of outliers demonstrate that our improved ν-TSVM is robust and stable. Finally, we apply our improved ν-TSVM to two BCI competition datasets, and also obtain better prediction accuracy.
Similar content being viewed by others
References
Vapnik V (1995) The nature of statistical learning theory. Springer, New York
Xu Y, Wang L (2005) Fault diagnosis system based on rough set theory and support vector machine. Lect Notes Comput Sci 3614:981–988
Zhang W, Yoshida T, Tang X (2008) Text classification based on multi-word with support vector machine. Knowl-Based Syst 21(8):879–886
Kaper M, Meinicke P, Grossekathoefer U et al (2004) BCI competition 2003—data set IIb: support vector machines for the P300 speller paradigm. IEEE Trans Biomed Eng 51(6):1073–1076
Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29:905–910
Fung G, Mangasarian O (2001) Proximal support vector machine classifiers. In: Seven international proceedings on knowledge discovery and data mining, pp 77–86
Ghorai S, Mukherjee A, Dutta P (2009) Nonparallel plane proximal classifier. Signal Process 89:510–522
Fung G, Mangasarian O (2005) Multicategory proximal support vector machine classifiers. Mach Learn 59:77–97
Peng X (2010) A ν twin support vector machine (ν-TSVM) classifier and its geometric algorithms. Inf Sci 180:3863–3875
Xu Y, Wang L, Zhong P (2012) A rough margin-based ν-twin support vector machine. Neural Comput Appl 21:1307–1317
Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36:7535–7543
Shao H, Wang Z, Chen W, Deng N (2013) Least squares twin parametric-margin support vector machine for classification. Appl Intell 39:451–464
Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23:365–372
Xu Y, Wang L (2012) A weighted twin support vector regression. Knowl-Based Syst 33:92–101
Xu Y, Guo R, Wang L (2012) A twin multi-class classification support vector machine. Cogn Comput. doi:10.1007/s12559-012-9179-7
Thongkam J, Xu G, Zhang Y (2008) Support vector machine for outlier detection in breast cancer survivability prediction. Lect Notes Comput Sci 4977:99–109
Cui W, Yan X (2009) Adaptive weighted least square support vector machine regression integrated with outlier detection and its application in QSAR. Chemom Intell Lab Syst 98:130–135
Zhang Y, Meratnia N, Havinga P (2009) Adaptive and online one-class support vector machine-based outlier detection techniques for wireless sensor networks. In: Proceeding of international conference on advanced information networking and applications workshops, pp 990–995
Xu Y, Liu C (2013) A rough margin-based one class support vector machine. Neural Comput Appl 22:1077–1084
Scholkopf B, Smola A, Bartlett P, Williamson R (2000) New support vector algorithms. Neural Comput 12:1207–1245
Peng X (2011) Building sparse twin support vector machine classifiers in primal space. Inf Sci 181:3967–3980
Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machine. IEEE Trans Neural Netw 22:962–968
Khemchandani R, Jayadeva, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3:77–88
Diosan L, Rogozan A, Pecuchet J-P (2013) Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters. Appl Intell 36:280–294
Kim M (2013) Accelerated max-margin multiple kernel learning. Appl Intell 38:45–57
Zhang J, Wang Y (2008) A rough margin based on support vector machine. Inf Sci 178:2204–2214
Wang C, You W (2013) Boosting-SVM: effective learning with reduced data dimension. Appl Intell 39:465–474
Acknowledgements
The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. This work was supported by the National Natural Science Foundation of China (No. 61153003) and China Scholarship Fund (No. 201208110282).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xu, Y., Guo, R. An improved ν-twin support vector machine. Appl Intell 41, 42–54 (2014). https://doi.org/10.1007/s10489-013-0500-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-013-0500-2