Abstract
Support vector machine (SVM) is an important classification tool in the pattern recognition and machine learning community, but its training is a time-consuming process. To deal with this problem, we propose a novel method to mine the useful information about classification hidden in the training sample for improving the training algorithm, and every training point is assigned to a value that represents the classification information, respectively, where training points with the higher values are chosen as candidate support vectors for SVM training. The classification information value for a training point is computed based on the classification accuracy of an appropriate hyperplane for the training sample, where the hyperplane goes through the mapped target of the training point in feature space defined by a kernel function. Experimental results on various benchmark datasets show the effectiveness of our algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Vapnik V. The Nature of Statistical Learning Theory[M]. New York: Springer-Verlag, 1998.
Burges C. A tutorial on support vector machines for pattern recognition[J]. Data Mining and Knowledge Discovery, 1998, 2(2): 121–167.
Cristianini N, Shawe-Taylor J. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods[ M]. Cambridge: Cambridge University Press, 2000.
Malon C, Uchida S, Suzuki M. Mathematical symbol recognition with support vector machines[J]. Pattern Recognition Letters, 2008, 29: 1326–1332.
Platt J. Fast Training Support Vector Machines Using Sequential Minimal Optimization [C]//Advances in Kernel Methods—Support Vector Learning. Cambridge: MIT Press, 1998.
Joachims T. Making large-scale support vector machine learning practical[C]//Advances in Kernel Methods-Support Vector Learning. Cambridge: MIT Press, 1998.
Syed N, Liu H, Sung K. Incremental learning with support vector machines[C]// Proceedings of the International Joint Conference on Artificial Intelligence. Sweden: Morgan Kaufmann Publishers, 1999: 352–356.
Mitra P, Murthy, Sankar K P. Data condensation in large databases by incremental learning with support vector machines[ C]//Proc Internat Conf on Pattern Recognition, 2000, 2: 2708–2711.
Xiao Xiaobo, Hu Guangshu. An incremental support vector machine based speech activity detection algorithm[C]//Proc of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conf, Shanghai: IEEE Press, 2005: 4224–4226.
Domeniconi C, Gunopulos D. Incremental support vector machine construction[C]// Proc ICDM, 2001: 589–592.
An Jinlong, Wang Zhengou, Ma Zhenping. An incremental learning algorithm for support vector machine[C]// Proc of the Second Internat Conf on Machine Learning and Cybernetics, Xi’an: IEEE Press, 2003: 1153–1156.
Katagiri S, Abe S. Incremental training of support vector machines using hyperspheres[J]. Pattern Recognition Letters, 2006, 27: 1495–1507.
Shawe-Taylor J, Cristianini N. Kernel Methods for Pattern Analysis[M]. Cambridge: Cambridge University Press, 2004.
Cauwenberghs G, Poggio T. Incremental and decremental support vector machine learning[J]. Advances in Neural Information Processing Systems, 2001, 13: 409–415.
Laskov P, Gehl C, Kruger S, et al. Incremental support vector learning: analysis, implementation and application[J]. Journal of Machine Learning Research, 2006, 7: 1909–1936.
Shilton A, Palamiswami M, Ralph D, et al. Incremental training of support vector machines[J]. IEEE Transactions on Neural Networks, 2005, 16: 114–131.
Cheng Shouxian, Shih F. An improved incremental training algorithm for support vector machines using active query[J]. Pattern Recognition, 2007, 40: 964–971.
Liang Zhizheng, Li Youfu. Incremental support vector machine learning in the primal and applications[J]. Neurocomputing, 2009, 72: 2249–2258.
Duan Hua, Shao Xiaojian, Hou Weizhen, et al. An incremental learning algorithm for Lagrangian support vector machines[J]. Pattern Recognition Letters, 2009, 30: 1384–1391.
Blake C, Keogh E, Merz C. UCI Repository of machine learning databases [ED/OL]. [2011-03-10]. http://www.ics.uci.edu/_mlearn/MLRepository.html.
Cheng Weiyuan, Juang Chiafeng. An incremental support vector machine-trained TS-type fuzzy system for online classification problems[J]. Fuzzy Sets and Systems, 2011, 163: 24–44.
Yi Yang, Wu Jiansheng, Xu Wei. Incremental SVM based on reserved set for network intrusion detection[J]. Expert Systems with Applications, 2011, 38: 7698–7707.
Andrzej P, Luo Jie, Barbara C. The more you learn, the less you store: Memory-controlled incremental SVM for visual place recognition[J]. Image and Vision Computing, 2010, 28: 1080–1097.
Author information
Authors and Affiliations
Corresponding author
Additional information
Foundation item: Supported by the National Natural Science Foundation of China (61070137, 60933009) and the Science and Technology Research Development Program in Shaanxi Province of China (2009K01-56)
Biography: SHEN Fengshan, male, Ph.D. candidate, research direction: pattern recognition and machine learning.
Rights and permissions
About this article
Cite this article
Shen, F., Zhang, J. & Yuan, X. Novel method of mining classification information for SVM training. Wuhan Univ. J. Nat. Sci. 16, 475–480 (2011). https://doi.org/10.1007/s11859-011-0784-1
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11859-011-0784-1