Abstract
Support Vector Machines (SVMs) have gained outstanding generalization in many fields. However, standard SVM and most of modified SVMs are in essence batch learning, which make them unable to handle incremental learning or online learning well. Also, such SVMs are not able to handle large-scale data effectively because they are costly in terms of memory and computing consumption. In some situations, plenty of Support Vectors (SVs) are produced, which generally means a long testing time. In this paper, we propose an online incremental learning SVM for large data sets. The proposed method mainly consists of two components: the learning prototypes (LPs) and the learning Support Vectors (LSVs). LPs learn the prototypes and continuously adjust prototypes to the data concept. LSVs are to get a new SVM by combining learned prototypes with trained SVs. The proposed method has been compared with other popular SVM algorithms and experimental results demonstrate that the proposed algorithm is effective for incremental learning problems and large-scale problems.
Similar content being viewed by others
Notes
LibSVM are available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
CVM and BVM are available at http://www.cse.ust.hk/%7Eivor/cvm.htm.
Pendigits and optidigits are available at http://archive.ics.uci.edu/ml/.
Satimage, w3a, ijcnn, webpage, usps, sensit vehicle (combined), are available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
References
Vapnik V (1998) Statistical learning theory. Wiley, Chichester
Langford J, Li L, Zhang T (2008) Sparse online learning via truncated gradient. In: Advances in neural information processing systems, p 21
Zhou ZH, Chen ZQ (2002) Hybrid decisions tree. Knowl Based Syst 15:515–528
Syed N, Liu H, Sung K (1999) Incremental learning with support vector machines. In: Proceedings of the workshop on support vector machines at the international joint conference on artificial intelligence (IJCAI-99), Stockholm, Sweden
Rüping S (2001) Incremental learning with support vector machines. In: First IEEE international conference on data mining (ICDM’01)
Laskov P, Gehl C, Krüger S, Müller K (2006) Incremental support vector learning: analysis implementation and applications. J Mach Learn Res 7:1909–1936
Schohn G, Cohn D (2000) Less is more: active learning with support vector machines. In: Proceedings of the international conference on machine learning
Yu H, Yang J, Han J (2003) Classifying large data sets using SVMs with hierarchical clusters. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 306–C331
Boley D, Cao D (2004) Training support vector machine using adaptive clustering. In: 4th SIAM international conference on data mining, pp 126–137
Tsang IW, Kwok JT, Cheung P-M (2005) Core vector machines: fast SVM training on very large data sets. J Mach Learn Res 6:363–392
Tsang IW, Kocsor A, Kwok JT (2007) Simpler core vector machines with enclosing balls. In: 24th International conference on machine learning, pp 911–918
Li B, Chi M, Fan J, Xue X (2007) Support cluster machine. In: 24th International conference on machine learning, pp 505–512
Shen F, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19:90–106
Shen F, Ogura T, Hasgeawa O (2007) An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw 20:893–903
Kohonen T (1990) Improved versions of learning vector quantization. In: IJCNN90, pp 545–550
Xu Y et al (2009) An online incremental learning vector quantization. In: PAKDD 2009, pp 1046–1053
Zhang H et al (2006) SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: CVPR06, pp 2126–2136
Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm
Acknowledgments
This work was supported in part by the Fund of the National Natural Science Foundation of China (Grant No. 60975047, 60723003, 60721002), 973 Program (2010CB327903), and Jiangsu NSF grant (BK2009080, BK2011567).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zheng, J., Shen, F., Fan, H. et al. An online incremental learning support vector machine for large-scale data. Neural Comput & Applic 22, 1023–1035 (2013). https://doi.org/10.1007/s00521-011-0793-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-011-0793-1