Neural Computing and Applications

, Volume 22, Issue 5, pp 1023–1035 | Cite as

An online incremental learning support vector machine for large-scale data

  • Jun Zheng
  • Furao Shen
  • Hongjun Fan
  • Jinxi Zhao
Original Article


Support Vector Machines (SVMs) have gained outstanding generalization in many fields. However, standard SVM and most of modified SVMs are in essence batch learning, which make them unable to handle incremental learning or online learning well. Also, such SVMs are not able to handle large-scale data effectively because they are costly in terms of memory and computing consumption. In some situations, plenty of Support Vectors (SVs) are produced, which generally means a long testing time. In this paper, we propose an online incremental learning SVM for large data sets. The proposed method mainly consists of two components: the learning prototypes (LPs) and the learning Support Vectors (LSVs). LPs learn the prototypes and continuously adjust prototypes to the data concept. LSVs are to get a new SVM by combining learned prototypes with trained SVs. The proposed method has been compared with other popular SVM algorithms and experimental results demonstrate that the proposed algorithm is effective for incremental learning problems and large-scale problems.


Online incremental SVM Incremental learning Large-scale data 



This work was supported in part by the Fund of the National Natural Science Foundation of China (Grant No. 60975047, 60723003, 60721002), 973 Program (2010CB327903), and Jiangsu NSF grant (BK2009080, BK2011567).


  1. 1.
    Vapnik V (1998) Statistical learning theory. Wiley, ChichesterzbMATHGoogle Scholar
  2. 2.
    Langford J, Li L, Zhang T (2008) Sparse online learning via truncated gradient. In: Advances in neural information processing systems, p 21Google Scholar
  3. 3.
    Zhou ZH, Chen ZQ (2002) Hybrid decisions tree. Knowl Based Syst 15:515–528CrossRefGoogle Scholar
  4. 4.
    Syed N, Liu H, Sung K (1999) Incremental learning with support vector machines. In: Proceedings of the workshop on support vector machines at the international joint conference on artificial intelligence (IJCAI-99), Stockholm, SwedenGoogle Scholar
  5. 5.
    Rüping S (2001) Incremental learning with support vector machines. In: First IEEE international conference on data mining (ICDM’01)Google Scholar
  6. 6.
    Laskov P, Gehl C, Krüger S, Müller K (2006) Incremental support vector learning: analysis implementation and applications. J Mach Learn Res 7:1909–1936MathSciNetzbMATHGoogle Scholar
  7. 7.
    Schohn G, Cohn D (2000) Less is more: active learning with support vector machines. In: Proceedings of the international conference on machine learningGoogle Scholar
  8. 8.
    Yu H, Yang J, Han J (2003) Classifying large data sets using SVMs with hierarchical clusters. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 306–C331Google Scholar
  9. 9.
    Boley D, Cao D (2004) Training support vector machine using adaptive clustering. In: 4th SIAM international conference on data mining, pp 126–137Google Scholar
  10. 10.
    Tsang IW, Kwok JT, Cheung P-M (2005) Core vector machines: fast SVM training on very large data sets. J Mach Learn Res 6:363–392MathSciNetzbMATHGoogle Scholar
  11. 11.
    Tsang IW, Kocsor A, Kwok JT (2007) Simpler core vector machines with enclosing balls. In: 24th International conference on machine learning, pp 911–918Google Scholar
  12. 12.
    Li B, Chi M, Fan J, Xue X (2007) Support cluster machine. In: 24th International conference on machine learning, pp 505–512Google Scholar
  13. 13.
    Shen F, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19:90–106zbMATHCrossRefGoogle Scholar
  14. 14.
    Shen F, Ogura T, Hasgeawa O (2007) An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw 20:893–903Google Scholar
  15. 15.
    Kohonen T (1990) Improved versions of learning vector quantization. In: IJCNN90, pp 545–550Google Scholar
  16. 16.
    Xu Y et al (2009) An online incremental learning vector quantization. In: PAKDD 2009, pp 1046–1053Google Scholar
  17. 17.
    Zhang H et al (2006) SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: CVPR06, pp 2126–2136Google Scholar
  18. 18.
    Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines.

Copyright information

© Springer-Verlag London Limited 2012

Authors and Affiliations

  • Jun Zheng
    • 1
  • Furao Shen
    • 1
    • 2
  • Hongjun Fan
    • 1
  • Jinxi Zhao
    • 1
  1. 1.National Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.Jiangyin Information Technology Research InstituteNanjing UniversityNanjingChina

Personalised recommendations