Skip to main content
SpringerLink
Log in

An online incremental learning support vector machine for large-scale data

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Support Vector Machines (SVMs) have gained outstanding generalization in many fields. However, standard SVM and most of modified SVMs are in essence batch learning, which make them unable to handle incremental learning or online learning well. Also, such SVMs are not able to handle large-scale data effectively because they are costly in terms of memory and computing consumption. In some situations, plenty of Support Vectors (SVs) are produced, which generally means a long testing time. In this paper, we propose an online incremental learning SVM for large data sets. The proposed method mainly consists of two components: the learning prototypes (LPs) and the learning Support Vectors (LSVs). LPs learn the prototypes and continuously adjust prototypes to the data concept. LSVs are to get a new SVM by combining learned prototypes with trained SVs. The proposed method has been compared with other popular SVM algorithms and experimental results demonstrate that the proposed algorithm is effective for incremental learning problems and large-scale problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. LibSVM are available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

  2. CVM and BVM are available at http://www.cse.ust.hk/%7Eivor/cvm.htm.

  3. Pendigits and optidigits are available at http://archive.ics.uci.edu/ml/.

  4. Satimage, w3a, ijcnn, webpage, usps, sensit vehicle (combined), are available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

References

  1. Vapnik V (1998) Statistical learning theory. Wiley, Chichester

    MATH  Google Scholar 

  2. Langford J, Li L, Zhang T (2008) Sparse online learning via truncated gradient. In: Advances in neural information processing systems, p 21

  3. Zhou ZH, Chen ZQ (2002) Hybrid decisions tree. Knowl Based Syst 15:515–528

    Article  Google Scholar 

  4. Syed N, Liu H, Sung K (1999) Incremental learning with support vector machines. In: Proceedings of the workshop on support vector machines at the international joint conference on artificial intelligence (IJCAI-99), Stockholm, Sweden

  5. Rüping S (2001) Incremental learning with support vector machines. In: First IEEE international conference on data mining (ICDM’01)

  6. Laskov P, Gehl C, Krüger S, Müller K (2006) Incremental support vector learning: analysis implementation and applications. J Mach Learn Res 7:1909–1936

    MathSciNet  MATH  Google Scholar 

  7. Schohn G, Cohn D (2000) Less is more: active learning with support vector machines. In: Proceedings of the international conference on machine learning

  8. Yu H, Yang J, Han J (2003) Classifying large data sets using SVMs with hierarchical clusters. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 306–C331

  9. Boley D, Cao D (2004) Training support vector machine using adaptive clustering. In: 4th SIAM international conference on data mining, pp 126–137

  10. Tsang IW, Kwok JT, Cheung P-M (2005) Core vector machines: fast SVM training on very large data sets. J Mach Learn Res 6:363–392

    MathSciNet  MATH  Google Scholar 

  11. Tsang IW, Kocsor A, Kwok JT (2007) Simpler core vector machines with enclosing balls. In: 24th International conference on machine learning, pp 911–918

  12. Li B, Chi M, Fan J, Xue X (2007) Support cluster machine. In: 24th International conference on machine learning, pp 505–512

  13. Shen F, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19:90–106

    Article  MATH  Google Scholar 

  14. Shen F, Ogura T, Hasgeawa O (2007) An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw 20:893–903

    Google Scholar 

  15. Kohonen T (1990) Improved versions of learning vector quantization. In: IJCNN90, pp 545–550

  16. Xu Y et al (2009) An online incremental learning vector quantization. In: PAKDD 2009, pp 1046–1053

  17. Zhang H et al (2006) SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: CVPR06, pp 2126–2136

  18. Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm

Download references

Acknowledgments

This work was supported in part by the Fund of the National Natural Science Foundation of China (Grant No. 60975047, 60723003, 60721002), 973 Program (2010CB327903), and Jiangsu NSF grant (BK2009080, BK2011567).

Author information

Authors and Affiliations

  1. National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Jun Zheng, Furao Shen, Hongjun Fan & Jinxi Zhao

  2. Jiangyin Information Technology Research Institute, Nanjing University, Nanjing, China

    Furao Shen

Authors
  1. Jun Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Furao Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongjun Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinxi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Furao Shen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zheng, J., Shen, F., Fan, H. et al. An online incremental learning support vector machine for large-scale data. Neural Comput & Applic 22, 1023–1035 (2013). https://doi.org/10.1007/s00521-011-0793-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0793-1

Keywords

Search

Navigation