Online Support Vector Machine: A Survey

  • Xujun Zhou
  • Xianxia Zhang
  • Bing Wang
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 382)


Support Vector Machine (SVM) is one of the fastest growing methods of machine learning due to its good generalization ability and good convergence performance; it has been successfully applied in various fields, such as text classification, statistics, pattern recognition, and image processing. However, for real-time data collection systems, the traditional SVM methods could not perform well. In particular, they cannot well cope with the increasing new samples. In this paper, we give a survey on online SVM. Firstly, the description of SVM is introduced, then the brief summary of online SVM is given, and finally the research and development of online SVM are presented.


Support vector machine Machine learning Generalization ability Convergence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Burges, J.C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc., 121-167 (1998)Google Scholar
  2. 2.
    Schölkopf, B.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press (2002)Google Scholar
  3. 3.
    Vapnik, N.V.: The Nature of Statistical Learning. Springer, New York (1995)zbMATHCrossRefGoogle Scholar
  4. 4.
    Vapnik, N.V., Lerner, A.: Pattern recognition using generalized portrait method. Automat. Rem. Contr+. 24, 774–780 (1963)Google Scholar
  5. 5.
    Vapnik, N.V., Chervonenkis A.: A note on one class of perceptrons. Automat. Rem. Contr+. 25 (1964)Google Scholar
  6. 6.
    Cristianini, N., Shawe Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000)Google Scholar
  7. 7.
    Dumais, S., Platt, J., Heckerman, D., Sahami, M.: Inductive learning algorithms and representations for text categorization. In: 7th International Conference on Information and Knowledge Management, ACM-CIKM 1998 (1998)Google Scholar
  8. 8.
    Osuna, E., Freund, R., Girosi, F.: Training support vector machines: an application to face detection. In: International Conference on Computer Vision and Pattern Recognition, CVPR 1997 (1997)Google Scholar
  9. 9.
    Schölkopf, B., Smola, A., Muller, K.R.: Kernel Principal Component Analysis. In: Schölkopf, B., Burges, C.J.C., Smola, A. (eds.) Advances in Kernel Methods-Support Vector Learning, pp. 327–352. MIT Press, Cambridge (1999)Google Scholar
  10. 10.
    Ahmad, A, Khalid, M., Yusof, R.: Kernel methods and support vector machines for handwriting recognition. In: IEEE Student Conference on Research and Development Proceedings (SCOReD 2002), pp. 309-312 (2002)Google Scholar
  11. 11.
    Cortes, C., Vapnik, N.V.: Support vector networks. Mach. Learn. 20, 273–297 (1995)zbMATHGoogle Scholar
  12. 12.
    Shevade, S.K., Keerthi, S.S., Bhattacharyya, C., Murthy, K.R.K.: Improvements to the SMO algorithm for SVM regression. IEEE Trans. Neural Netw. 11(5), 1188–1193 (2000)CrossRefGoogle Scholar
  13. 13.
    Chen, P., Fan, R., Lin, C.: A study on SMO-type decomposition methods for support vector machines. IEEE Trans. Neural Netw. 17(4), 893–908 (2006)CrossRefGoogle Scholar
  14. 14.
    Cai, F., Cherkassky, V.: Generalized SMO algorithm for SVM-based multitask learning. IEEE Trans. Neural Netw. Learn. Syst. 23(6), 997–1003 (2012)CrossRefGoogle Scholar
  15. 15.
    Cortes, C., Vapnik, V.: Support Vector Networks. Mach. Learn. 20, 273–297 (1995)zbMATHGoogle Scholar
  16. 16.
    Bordes, A., Bottou, L., Gallinari, P.: SGD-QN: Careful quasi-Newton stochastic gradient descent. J. Mach. Learn. 10, 1737–1754 (2009)zbMATHMathSciNetGoogle Scholar
  17. 17.
    Souza, R.C., Leite, S.C., Borges, C.C., Neto, R.F.: Online algorithm based on support vectors for orthogonal regression. Pattern Recognition Letters 34(12), 1394–1404 (2013)Google Scholar
  18. 18.
    Kivinen, J., Smola, A.J., Williamson, R.C.: Online learning with Kernels. IEEE Transactions on Signal Processing, 2165-2176 (2004)Google Scholar
  19. 19.
    Syed, N., Liu, H., Sung, K.: Handling concept drifts in incremental learning with support vector machines. In: 5th ACM SIGKDD International Conference Knowledge Discovery Data Mining, pp. 317–321 (1999)Google Scholar
  20. 20.
    Singh, R., Vatsa, M., Ross, A., Noore, A.: Biometric classifier update using online learning: A case study in near infrared face verification. Image Vis. Comput. 28(7), 1098–1105 (2010)CrossRefGoogle Scholar
  21. 21.
    Wang, M., Zhou, X., Li, F., Huckins, J., King, R., Wong, S.T.C.: Novel cell segmentation and online SVM for cell cycle phase identification in automated microscopy. Bioinformatics. 24(1), 94–101 (2008)CrossRefGoogle Scholar
  22. 22.
    Cauwenberghs, G., Poggio, T.: Incremental and decremental support vector machine learning. In: Dietterich, T.G., Leen, T.K., Tresp, V. (eds.) Advances in Neural Infomation Processing Systems, vol.13, pp. 409-415. MIT Press (2001)Google Scholar
  23. 23.
    Martin, M.: On-Line support vector machine regression. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 282–294. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  24. 24.
    Bordes, A., Bottou, L.: The Huller: a simple and efficient online SVM. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 505–512. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  25. 25.
    Wang, D.I.: Online Support Vector Machine Based on Convex Hull Vertices Selection. IEEE Transactions on Neural Networks Learning Systems 24(4), 593–608 (2013)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Shanghai Key Laboratory of Power Station Automation Technology, College of Mechatronics Engineering and AutomationShanghai UniversityShanghaiChina

Personalised recommendations