An Aggressive Margin-Based Algorithm for Incremental Learning

  • JuiHsi Fu
  • SingLing Lee
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7301)


In incremental learning, the classification model is incrementally updated using the small datasets. Different with existing methods, our approach updates the current classifier according to each sample in the dataset, respectively. The classifier is updated by adjusting more than the margin of each sample. Then the new classifier is generated by carefully analyzing classifier adjustments caused for labeled samples. Additionally the new classifier shall correct prediction mistakes of the previous classifier as many as possible. In details, we formulate simple constrained optimization problems and then the updated classifier is the solution derived using Lagrange multipliers. In our experiments, 13 real-world dataset are used to present the effectiveness of the proposed approach. The experimental results are shown that our update strategy is able to adjust the classifier properly. And it is also shown that the proposed incremental learning approach is suitable to be applied for the requirement of frequently adjusting the existing classifiers.


Incremental Learning Margin-based Approaches Passive-Aggressive (PA) Algorithm Period Datasets Classifier Adjustment 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sena, G.G., Belzarena, P.: Early traffic classification using support vector machines. In: 5th International Latin American Networking Conference, pp. 60–66. ACM, New York (2009)CrossRefGoogle Scholar
  2. 2.
    Robertson, W.K., Maggi, F., Kruegel, C., Vigna, G.: Effective Anomaly Detection with Scarce Training Data. In: The Network and Distributed System Security Symposium. ISOC (2010)Google Scholar
  3. 3.
    Du, H., Teng, S., Yang, M., Zhu, Q.: Intrusion Detection System Based on Improved SVM Incremental Learning. In: International Conference on Artificial Intelligence and Computational intelligence, pp. 23–28. IEEE Press (2009)Google Scholar
  4. 4.
    Utgoff, P.E.: Incremental Induction of Decision Trees. J. Machine Learning 4, 161–186 (1989)CrossRefGoogle Scholar
  5. 5.
    Mohamed, S., Rubin, D., Marwala, T.: Incremental Learning for Classification of Protein Sequences. In: International Joint Conference on Neural Networks, pp. 19–24. IEEE Press (2007)Google Scholar
  6. 6.
    Chen, Z., Huang, L., Murphey, Y.L.: Incremental Learning for Text Document Classification. In: International Joint Conference on Neural NetWorks, pp. 2592–2597. IEEE Press (2007)Google Scholar
  7. 7.
    Ruping, S.: Incremental Learning with Support Vector Machines. In: International Conference on Data Mining, pp. 641–642. IEEE Press (2001)Google Scholar
  8. 8.
    Xiao, R., Wang, J., Zhang, F.: An Approach to Incremental SVM Learning Algorithm. In: International Conference on Tools with Artificial Intelligence, pp. 268–273. IEEE Press (2000)Google Scholar
  9. 9.
    Cauwenberghs, G., Poggio, T.: Incremental and Decremental Support Vector Machine Learning. In: Neural Information Processing Systems, vol. 13. MIT Press, Cambridge (2001)Google Scholar
  10. 10.
    Liu, Y., He, Q., Chen, Q.: Incremental Batch Learning with Support Vector Machines. In: 5th World Congress on Intelligent Control and Automation, pp. 1857–1861. IEEE Press (2004)Google Scholar
  11. 11.
    Crammer, K., Dekel, O., Keshet, J., Shwartz, S.S., Singer, Y.: Online Passive-Aggressive Algorithms. J. Machine Learning Research 7, 551–585 (2006)zbMATHGoogle Scholar
  12. 12.
    Zhu, X.: Lazy Bagging for Classifying Imbalanced Data. In: 7th IEEE International Conference on Data Mining, pp. 763–768 (2007)Google Scholar
  13. 13.
    Freund, Y., Schapire, R.E.: Large Margin Classification Using the Perceptron Algorithm. J. Machine Learning 37, 277–296 (1999)zbMATHCrossRefGoogle Scholar
  14. 14.
    Ng, H.T., Goh, W.B., Low, K.L.: Feature selection, perceptron learning, and a usability case study for text categorization. In: International Conference on Research and Development in Information Retrieval, pp. 67–73. ACM, New York (1997)Google Scholar
  15. 15.
    Cesa-Bianchi, N., Conconi, A., Gentile, C.: A Second-Order Perceptron Algorithm. J. Computing 34(3), 640–668 (2005)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Wang, S., San, Y., Wang, S.: An Online Modeling Method Based on Support Vector Machine. In: International Conference on COmputer Science and Software Engineering, pp. 98–101. IEEE Press (2008)Google Scholar
  17. 17.
    Sculley, D., Wachman, G.M.: Relaxed Online SVMs for spam filtering. In: International Conference on Research and Development in Information Retrieval, pp. 415–422. ACM, New York (2007)Google Scholar
  18. 18.
    Dredze, M., Crammer, K., Pereira, F.: Confidence-Weighted Linear Classification. In: International Conference on Machine Learning, pp. 264–271. ACM, New York (2008)Google Scholar
  19. 19.
    Crammer, K., Kulesza, A., Dredze, M.: Adaptive Regularization of Weight Vectors. In: Neural Information Processing Systems. MIT Press, Cambridge (2009)Google Scholar
  20. 20.
    Lin, P., Yen, T., Fu, J., Yu, C.: Analyzing Anomalous Spamming Activities in a Campus Network. In: TANET (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • JuiHsi Fu
    • 1
  • SingLing Lee
    • 1
  1. 1.National Chung Cheng UniversityChiayiTaiwan, R.O.C.

Personalised recommendations