Incremental Learning and Forgetting in One-Class Classifiers for Data Streams

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 226)


One-class classification and novelty detection is an important task in processing data streams. Standard algorithms used for this task cannot efficiently handle the changing environment to which they are applied. In this paper we present a modification of Weighted One-Class Support Vector Machine that is able to swiftly adapt to changes in data. This was achieved by extending this classifier by adding incremental learning and forgetting procedures. Both addition of new incoming data and removal of outdated objects is carried out on the basis of modifying weights assigned to each observation. We propose two methods for assigning weights to incoming data and two methods for removing the old objects. These approaches work gradually, therefore preserving useful characteristic of the examined dataset from previous iterations. Our approach was tested on two real-life dynamic datasets and the results prove the quality of our proposal.


machine learning one-class classification data streams concept drift incremental learning forgetting 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aggarwal, C.C., Han, J., Wang, J., Yu, P.S.: On demand classification of data streams. In: KDD 2004 Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 503–508 (2004)Google Scholar
  2. 2.
    Alpaydin, E.: Combined 5 x 2 cv f test for comparing supervised classification learning algorithms. Neural Computation 11(8), 1885–1892 (1999)CrossRefGoogle Scholar
  3. 3.
    Bicego, M., Figueiredo, M.A.T.: Soft clustering using weighted one-class support vector machines. Pattern Recognition 42(1), 27–32 (2009)zbMATHCrossRefGoogle Scholar
  4. 4.
    Bishop, C.M.: Novelty detection and neural network validation. IEE Proceedings: Vision, Image and Signal Processing 141(4), 217–222 (1994)CrossRefGoogle Scholar
  5. 5.
    Cyganek, B.: One-class support vector ensembles for image segmentation and classification. Journal of Mathematical Imaging and Vision 42(2-3), 103–117 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Gent, I.P., Miguel, I., Moore, N.C.A.: An empirical study of learning and forgetting constraints. AI Communications 25(2), 191–208 (2012)MathSciNetGoogle Scholar
  7. 7.
    Gomez-Verdejo, V., Arenas-Garcia, J., Lazaro-Gredilla, M., Navia-Vazquez, A.: Adaptive one-class support vector machine. IEEE Transactions on Signal Processing 59(6), 2975–2981 (2011)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Hashemi, S., Yang, Y., Mirzamomen, Z., Kangavari, M.: Adapted one-versus-all decision trees for data stream classification. IEEE Transactions on Knowledge and Data Engineering 21(5), 624–637 (2009)CrossRefGoogle Scholar
  9. 9.
    Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 97–106 (2001)Google Scholar
  10. 10.
    Koch, M.W., Moya, M.M., Hostetler, L.D., Fogler, R.J.: Cueing, feature discovery, and one-class learning for synthetic aperture radar automatic target recognition. Neural Networks 8(7-8), 1081–1102 (1995)CrossRefGoogle Scholar
  11. 11.
    Krawczyk, B.: Diversity in ensembles for one-class classification. In: Pechenizkiy, M., Wojciechowski, M. (eds.) New Trends in Databases & Inform. AISC, vol. 185, pp. 119–129. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Krawczyk, B., Woźniak, M.: Combining diverse one-class classifiers. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012, Part II. LNCS, vol. 7209, pp. 590–601. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  13. 13.
    Krawczyk, B., Woźniak, M.: Experiments on distance measures for combining one-class classifiers. In: Proceedings of the FEDCISIS 2012 Conference, pp. 88–92 (2012)Google Scholar
  14. 14.
    Masud, M., Gao, J., Khan, L., Han, J., Thuraisingham, B.M.: Classification and novel class detection in concept-drifting data streams under time constraints. IEEE Transactions on Knowledge and Data Engineering 23(6), 859–874 (2011)CrossRefGoogle Scholar
  15. 15.
    Ross, D.A., Lim, J., Lin, R., Yang, M.: Incremental learning for robust visual tracking. International Journal of Computer Vision 77(1-3), 125–141 (2008)CrossRefGoogle Scholar
  16. 16.
    Schölkopf, B., Smola, A.J.: Learning with kernels: support vector machines, regularization, optimization, and beyond. In: Adaptive Computation and Machine Learning. MIT Press (2002)Google Scholar
  17. 17.
    R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2008)Google Scholar
  18. 18.
    Zmyslony, M., Krawczyk, B., Woźniak, M.: Combined classifiers with neural fuser for spam detection. In: Herrero, A., Snasel, V., Abraham, A., Zelinka, I., Baruque, B., Quintin, H., Calvo, J.L., Sedano, J., Corchado, E. (eds.) International Joint Conference CISIS12-ICEUTE12-SOCO12 Special Sessions. AISC, vol. 189, pp. 245–252. Springer, Heidelberg (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  1. 1.Department of Systems and Computer NetworksWroclaw University of TechnologyWroclawPoland

Personalised recommendations