Two One-Pass Algorithms for Data Stream Classification Using Approximate MEBs

  • Ricardo Ñanculef
  • Héctor Allende
  • Stefano Lodi
  • Claudio Sartori
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6594)

Abstract

It has been recently shown that the quadratic programming formulation underlying a number of kernel methods can be treated as a minimal enclosing ball (MEB) problem in a feature space where data has been previously embedded. Core Vector Machines (CVMs) in particular, make use of this equivalence in order to compute Support Vector Machines (SVMs) from very large datasets in the batch scenario. In this paper we study two algorithms for online classification which extend this family of algorithms to deal with large data streams. Both algorithms use analytical rules to adjust the model extracted from the stream instead of recomputing the entire solution on the augmented dataset. We show that these algorithms are more accurate than the current extension of CVMs to handle data streams using an analytical rule instead of solving large quadratic programs. Experiments also show that the online approaches are considerably more efficient than periodic computation of CVMs even though warm start is being used.

Keywords

Data stream mining Online learning Kernel methods Minimal enclosing balls 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aggarwal, C. (ed.): Data Streams, Models and Algorithms. Springer, Heidelberg (2007)MATHGoogle Scholar
  2. 2.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2010)Google Scholar
  3. 3.
    Clarkson, K.: Coresets, sparse greedy approximation, and the frank-wolfe algorithm. In: Proceedings of SODA 2008, pp. 922–931. SIAM, Philadelphia (2008)Google Scholar
  4. 4.
    Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., Singer, Y.: Online passive-aggressive algorithms. J. of Machine Learning Research 7, 551–585 (2006)MathSciNetMATHGoogle Scholar
  5. 5.
    Dekel, O., Shalev-Shwartz, S., Singer, Y.: The forgetron: A kernel-based perceptron on a budget. SIAM Journal of Computing 37(5), 1342–1372 (2008)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Frandi, E., Gasparo, M.-G., Lodi, S., Ñanculef, R., Sartori, C.: A new algorithm for training sVMs using approximate minimal enclosing balls. In: Bloch, I., Cesar Jr., R.M. (eds.) CIARP 2010. LNCS, vol. 6419, pp. 87–95. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Hettich, S., Bay, S.: The UCI KDD Archive (2010), http://kdd.ics.uci.edu
  8. 8.
    Kivinen, J.: Online learning of linear classifiers, pp. 235–257 (2003)Google Scholar
  9. 9.
    Kivinen, J., Smola, A.J., Williamson, R.C.: Online learning with kernels. IEEE Transactions on Signal Processing 52(8), 2165–2176 (2004)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Lodi, S., Ñanculef, R., Sartori, C.: Single-pass distributed learning of multi-class svms using core-sets. In: Proceedings of the SDM 2010, pp. 257–268. SIAM, Philadelphia (2010)Google Scholar
  11. 11.
    Léon Bottou, D.D., Chapelle, O., Weston, J. (eds.): Large Scale Kernel Machines. MIT Press, Cambridge (2007)Google Scholar
  12. 12.
    Rai, P., Daumé, H., Venkatasubramanian, S.: Streamed learning: one-pass svms. In: IJCAI 2009: Proceedings of the 21st International Jont Conference on Artifical Intelligence, pp. 1211–1216. Morgan Kaufmann Publishers, San Francisco (2009)Google Scholar
  13. 13.
    Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)Google Scholar
  14. 14.
    Shalev-Shwartz, S., Singer, Y.: A primal-dual perspective of online learning algorithms. Machine Learning 69(2-3), 115–142 (2007)CrossRefGoogle Scholar
  15. 15.
    Tsang, I., Kocsor, A., Kwok, J.: Simpler core vector machines with enclosing balls. In: ICML 2007, pp. 911–918. ACM, New York (2007)Google Scholar
  16. 16.
    Tsang, I., Kocsor, A., Kwok, J.: LibCVM Toolkit (2009)Google Scholar
  17. 17.
    Tsang, I., Kwok, J., Cheung, P.-M.: Core vector machines: Fast svm training on very large data sets. Journal of Machine Learning Research 6, 363–392 (2005)MathSciNetMATHGoogle Scholar
  18. 18.
    Tsang, I., Kwok, J., Zurada, J.: Generalized core vector machines. IEEE Transactions on Neural Networks 17(5), 1126–1140 (2006)CrossRefGoogle Scholar
  19. 19.
    Wang, D., Zhang, B., Zhang, P., Qiao, H.: An online core vector machine with adaptive meb adjustment. Pattern Recognition 43(10), 3468–3482 (2010)CrossRefMATHGoogle Scholar
  20. 20.
    Yildirim, E.A.: Two algorithms for the minimum enclosing ball problem. SIAM Journal on Optimization 19(3), 1368–1391 (2008)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Zarrabi-Zadeh, H., Chan, T.M.: A simple streaming algorithm for minimum enclosing balls. In: Proceedings of the CCCG 2006 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ricardo Ñanculef
    • 1
  • Héctor Allende
    • 1
  • Stefano Lodi
    • 2
  • Claudio Sartori
    • 2
  1. 1.Dept. of InformaticsFederico Santa María UniversityChile
  2. 2.Dept. of Electronics, Computer Science and SystemsUniversity of BolognaItaly

Personalised recommendations