We propose a strategy for updating the learning rate parameter of online linear classifiers for streaming data with concept drift. The change in the learning rate is guided by the change in a running estimate of the classification error. In addition, we propose an online version of the standard linear discriminant classifier (O-LDC) in which the inverse of the common covariance matrix is updated using the Sherman-Morrison-Woodbury formula. The adaptive learning rate was applied to four online linear classifier models on generated and real streaming data with concept drift. O-LDC was found to be better than balanced Winnow, the perceptron and a recently proposed online linear discriminant analysis.


Linear Discriminant Analysis Electricity Price Concept Drift Streaming Data Online Training 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Rosenblatt, F.: Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books, Washington (1962)zbMATHGoogle Scholar
  2. 2.
    Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear threshold algorithm. Machine Learning 2, 285–318 (1988)Google Scholar
  3. 3.
    Grove, A., Littlestone, N., Schuurmans, D.: General convergence results for linear discriminant updates. Machine Learning 43(3), 179–210 (2001)CrossRefzbMATHGoogle Scholar
  4. 4.
    Wang, B., Jones, G.J.F., Pan, W.: Using online linear classifiers to filter spam emails. Pattern Analysis and Applications 9(4), 339–351 (2006)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Carvalho, V.R., Cohen, W.W.: Notes on single-pass online learning. Technical Report CMU-LTI-06-002, Carnegie Mellon University (2006)Google Scholar
  6. 6.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, Chichester (2001)zbMATHGoogle Scholar
  7. 7.
    Hiraoka, K., Yoshizawa, S., Hidai, K., Hamahira, M., Mizoguchi, H., Mishima, T.: Convergence analysis of online linear discriminant analysis. In: Proc. Int. Joint Conf. on Neural Networks, vol. 3, pp. 387–391 (2000)Google Scholar
  8. 8.
    Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23, 69–101 (1996)Google Scholar
  9. 9.
    Narasimhamurthy, A., Kuncheva, L.I.: A framework for generating data to simulate changing environments. In: Proc. IASTED, Artificial Intelligence and Applications, Innsbruck, Austria, pp. 384–389 (2007)Google Scholar
  10. 10.
    Gama, J., Medas, P., Castillo, G., Rodrigues, P.: Learning with drift detection. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS, vol. 3171, pp. 286–295. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Harries, M.: Splice-2 comparative evaluation: Electricity pricing (1999)Google Scholar
  12. 12.
    Bifet, A., Gavaldà, R.: Learning from time-changing data with adaptive windowing. In: Proceedings of the Seventh SIAM International Conference on Data Mining, Minneapolis, Minnesota, USA, pp. 443–448 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Ludmila I. Kuncheva
    • 1
  • Catrin O. Plumpton
    • 1
  1. 1.School of Computer ScienceBangor UniversityBangor GwyneddUK

Personalised recommendations