Learning with Local Drift Detection

  • João Gama
  • Gladys Castillo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4093)


Most of the work in Machine Learning assume that examples are generated at random according to some stationary probability distribution. In this work we study the problem of learning when the distribution that generates the examples changes over time. We present a method for detection of changes in the probability distribution of examples. The idea behind the drift detection method is to monitor the online error-rate of a learning algorithm looking for significant deviations. The method can be used as a wrapper over any learning algorithm. In most problems, a change affects only some regions of the instance space, not the instance space as a whole. In decision models that fit different functions to regions of the instance space, like Decision Trees and Rule Learners, the method can be used to monitor the error in regions of the instance space, with advantages of fast model adaptation. In this work we present experiments using the method as a wrapper over a decision tree and a linear model, and in each internal-node of a decision tree. The experimental results obtained in controlled experiments using artificial data and a real-world problem show a good performance detecting drift and in adapting the decision model to the new concept.


Decision Tree Learning Algorithm Data Stream Decision Model Concept Change 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Basseville, M., Nikiforov, I.: Detection of Abrupt Changes: Theory and Applications. Prentice-Hall, Englewood Cliffs (1987)Google Scholar
  2. 2.
    Blake, C., Keogh, E., Merz, C.J.: UCI repository of Machine Learning databases (1999)Google Scholar
  3. 3.
    Dawid, A.P.: Statistical theory: The prequential approach. Journal of the Royal Statistical Society-A 147, 278–292 (1984)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Fan, W.: Systematic data selection to mine concept-drifting data streams. In: Gehrke, J., DuMouchel, W. (eds.) Proceedings of the Tenth International Conference on Knowledge Discovery and Data Mining, ACM Press, New York (2004)Google Scholar
  5. 5.
    Gama, J., Fernandes, R., Rocha, R.: Decision trees for mining data streams. Intelligent Data Analysis 10(1), 23–46 (2006)Google Scholar
  6. 6.
    Gama, J., Medas, P., Rodrigues, P.: Learning decision trees from dynamic data streams. In: Haddad, H., Liebrock, L.M., Omicini, A., Wainwright, R.L. (eds.) Proceedings of the 2005 ACM Symposium on Applied Computing, March 2005, pp. 573–577. ACM Press, New York (2005)CrossRefGoogle Scholar
  7. 7.
    Harries, M., Sammut, C., Horn, K.: Extracting hidden context. Machine Learning 32, 101 (1998)CrossRefMATHGoogle Scholar
  8. 8.
    Harries, M.: Splice-2 comparative evaluation: Electricity pricing. Technical report, The University of South Wales (1999)Google Scholar
  9. 9.
    Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 97–106. ACM Press, New York (2001)CrossRefGoogle Scholar
  10. 10.
    Kifer, D., Ben-David, S., Gehrke, J.: Detecting change in data streams. In: VLDB 2004: Proceedings of the 30th International Conference on Very Large Data Bases, Morgan Kaufmann Publishers Inc., San Francisco (2004)Google Scholar
  11. 11.
    Klinkenberg, R.: Learning drifting concepts: Example selection vs. example weighting. Intelligent Data Analysis (2004)Google Scholar
  12. 12.
    Klinkenberg, R., Joachims, T.: Detecting concept drift with support vector machines. In: Langley, P. (ed.) Proceedings of ICML 2000, 17th International Conference on Machine Learning, Stanford, US, pp. 487–494. Morgan Kaufmann, San Francisco (2000)Google Scholar
  13. 13.
    Klinkenberg, R., Renz, I.: Adaptive information filtering: Learning in the presence of concept drifts. In: Learning for Text Categorization, pp. 33–40. AAAI Press, Menlo Park (1998)Google Scholar
  14. 14.
    Kolter, J., Maloof, M.: Using additive expert ensembles to cope with concept drift. In: Raedt, L., Wrobel, S. (eds.) Machine Learning, Proceedings of the 22th International Conference, Omni Press (2005)Google Scholar
  15. 15.
    Kolter, J.Z., Maloof, M.A.: Dynamic weighted majority: A new ensemble method for tracking concept drift. In: Proceedings of the Third International IEEE Conference on Data Mining, pp. 123–130. IEEE Computer Society Press, Los Alamitos (2003)CrossRefGoogle Scholar
  16. 16.
    Koychev, I.: Gradual forgetting for adaptation to concept drift. In: Proceedings of ECAI 2000 Workshop Current Issues in Spatio-Temporal Reasoning, Berlin, Germany, pp. 101–106 (2000)Google Scholar
  17. 17.
    Koychev, I.: Learning about user in the presence of hidden context. In: Proceedings of Machine Learning for User Modeling: UM-2001 (2001)Google Scholar
  18. 18.
    Lanquillon, C.: Enhancing Text Classification to Improve Information Filtering. PhD thesis, University of Madgdeburg, Germany (2001)Google Scholar
  19. 19.
    Maloof, M., Michalski, R.: Selecting examples for partial memory learning. Machine Learning 41, 27–52 (2000)CrossRefGoogle Scholar
  20. 20.
    Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)MATHGoogle Scholar
  21. 21.
    Domingos, P., Hulten, G.: Mining High-Speed Data Streams. In: Proceedings of the Association for Computing Machinery Sixth International Conference on Knowledge Discovery and Data Mining, pp. 71–80. ACM Press, New York (2000)Google Scholar
  22. 22.
    Nick Street, W., Kim, Y.S.: A streaming ensemble algorithm SEA for large-scale classification. In: Proc. 7th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 377–382. ACM Press, New York (2001)Google Scholar
  23. 23.
    Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23, 69–101 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • João Gama
    • 1
    • 2
  • Gladys Castillo
    • 1
    • 3
  1. 1.LIACCUniversity of PortoPortoPortugal
  2. 2.Fac. EconomicsUniversity of PortoPortugal
  3. 3.University of AveiroPortugal

Personalised recommendations