ACE: Adaptive Classifiers-Ensemble System for Concept-Drifting Environments

  • Kyosuke Nishida
  • Koichiro Yamauchi
  • Takashi Omori
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3541)


Most machine learning algorithms assume stationary environments, require a large number of training examples in advance, and begin the learning from scratch. In contrast, humans learn in changing environments with sequential training examples and leverage prior knowledge in new situations. To deal with real-world problems in changing environments, the ability to make human-like quick responses must be developed in machines.

Many researchers have presented learning systems that assume the presence of hidden context and concept drift. In particular, several systems have been proposed that use ensembles of classifiers on sequential chunks of training examples. These systems can respond to gradual changes in large-scale data streams but have problems responding to sudden changes and leveraging prior knowledge of recurring contexts. Moreover, these are not pure online learning systems.

We propose an online learning system that uses an ensemble of classifiers suited to recent training examples. We use experiments to show that this system can leverage prior knowledge of recurring contexts and is robust against various noise levels and types of drift.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aha, D.W., Kibler, D., Albert, M.K.: Instance-based learning algorithms. Machine Learning 6, 37–66 (1991)Google Scholar
  2. 2.
    Cunningham, P., Nowlan, N., Delany, S.J., Haahr, M.: A case-based approach to spam filtering that can track concept drift. In: ICCBR 2003 Workshop on Long-Lived CBR Systems, Trondheim, Norway (June 2003)Google Scholar
  3. 3.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, Hoboken (2000)Google Scholar
  4. 4.
    Delany, S.J., Cunningham, P., Tsymbal, A., Coyle, L.: A case-based technique for tracking concept drift in spam filtering. Technical Report TCD-CS-2004-30, Department of Computer Science, Trinity College Dublin (August 2004)Google Scholar
  5. 5.
    Harries, M.B., Sammut, C., Horn, K.: Extracting hidden context. Machine Learning 32, 101–126 (1998)zbMATHCrossRefGoogle Scholar
  6. 6.
    Hogg, R.V., Tanis, E.A.: Probability and Statistical Inference, 5th edn. Prentice Hall, Englewood Cliffs (1997)Google Scholar
  7. 7.
    Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proc. of the 7th ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, pp. 97–106 (2001)Google Scholar
  8. 8.
    Kuncheva, L.I.: Classifier ensembles for changing environments. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 1–15. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  10. 10.
    Schlimmer, J., Granger, R.H.: Incremental learning from noisy data. Machine Learning 1(3), 317–354 (1986)Google Scholar
  11. 11.
    Street, W.N., Kim, Y.S.: A streaming ensemble algorithm (SEA) for large-scale classification. In: Proc. of the 7th ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, pp. 377–382 (2001)Google Scholar
  12. 12.
    Tsymbal, A.: The problem of concept drift: definitions and related work. Technical Report TCD-CS-2004-15, Department of Computer Science, Trinity College Dublin (April 2004)Google Scholar
  13. 13.
    Wang, H., Fan, W., Yu, P.S., Han, J.: Mining concept-drifting data streams using ensemble classifiers. In: Proc. of the 9th ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, pp. 226–235 (2003)Google Scholar
  14. 14.
    Widmer, G., Kubat, M.: Effective learning in dynamic environments by explicit concept tracking. In: Proc. of the Sixth European Conference on Machine Learning, pp. 227–243. Springer, Heidelberg (1993)Google Scholar
  15. 15.
    Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23, 69–101 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Kyosuke Nishida
    • 1
  • Koichiro Yamauchi
    • 1
  • Takashi Omori
    • 1
  1. 1.Graduate School of Information Science and TechnologyHokkaido UniversitySapporoJapan

Personalised recommendations