ACE: Adaptive Classifiers-Ensemble System for Concept-Drifting Environments
Most machine learning algorithms assume stationary environments, require a large number of training examples in advance, and begin the learning from scratch. In contrast, humans learn in changing environments with sequential training examples and leverage prior knowledge in new situations. To deal with real-world problems in changing environments, the ability to make human-like quick responses must be developed in machines.
Many researchers have presented learning systems that assume the presence of hidden context and concept drift. In particular, several systems have been proposed that use ensembles of classifiers on sequential chunks of training examples. These systems can respond to gradual changes in large-scale data streams but have problems responding to sudden changes and leveraging prior knowledge of recurring contexts. Moreover, these are not pure online learning systems.
We propose an online learning system that uses an ensemble of classifiers suited to recent training examples. We use experiments to show that this system can leverage prior knowledge of recurring contexts and is robust against various noise levels and types of drift.
Unable to display preview. Download preview PDF.
- 1.Aha, D.W., Kibler, D., Albert, M.K.: Instance-based learning algorithms. Machine Learning 6, 37–66 (1991)Google Scholar
- 2.Cunningham, P., Nowlan, N., Delany, S.J., Haahr, M.: A case-based approach to spam filtering that can track concept drift. In: ICCBR 2003 Workshop on Long-Lived CBR Systems, Trondheim, Norway (June 2003)Google Scholar
- 3.Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, Hoboken (2000)Google Scholar
- 4.Delany, S.J., Cunningham, P., Tsymbal, A., Coyle, L.: A case-based technique for tracking concept drift in spam filtering. Technical Report TCD-CS-2004-30, Department of Computer Science, Trinity College Dublin (August 2004)Google Scholar
- 6.Hogg, R.V., Tanis, E.A.: Probability and Statistical Inference, 5th edn. Prentice Hall, Englewood Cliffs (1997)Google Scholar
- 7.Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proc. of the 7th ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, pp. 97–106 (2001)Google Scholar
- 9.Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
- 10.Schlimmer, J., Granger, R.H.: Incremental learning from noisy data. Machine Learning 1(3), 317–354 (1986)Google Scholar
- 11.Street, W.N., Kim, Y.S.: A streaming ensemble algorithm (SEA) for large-scale classification. In: Proc. of the 7th ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, pp. 377–382 (2001)Google Scholar
- 12.Tsymbal, A.: The problem of concept drift: definitions and related work. Technical Report TCD-CS-2004-15, Department of Computer Science, Trinity College Dublin (April 2004)Google Scholar
- 13.Wang, H., Fan, W., Yu, P.S., Han, J.: Mining concept-drifting data streams using ensemble classifiers. In: Proc. of the 9th ACM SIGKDD Int. Conference on Knowledge Discovery and Data Mining, pp. 226–235 (2003)Google Scholar
- 14.Widmer, G., Kubat, M.: Effective learning in dynamic environments by explicit concept tracking. In: Proc. of the Sixth European Conference on Machine Learning, pp. 227–243. Springer, Heidelberg (1993)Google Scholar
- 15.Widmer, G., Kubat, M.: Learning in the presence of concept drift and hidden contexts. Machine Learning 23, 69–101 (1996)Google Scholar