Abstract
This paper addresses the task of learning concept descriptions from streams of data. As new data are obtained the concept description has to be updated regularly to include the new data. In this case we can face the problem that the concept changes over time. Hence the old data become irrelevant to the current concept and have to be removed from the training dataset. This problem is known in the area of machine learning as concept drift. We develop a mechanism that tracks changing concepts using an adaptive time window. The method uses a significance test to detect concept drift and then optimizes the size of the time window, aiming to maximise the classification accuracy on recent data. The method presented is general in nature and can be used with any learning algorithm. The method is tested with three standard learning algorithms (kNN, ID3 and NBC). Three datasets have been used in these experiments. The experimental results provide evidence that the suggested forgetting mechanism is able significantly to improve predictive accuracy on changing concepts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aha, D., Kibler, D. and Albert, M.: Instance-Based Learning Algorithms. Machine Learning 6, (1991)37–66
Burghes, D. and Graham, A.: Introduction to Control Theory including Optimal Control: Ellis Horwood Series Mathematics and its Applications. John Wiley & Sons (1980)
Delany, SJ., Cunningham. P., Tsymbal, A. and Coyle, L.: A Case-Based Technique for Tracking Concept Drift in Spam Filtering. In: Macintosh, A., Ellis, R. & Allen T. (eds.) Applications and Innovations in Intelligent Systems XII, Proceedings of AI2004, Lecture Notes in Computer Science, Springer (2004) 3–16
Gama, J., Medas, P., Castillo, G. and Rodrigues, P.: Learning with Drift Detection. In: Ana, C, Bazzan, S. and Labidi (Eds.): Proceedings of the 17th Brazilian Symposium on Artificial Intelligence. Lecture Notes in Computer Science, Vol. 3171, Springer, (2004) 286–295
Harries, M. and Sammut, C.: Extracting Hidden Context. Machine Learning 32 (1998) 101–126
Koychev, I.: Gradual Forgetting for Adaptation to Concept Drift. Proceedings of ECAI 2000 Workshop on Current Issues in Spatio-Temporal Reasoning, Berlin, (2000) 101–107
Koychev, I.: Tracking Changing User Interests through Prior-Learning of Context. In: de Bra, P., Brusilovsky, P., Conejo, R. (eds.): Adaptive Hypermedia and Adaptive Web Based Systems. Lecture Notes in Computer Science, Vol. 2347, Springer-Verlag (2002) 223–232
Klinkenberg, R.: Learning Drifting Concepts: Example Selection vs. Example Weighting. In Intelligent Data Analysis, Special Issue on Incremental Learning Systems Capable of Dealing with Concept Drift, Vol. 8, No. 3, (2004) 281–300
Kukar, M.: Drifting Concepts as Hidden Factors in Clinical Studies. In Dojat, D., Elpida T. Keravnou, Pedro Barahona (Eds.): Proceedings of 9th Conference on Artificial Intelligence in Medicine in Europe, AIME 2003, Protaras, Cyprus, October 18–22, 2003, Lecture Notes in Computer Science, Vol. 2780, Springer-Verlag (2003) 355–364
Chu, F.and Zaniolo, C: Fast and light boosting for adaptive mining of data streams. In: Proc. of the 8th Pacific-Asia Conference on Knowledge Discovery and Data Mining. Lec ture Notes in Computer Science, Vol.3056, Springer-Verlag, (2004) 282–292
Lazarescu, M., Venkatesh, S. and Bui H. H.: Using Multiple Windows to Track Concept Drift. In the Intelligent Data Analysis Journal, Vol 8(1), (2004) 29–59
Maloof, M. and Michalski, R.: Selecting examples for partial memory learning. Machine Learning 41 (2000) 27–52
Mitchell, T., Caruana, R., Freitag, D., McDermott, J. and Zabowski, D.: Experience with a Learning Personal Assistant. Communications of the ACM 37(7) (1994) 81–91
Mitchell T. Machine Learning. McGraw-Hill (1997)
Quintan, R.: Induction of Decision Trees. Machine Learning 1 (1986) 81–106
Schlimmer, J. and Granger, R.: Incremental Learning from Noisy Data. Machine Learning 3, (1986), 317–357
Widmer, G.: Tracking Changes through Meta-Leaming. Machine Learning 27 (1997) 256–286
Widmer, G. and Kubat, M.: Learning in the presence of concept drift and hidden contexts: Machine Learning 23 (1996) 69–101
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag London Limited
About this paper
Cite this paper
Koychev, I., Lothian, R. (2006). Tracking Drifting Concepts by Time Window Optimisation. In: Bramer, M., Coenen, F., Allen, T. (eds) Research and Development in Intelligent Systems XXII. SGAI 2005. Springer, London. https://doi.org/10.1007/978-1-84628-226-3_5
Download citation
DOI: https://doi.org/10.1007/978-1-84628-226-3_5
Publisher Name: Springer, London
Print ISBN: 978-1-84628-225-6
Online ISBN: 978-1-84628-226-3
eBook Packages: Computer ScienceComputer Science (R0)