Drift Detector for Memory-Constrained Environments
- 1.5k Downloads
Current approaches to drift detection assume that stable memory consumption with slight variations with each stream is suitable for all programs. This is not always the case and there are situations where small variations in memory are undesirable such as drift detectors on medical vital sign monitoring systems. Under these circumstances, it is not sufficient to have a memory use that is predictable on average, but instead memory use must be fixed. To detect drift using fixed memory in a stream, we propose DualWin: a technique that keeps two samples of controllable size, one is stored in a sliding window, which represents the most recent stream elements, and the other is stored in a reservoir, which uses reservoir sampling to maintain an image of the stream since the previous drift was detected. Through experimentation, we find that DualWin obtains a rate of true positive detection which is comparable to ADWIN2, a rate of false positive detection which is much lower, an execution time which is faster, and a fixed memory consumption.
KeywordsData Streams Drift Detection Fixed Memory Reservoir Sampling
Unable to display preview. Download preview PDF.
- 1.Aggarwal, C.C., Han, J., Wang, J., Yu, P.S.: A framework for clustering evolving data streams. In: Proceedings of the 29th International Conference on Very Large Data Bases, VLDB 2003, vol. 29, pp. 81–92. VLDB Endowment (2003)Google Scholar
- 2.Bifet, A., Gavaldà, R.: Learning from time-changing data with adaptive windowing. In: Proceedings of the Seventh SIAM International Conference on Data Mining. SIAM (2007)Google Scholar
- 4.Ho, S.S.: A martingale framework for concept change detection in time-varying data streams. In: Proceedings of the 22nd International Conference on Machine Learning, ICML 2005, pp. 321–327. ACM, New York (2005)Google Scholar
- 5.Jose, M.B., Campo-Ávila, J.D., Fidalgo, R., Bifet, A., Gavaldà, R., Morales-bueno, R.: Early Drift Detection Method. In: Proceedings of the 4th ECML PKDD Int. Workshop on Knowledge Discovery from Data Streams, Berlin, pp. 77–86 (2006)Google Scholar
- 6.Kifer, D., Ben-David, S., Gehrke, J.: Detecting change in data streams. In: Proceedings of the Thirtieth International Conference on Very Large Data Bases, VLDB 2004, vol. 30, pp. 180–191. VLDB Endowment (2004)Google Scholar
- 7.Klinkenberg, R., Joachims, T.: Detecting concept drift with support vector machines. In: Proceedings of the Seventeenth International Conference on Machine Learning, ICML 2000, pp. 487–494. Morgan Kaufmann Publishers Inc., San Francisco (2000)Google Scholar
- 9.Sebastiao, R., Gama, J.: A study on change detection methods. In: Proceedings of the 14th Portuguese Conference on Artificial Intelligence, EPIA 2009. pp. 353–364. Springer, Heidelberg (2009)Google Scholar
- 10.Vitter, J.: Random sampling with a reservoir. ACM Transactions on Mathematical Software, 37–57 (1985)Google Scholar