New Drift Detection Method for Data Streams

  • Parinaz Sobhani
  • Hamid Beigy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6943)

Abstract

Correctly detecting the position where a concept begins to drift is important in mining data streams. In this paper, we propose a new method for detecting concept drift. The proposed method, which can detect different types of drift, is based on processing data chunk by chunk and measuring differences between two consecutive batches, as drift indicator. In order to evaluate the proposed method we measure its performance on a set of artificial datasets with different levels of severity and speed of drift. The experimental results show that the proposed method is capable to detect drifts and can approximately find concept drift locations.

Keywords

Concept drift stream mining drift detection evolving data 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Markou, M., Singh, S.: Novelty detection: A review, Part I: Statistical approaches. Signal Processing 83(12), 2481–2521 (2003)CrossRefMATHGoogle Scholar
  2. 2.
    Kuncheva, L.I.: Classifier ensembles for detecting concept change in streaming data: Overview and perspectives. In: Proc. of the Second Workshop SUEMA, Patras, Greece, pp. 5–9 (2008)Google Scholar
  3. 3.
    Gama, J., Medas, P., Castillo, G., Rodrigues, P.: Learning with drift detection. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 286–295. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Baena-Garcia, M., Del Campo-Avila, J., Fidalgo, R., Bifet, A.: Early drift detection method. In: Proc. of the 4th ECML PKDD International Workshop on Knowledge Discovery from Data Streams, Berlin, Germany, pp. 77–86 (2006)Google Scholar
  5. 5.
    Wilson, D.R., Martinez, T.R.: Improved heterogeneous distance functions. Journal of Artificial Intelligence Research 6(1), 1–34 (1997)MathSciNetMATHGoogle Scholar
  6. 6.
    Nishida, K., Yamauchi, K.: Detecting concept drift using statistical testing. In: Corruble, V., Takeda, M., Suzuki, E. (eds.) DS 2007. LNCS (LNAI), vol. 4755, pp. 264–269. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  7. 7.
    Minku, F.L., White, A., Yao, X.: The impact of diversity on on-line ensemble learning in the presence of concept drift. In: IEEE TKDE, vol. 22, pp. 730–742 (2010)Google Scholar
  8. 8.
    Salganicoff, M.: Density-adaptive learning and forgetting. In: Proc. of the 10th Int. Conf. on Machine Learning, pp. 276–283 (1993)Google Scholar
  9. 9.
    Klinkenberg, R., Joachims, T.: Detecting concept drift with support vector machines. In: Proc. of the 17th Int. Conf. on Machine Learning, pp. 487–494 (2000)Google Scholar
  10. 10.
    Dries, A., Ruckert, U.: Adaptive concept drift detection. In: SDM, pp. 233–244. SIAM, Philadelphia (2009)Google Scholar
  11. 11.
    Gama, J.: Knowledge discovery from data streams. Data Mining and Knowledge Discovery Series. USA (2010)Google Scholar
  12. 12.
    Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)MATHGoogle Scholar
  13. 13.
    Chu, F., Zaniolo, C.: Fast and light boosting for adaptive mining of data streams. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 282–292. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  14. 14.
    Friedman, J.H., Rafsky, L.C.: Multivariate generalizations of the Wald-Wolfowitz and Smirnov two-sample tests. Annals of Statistics 7(4), 697–717 (1979)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Parinaz Sobhani
    • 1
  • Hamid Beigy
    • 1
  1. 1.Department of Computer EngineeringSharif University of TechnologyTehranIran

Personalised recommendations