Advertisement

EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments

  • Haider Raza
  • Girijesh Prasad
  • Yuhua Li
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 412)

Abstract

Dataset shift is a major challenge in the non-stationary environments wherein the input data distribution may change over time. In a time-series data, detecting the dataset shift point, where the distribution changes its properties is of utmost interest. Dataset shift exists in a broad range of real-world systems. In such systems, there is a need for continuous monitoring of the process behavior and tracking the state of the shift so as to decide about initiating adaptive corrections in a timely manner. This paper presents a novel method to detect the shift-point based on a two-stage structure involving Exponentially Weighted Moving Average (EWMA) chart and Kolmogorov-Smirnov test, which substantially reduces type-I error rate. The algorithm is suitable to be run in real-time. Its performance is evaluated through experiments using synthetic and real-world datasets. Results show effectiveness of the proposed approach in terms of decreased type-I error and tolerable increase in detection time delay.

Keywords

Non-stationary Dataset shift EWMA Online Shift-detection 

References

  1. 1.
    Liu, S., Yamada, M., Collier, N., Sugiyama, M.: Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation. Neural Networks, 1–25 (2013)Google Scholar
  2. 2.
    Shimodaira, H.: Improving Predictive Inference Under Covariate Shift by Weighting the Log-Likelihood Function. Journal of Statistical Planning and Inference 90(2), 227–244 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Alippi, C., Boracchi, G., Roveri, M.: Change Detection Tests Using the ICI Rule. In: The International Joint Conference on Neural Networks (IJCNN), pp. 1–7 (July 2010)Google Scholar
  4. 4.
    Alippi, C., Roveri, M.: Just-in-Time Adaptive Classifier–Part I: Detecting Nonstationary Changes. IEEE Transactions on Neural Networks 19(7), 1145–1153 (2008)CrossRefGoogle Scholar
  5. 5.
    Basseville, M., Nikiforov, I.: Detection of Abrupt Changes: Theory and Application. Prentice-Hall (1993)Google Scholar
  6. 6.
    Alippi, C., Boracchi, G., Roveri, M.: A Just-In-Time Adaptive Classification System Based on the Intersection of Confidence Intervals Rule. Neural Networks: The Official Journal of the International Neural Network Society 24(8), 791–800 (2011)CrossRefGoogle Scholar
  7. 7.
    Sugiyama, M., Suzuki, T., Kanamori, T.: Density Ratio Estimation in Machine Learning, p. 344. Cambridge University Press (2012)Google Scholar
  8. 8.
    Raza, H., Prasad, G., Li, Y.: Dataset Shift Detection in Non-Stationary Environments using EWMA Charts. In: IEEE International Conference on Systems, Man, and Cybernetics (accepted, 2013)Google Scholar
  9. 9.
    Moreno-Torres, J.G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N.V., Herrera, F.: A Unifying View on Dataset Shift in Classification. Pattern Recognition 45(1), 521–530 (2012)CrossRefGoogle Scholar
  10. 10.
    Sugiyama, M., Schwaighofer, A., Lawrence, N.D.: Dataset Shift in Machine Learning. MIT Press (2009)Google Scholar
  11. 11.
    Roberts, S.W.: Control chart tests Based on Geometric Moving Averages. Technometrics (1959)Google Scholar
  12. 12.
    Dougla, C.M.: Introduction to Statistical Quality Control, 5th edn. John Wiley & Sons (2007)Google Scholar
  13. 13.
    Ye, N., Vilbert, S., Chen, Q.: Computer Intrusion Detection Through EWMA for Autocorrelated and Uncorrelated Data. IEEE Transaction on Reliability 52(1), 75–82 (2003)CrossRefGoogle Scholar
  14. 14.
    Connie, M., Douglas, C., George, C.: Robustness of the EWMA Control Chart to Non-Normality. Journal of Quality Technology 31(3), 309 (1999)Google Scholar
  15. 15.
    Snedecor, G.W., Cochran, W.G.: Statistical Methods, Eight. Iowa State University Press (1989)Google Scholar
  16. 16.
    Table of Critical Values for the Two-Sample Test, http://www.soest.hawaii.edu/wessel/courses/gg313/Critical_KS.pdf
  17. 17.
    Klaus-Robert Müller, B.B.: BCI Competition III: Data set IVb (2005), http://www.bbci.de/competition/iii/desc_IVb.html

Copyright information

© IFIP International Federation for Information Processing 2013

Authors and Affiliations

  • Haider Raza
    • 1
  • Girijesh Prasad
    • 1
  • Yuhua Li
    • 1
  1. 1.Intelligent Systems Research CenterUniversity of UlsterUK

Personalised recommendations