Abstract
Detection of changes in streaming data is an important mining task, with a wide range of real-life applications. Numerous algorithms have been proposed to efficiently detect changes in streaming data. However, the limitation of existing algorithms is that they assume that data are generated independently. In particular, temporal dependencies of data in a stream are still not thoroughly studied. Motivated by this, in this work we propose a new efficient method to detect changes in streaming data by exploring the temporal dependencies of data in the stream. As part of this, we introduce a new statistical model called the Candidate Change Point (CCP) model, with which the main idea is to compute the probabilities of finding change points in the stream. The computed probabilities are used to generate a distribution, which is, in turn, used in statistical hypothesis tests to determine the candidate changes. We use the CCP model to develop a new algorithm called Candidate Change Point Detector (CCPD), which detects change points in linear time, and is thus applicable for real-time applications. Our extensive experimental evaluation demonstrates the efficiency and the feasibility of our approach.
Similar content being viewed by others
Notes
Version 4.0.0, June 2017.
Source code of the FHDDM is provided by the authors of the algorithm
Source code of the RDDM is obtained from the authors personal website
References
Adä I, Berthold MR (2013) EVE: a framework for event detection. Evolving Systems 4(1):61–70
Adhikari U, Morris T, Pan S (2017) Applying Hoeffding adaptive trees for real-time cyber-power event and intrusion classification. IEEE Transactions on Smart Grid PP(99):1–12
Anagnostopoulos C, Tasoulis DK, Adams NM, Pavlidis NG, Hand DJ (2012) Online linear and quadratic discriminant analysis with adaptive forgetting for streaming classification. Statistical Analysis and Data Mining 5(2):139–166
Baena-García M, del Campo-Ȧvila J, Fidalgo R, Bifet A, Gavaldȧ R, Morales-Bueno R (2006) Early drift detection method. In: The 4th international workshop on knowledge discovery from data streams
Barros RS, Cabral DR, Gonçalves PM, Santos SG (2017) RDDM: Reactive drift detection method. Expert Syst Appl 90(Supplement C):344–355
Bifet A (2017) Classifier concept drift detection and the illusion of progress. In: Artificial intelligence and soft computing. Springer International Publishing, Cham, pp 715–725
Bifet A, Gavaldà R (2007) Learning from time-changing data with adaptive windowing. In: Proceedings of the 2007 SIAM international conference on data mining, pp 443–448
Bifet A, Gavaldà R (2009) Adaptive learning from evolving data streams. In: Proceedings of the 8th international symposium on intelligent data analysis, pp 249–260
Bifet A, Holmes G, Kirkby R, Pfahringer B (2010) MOA: Massive online analysis. J Mach Learn Res 11:1601–1604
Bifet A, Read J, žliobaitė I, Pfahringer B, Holmes G (2013) Pitfalls in benchmarking data stream classification and how to avoid them. In: Proceedings of the european conference on machine learning and knowledge discovery in databases, ECML PKDD, pp 465–479
Bodenham DA, Adams NM (2017) Continuous monitoring for changepoints in data streams using adaptive estimation. Stat Comput 27(5):1257–1270
Bryc W (2002) A uniform approximation to the right normal tail integral. Appl Math Comput 127(2):365–374
Chattopadhyay S, Murthy C, Pal SK (2014) Fitting truncated geometric distributions in large scale real world networks. Theor Comput Sci 551:22–38
Condat L (2016) Fast projection onto the simplex and the and the \(\ell _{1}\) ball. Math Program 158(1):575–585
Demšar J (2006) Statistical Comparisons of Classifiers over Multiple Data Sets. J Mach Learn Res 7:1–30
Duchi J, Shalev-Shwartz S, Singer Y, Chandra T (2008) Efficient projections onto the \(\ell _{1}\)-ball for learning in high dimensions. In: Proceedings of the 25th international conference on machine learning, ICML, pp 272–279
Frías-Blanco II, del Campo-Ávila J, Ramos-Jiménez G, Carvalho ACPLF, Díaz AAO, Morales-Bueno R (2016) Online adaptive decision trees based on concentration inequalities. Knowl-Based Syst 104:179–194
Frías-Blanco II, del Campo-Ávila J, Ramos-Jiménez G, Morales-Bueno R, Ortiz-Díaz AA, Caballero-Mota Y (2015) Online and non-parametric drift detection methods based on Hoeffding’s bounds. IEEE Trans Knowl Data Eng 27(3):810–823
Gama J, Medas P, Castillo G, Rodrigues PP (2004) Learning with drift detection. In: Proceedings of brazilian symposium on artificial intelligence, pp 286–295
Gama J, Sebastião R, Rodrigues PP (2013) On evaluating stream learning algorithms. Mach Learn 90 (3):317–346
Gama JA, žliobaitė I, Bifet A, Pechenizkiy M, Bouchachia A (2014) A survey on concept drift adaptation. ACM Comput Surv 46(4):1–37
Gomes HM, Barddal JP, Enembreck F, Bifet A (2017) A survey on ensemble learning for data stream classification. ACM Computing Surveys 50(2):23:1–23:36
Gomes HM, Bifet A, Read J, Barddal JP, Enembreck F, Pfharinger B, Holmes G, Abdessalem T (2017) Adaptive random forests for evolving data stream classification. Mach Learn 106(9):1469–1495
Harries MB, Sammut C, Horn K (1998) Extracting hidden context. Mach Learn 32(2):101–126
Hoeffding W (1963) Probability inequalities for sums of bounded random variables. J Am Stat Assoc 58 (301):13–30
Kifer D, Ben-David S, Gehrke J (2004) Detecting change in data streams
Kolter JZ, Maloof MA (2007) Dynamic weighted majority: an ensemble method for drifting concepts. J Mach Learn Res 8:2755–2790
Kumar R, Raghu M, Sarlós T, Tomkins A (2017) Linear additive markov processes. In: Proceedings of the 26th international conference on World Wide Web, pp 411–419
Li P, Wu X, Hu X (2010) Mining recurring concept drifts with limited labeled streaming data. In: Proceedings of the 2nd Asian conference on machine learning. PMLR, vol 13, pp 241–252
Liu J, Ye J (2009) Efficient euclidean projections in linear time. In: Proceedings of the 26th annual international conference on machine learning. ICML, pp 657–664
Markov A (1971) Extension of the Limit Theorems of Probability Theory to a Sum of Variables Connected in a Chain. In: Appendix B, dynamic probabilistic systems (Volume I: Markov models), pp 552–577
Page ES (1954) Continuous Inspection Schemes. Biometrika 41(1/2):100–115
Pears R, Sakthithasan S, Koh YS (2014) Detecting concept change in dynamic data streams. Mach Learn 97(3):259–293
Pesaranghader A, Viktor H, Paquet E (2017) McDiarmid drift detection methods for evolving data streams. CoRR arXiv:1710.02030
Pesaranghader A, Viktor H, Paquet E (2017) Reservoir of diverse adaptive learners and stacking fast Hoeffding drift detection methods for evolving data streams. CoRR arXiv:1709.02457
Pesaranghader A, Viktor HL (2016) Fast Hoeffding Drift Detection Method for Evolving Data Streams. In: Proceedings of the 2016 machine learning and knowledge discovery in databases. ECML PKDD, pp 96–111
Roberts SW (1959) Control chart tests based on geometric moving averages. Technometrics 1(3):239–250
Rösler O, Suendermann D (2013) A first step towards eye state prediction using EEG. In: Proceedings of the international conference on applied informatics for health and life sciences (AIHLS 2013)
Ross GJ, Adams NM, Tasoulis DK, Hand DJ (2012) Exponentially weighted moving average charts for detecting concept drift. Pattern Recogn Lett 33(2):191–198
Schlimmer JC, Granger RH (1986) Incremental learning from noisy data. Mach Learn 1(3):317–354
Sebastião R, Gama J, Mendonça T (2017) Fading histograms in detecting distribution and concept changes. International Journal of Data Science and Analytics 3(3):183–212
Tibshirani RJ, Taylor J, Lockhart R, Tibshirani R (2016) Exact Post-Selection inference for sequential regression procedures. J Am Stat Assoc 111(514):600–620
Weissman T, Ordentlich E, Seroussi G, Verdu S, Weinberger MJ (2003) Inequalities for the \(\ell _{1}\) Deviation of the Empirical Distribution. Technical report, Hewlett-Packard Labs
Wu T, Gleich DF (2017) Retrospective higher-order markov processes for user trails. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. KDD, pp 1185–1194
žliobaitė I, Bifet A, Read J, Pfahringer B, Holmes G (2015) Evaluation methods and decision theory for classification of streaming data with temporal dependence. Mach Learn 98(3):455–482
Acknowledgements
This research is funded by the Norwegian University of Science and Technology (NTNU) through the MUSED project.
Author information
Authors and Affiliations
Corresponding author
Appendix: Proof
Appendix: Proof
1.1 A.1 Proof of property 1
Proof
Equation 7 can be easily proved by induction. When \(t_2 = t_1\), we have:
When \(t_2 = t_1 + 1 \), we have:
Assume that Eq. 7 is satisfied when \(t_2 = t_1 + m\), with \(m \in \mathbb {N}, m > 0\). We prove that Eq. 7 is also satisfied with \(t_2 = t_1 + m + 1\). We have:
□
1.2 A.2 Proof of proposition 1
Proof
We have:
□
1.3 A.3 Proof of proposition 2
Proof
We have:
□
Rights and permissions
About this article
Cite this article
Duong, QH., Ramampiaro, H. & Nørvåg, K. Applying temporal dependence to detect changes in streaming data. Appl Intell 48, 4805–4823 (2018). https://doi.org/10.1007/s10489-018-1254-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-018-1254-7