Advertisement

Statistics and Computing

, Volume 25, Issue 5, pp 913–928 | Cite as

Robust online-surveillance of trend-coherence in multivariate data streams: the similar trend monitoring (STM) procedure

  • Matthias Borowski
  • Dennis Busse
  • Roland Fried
Article

Abstract

When several data streams are observed simultaneously, it is often of great interest to monitor the coherences between all pairs of streams. We propose a new technique called Similar Trend Monitoring (STM) for this task: The current slopes of all univariate streams are estimated and compared pairwise at each time point. The STM statistic is the standardized slope difference, so that decisions about coherence can be made by means of the six-sigma-rule, for instance. The STM meets the high demands that come along with the online monitoring of multivariate data streams: it is fast to compute, robust against outliers, applicable when observations are missing, and does not require stationarity of the processes. We investigate the distribution and the performance of the STM and demonstrate its capabilities considering blood pressure time series from intensive care patient monitoring as an example.

Keywords

Multivariate data streams Coherence Trend estimation Online Robustness 

References

  1. Aue, A., Hörmann, S., Horváth, L., Reimherr, M.: Break detection in the covariance structure of multivariate time series models. Ann. Stat. 37(6B), 4046–4087 (2009)CrossRefzbMATHGoogle Scholar
  2. Beringer, J., Hüllermeier, E.: Online clustering of parallel data streams. Data Knowl. Eng. 58, 180–204 (2006)CrossRefGoogle Scholar
  3. Bernholt, T., Fried, R.: Computing the update of the repeated median regression line in linear time. Inf. Process. Lett. 88(3), 111–117 (2003)CrossRefMathSciNetzbMATHGoogle Scholar
  4. Bodnar, O., Schmid, W.: Surveillance of the mean behavior of multivariate time series. Stat. Neerl. 61(4), 383–406 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  5. Borowski, M., Fried, R.: Online signal extraction by robust regression in moving windows with data-adaptive width selection. Stat. Comput. (2013)Google Scholar
  6. Borowski, M., Schettlinger, K., Gather, U.: Multivariate real time signal processing by a robust adaptive regression filter. Commun. Stat. Simul. 38(2), 426–440 (2009)CrossRefMathSciNetzbMATHGoogle Scholar
  7. Bulut, A., Singh, A.: A unified framework for monitoring data streams in real time. In: Proceedings of the 21st International Conference on Data Engineering (ICDE 2005) (2005)Google Scholar
  8. Busse, D.: Robuste Echtzeit-Überwachung der Abhängigkeiten multivariater nichtstationärer Zeitreihen. Diploma thesis, Faculty of Statistics, TU Dortmund University (in German) (2012)Google Scholar
  9. Chan, L., Zhang, J.: Cumulative sum control charts for the covariance matrix. Stat. Sin. 11, 767–790 (2001)MathSciNetzbMATHGoogle Scholar
  10. Cole, R., Shasha, D., Zhao, X.: Fast window correlations over uncooperative time series. In: Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pp. 743–749. KDD ’05 (2005)Google Scholar
  11. Dai, B., Huang, J., Yeh, M., Chen, M.: Adaptive clustering for multiple evolving streams. IEEE Trans. Knowl. Data Eng. 18(9), 1166–1180 (2006) Google Scholar
  12. Davies, P., Gather, U.: Breakdown and groups. Ann. Stat. 33(3), 977–1035 (2005)CrossRefMathSciNetzbMATHGoogle Scholar
  13. Davies, P., Fried, R., Gather, U.: Robust signal extraction for on-line monitoring data. J. Stat. Plan. Inference 122, 65–78 (2004)CrossRefMathSciNetzbMATHGoogle Scholar
  14. Fried, R.: Robust filtering of time series with trends. J. Nonparametr. Stat. 16(3–4), 313–328 (2004)CrossRefMathSciNetzbMATHGoogle Scholar
  15. Fried, R., Schettlinger, K., Borowski, M.: robfilter: Robust Time Series Filters (2012), http://CRAN.R-project.org/package=robfilter, r package version 4.0
  16. Gather, U., Schettlinger, K., Fried, R.: Online signal extraction by robust linear regression. Comput. Stat. 21(1), 33–51 (2006)CrossRefMathSciNetzbMATHGoogle Scholar
  17. Gelper, S., Schettlinger, K., Croux, C., Gather, U.: Robust online scale estimation in time series: A model-free approach. J. Stat. Plan. Inference 139, 335–349 (2009)CrossRefMathSciNetzbMATHGoogle Scholar
  18. Gnanadesikan, R., Kettenring, J.: Robust estimates, residuals, and outlier detection with multiresponse data. Biometrics 28, 81–124 (1972)CrossRefGoogle Scholar
  19. Idé, T., Papadimitriou, S., Vlachos, M.: Computing correlation anomaly scores using stochastic nearest neighbors. In: Proceedings of the 7th IEEE International Conference on Data Mining 2007. pp. 523–528 (2007)Google Scholar
  20. Jiang, T., Feng, Y., Zhang, B., Cao, Z., Fu, G., Shi, J.: Monitoring correlative financial data streams by local pattern similarity. J. Zhejiang Univ. Sci. A 10(7), 937–951 (2009)CrossRefzbMATHGoogle Scholar
  21. Lanius, V., Gather, U.: Robust online signal extraction from multivariate time series. Comput. Stat. Data Anal. 54, 966–975 (2010)CrossRefMathSciNetzbMATHGoogle Scholar
  22. Liu, X., Ferhatosmanoglu, H.: Efficient k-NN search on streaming data series. In: Hadzilacos, T., Manolopoulos, Y., Roddick, J., Theodoridis, Y. (eds.) Advances in Spatial and Temporal Databases, pp. 83–101. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg (2003)Google Scholar
  23. Maronna, A., Zamar, R.: Robust estimates of location and dispersion for high-dimensional data sets. Technometrics 44(4), 307–317 (2002)CrossRefMathSciNetGoogle Scholar
  24. Papadimitriou, S., Sun, J., Yu, P.: Local correlation tracking in time series. In: Proceedings of the Sixth International Conference on Data Mining (ICDM’06), pp. 456–465 (2006)Google Scholar
  25. Rodrigues, P., Gama, J., Pedroso, J.: Hierarchical clustering of time-series data streams. IEEE T. Knowl. Data Eng. 20(5), 615–627 (2008)CrossRefGoogle Scholar
  26. Rousseeuw, P., Croux, C.: Alternatives to the median absolute deviation. J. Am. Stat. Assoc. 88(424), 1273–1283 (1993)CrossRefMathSciNetzbMATHGoogle Scholar
  27. Rousseeuw, P., Hubert, M.: Regression-free and robust estimation of scale for bivariate data. Comput. Stat. Data Anal. 21, 67–85 (1996)CrossRefMathSciNetzbMATHGoogle Scholar
  28. Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley, New York (1987)CrossRefzbMATHGoogle Scholar
  29. Siegel, A.: Robust regression using repeated medians. Biometrika 69(1), 242–244 (1982)CrossRefzbMATHGoogle Scholar
  30. Śliwa, P., Schmid, W.: Monitoring the cross-covariances of a multivariate time series. Metrika 61, 89–115 (2005a)CrossRefMathSciNetzbMATHGoogle Scholar
  31. Śliwa, P., Schmid, W.: Surveillance of the covariance matrix of multivariate nonlinear time series. Statistics 39(3), 221–246 (2005b)CrossRefMathSciNetzbMATHGoogle Scholar
  32. Wied, D., Galeano, P.: Monitoring correlation change in a sequence of random variables. J. Stat. Plan. Inference 143(1), 186–196 (2013)CrossRefMathSciNetzbMATHGoogle Scholar
  33. Yang, J.: Dynamic clustering of evolving streams with a single pass. In: Proceedings of the 19th International Conference on Data, Engineering 2003, pp. 695–697 (2003)Google Scholar
  34. Yeh, M., Dai, B., Chen, M.: Clustering over multiple evolving streams by events and correlations. IEEE T. Knowl. Data Eng. 19(10), 1349–1362 (2007)CrossRefGoogle Scholar
  35. Zhu, Y., Shasha, D.: Statstream: Statistical monitoring of thousands of data streams in real time. In: Proceedings of the 28th international conference on Very Large Data Bases, pp. 358–369 (2002)Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Institute of Biostatistics and Clinical ResearchUniversity of MünsterMünsterGermany
  2. 2.Chrestos Concept GmbH & Co. KGRatingenGermany
  3. 3.Faculty of StatisticsTU Dortmund UniversityDortmundGermany

Personalised recommendations