Statistics and Computing

, Volume 25, Issue 5, pp 913–928 | Cite as

Robust online-surveillance of trend-coherence in multivariate data streams: the similar trend monitoring (STM) procedure

Article

Abstract

When several data streams are observed simultaneously, it is often of great interest to monitor the coherences between all pairs of streams. We propose a new technique called Similar Trend Monitoring (STM) for this task: The current slopes of all univariate streams are estimated and compared pairwise at each time point. The STM statistic is the standardized slope difference, so that decisions about coherence can be made by means of the six-sigma-rule, for instance. The STM meets the high demands that come along with the online monitoring of multivariate data streams: it is fast to compute, robust against outliers, applicable when observations are missing, and does not require stationarity of the processes. We investigate the distribution and the performance of the STM and demonstrate its capabilities considering blood pressure time series from intensive care patient monitoring as an example.

Keywords

Multivariate data streams Coherence Trend estimation Online Robustness 

References

  1. Aue, A., Hörmann, S., Horváth, L., Reimherr, M.: Break detection in the covariance structure of multivariate time series models. Ann. Stat. 37(6B), 4046–4087 (2009)CrossRefMATHGoogle Scholar
  2. Beringer, J., Hüllermeier, E.: Online clustering of parallel data streams. Data Knowl. Eng. 58, 180–204 (2006)CrossRefGoogle Scholar
  3. Bernholt, T., Fried, R.: Computing the update of the repeated median regression line in linear time. Inf. Process. Lett. 88(3), 111–117 (2003)CrossRefMathSciNetMATHGoogle Scholar
  4. Bodnar, O., Schmid, W.: Surveillance of the mean behavior of multivariate time series. Stat. Neerl. 61(4), 383–406 (2007)CrossRefMathSciNetMATHGoogle Scholar
  5. Borowski, M., Fried, R.: Online signal extraction by robust regression in moving windows with data-adaptive width selection. Stat. Comput. (2013)Google Scholar
  6. Borowski, M., Schettlinger, K., Gather, U.: Multivariate real time signal processing by a robust adaptive regression filter. Commun. Stat. Simul. 38(2), 426–440 (2009)CrossRefMathSciNetMATHGoogle Scholar
  7. Bulut, A., Singh, A.: A unified framework for monitoring data streams in real time. In: Proceedings of the 21st International Conference on Data Engineering (ICDE 2005) (2005)Google Scholar
  8. Busse, D.: Robuste Echtzeit-Überwachung der Abhängigkeiten multivariater nichtstationärer Zeitreihen. Diploma thesis, Faculty of Statistics, TU Dortmund University (in German) (2012)Google Scholar
  9. Chan, L., Zhang, J.: Cumulative sum control charts for the covariance matrix. Stat. Sin. 11, 767–790 (2001)MathSciNetMATHGoogle Scholar
  10. Cole, R., Shasha, D., Zhao, X.: Fast window correlations over uncooperative time series. In: Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pp. 743–749. KDD ’05 (2005)Google Scholar
  11. Dai, B., Huang, J., Yeh, M., Chen, M.: Adaptive clustering for multiple evolving streams. IEEE Trans. Knowl. Data Eng. 18(9), 1166–1180 (2006) Google Scholar
  12. Davies, P., Gather, U.: Breakdown and groups. Ann. Stat. 33(3), 977–1035 (2005)CrossRefMathSciNetMATHGoogle Scholar
  13. Davies, P., Fried, R., Gather, U.: Robust signal extraction for on-line monitoring data. J. Stat. Plan. Inference 122, 65–78 (2004)CrossRefMathSciNetMATHGoogle Scholar
  14. Fried, R.: Robust filtering of time series with trends. J. Nonparametr. Stat. 16(3–4), 313–328 (2004)CrossRefMathSciNetMATHGoogle Scholar
  15. Fried, R., Schettlinger, K., Borowski, M.: robfilter: Robust Time Series Filters (2012), http://CRAN.R-project.org/package=robfilter, r package version 4.0
  16. Gather, U., Schettlinger, K., Fried, R.: Online signal extraction by robust linear regression. Comput. Stat. 21(1), 33–51 (2006)CrossRefMathSciNetMATHGoogle Scholar
  17. Gelper, S., Schettlinger, K., Croux, C., Gather, U.: Robust online scale estimation in time series: A model-free approach. J. Stat. Plan. Inference 139, 335–349 (2009)CrossRefMathSciNetMATHGoogle Scholar
  18. Gnanadesikan, R., Kettenring, J.: Robust estimates, residuals, and outlier detection with multiresponse data. Biometrics 28, 81–124 (1972)CrossRefGoogle Scholar
  19. Idé, T., Papadimitriou, S., Vlachos, M.: Computing correlation anomaly scores using stochastic nearest neighbors. In: Proceedings of the 7th IEEE International Conference on Data Mining 2007. pp. 523–528 (2007)Google Scholar
  20. Jiang, T., Feng, Y., Zhang, B., Cao, Z., Fu, G., Shi, J.: Monitoring correlative financial data streams by local pattern similarity. J. Zhejiang Univ. Sci. A 10(7), 937–951 (2009)CrossRefMATHGoogle Scholar
  21. Lanius, V., Gather, U.: Robust online signal extraction from multivariate time series. Comput. Stat. Data Anal. 54, 966–975 (2010)CrossRefMathSciNetMATHGoogle Scholar
  22. Liu, X., Ferhatosmanoglu, H.: Efficient k-NN search on streaming data series. In: Hadzilacos, T., Manolopoulos, Y., Roddick, J., Theodoridis, Y. (eds.) Advances in Spatial and Temporal Databases, pp. 83–101. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg (2003)Google Scholar
  23. Maronna, A., Zamar, R.: Robust estimates of location and dispersion for high-dimensional data sets. Technometrics 44(4), 307–317 (2002)CrossRefMathSciNetGoogle Scholar
  24. Papadimitriou, S., Sun, J., Yu, P.: Local correlation tracking in time series. In: Proceedings of the Sixth International Conference on Data Mining (ICDM’06), pp. 456–465 (2006)Google Scholar
  25. Rodrigues, P., Gama, J., Pedroso, J.: Hierarchical clustering of time-series data streams. IEEE T. Knowl. Data Eng. 20(5), 615–627 (2008)CrossRefGoogle Scholar
  26. Rousseeuw, P., Croux, C.: Alternatives to the median absolute deviation. J. Am. Stat. Assoc. 88(424), 1273–1283 (1993)CrossRefMathSciNetMATHGoogle Scholar
  27. Rousseeuw, P., Hubert, M.: Regression-free and robust estimation of scale for bivariate data. Comput. Stat. Data Anal. 21, 67–85 (1996)CrossRefMathSciNetMATHGoogle Scholar
  28. Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley, New York (1987)CrossRefMATHGoogle Scholar
  29. Siegel, A.: Robust regression using repeated medians. Biometrika 69(1), 242–244 (1982)CrossRefMATHGoogle Scholar
  30. Śliwa, P., Schmid, W.: Monitoring the cross-covariances of a multivariate time series. Metrika 61, 89–115 (2005a)CrossRefMathSciNetMATHGoogle Scholar
  31. Śliwa, P., Schmid, W.: Surveillance of the covariance matrix of multivariate nonlinear time series. Statistics 39(3), 221–246 (2005b)CrossRefMathSciNetMATHGoogle Scholar
  32. Wied, D., Galeano, P.: Monitoring correlation change in a sequence of random variables. J. Stat. Plan. Inference 143(1), 186–196 (2013)CrossRefMathSciNetMATHGoogle Scholar
  33. Yang, J.: Dynamic clustering of evolving streams with a single pass. In: Proceedings of the 19th International Conference on Data, Engineering 2003, pp. 695–697 (2003)Google Scholar
  34. Yeh, M., Dai, B., Chen, M.: Clustering over multiple evolving streams by events and correlations. IEEE T. Knowl. Data Eng. 19(10), 1349–1362 (2007)CrossRefGoogle Scholar
  35. Zhu, Y., Shasha, D.: Statstream: Statistical monitoring of thousands of data streams in real time. In: Proceedings of the 28th international conference on Very Large Data Bases, pp. 358–369 (2002)Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Institute of Biostatistics and Clinical ResearchUniversity of MünsterMünsterGermany
  2. 2.Chrestos Concept GmbH & Co. KGRatingenGermany
  3. 3.Faculty of StatisticsTU Dortmund UniversityDortmundGermany

Personalised recommendations