Advertisement

Metrika

, Volume 81, Issue 6, pp 589–607 | Cite as

The multiple filter test for change point detection in time series

  • Michael Messer
  • Stefan Albert
  • Gaby Schneider
Article

Abstract

A framework for the detection of change points in the expectation in sequences of random variables is presented. Specifically, we investigate time series with general distributional assumptions that may show an unknown number of change points in the expectation occurring on multiple time scales and that may also contain change points in other parameters. To that end we propose a multiple filter test (MFT) that tests the null hypothesis of constant expectation and, in case of rejection of the null hypothesis, an algorithm that estimates the change points.

The MFT has three important benefits. First, it allows for general distributional assumptions in the underlying model, assuming piecewise sequences of i.i.d. random variables, where also relaxations with regard to identical distribution or independence are possible. Second, it uses a MOSUM type statistic and an asymptotic setting in which the MOSUM process converges weakly to a functional of a Brownian motion which is then used to simulate the rejection threshold of the statistical test. This approach enables a simultaneous application of multiple MOSUM processes which improves the detection of change points that occur on different time scales. Third, we also show that the method is practically robust against changes in other distributional parameters such as the variance or higher order moments which might occur with or even without a change in expectation. A function implementing the described test and change point estimation is available in the R package MFT.

Keywords

Change point Multiscale MOSUM MFT 

Notes

Acknowledgements

This work was supported by the German Federal Ministry of Education and Research (www.bmbf.de) within the framework of the e:Med research and funding concept (grant number 01ZX1404B to SA, MM and GS) and by the Priority Program 1665 of the Deutsche Forschungsgemeinschaft (grant number SCHN 1370/02-1 to MM and GS, www.dfg.de). We thank Solveig Plomer for helpful comments to the manuscript.

Compliance with ethical standards

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

References

  1. Albert S, Messer M, Schiemann J, Roeper J, Schneider G (2017) Multi-scale detection of variance changes in renewal processes in the presence of rate change points. J Time Ser Anal 38(6):1028–1052MathSciNetCrossRefzbMATHGoogle Scholar
  2. Arlot S, Celisse A (2011) Segmentation of the mean of heteroscedastic data via cross-validation. Stat Comput 21(4):613–632MathSciNetCrossRefzbMATHGoogle Scholar
  3. Aue A, Horváth L (2013) Structural breaks in time series. J Time Ser Anal 34(1):1–16MathSciNetCrossRefzbMATHGoogle Scholar
  4. Basseville M, Nikiforov I (1993) Detection of abrupt changes: theory and application. Prentice Hall Information and System Sciences Series. Prentice Hall Inc., Englewood CliffsGoogle Scholar
  5. Bertrand PR (2000) A local method for estimating change points: the hat-function. Statistics 34(3):215–235MathSciNetCrossRefzbMATHGoogle Scholar
  6. Bertrand PR, Fhima M, Guillin A (2011) Off-line detection of multiple change points by the filtered derivative with p-value method. Seq Anal 30(2):172–207MathSciNetCrossRefzbMATHGoogle Scholar
  7. Billingsley P (1999) Convergence of probability measures. Wiley Series in Probability and Statistics: Probability and Statistics, 2nd edn. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  8. Brodsky B (2017) Change-point analysis in nonstationary stochastic models. CRC Press, Boca RatonzbMATHGoogle Scholar
  9. Brodsky BE, Darkhovsky BS (1993) Nonparametric methods in change-point problems, Mathematics and its Applications, vol 243. Kluwer Academic Publishers, DordrechtCrossRefzbMATHGoogle Scholar
  10. Eichinger B, Kirch C (2018) A MOSUM procedure for the estimation of multiple random change points. Bernoulli 24(1):526–564MathSciNetCrossRefzbMATHGoogle Scholar
  11. Frick K, Munk A, Sieling H (2014) Multiscale change point inference. J R Stat Soc Ser B Stat Methodol 76(3):495–580 (with 32 discussions by 47 authors and a rejoinder by the authors)MathSciNetCrossRefGoogle Scholar
  12. Fryzlewicz P (2014) Wild binary segmentation for multiple change-point-detection. Ann Stat 42(6):2243–2281MathSciNetCrossRefzbMATHGoogle Scholar
  13. Harchaoui Z, Lévy-Leduc C (2008) Catching change-points with lasso. Adv Neural Inf Process Syst 20:617–624Google Scholar
  14. Horváth L, Horvath Z, Huskova M (2008) Ratio test for change point detection. In: Beyond parametrics in interdisciplinary research, vol 1. IMS, Collections, pp 293–304Google Scholar
  15. Hušková M, Slabý A (2001) Permutation tests for multiple changes. Kybernetika (Prague) 37:605–622MathSciNetzbMATHGoogle Scholar
  16. Jandhyala V, Fotopoulos S, MacNeill I, Liu P (2013) Inference for single and multiple change-points in time series. J Time Ser Anal 34(4):423–446MathSciNetCrossRefzbMATHGoogle Scholar
  17. Killick R, Eckley I, Ewans K, Jonathan P (2010) Detection of changes in variance of oceanographic time-series using changepoint analysis. Ocean Eng 37(13):1120–1126CrossRefGoogle Scholar
  18. Matteson DS, James NA (2014) A nonparametric approach for multiple change point analysis of multivariate data. J Am Stat Assoc 109(505):334–345MathSciNetCrossRefzbMATHGoogle Scholar
  19. Messer M, Schneider G (2017) The shark fin function—asymptotic behavior of the filtered derivative for point processes in case of change points. Stat Inference Stoch Process 20(2):253–272.  https://doi.org/10.1007/s11203-016-9138-0 MathSciNetCrossRefzbMATHGoogle Scholar
  20. Messer M, Kirchner M, Schiemann J, Roeper J, Neininger R, Schneider G (2014) A multiple filter test for change point detection in renewal processes with varying variance. Ann Appl Stat 8(4):2027–2067MathSciNetCrossRefzbMATHGoogle Scholar
  21. Messer M, Albert S, Plomer S, Schneider G (2017a) MFT: the multiple filter test for change point detection. https://cran.r-project.org/package=MFT, R package version 1.3 available via https://cran.r-project.org/package=MFT
  22. Messer M, Costa K, Roeper J, Schneider G (2017b) Multi-scale detection of rate changes in spike trains with weak dependencies. J Comput Neurosci 42(2):187–201MathSciNetCrossRefzbMATHGoogle Scholar
  23. Muggeo V, Adelfio G (2011) Efficient change point detection for genomic sequences of continuous measurements. Bioinformatics 27(2):161–166CrossRefGoogle Scholar
  24. Pein F, Sieling H, Munk A (2017) Heterogeneous change point inference. J R Stat Soc Ser B Stat Methodol 79(4):1207–1227MathSciNetCrossRefzbMATHGoogle Scholar
  25. Staude B, Grün S, Rotter S (2010) Higher-order correlations in non-stationary parallel spike trains: statistical modeling and inference. Front Comput Neurosci 4:16CrossRefGoogle Scholar
  26. Zhang H, Dantu R, Cangussu J (2009) Change point detection based on call detail records. In: Intelligence and Security Informatics. Institute of Electrical and Electronics Engineers, New York, pp 55–60Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute for MathematicsGoethe UniversityFrankfurt (Main)Germany

Personalised recommendations