The multiple filter test for change point detection in time series

Metrika volume 81, pages589–607(2018)Cite this article

Abstract

A framework for the detection of change points in the expectation in sequences of random variables is presented. Specifically, we investigate time series with general distributional assumptions that may show an unknown number of change points in the expectation occurring on multiple time scales and that may also contain change points in other parameters. To that end we propose a multiple filter test (MFT) that tests the null hypothesis of constant expectation and, in case of rejection of the null hypothesis, an algorithm that estimates the change points.

The MFT has three important benefits. First, it allows for general distributional assumptions in the underlying model, assuming piecewise sequences of i.i.d. random variables, where also relaxations with regard to identical distribution or independence are possible. Second, it uses a MOSUM type statistic and an asymptotic setting in which the MOSUM process converges weakly to a functional of a Brownian motion which is then used to simulate the rejection threshold of the statistical test. This approach enables a simultaneous application of multiple MOSUM processes which improves the detection of change points that occur on different time scales. Third, we also show that the method is practically robust against changes in other distributional parameters such as the variance or higher order moments which might occur with or even without a change in expectation. A function implementing the described test and change point estimation is available in the R package MFT.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Albert S, Messer M, Schiemann J, Roeper J, Schneider G (2017) Multi-scale detection of variance changes in renewal processes in the presence of rate change points. J Time Ser Anal 38(6):1028–1052

    MathSciNet  Article  MATH  Google Scholar 

  2. Arlot S, Celisse A (2011) Segmentation of the mean of heteroscedastic data via cross-validation. Stat Comput 21(4):613–632

    MathSciNet  Article  MATH  Google Scholar 

  3. Aue A, Horváth L (2013) Structural breaks in time series. J Time Ser Anal 34(1):1–16

    MathSciNet  Article  MATH  Google Scholar 

  4. Basseville M, Nikiforov I (1993) Detection of abrupt changes: theory and application. Prentice Hall Information and System Sciences Series. Prentice Hall Inc., Englewood Cliffs

    Google Scholar 

  5. Bertrand PR (2000) A local method for estimating change points: the hat-function. Statistics 34(3):215–235

    MathSciNet  Article  MATH  Google Scholar 

  6. Bertrand PR, Fhima M, Guillin A (2011) Off-line detection of multiple change points by the filtered derivative with p-value method. Seq Anal 30(2):172–207

    MathSciNet  Article  MATH  Google Scholar 

  7. Billingsley P (1999) Convergence of probability measures. Wiley Series in Probability and Statistics: Probability and Statistics, 2nd edn. Wiley, New York

    Google Scholar 

  8. Brodsky B (2017) Change-point analysis in nonstationary stochastic models. CRC Press, Boca Raton

    Google Scholar 

  9. Brodsky BE, Darkhovsky BS (1993) Nonparametric methods in change-point problems, Mathematics and its Applications, vol 243. Kluwer Academic Publishers, Dordrecht

    Google Scholar 

  10. Eichinger B, Kirch C (2018) A MOSUM procedure for the estimation of multiple random change points. Bernoulli 24(1):526–564

    MathSciNet  Article  MATH  Google Scholar 

  11. Frick K, Munk A, Sieling H (2014) Multiscale change point inference. J R Stat Soc Ser B Stat Methodol 76(3):495–580 (with 32 discussions by 47 authors and a rejoinder by the authors)

    MathSciNet  Article  Google Scholar 

  12. Fryzlewicz P (2014) Wild binary segmentation for multiple change-point-detection. Ann Stat 42(6):2243–2281

    MathSciNet  Article  MATH  Google Scholar 

  13. Harchaoui Z, Lévy-Leduc C (2008) Catching change-points with lasso. Adv Neural Inf Process Syst 20:617–624

    Google Scholar 

  14. Horváth L, Horvath Z, Huskova M (2008) Ratio test for change point detection. In: Beyond parametrics in interdisciplinary research, vol 1. IMS, Collections, pp 293–304

  15. Hušková M, Slabý A (2001) Permutation tests for multiple changes. Kybernetika (Prague) 37:605–622

    MathSciNet  MATH  Google Scholar 

  16. Jandhyala V, Fotopoulos S, MacNeill I, Liu P (2013) Inference for single and multiple change-points in time series. J Time Ser Anal 34(4):423–446

    MathSciNet  Article  MATH  Google Scholar 

  17. Killick R, Eckley I, Ewans K, Jonathan P (2010) Detection of changes in variance of oceanographic time-series using changepoint analysis. Ocean Eng 37(13):1120–1126

    Article  Google Scholar 

  18. Matteson DS, James NA (2014) A nonparametric approach for multiple change point analysis of multivariate data. J Am Stat Assoc 109(505):334–345

    MathSciNet  Article  MATH  Google Scholar 

  19. Messer M, Schneider G (2017) The shark fin function—asymptotic behavior of the filtered derivative for point processes in case of change points. Stat Inference Stoch Process 20(2):253–272. https://doi.org/10.1007/s11203-016-9138-0

    MathSciNet  Article  MATH  Google Scholar 

  20. Messer M, Kirchner M, Schiemann J, Roeper J, Neininger R, Schneider G (2014) A multiple filter test for change point detection in renewal processes with varying variance. Ann Appl Stat 8(4):2027–2067

    MathSciNet  Article  MATH  Google Scholar 

  21. Messer M, Albert S, Plomer S, Schneider G (2017a) MFT: the multiple filter test for change point detection. https://cran.r-project.org/package=MFT, R package version 1.3 available via https://cran.r-project.org/package=MFT

  22. Messer M, Costa K, Roeper J, Schneider G (2017b) Multi-scale detection of rate changes in spike trains with weak dependencies. J Comput Neurosci 42(2):187–201

    MathSciNet  Article  MATH  Google Scholar 

  23. Muggeo V, Adelfio G (2011) Efficient change point detection for genomic sequences of continuous measurements. Bioinformatics 27(2):161–166

    Article  Google Scholar 

  24. Pein F, Sieling H, Munk A (2017) Heterogeneous change point inference. J R Stat Soc Ser B Stat Methodol 79(4):1207–1227

    MathSciNet  Article  MATH  Google Scholar 

  25. Staude B, Grün S, Rotter S (2010) Higher-order correlations in non-stationary parallel spike trains: statistical modeling and inference. Front Comput Neurosci 4:16

    Article  Google Scholar 

  26. Zhang H, Dantu R, Cangussu J (2009) Change point detection based on call detail records. In: Intelligence and Security Informatics. Institute of Electrical and Electronics Engineers, New York, pp 55–60

Download references

Acknowledgements

This work was supported by the German Federal Ministry of Education and Research (www.bmbf.de) within the framework of the e:Med research and funding concept (grant number 01ZX1404B to SA, MM and GS) and by the Priority Program 1665 of the Deutsche Forschungsgemeinschaft (grant number SCHN 1370/02-1 to MM and GS, www.dfg.de). We thank Solveig Plomer for helpful comments to the manuscript.

Author information

Affiliations

  1. Institute for Mathematics, Goethe University, Robert-Mayer-Str. 10, 60325, Frankfurt (Main), Germany

    Michael Messer, Stefan Albert & Gaby Schneider

Authors
  1. Michael Messer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Albert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gaby Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaby Schneider.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Messer, M., Albert, S. & Schneider, G. The multiple filter test for change point detection in time series. Metrika 81, 589–607 (2018). https://doi.org/10.1007/s00184-018-0672-1

Download citation

Keywords

  • Change point
  • Multiscale
  • MOSUM
  • MFT