Abstract
A framework for the detection of change points in the expectation in sequences of random variables is presented. Specifically, we investigate time series with general distributional assumptions that may show an unknown number of change points in the expectation occurring on multiple time scales and that may also contain change points in other parameters. To that end we propose a multiple filter test (MFT) that tests the null hypothesis of constant expectation and, in case of rejection of the null hypothesis, an algorithm that estimates the change points.
The MFT has three important benefits. First, it allows for general distributional assumptions in the underlying model, assuming piecewise sequences of i.i.d. random variables, where also relaxations with regard to identical distribution or independence are possible. Second, it uses a MOSUM type statistic and an asymptotic setting in which the MOSUM process converges weakly to a functional of a Brownian motion which is then used to simulate the rejection threshold of the statistical test. This approach enables a simultaneous application of multiple MOSUM processes which improves the detection of change points that occur on different time scales. Third, we also show that the method is practically robust against changes in other distributional parameters such as the variance or higher order moments which might occur with or even without a change in expectation. A function implementing the described test and change point estimation is available in the R package MFT.
Similar content being viewed by others
References
Albert S, Messer M, Schiemann J, Roeper J, Schneider G (2017) Multi-scale detection of variance changes in renewal processes in the presence of rate change points. J Time Ser Anal 38(6):1028–1052
Arlot S, Celisse A (2011) Segmentation of the mean of heteroscedastic data via cross-validation. Stat Comput 21(4):613–632
Aue A, Horváth L (2013) Structural breaks in time series. J Time Ser Anal 34(1):1–16
Basseville M, Nikiforov I (1993) Detection of abrupt changes: theory and application. Prentice Hall Information and System Sciences Series. Prentice Hall Inc., Englewood Cliffs
Bertrand PR (2000) A local method for estimating change points: the hat-function. Statistics 34(3):215–235
Bertrand PR, Fhima M, Guillin A (2011) Off-line detection of multiple change points by the filtered derivative with p-value method. Seq Anal 30(2):172–207
Billingsley P (1999) Convergence of probability measures. Wiley Series in Probability and Statistics: Probability and Statistics, 2nd edn. Wiley, New York
Brodsky B (2017) Change-point analysis in nonstationary stochastic models. CRC Press, Boca Raton
Brodsky BE, Darkhovsky BS (1993) Nonparametric methods in change-point problems, Mathematics and its Applications, vol 243. Kluwer Academic Publishers, Dordrecht
Eichinger B, Kirch C (2018) A MOSUM procedure for the estimation of multiple random change points. Bernoulli 24(1):526–564
Frick K, Munk A, Sieling H (2014) Multiscale change point inference. J R Stat Soc Ser B Stat Methodol 76(3):495–580 (with 32 discussions by 47 authors and a rejoinder by the authors)
Fryzlewicz P (2014) Wild binary segmentation for multiple change-point-detection. Ann Stat 42(6):2243–2281
Harchaoui Z, Lévy-Leduc C (2008) Catching change-points with lasso. Adv Neural Inf Process Syst 20:617–624
Horváth L, Horvath Z, Huskova M (2008) Ratio test for change point detection. In: Beyond parametrics in interdisciplinary research, vol 1. IMS, Collections, pp 293–304
Hušková M, Slabý A (2001) Permutation tests for multiple changes. Kybernetika (Prague) 37:605–622
Jandhyala V, Fotopoulos S, MacNeill I, Liu P (2013) Inference for single and multiple change-points in time series. J Time Ser Anal 34(4):423–446
Killick R, Eckley I, Ewans K, Jonathan P (2010) Detection of changes in variance of oceanographic time-series using changepoint analysis. Ocean Eng 37(13):1120–1126
Matteson DS, James NA (2014) A nonparametric approach for multiple change point analysis of multivariate data. J Am Stat Assoc 109(505):334–345
Messer M, Schneider G (2017) The shark fin function—asymptotic behavior of the filtered derivative for point processes in case of change points. Stat Inference Stoch Process 20(2):253–272. https://doi.org/10.1007/s11203-016-9138-0
Messer M, Kirchner M, Schiemann J, Roeper J, Neininger R, Schneider G (2014) A multiple filter test for change point detection in renewal processes with varying variance. Ann Appl Stat 8(4):2027–2067
Messer M, Albert S, Plomer S, Schneider G (2017a) MFT: the multiple filter test for change point detection. https://cran.r-project.org/package=MFT, R package version 1.3 available via https://cran.r-project.org/package=MFT
Messer M, Costa K, Roeper J, Schneider G (2017b) Multi-scale detection of rate changes in spike trains with weak dependencies. J Comput Neurosci 42(2):187–201
Muggeo V, Adelfio G (2011) Efficient change point detection for genomic sequences of continuous measurements. Bioinformatics 27(2):161–166
Pein F, Sieling H, Munk A (2017) Heterogeneous change point inference. J R Stat Soc Ser B Stat Methodol 79(4):1207–1227
Staude B, Grün S, Rotter S (2010) Higher-order correlations in non-stationary parallel spike trains: statistical modeling and inference. Front Comput Neurosci 4:16
Zhang H, Dantu R, Cangussu J (2009) Change point detection based on call detail records. In: Intelligence and Security Informatics. Institute of Electrical and Electronics Engineers, New York, pp 55–60
Acknowledgements
This work was supported by the German Federal Ministry of Education and Research (www.bmbf.de) within the framework of the e:Med research and funding concept (grant number 01ZX1404B to SA, MM and GS) and by the Priority Program 1665 of the Deutsche Forschungsgemeinschaft (grant number SCHN 1370/02-1 to MM and GS, www.dfg.de). We thank Solveig Plomer for helpful comments to the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Rights and permissions
About this article
Cite this article
Messer, M., Albert, S. & Schneider, G. The multiple filter test for change point detection in time series. Metrika 81, 589–607 (2018). https://doi.org/10.1007/s00184-018-0672-1
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00184-018-0672-1