Abstract
First part of this paper deals with tests on the stability of statistical models. The problem is formulated in terms of testing the null hypothesis H against the alternative hypothesis A. The null hypothesis H claims that the model remains the same during the whole observational period, usually it means that the parameters of the model do not change. The alternative hypothesis A claims that, at an unknown time point, the model changes, which means that some of the parameters of the model are subject to a change. In case we reject the null hypothesis H, i.e. when we decide that there is a change in the model, we concentrate on a number of questions that arise:
-
when has the model changed;
-
is there just one change or are there more changes;
-
what is the total number of changes etc.
The time moment when the model has changed is usually called change point. Aside testing for a change, our interest is to estimate change point(s) in different models. The least squares, M-, R- and MOSUM estimators are introduced and studied. Of course, we also estimate other parameters of the model(s), show approximations to the distributions of the change point estimators and show that the estimators of the change points are usually closely related to some of the test statistics treated in the first part.
Three types of confidence intervals are developed, one based on the limit distribution of the (point) estimators of m and two based on the bootstrap methods. All three methods are suitable for local changes while only the bootstrap constructions apply also to fixed changes.
The test statistics described below are typically certain functionals of partial sums of independent, identically distributed variables and their distribution is very complex. Therefore, we present selected limit results that form the basis for establishing the limit distribution of considered test statistics, functionals of partial sums and change point estimators.
Several Matlab codes, that implement selected methods described below and include detailed description and links to the previous sections, are presented to illustrate the possibilities of the studied methods. Complete Matlab codes are available from the authors on request.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Selected books and proceedings related to the problem
Basseville M. and Benveniste A. (eds.) (1986). Detection of Abrupt Changes in Signals and Dynamic Systems. Springer Verlag, Heidelberg.
Basseville M. and Nikiforov I. V. (1993). Detection of Abrupt Changes. Theory and Applications. Prentice Hall, London.
Broemling L. and Tsurumi H. (1987). Econometrics and Structural Change over Time. M. Dekker, New York.
Brodsky B. S. and Darkhovsky B. E. (1993). Nonparametric Methods in Change Point Problem. Kluwer, Dordrecht.
Carlstein E., Müller H. G. and Siegmund D. (eds.) (1994). Change Point Problem. IMS Lecture Notes-Monograph Series 23.
Csörgö M. and Horvath L. (1997). Limit Theorems in Change Point Analysis. J. Wiley, New York.
Hackl P. (1980). Testing the Constancy of Regression Relationship over Time. Vandenhoeck and Ruprecht, Göttingen.
Hackl P. (ed.) (1989). Statistical Analysis and Forecasting of Economic Structural Change. Springer Verlag, New York.
Hackl P. and Westlund A. H. (eds.) (1991). Economic Structural Changes. Analysis and Forecastings. IIASA & Springer Verlag, Heidelberg.
Selected articles in journals and proceedings; additional monographs
Albin J. M. P. (1990). On extremal theory for stationary processes. Ann. Probab. 18 92–108.
Albin J. M. P., Carlsson B. and Jaruskovâ D. (2000). On the asymptotic distribution of a statistic for test of appearance of linear trend. Extremes (submitted).
Anderson T. W. (1971). The Statistical Analysis of Time Series. J. Wiley, New York.
Anderson T. W. and Darling D. A. (1952). Asymptotic theory of certain goodness of fit criteria based on stochastic processes. Ann. Math. Statist. 23 193–212.
Anderson T. W. and Darling D. A. (1954). A test of goodness of fit. J. American Statistical Association 49 765–769.
Antoch J. and Ekblom H. (1993). Recursive robust regression - computational aspects and comparison. Computational Statistics and Data Analysis 19 115–128.
Antoch J. and Huskovâ M. (1993). Change point problem. In: Computational Aspects of Model Choice, Antoch J. (ed.), Physica-Verlag, Heidelberg, 11–38.
Antoch J. and Huskovâ. M. (1994). Procedures for detection of multiple changes in series of independent observations. In: 5th Prague Symp. on Asymptotic Statistics, Hus“kovâ. M. and Mandl P. (eds.), Physica-Verlag, Heidelberg, 3–20.
Antoch J. and Hu•kovâ M. (1996). Tests and estimators for epidemic alternatives. In: Probastat’94, Witkovsky V. and Pkzman A. (eds.), Tatra Mountains Math. Publ. 7 316–329.
Antoch J. and Huškovâ M. (1998). Estimators of changes. In: Nonparametrics, Asymp- totics and Time Series, Ghosh S. (ed.), M. Dekker, New York, 533–578.
Antoch J. and Hu•kovâ, M. (2000). Bayesian type estimators for change point. J. Stat. Plan. Infer. 91 195–208.
Antoch J. and Hu•kovâ M. (2000). Bayesian like R- and M-estimators of change points. Discussiones Mathematicae 20 115–134.
Antoch J. and Huškovâ, M. (2001). Permutation tests in change point analysis. Statist. Probab. Letters (to appear).
Antoch J., Huškovâ M. and Jaruskovâ D. (2000). In: Lecture Notes of the 5th IASC Summer School, Lauro N.C. et al eds., ISI, Voorburg, 1–75.
Antoch J., Huékovâ M. and Préäkovâ Z. (1997). Effect of dependence on statistics for determination of change. J. Stat. Plan. Infer. 60 291–310.
Antoch J., Hu•kovâ. M. and Veraverbeke N. (1995). Change point problem and bootstrap. J. Nonparametric Statistics 5 123–144.
Antoch J. and VIIek J.A. (1992). Robust estimation in linear model and its computational aspects. Computational Aspects of Model Choice, Antoch J. (ed.), Physica-Verlag, Heidelberg, 39–104.
Bai J. (1991). On the partial sums of residuals in autoregressive and moving average models. J. Time Series Analysis 14 247–259.
Bai J. (1994). Least squares estimation of a shift in linear processes. J. Time Series Analysis 15 453–472.
Berman S M (1982). Sojourns and extremes of stationary processes. Ann. Probab. 10 1–46.
Bhattacharya P. K. and Brockwell P. J. (1976). The minimum of additive process with applications to signal estimation and storage theory. Z. Wahrsch. Verw. Gebiete 37 51–75.
Billingsley P. (1968). Convergence of Probability Measures. J. Wiley, New York.
Bräker H. U. (1993). High boundary excursions of locally stationary Gaussian processes. Dissertation Thesis, University of Bern.
Brown R. L., Durbin J. and Evans J. M. (1975). Techniques for testing the constancy of regression relationships over time (with discussion). JRSS B 37 149–192.
Carlstein E. (1988). Nonparametric estimation of a change point. Ann. Statist. 16 188–197.
Chen X. (1988). Inference in a simple change point problem. Scientia Sinica A 31 654–667.
Chernoff H. and Zacks S. (1964). Estimating the current mean of normal distribution which is subjected to changes in time. Ann. Math. Statist. 35 999–1018.
Chib J. (1999). Bayesian estimation and comparison of multiple change point models. Manuscrip
Chow Y. S. and Teicher H. (1988). Probability Theory. Springer Verlag, New York.
Chung-Bow Lee (1995). Estimation of the number of change points in exponential families distributions. Preprint.
Chung-Bow Lee (1995). Estimation of the number of change points in normal random variables. Preprint.
Csörgö M. and Horvath L. (1988). Nonparametric methods for the change point problem. In: Handbook of Statistics 7, Krishnaiah P.R. and Rao C. R. (eds.), North Holland, Amsterdam, 403–425.
Chow Yu. S. and Teicher H. (1988). Probability Theory. Springer Verlag, Heidelberg.
Cobb G. W. (1978). The problem of the Nile. Conditional solution to a change point problem. Biometrika 65 243–251.
Cramér H. and Leadbetter M. R. (1967). Stationary and Related Stochastic Processes. J. Wiley, New York.
Csörgö M. and Horvath L. (1993). Weighted Approximations in Probability and Statistics. J. Wiley, New York.
Darkhovsky B. E. (1976). A nonparametric method for a posteriori detection of the “disorder” time of a sequence of independent random variables. Theory Probab. Appl. 21 178–183.
Darling D.A. and Erdös P. (1956). A limit theorem for the maximum of normalized sums of independent random variables. Duke Math. J. 23 143–155.
Davies R. B. (1987).Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika 74 33–43.
Davis R. A., Huang D. and Yao Y. C. (1995). Testing for a change in the parameter values and order of an autoregressive model. Ann. Statist. 23 282–304.
De Long D. (1981). Crossing probabilities for a square root boundary by a Bessel process. Commun. in Statist. -Theory and Method. A10 2191–2213.
Deshayes J. and Picard D. (1986). Off-line statistical analysis of change point models using nonparametric and likelihood methods. In: Lecture Notes in Control and Information Sciences 77, Basseville M. et al. (eds.), Springer Verlag, New York, 103–168.
Dümbgen L. (1991). The asymptotic behavior of some nonparametric change point esti-mators. Ann. Statist. 19 1471–1495.
Ferger D. (1994a). Change point estimators in case of small disorders.J. Stat. Plan. Infer. 40 33–49.
Ferger D. (1994b). On the rate of almost sure convergence of Diimbgen change point estimators. Statist. Probab. Letters 25 27–31.
Ferger D. (1994c). Asymptotic distribution theory of change point estimators and confi-dence intervals based on bootstrap approximation. Math. Meth. Statist. 3 362–378.
Ferger D. (1995). Nonparametric tests for nonstandard change point problems. Ann.Statist. 23 1848–1861.
Ferger D. and Stute W. (1992). Convergence of change point estimators. Stoch. Proc. Appl. 42 345–351.
Gardner L. A. (1969). On detecting changes in the mean of normal variates. Ann. Math. Statist. 40 116–126.
Gombay E. and Horvâth L. (1990). Asymptotic distributions of maximum likelihood tests for change in the mean. Biometrika 77 411–414.
Gombay E. and Horvâth L. (1994). Limit theorems for change in linear regression. J. Multivariate Analysis 48 43–69.
Gombay E., Horvâth L. and Hus’kovâ. M. (1996). Estimators and tests for change in variances. Statistics & Decisions 14 145–159.
Gombay E. and Horvâth L. (1996). Approximations for the time of change and the power function in change point models. J. Stat. Plan. Infer. 52 43–66.
Gombay E. and Horvâth L. (1996). On the rate of approximations for maximum likelihood tests in change point models. J. Mult. Anal. 56 120–152.
Gombay E. and Huskovâ M. (1998). Rank based estimators of the change point. J. Stat. Plan. Infer. 67 137–154.
Hampel F. R., Ronchetti E. M., Rousseeuw P. and Stahel W. A. (1981). Robust Statistics. The Approach Based on Influence Functions. J. Wiley, New York.
Hawkins D. H. (1977). Testing a sequence of observations for a shift in location. J. American Statistical Association 72 180–186.
Hinkley D. V. (1969). Inference about the intersection in two-phase regression. Biometrika 56 495–504.
Hinkley D. V. (1970a). Inference about the change point in a sequence of random variables. Biometrika 57 1–17.
Hinkley D. V. and Hinkley E. A. (1970). Inference about the change point in a sequence of binomial variables. Biometrika 57 477–488.
Hinkley D. and Schechtman E. (1987). Conditional bootstrap methods in the mean-shift model. Biometrika 74 85–93.
Horvâth L. (1993). The maximum likelihood method for testing changes in the parameters of normal observations. Ann. Statist. 21 671–680.
Horvâth L. (1995). Detecting changes in linear regression. Statistics 26 189–208.
Horvâth L. and Gombay E. (1994). Limit theorems for change in linear regression. J. Multivariate Analysis 48 43–69.
Horvâth L., Hus“kovâ M. and Serbinovska M. (1997). Estimators for the time of change in linear models. Statistics 27 109–130.
Horvath L. and Kokozska P. (1997). The effect of long-range dependence on change point estimators J. Stat. Plan. Infer. 64 57–81.
Horyath L. and Qi-Man Shao (1996). Darling-Erdds type theorems for sums of Gaussian variables with long range memory. Stoch. Proc. Appl. 83 117–137.
Huber P. J. (1981). Robust Statistics. J. Wiley, New York.
Hasler J. (1990). Extreme values and high boundary crossings of locally stationary Gaussian processes. Ann. Probability 18 1141–1158.
Hasler J. (1993). A note on extreme values of locally stationary Gaussian processes. J. Stat. Plan. Infer. 45, 203–213.
Hus“kovâ. M. (1994). Nonrecursive procedures for detecting change in simple linear regression models. In: New Progress in Probability and Statistics, Puri M. L. and Vilaplana J. P. (eds.), 105–126.
Hus“kovâ. M. (1995). Estimators for epidemic alternatives. Comment. Math. Univ. Carolinae 36 279–291.
Huskovâ M. and Sen P. K. (1989). Nonparametric tests for shift and change in regression at an unknown time point. In: Statistical Analysis and Forecasting of Economic Structural Change, Haakl P. (ed.), Springer Verlag, New York, 71–85.
James B., James K. L. and Siegmund D. (1987). Tests for change points. Biometrika 74 71–84.
Jandhyala V. K. and MacNeill I. B. (1990). Residual partial sum limit process for regression models with applications to detecting parameter changes at unknown times. Stoch. Proc. Appl. 33 309–323.
Jandhyala V. K. and MacNeill I. B. (1991). Tests for parameter changes at unknown times in linear regression models J. Stat. Plan. Infer. 27 291–316.
Jandhyala V. K. and MacNeill I. B. (1992). On testing for the constancy of regression coefficients under random walk and change point alternatives Econometric Theory 8 501–517.
Jaruškovâ D. (1996). Changè point detection in meteorological measurement Monthly Weather Review 124 1535–1543.
Jaruškovâ D. (1997). Some problems with application of change point detection methods to environmental data Environmetrics 8 469–483.
Jaruškovâ D. (1998a). Change point estimator in gradually changing sequences Comment. Math. Univ. Carolinae 39 551–561.
Jaruškovâ D. (1998b). Testing appearance of linear trend J. Stat. Plan. Infer. 70 263–276.
Jaruškovâ, D. (1999). Testing appearance of polynomial trend Extremes 2 25–37.
Jaruškovâ D. and Antoch J. (1994).Detection of change in variance. In: 5th PragueSymposium on Asymptotic Statistics, Mandl P. and Huškovâ M. (eds.), Physica-Verlag, Heidelberg, 297–301.
Jones P. D., Wigley M. L. and Briffa K. R. (1994). Global and hemispheric temperature anomalies–land and marine instrumental records In: Trends’93: A Compendium of Data on Global Change, Boden T. A. et al. (eds.), ORNL/CDIAC–65, Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory, Oak Ridge, Term., USA, 603–608.
Kander Z. and Zacks S. (1966). Test procedures for possible changes in parameters of statistical distributions occurring at unknown time points Ann. Math. Statist. 37 1196–1210.
Kiefer J. (1960). K-sample analogues of the Kolmogorov-Smirnov’s and Cramér-von Mises tests Ann. Math. Statist. 30 420–447.
Kim Hyune-Ju (1995). Detection of a change in longitudinal data. In: Int. Conference on Statistical Methods and Statistical Computing for Quality and Productivity Improvement, 748–754.
Kim Hyune-Ju and Siegmund D. (1989). The likelihood ratio test for a change point in simple linear regression Biometrika 76 409–423.
Knowles M., Siegmund D. and Zhang H. (1991). Confidence regions in semilinear regression Biometrika 78 15–31.
Leadbetter M. R., Lindgren G. and Rootzén H. (1983). Extremes and related properties of random sequences and processes. Springer Verlag, Heidelberg.
MacNeill I. B. (1974). Tests for change of parameter at unknown time and distribution of some related functionals of Brownian motion Ann. Statist. 2 950–962.
MacNeill I. B. (1978). Properties of sequences of partial sums of polynomial regression residuals with applications to tests for change of regression at unknown times Ann.Statist. 6 422–433.
Michels P. and Trenkler G. (1990). Testing the stability of regression coefficients using generalized recursive residuals. Austr. J. Statist. 22, 293–312.
Nicholson S. E. (1994). Century-scale series of standardised annual departure of African rainfall. In: Trends’93: A Compendium of Data on Global Change, Boden T. A. et al. (eds.), ORNL/CDIAC–65, Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory, Oak Ridge, Tenn., USA, 952–962.
Ploberger W. and Krämer W. (1992).The CUSUM test with OLS residuals Econometrica 60 217–285.
Ritov Y. (1990). Asymptotic efficient estimation of the change point with unknown distribution. Ann. Statist. 18, 1829–1839.
Schechtman E. (1983). A conservative nonparametric distribution-free confidence bound for the shift in the change point problem Comment. Statist. Al2 2455–2464.
Sen P. K. (1982). Asymptotic theory of some tests for constancy of regression relationships over time Statistics 13 21–31.
Sen P. K. (1984). Recursive M-tests for constancy of multivariate regression relationships over time. Sequential Analysis 3, 191–211.
Shepp L. A. (1979). The joint density of the maximum and its location for a Wiener process with drift. J. Appl. Probab. 16, 423–427.
Shorack G. R. and Wellner J. A. (1986). Empirical Processes with Applications to Statistics. J. Wiley, New York.
Siegmund D. (1986). Boundary crossing probabilities and statistical applications Ann. Statist. 14 361–404.
Siegmund D. (1988). Confidence sets in change point problems Int. Statist. Review 56 31–48.
Siegmund D. and Zhang H. (1994). Confidence region in broken line regression In: Change point problems, IMS Lecture Notes-Monograph Series 23 Carlstein E. et al. (eds.). 292–316.
Steinebach J. and Eastwood V. R. (1996). Extreme value asymptotics for multivariate renewal processes J. Multivariate Analysis 56 284–302.
Stryhn H. (1996). The location of maximum of asymmetric two-sided Brownian motion with triangular drift Statist. Probab. Letters 29 279–284.
Tang S.M. and Mac Neill I.B. (1993). The effect of serial correlation on tests for parameter change at unknown time Ann. Statist. 21 552–575.
Vostrikova L. Y. (1981). Detection of “disorder” in multidimensional random processes Soy. Math. Dokl. 24 55–59.
Wolfe D. A. and Schechtman E. (1984). Nonparametric procedures for the change point problem J. Stat. Plan. Infer. 9 389–396.
Worsley K. J. (1983). Testing for a two-phase multiple regression Technometrics 25 35–42.
Worsley K. J. (1986). Confidence regions and tests for a change point in a sequence of exponential family random variables Biometrika ’73 91–104.
Yao Q. (1993). Tests for change points with epidemic alternatives Biometrika 80 179–191.
Yao Y.-C. (1988). Estimatingnumber of change points via Schwarz criterion Statist. Probab. Letters 6 181–189.
Yao Y.-C. (1990). On asymptotic behavior of a class of nonparametric tests Statist. Probab. Letters 19 173–177.
Yao Y.-C. (1987). Approximating the distribution of maximum likelihood estimate of the change point in a sequence of independent random variables Ann. Statist. 15 1321–1328.
Yao Yi-Ching and Davis R. A. (1986). The asymptotic behavior of the likelihood ratio statistic for testing a shift in mean in a sequence of independent normal variates Sankhya 48 339–353.
Zacks S. (1983). Survey of classical and Bayesian approach to the change point problem. Fixed sample and sequential procedures of testing and sequential procedures of testing and estimation. In: Recent Advances in Statistics, Papers in Honor of Herman Chernoff’s Sixties Birthday, Rizvi M. H. (ed.), Academic Press, New York, 245–269.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Physica-Verlag Heidelberg
About this chapter
Cite this chapter
Antoch, J., Hušková, M., Jarušková, D. (2002). Off-Line Statistical Process Control. In: Lauro, C., Antoch, J., Vinzi, V.E., Saporta, G. (eds) Multivariate Total Quality Control. Contributions to Statistics. Physica-Verlag HD. https://doi.org/10.1007/978-3-642-48710-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-48710-1_1
Publisher Name: Physica-Verlag HD
Print ISBN: 978-3-7908-1383-8
Online ISBN: 978-3-642-48710-1
eBook Packages: Springer Book Archive