A sequential multiple change-point detection procedure via VIF regression

Abstract

In this paper, we propose a procedure for detecting multiple change-points in a mean-shift model, where the number of change-points is allowed to increase with the sample size. A theoretic justification for our new method is also given. We first convert the change-point problem into a variable selection problem by partitioning the data sequence into several segments. Then, we apply a modified variance inflation factor regression algorithm to each segment in sequential order. When a segment that is suspected of containing a change-point is found, we use a weighted cumulative sum to test if there is indeed a change-point in this segment. The proposed procedure is implemented in an algorithm which, compared to two popular methods via simulation studies, demonstrates satisfactory performance in terms of accuracy, stability and computation time. Finally, we apply our new algorithm to analyze two real data examples.

Appendix: Proof of Theorem 1

Since $\varepsilon _i$, $i=1,2,\ldots$, are iid zero-mean variables with variance $\sigma ^2$, it follows from the definition of $\rho _{i+1}$ in (8) and the idempotence of $I- X^{(i+1)}[( X^{(i+1)})^T X^{(i+1)}]^{-1}( X^{(i+1)})^T$ that the variance of $\rho _{i+1}^{-1}(\varvec{x}_{\mathrm{new}}^{(i+1)})^T\{I- X^{(i+1)}[( X^{(i+1)})^T X^{(i+1)}]^{-1}( X^{(i+1)})^T\}{\varvec{\varepsilon }}^{(i+1)}$ is still $\sigma ^2$. By the central limit theorem, we obtain that

$$\begin{aligned}&\rho _{i+1}^{-1}\left( \varvec{x}_{\mathrm{new}}^{(i+1)}\right) ^T\left\{ I- X^{(i+1)}\left[ \left( X^{(i+1)}\right) ^T X^{(i+1)}\right] ^{-1}\left( X^{(i+1)}\right) ^T\right\} {\varvec{\varepsilon }}^{(i+1)} \xrightarrow {d} N\left( 0,\sigma ^2\right) . \end{aligned}$$

Note that $( X^{(i+1)})^T X^{(i+1)}$ can be expressed as $( U^{(i+1)})^T \varLambda ^{(i+1)} U^{(i+1)}$, where $U^{(i+1)}$ is the lower triangular matrix of order $k+1$ whose nonzero entries are all 1’s, and $\varLambda ^{(i+1)}$ is a diagonal matrix with diagonal entries being $k_1-k_0,\ k_2-k_1,\ \ldots ,\ k_m-k_{m-1},\ 1+(i+1)l-k_m$. Since the change-points are well-separated, i.e., $k_r-k_{r-1}=O(n)$, $(\varLambda ^{(i+1)})^{-1}$ is of order O(1 / n), we have that $[( X^{(i+1)})^T X^{(i+1)}]^{-1}$ is also of order O(1 / n).

Next, we prove that $\rho _{i+1}$ defined in (8) is asymptotically equal to $\sqrt{l}$. Note that $\varvec{x}_{\mathrm{new}}^{(i+1)}={{\varvec{\ell }}}_{il,l}$ is the vector with only the last l elements being ones, and all other elements are zeros. It can be seen that $(\varvec{x}_{\mathrm{new}}^{(i+1)})^T\varvec{x}_{\mathrm{new}}^{(i+1)}=l$ and $(\varvec{x}_{\mathrm{new}}^{(i+1)})^T X^{(n+1)}=O(l)$. Therefore, as $n\rightarrow \infty$, it is readily seen from $[( X^{(i+1)})^T X^{(i+1)}]^{-1}=O(1/n)$ that

$$\begin{aligned} \rho _{i+1}^2&=\left( \varvec{x}_{\mathrm{new}}^{(i+1)}\right) ^T\varvec{x}_{\mathrm{new}}^{(i+1)}-\left( \varvec{x}_{\mathrm{new}}^{(i+1)}\right) ^T \\&\quad \times \left\{ I- X^{(i+1)}\left[ \left( X^{(i+1)}\right) ^T X^{(i+1)}\right] ^{-1}\left( X^{(i+1)}\right) ^T\right\} \varvec{x}_{\mathrm{new}}^{(i+1)}\\&=l-O\left( l^2/n\right) \sim l. \end{aligned}$$

Under the null hypothesis, there exists no change-point in the interval $[1+il,(i+1)l]$. It can be shown that the last l elements of the correction vector ${\varvec{\eta }}^{(i+1)}$ are zeros, which implies that $(\varvec{x}_{\mathrm{new}}^{(i+1)})^T{\varvec{\eta }}^{(i+1)}=0$. Since $(\varvec{x}_{\mathrm{new}}^{(i+1)})^T X^{(i+1)}=O(l)$, $(X^{(i+1)})^T{\varvec{\eta }}^{(i+1)}=o_p(bl)$, $[ (X^{(i+1)})^T X^{(i+1)}]^{-1}=O(1/n)$ and $\rho _{i+1}/\sqrt{l}\rightarrow 1$, by Assumption A1, it follows that

$$\begin{aligned} \rho _{i+1}^{-1}\left( \varvec{x}_{\mathrm{new}}^{(i+1)}\right) ^T\left\{ I- X^{(i+1)}\left[ \left( X^{(i+1)}\right) ^T X^{(i+1)}\right] ^{-1}\left( X^{(i+1)}\right) ^T\right\} {\varvec{\eta }}^{(i+1)}=o(1). \end{aligned}$$

In view of the fact that $\beta _{\mathrm{new}}^{(i+1)}=0$, i.e., there is no change-point in $[1+il,(i+1)l]$, and $\rho _{i+1}\rightarrow \infty$, by (7) and (9), we obtain that

$$\begin{aligned} \rho _{i+1}\hat{\beta }_{\mathrm{new}}^{(i+1)}\xrightarrow {d} N(0,\sigma ^2). \end{aligned}$$

This proves Theorem 1(a).

Under the alternative hypothesis, there exists a change-point, say $k_m$, in the segment $[1+il,(i+1)l]$. Moreover, $k_m-il$ many of the last l elements of the correction vector ${\varvec{\eta }}^{(i+1)}$ are equal to $\beta _{\mathrm{new}}^{(i+1)}$, and $\beta _{\mathrm{new}}^{(i+1)} \not =0$, which implies $(\varvec{x}_{\mathrm{new}}^{(i+1)})^T{\varvec{\eta }}^{(i+1)}=\beta _{\mathrm{new}}^{(i+1)}\left( k_m-il\right)$.

Moreover, we have

$$\begin{aligned} \rho _{i+1}^{-2}\left( \varvec{x}_{\mathrm{new}}^{(i+1)}\right) ^TX^{(i+1)}\left[ \left( X^{(i+1)}\right) ^T X^{(i+1)}\right] ^{-1}\left( X^{(i+1)}\right) ^T{\varvec{\eta }}^{(i+1)}=o_p(1) \end{aligned}$$

from the Proof of Theorem 1(a). In view of (11), we obtain that

$$\begin{aligned} \rho _{i+1}^{-2}\left( \varvec{x}_{\mathrm{new}}^{(i+1)}\right) ^T\left\{ I- X^{(i+1)}\left[ \left( X^{(i+1)}\right) ^T X^{(i+1)}\right] ^{-1}\left( X^{(i+1)}\right) ^T\right\} {\varvec{\varepsilon }}^{(i+1)}=o_p(1). \end{aligned}$$

Applying these results to (7) yields

$$\begin{aligned} \hat{\beta }_{\mathrm{new}}^{(i+1)}=\beta _{\mathrm{new}}^{(i+1)}\left[ 1-\rho _{i+1}^{-2}(k_m-il)\right] +o_p(1). \end{aligned}$$

Furthermore, if the change-point $k_m$ is located in the artificial interval $[1+(i-1)l,il]$ (i.e., the change-point was previously undetected), then the correction vector ${\varvec{\eta }}^{(i+1)}$ has zero components in the last l rows, which implies that $(\varvec{x}_{\mathrm{new}}^{(i+1)})^T{\varvec{\eta }}^{(i+1)}=0$. A similar argument as above yields that $\hat{\beta }_{\mathrm{new}}^{(i+1)}=\beta _{\mathrm{new}}^{(i+1)}+o_p(1).$ This ends the proof of Theorem 1(b).