Jump detection in time series nonparametric regression models: a polynomial spline approach

Abstract

For time series nonparametric regression models with discontinuities, we propose to use polynomial splines to estimate locations and sizes of jumps in the mean function. Under reasonable conditions, test statistics for the existence of jumps are given and their limiting distributions are derived under the null hypothesis that the mean function is smooth. Simulations are provided to check the powers of the tests. A climate data application and an application to the US unemployment rates of men and women are used to illustrate the performance of the proposed method in practice.

Appendix

The following notations are used throughout the proof. We define $||\cdot ||_{\infty }$ as the supremum norm of a function $r$ on $[a,b]$, i.e. $||r||_{\infty }=\sup _{x \in [a,b]}|r(x)|.$ We will use $c,C$ to denote some positive constants in a generic sense through the proof.

To prove Theorem 1, we decompose the estimation error $\hat{m}_p(x)-m_p(x)$ into a bias term and a noise term. Denoting $\mathbf{m}=\left( m\left( X_{1}\right) ,\ldots ,m\left( X_{n}\right) \right) ^\mathrm{T}$ and $\mathbf{E}_{\sigma }=\left( \sigma \left( X_{1}\right) \varepsilon _{1},\ldots , \sigma \left( X_{n}\right) \varepsilon _{n}\right) ^\mathrm{T}$, we can rewrite $\mathbf{Y}$ as $\mathbf{Y}=\mathbf{m}+\mathbf{E}_{\sigma }$. We project the response $\mathbf{Y}$ onto the spline space $G^{(p-2)}_N$ spanned by $\{\mathbf{B}_{j,p}(\mathbf{X})\}_{j=1-p}^N$, where $\mathbf{B}_{j,p}\left( \mathbf{X}\right)$ is denoted as

$$\begin{aligned} \mathbf{B}_{j,p}\left( \mathbf{X}\right) =\left\{ B_{j,p}\left( X_1\right) , \ldots ,B_{j,p}\left( X_n\right) \right\} ^\mathrm{T},\quad j=1-p,\ldots ,N, \end{aligned}$$

with $B_{j,p}\left( x\right)$ introduced in Sect. 2.1. We obtain the following decomposition

$$\begin{aligned} \hat{m}_p(x)=\tilde{m}_p(x)+\tilde{\varepsilon }_p(x), \end{aligned}$$

where

$$\begin{aligned} \tilde{m}_p(x)=\{B_{j,p}(x)\}_{1-p\le j \le N}^\mathrm{T} \mathbf{V}_{n,p}^{-1}\{\langle \mathbf{m},B_{j,p}\rangle _n \}_{j=1-p}^N, \nonumber \\ \tilde{\varepsilon }_p(x)=\{B_{j,p}(x)\}_{1-p\le j \le N}^\mathrm{T} \mathbf{V}_{n,p}^{-1}\{\langle \mathbf{E}_\sigma ,B_{j,p}\rangle _n\}_{j=1-p}^N. \end{aligned}$$

(10)

The bias term is $\tilde{m}_p(x)-m_p(x)$ and the noise term is $\tilde{\varepsilon }_p(x)$.

Lemma 1

As $n \rightarrow \infty$,

$$\begin{aligned} \left| \left| b_{j,1}\right| \right| _2^2&= \!f(t_j)h (1\!+\!r_{j,n,1}),\quad \left| \left| b_{j,2}\right| \right| _2^2\!=\!\frac{2f(t_{j+1} )h}{3}\left\{ \begin{array}{rl} 1\!+\!r_{j,n,2},&{} 0 \!\le \! j \!\le \! N\!-\!1,\\ \frac{1}{2}\!+\!r_{j,n,2}, &{} j\!=\!-1,N,\\ \end{array} \right. \\ \langle b_{j,1},b_{j^{\prime },1}\rangle \!&= \!\left\{ \begin{array}{rl} 1, &{} j\!=\!j^{\prime },\\ 0, &{} j\!\ne \! j^{\prime }, \end{array}\right. \quad \langle b_{j,2},b_{j^{\prime },2}\rangle =\!\frac{1}{6}f\left( t_{j+1}\right) h\left\{ \begin{array}{ll} 1\!+\!\tilde{r}_{j,n,2}, &{} \left| j^{\prime }\!-\!j\right| \!=\!1,\\ 0, &{} \left| j^{\prime }\!-\!j\right| \!>\!1, \end{array}\right. \end{aligned}$$

where $\max _{0 \le j \le N}\left| r_{j,n,1}\right| +\max _{-1 \le j \le N}\left| r_{j,n,2}\right| +\max _{-1 \le j \le N-1}\left| \tilde{r}_{j,n,2}\right| \le C \omega \left( f,h\right)$ and $\omega (f,h)= \max _{x,x^{\prime }\in [a,b],|x-x^{\prime }| \le h}|f(x)-f(x^{\prime })|$ is the moduli of continuity of a continuous function $f$ on $[a,b]$. Furthermore,

$$\begin{aligned} \frac{1}{3}f(t_{j+1})h\left\{ 1-C\omega (f,h)\right\} \le \left| \left| b_{j,2}\right| \right| _2^2 \le \frac{2}{3}f(t_{j+1})h\left\{ 1+C\omega (f,h)\right\} . \end{aligned}$$

Proof of Theorem 1

for $p=1$   When $p=1,\,\mathbf{V}_{n,1}^{-1}$ is a diagonal matrix, and $\tilde{\varepsilon }_1(x)$ in Eq. (10) can be rewritten as

$$\begin{aligned} \tilde{\varepsilon }_1(x)=\sum _{j=0}^N \varepsilon _j^* B_{j,1}(x)\left| \left| B_{j,1}\right| \right| _{2,n}^{-2}, \quad \varepsilon _j^*= n^{-1} \sum _{i=1}^n B_{j,1}(X_i)\sigma (X_i)\varepsilon _i,\quad x \in [a,b]. \end{aligned}$$

We define $\hat{\varepsilon }_1(x)=\sum _{j=0}^N \varepsilon _j^* B_{j,1}(x)$, and it is straightforward that $\hat{\varepsilon }_1(t_{j})=B_{j,1}(t_{j})\varepsilon _{j}^*, \, j=0,\ldots ,N$. We treat the variance of $\hat{\varepsilon }_1(t_{j+1})-\hat{\varepsilon }_1(t_{j})$ as follows.

Lemma 2

The variance of $\hat{\varepsilon }_1(t_{j+1})-\hat{\varepsilon }_1(t_{j}),\,j=0, \ldots ,N-1,$ is $\sigma ^2_{n,1,j}$ in Eq. (5), which satisfies

$$\begin{aligned} \sigma ^2_{n,1,j}=E\{\hat{\varepsilon }_1(t_{j+1})-\hat{ \varepsilon }_1(t_{j})\}^2=\sigma ^2(t_{j+1})(f(t_{j+1})nh)^{-1}+\sigma ^2(t_{j})(f(t_{j})nh)^{-1}. \end{aligned}$$

Accordingly, under Assumption $($A2$)$, one has $c(nh)^{-1/2}\le \sigma _{n,1,j}\le C(nh)^{-1/2}$ for any $j=0,\ldots ,N-1$ as $n$ sufficiently large.

The proof can be easily obtained by Lemma A.1 combining with the fact that $\langle B_{j,1},B_{j+1,1}\rangle =0$.

Denote, for $0\le j \le N-1,\,\tilde{\xi }_{n,1,j}=\sigma _{n,1,j}^{-1}\{\tilde{ \varepsilon }_1(t_{j+1} )-\tilde{\varepsilon }_1(t_j)\}$ and $\hat{\xi }_{n,1,j}=\sigma _{n,1,j}^{-1}\{\hat{\varepsilon }_1 (t_{j+1})-\hat{\varepsilon }_1(t_j)\}$. The next lemma follows from Lemma A.6 of Wang and Yang (2010).

Lemma 3

Under Assumptions $(\mathrm{A2}{-}\mathrm{A4})$, as $n\rightarrow \infty$,

$$\begin{aligned} \left| \sup _{0\le j \le N-1}\left| \hat{\xi }_{n,1,j}\right| -\sup _{0\le j \le N-1}\left| \tilde{\xi }_{n,1,j}\right| \right| =O_p\left\{ \left( nh\right) ^{-1/2}\log n \right\} . \end{aligned}$$

Proof

Rewrite $\hat{\varepsilon }_1(t_{j+1})-\hat{\varepsilon }_1(t_{j})$ as $\hat{\varepsilon }_1(t_{j+1})-\hat{\varepsilon }_1(t_{j})={\mathbf{D}_{j,1}}^\mathrm{T}{{\varvec{\Lambda }}_{j,1}},\,j=0,\ldots ,N-1$, where

$$\begin{aligned} {\mathbf{D}_{j,1}}&= (-n^{-1/2}B_{j,1}(t_j), n^{-1/2}B_{j+1,1}(t_{j+1}))^\mathrm{T},\\{{\varvec{\Lambda }}_{j,1}}&= \left( \begin{array}{c} n^{-1/2}\sum _{i=1}^n B_{j,1}\left( X_i\right) \sigma \left( X_i\right) \varepsilon _i\\ [6pt] n^{-1/2}\sum _{i=1}^n B_{j+1,1}\left( X_i\right) \sigma \left( X_i\right) \varepsilon _i \end{array}\right) .\\ \end{aligned}$$

It follows that $\sigma ^2_{n,1,j}={\mathbf{D}_{j,1}}^\mathrm{T}Cov\left( {\varvec{\Lambda }}_{j,1} \right) \mathbf{D}_{j,1}$ with

$$\begin{aligned} Cov\left( {\varvec{\Lambda }}_{j,1} \right) =\left( \begin{array}{c@{\qquad }c} EB_{j,1}^2\left( X\right) \sigma ^2\left( X\right) &{} 0\\ 0&{} EB_{j+1,1}^2\left( X\right) \sigma ^2\left( X\right) \\ \end{array}\right) . \end{aligned}$$

Let $\mathbf{Z}_j=(Z_{j1},Z_{j2})^\mathrm{T}= {\varvec{\Lambda }}_{j,1}^\mathrm{T}\{Cov({\varvec{\Lambda }}_{j,1})\}^{-1/2}$. Specifically, for $0\le j\le N-1$,

$$\begin{aligned} Z_{j1}&= \left\{ nEB_{j,1}^2(X)\sigma ^2(X) \varepsilon ^2\right\} ^{-1/2}\left\{ \sum _{i=1}^nB_{j,1}(X_i) \sigma (X_i)\varepsilon _i\right\} ,\\ Z_{j2}&= \left\{ nEB_{j+1,1}^2\left( X\right) \sigma ^2 \left( X\right) \varepsilon ^2\right\} ^{-1/2}\left\{ \sum _{i=1}^nB_{j+1,1} \left( X_i\right) \sigma \left( X_i\right) \varepsilon _i \right\} . \end{aligned}$$

By Lemmas 3.2, 3.3 and A.7 of Wang and Yang (2010), we have uniformly in $j$,

$$\begin{aligned} P\left[ \left| Z_{j\gamma }\right| \le \left\{ 2\log \left( N+1\right) \right\} ^{1/2}d_n\left( \frac{\alpha }{2} \right) \right] =1-\frac{\alpha }{2\left( N+1\right) }+o \left( N^{-1}\right) ,\,\gamma =1,2. \end{aligned}$$

Therefore, for $\gamma =1,2$,

$$\begin{aligned} \limsup _{n\rightarrow \infty }P\left[ \max _{0\le j \le N}Z_{j\gamma }^2> 2\log \left( N+1\right) \left\{ d_n\left( \frac{\alpha }{2}\right) \right\} ^2\right] \le \frac{\alpha }{2}. \end{aligned}$$

Denote $\mathbf{Q}_{j,1}={\varvec{\Lambda }}_{j,1}^\mathrm{T}\left\{ Cov\left( {\varvec{\Lambda }}_{j,1} \right) \right\} ^{-1}{\varvec{\Lambda }}_{j,1}=\mathbf{Z}_j\mathbf{Z}_j^\mathrm{T}=\sum _{\gamma =1,2}Z_{j\gamma }^2,\,j=0,\ldots ,N-1$. According to the maximization lemma of Johnson and Wichern (1992),

$$\begin{aligned} \left\{ \sigma _{n,1,j}^{-1}[\hat{\varepsilon }_1 (t_{j+1})-\hat{\varepsilon }_1(t_j) ]\right\} ^2 \le {\varvec{\Lambda }}_{j,1}^\mathrm{T} \left\{ Cov({\varvec{\Lambda }}_{j,1})\right\} ^{-1} {\varvec{\Lambda }}_{j,1}=\mathbf{Q}_{j,1}. \end{aligned}$$

Hence,

$$\begin{aligned}&\liminf _{n \rightarrow \infty } P \left[ \sup _{0 \le j \le N-1}\left| \sigma _{n,1,j}^{-1}\left[ \hat{\varepsilon }_1 (t_{j+1})-\hat{\varepsilon }_1(t_j)\right] \right| \le 2\left\{ \log \left( N+1\right) \right\} ^{1/2}d_n \left( \frac{\alpha }{2}\right) \right] \\&\quad \ge \liminf _{n \rightarrow \infty } P\left[ \max _{0 \le j \le N-1} \mathbf{Q}_{j,1}\le 4 \log \left( N+1\right) \left\{ d_n\left( \frac{\alpha }{2}\right) \right\} ^2 \right] \\&\quad \ge 1-\sum _{\gamma =1,2}\limsup _{n \rightarrow \infty }P \left[ \max _{0 \le j \le N-1}Z_{j\gamma }^2 >2 \log \left( N+1\right) \left\{ d_n\left( \frac{\alpha }{2}\right) \right\} ^2 \right] \\&\quad \ge 1-\alpha . \end{aligned}$$

Note that $\hat{m}_1(t_{j+1})-\hat{m}_1(t_{j})=[\tilde{m}_1(t_{j+1})-m (t_{j+1})]-[\tilde{m}_1(t_j )-m(t_j)]+[m(t_{j+1}) -m(t_j)]+[\tilde{\varepsilon }_1 (t_{j+1})-\tilde{\varepsilon }_1(t_j)]$. The theorem of de Boor (2001) on page 149 and Theorem 5.1 of Huang (2003) entail that under $\mathcal{H }_0$ the orders of the first three terms are all $Op\left( h\right)$, which makes

$$\begin{aligned} \sigma ^{-1}_{n,1,j}[\log (N+1)]^{-1/2} \left| \left| \tilde{m}-m\right| \right| _{\infty }=O_p\left\{ \left( nh\right) ^{1/2}h[\log (N+1)]^{-1/2}\right\} =o_p(1). \end{aligned}$$

We finally apply Lemma 3 to get

$$\begin{aligned}&\limsup _{n \rightarrow \infty } P \left[ \sup _{0 \le j \le N-1}\sigma _{n,1,j}^{-1}\left| \hat{m}_1(t_{j+1} )-\hat{m}_1(t_{j})\right| > \left\{ 4\log \left( N+1\right) \right\} ^{1/2}d_n\left( \alpha /2\right) \right] \\&\quad = \limsup _{n \rightarrow \infty } P \left[ \sup _{0 \le j \le N-1}\sigma _{n,1,j}^{-1} \left| \hat{\varepsilon }_1(t_{j+1})-\hat{ \varepsilon }_1(t_j)\right| > \left\{ 4\log \left( N+1\right) \right\} ^{1/2}d_n\left( \alpha /2\right) \right] \\&\quad \le \alpha . \end{aligned}$$

$\square$

Proof of Theorem 1

for $p=2$      For $p=2$, we can rewrite the noise term $\tilde{\varepsilon }_2\left( x\right)$ in Eq. (10) as $\tilde{\varepsilon }_2\left( x\right) = \sum _{j=-1}^N \tilde{a}_j B_{j,2}\left( x\right)$, where

$$\begin{aligned} \tilde{\mathbf{a}}=\left( \tilde{a}_{-1},\ldots ,\tilde{a}_N \right) ^\mathrm{T}={\mathbf{V}}_{n,2}^{-1}\left\{ n^{-1} \sum _{i=1}^n B_{j,2} \left( X_i\right) \sigma \left( X_i\right) \varepsilon _i\right\} _{j=-1}^N. \end{aligned}$$

Similarly as before, we denote $\hat{\varepsilon }_2\left( x\right) =\sum _{j=-1}^N \hat{a}_j B_{j,2}\left( x\right)$, where $\hat{\mathbf{a}}= (\hat{a}_{-1},\ldots ,\hat{a}_N)^\mathrm{T}$ is defined by replacing ${\mathbf{V}}_{n,2}^{-1}$ in the above formula with $\mathbf{S}=\mathbf{V}_2^{-1}$, i.e.

$$\begin{aligned} \hat{\mathbf{a}}&= \mathbf{S}\left\{ n^{-1}\sum _{i=1}^n B_{j,2}\left( X_i\right) \sigma \left( X_i\right) \varepsilon _i \right\} _{j=-1}^N \\&= \left\{ \sum _{j=-1}^N s_{j^{\prime }j}n^{-1}\sum _{i=1}^n B_{j,2}\left( X_i\right) \sigma \left( X_i\right) \varepsilon _i\right\} _{j^{\prime }=-1}^N. \end{aligned}$$

Thus, for any $x \in [a,b]$,

$$\begin{aligned} \hat{\varepsilon }_2\left( x\right) =\sum _{j^{\prime }=-1}^N \hat{a}_j B_{j^{\prime },2}\left( x\right) =\sum _{j,j^{\prime }=-1}s_{j^{ \prime }j}n^{-1}\sum _{i=1}^n B_{j,2}\left( X_i\right) \sigma \left( X_i\right) \varepsilon _iB_{j^{\prime },2}\left( x\right) . \end{aligned}$$

Denote $\tilde{\xi }_{2,j}=\tilde{\varepsilon }_2(t_{j+1})-\tilde{ \varepsilon }_2(t_j),\, \hat{\xi }_{2,j}=\hat{\varepsilon }_2(t_{j+1})-\hat{ \varepsilon }_2(t_j)$, and $\tilde{\xi }_{n,2,j}=\sigma _{n,2,j}^{-1}\tilde{\xi }_{2,j},\, \hat{\xi }_{n,2,j}=\sigma _{n,2,j}^{-1}\hat{\xi }_{2,j}$. It follows that $\hat{\xi }_{2,j}={\mathbf{D}_{j,2}}^\mathrm{T}{{\varvec{\Lambda }}_{j,2}},\,j=0,\ldots ,N-1$, where

$$\begin{aligned} {\mathbf{D}_{j,2}}&= \left( -n^{-1/2}B_{j-1,2}(t_j) ,n^{-1/2}B_{j,2}(t_{j+1} )\right) ^\mathrm{T},\\ {{\varvec{\Lambda }}_{j,2}}&= \left( \begin{array}{c} n^{-1/2}\sum _{j^{\prime }=-1}^N\sum _{i=1}^n B_{j^{\prime },2}\left( X_i\right) \sigma \left( X_i\right) \varepsilon _is_{j-1,j^{\prime }}\\ [6pt] n^{-1/2}\sum _{j^{\prime }=-1}^N\sum _{i=1}^n B_{j^{\prime },2}\left( X_i\right) \sigma \left( X_i\right) \varepsilon _is_{j,j^{\prime }} \end{array}\right) . \end{aligned}$$

In the next lemma, we calculate the variance of $\hat{\xi }_{2,j}$.

Lemma 4

The variance of $\hat{\xi }_{2,j}$ is $\sigma ^2_{n,2,j}$ in Eq.  (6), which satisfies

$$\begin{aligned} \sigma _{n,2,j}^2=\sigma ^2(t_{j+1})\left( \frac{2f(t_{j+1})nh}{3}\right) ^{-1} {\varvec{\zeta }}_j^\mathrm{T}\mathbf{S}_{j} {\varvec{\zeta }}_j,\,j=0,\ldots ,N-1. \end{aligned}$$

And for large enough $n,\,c\left( nh\right) ^{-1/2}\le \sigma _{n,2,j} \le C \left( nh\right) ^{-1/2}$.

Proof

Since $\sigma ^2_{n,2,j}=E\hat{\xi }^2_{2,j}=\mathbf{D}_{j}^\mathrm{T}Cov({\varvec{\Lambda }}_j)\mathbf{D}_{j}$, by applying Lemma A.10 of Wang and Yang (2010), we can get the desired results.$\square$

Similar arguments used in Lemmas A.11 and A.12 of Wang and Yang (2010) yield that

$$\begin{aligned} \liminf _{n \rightarrow \infty } P \left[ \sup _{0 \le j \le N-1}\left| \hat{\xi }_{n,2,j} \right| \le 2\left\{ \log \left( N+1\right) \right\} ^{1/2}d_n\left( \frac{\alpha }{2} \right) \right] \ge 1-\alpha . \end{aligned}$$

Then we can finish the proof similarly as for $p=1$. $\square$