Complete consistency of estimators for regression models based on extended negatively dependent errors

Abstract

In this paper, we investigate the consistency of the estimators of nonparametric regression model and multiple linear regression model based on extended negatively dependent errors. The complete convergence rates of the estimators of nonparametric regression model are presented. In addition, the rth-mean consistency and complete consistency of the least squares estimator of the multiple linear regression model are obtained too. Finally, some examples and some simulations are illustrated.

Appendix

Lemma A

(cf. Wang et al. (2015), Lemma 2.2). Let $\{Z_n,n\ge 1\}$ be a sequence of END random variables such that $EZ_n=0$ and $|Z_n|\le d_n$ a.s. for each $n\ge 1$, where $\{d_n,n\ge 1\}$ is a sequence of positive constants. Assume that $t>0$ such that $t\max \limits _{1\le i\le n}d_i\le 1$. Then for every $\varepsilon >0$, there exists a constant $M>0$ such that

$$\begin{aligned} P\left( \left| \sum \limits _{i=1}^n Z_i\right| \ge \varepsilon \right) \le 2M \exp \left\{ -t\varepsilon +t^2\sum \limits _{i=1}^nEZ_i^2\right\} , \quad n\ge 1. \end{aligned}$$

(11)

Corollary A

Let $\{Z_n,n\ge 1\}$ be a sequence of END random variables such that $EZ_n=0$ and $|Z_n|\le d_n$ a.s. for each $n\ge 1$, where $\{d_n,n\ge 1\}$ is a sequence of positive constants. Denote $b_n=\max \limits _{1\le i\le n}d_i$ and $\Delta _n^2=\sum _{i=1}^n EZ_i^2$ for each $n\ge 1$. Then for every $\varepsilon >0$, there exists a constant $M>0$ such that

$$\begin{aligned} P\left( \left| \sum \limits _{i=1}^n Z_i\right| \ge \varepsilon \right) \le 2M\exp \left\{ -\frac{\varepsilon ^2}{2(2\Delta _n^2+b_n\varepsilon )}\right\} , \quad n\ge 1. \end{aligned}$$

(12)

Proof

Taking $t=\frac{\varepsilon }{2\Delta _n^2+b_n\varepsilon }$ in (11) of Lemma A, we immediately obtain (12). $\square$

Lemma B

(cf. Adler and Rosalsky (1987), Lemma 1, and Adler et al. (1989), Lemma 3). Let $\{Z_n,n\ge 1\}$ be a sequence of random variables, which is stochastically dominated by a nonnegative random variable Z. Then, for any $\alpha >0$ and $b>0$, the following two statements hold:

$$\begin{aligned} E\big [|Z_n|^{\alpha }I(|Z_n|\le b)\big ]\le C_1\left\{ E[Z^{\alpha }I(X\le b)]+b^{\alpha }P(Z>b)\right\} , \end{aligned}$$

$$\begin{aligned} E\big [|Z_n|^{\alpha }I(|Z_n|>b)\big ]\le C_2E\big [Z^{\alpha }I(Z>b)\big ], \end{aligned}$$

where $C_1$ and $C_2$ are positive constants. Consequently, it has $E|Z_n|^{\alpha }\le C_3EZ^{\alpha }$ for all $n\ge 1$.

Proof of Theorem 2.1:

In view of the proof of (4.5) in Yang et al. (2012), by the local Lipschitz condition of g(x) and the assumptions $(H_1)$–$(H_3)$, one can obtain that

$$\begin{aligned} |E\hat{g}_n(x)-g(x)|=O\left( n^{-\frac{1}{2r}}\log n\right) , \quad x\in A, \quad \text {as} \quad n\rightarrow \infty . \end{aligned}$$

(13)

With $x\in A$, to prove (6), we have to prove that

$$\begin{aligned} |\hat{g}_n(x)-E\hat{g}_n(x)|=\left| \sum \limits _{i=1}^n W_{ni}(x)\varepsilon _{ni}\right| =O\left( n^{-\frac{1}{2r}}\log n\right) , \quad \text {completely},\quad \text {as}\quad n\rightarrow \infty . \end{aligned}$$

(14)

In view of Lemma 3.1 of Liu (2010), for the fixed x, we can see that $\{W_{ni}^{+}(x)\varepsilon _i,1\le i\le n\}$ and $\{W_{ni}^{-}(x)\varepsilon _i,1\le i\le n\}$ are also sequences of END random variables. Noting that $W_{ni}(x)\varepsilon _i=W_{ni}^{+}(x)\varepsilon _i-W_{ni}^{-}(x)\varepsilon _i$, without loss of generality, we assume $W_{ni}(x)\ge 0$ in the proof. For all $n\ge 1$ and $1\le i\le n$, let

$$\begin{aligned} \varepsilon ^{(n)}_{1,i}= & {} -n^{\frac{1}{2r}}I\left( \varepsilon _{ni}<-n^{\frac{1}{2r}}\right) +\varepsilon _{ni}I\left( |\varepsilon _{ni}|\le n^{\frac{1}{2r}}\right) +n^{\frac{1}{2r}}I\left( \varepsilon _{ni}>n^{\frac{1}{2r}}\right) ,\\ \varepsilon ^{(n)}_{2,i}= & {} \varepsilon _{ni}-\varepsilon ^{(n)}_{1,i}=\left( \varepsilon _{ni}-n^{\frac{1}{2r}}\right) I\left( \varepsilon _{ni}>n^{\frac{1}{2r}}\right) +\left( \varepsilon _{ni}+n^{\frac{1}{2r}}\right) I\left( \varepsilon _{ni}<-n^{\frac{1}{2r}}\right) ,\\ \varepsilon _{1,i}(n)= & {} -n^{\frac{1}{2r}}I\left( \varepsilon _{i}<-n^{\frac{1}{2r}}\right) +\varepsilon _{i}I\left( |\varepsilon _{i}|\le n^{\frac{1}{2r}}\right) +n^{\frac{1}{2r}}I\left( \varepsilon _{i}>n^{\frac{1}{2r}}\right) ,\\ \varepsilon _{2,i}(n)= & {} \varepsilon _{i}-\varepsilon _{1,i}(n)=\left( \varepsilon _{i}-n^{\frac{1}{2r}}\right) I\left( \varepsilon _{i}>n^{\frac{1}{2r}}\right) +\left( \varepsilon _{i}+n^{\frac{1}{2r}}\right) I\left( \varepsilon _{i}<-n^{\frac{1}{2r}}\right) . \end{aligned}$$

Since $E\varepsilon _{ni}=E\varepsilon _i=0$ for $1\le i\le n$ and $n\ge 1$, it can be argued that

$$\begin{aligned} \hat{g}_n(x)-E\hat{g}_n(x)= & {} \sum \limits _{i=1}^n W_{ni}(x)\varepsilon _{ni}\nonumber \\= & {} \sum \limits _{i=1}^nW_{ni}(x)\left[ \varepsilon ^{(n)}_{1,i}-E\varepsilon ^{(n)}_{1,i}\right] +\sum \limits _{i=1}^n W_{ni}(x)\left[ \varepsilon ^{(n)}_{2,i}-E\varepsilon ^{(n)}_{2,i}\right] \nonumber \\:= & {} T_{n1}+T_{n2}. \end{aligned}$$

(15)

Obviously, $\{W_{ni}(x)(\varepsilon _{1,i}(n)-E\varepsilon _{1,i}(n))\}_{1\le i\le n}$ are END random variables with mean zero. From Lemma B, $(H_2)$ and $EZ^{2r+2}<\infty$ ($r\ge 1$), it follows

$$\begin{aligned}&\max \limits _{1\le i\le n}\left| W_{ni}(x)(\varepsilon _{1,i}(n)-E\varepsilon _{1,i}(n))\right| \le 2n^{\frac{1}{2r}}\max \limits _{1\le i\le n}|W_{ni}(x)|\le c_1n^{-\frac{1}{2r}},\\&\quad \sum \limits _{i=1}^n E\left[ W_{ni}(x)(\varepsilon _{1,i}(n)-E\varepsilon _{1,i}(n))\right] ^2\le \sum \limits _{i=1}^nW^2_{ni}(x) E\varepsilon ^2_{1,i}(n)\\&\qquad \le \sum \limits _{i=1}^nW^2_{ni}(x) E\varepsilon _{i}^2\le c_2EZ^2\max \limits _{1\le i\le n}|W_{ni}(x)|\sum \limits _{i=1}^n|W_{ni}(x)| \le c_3n^{-\frac{1}{r}}. \end{aligned}$$

Since that $(\varepsilon _{n1},\varepsilon _{n2},\ldots ,\varepsilon _{nn})$ has the same distribution as $(\varepsilon _{1},\varepsilon _{2},\ldots ,\varepsilon _{n})$ for each n, we apply Corollary A and obtain that for a sufficiently large $C>0$,

$$\begin{aligned}&\sum \limits _{n=1}^\infty P\left( |T_{n1}|\ge C n^{-\frac{1}{2r}}\log n\right) \nonumber \\&\quad =\sum \limits _{n=1}^\infty P\left( \left| \sum \limits _{i=1}^nW_{ni}(x)\left[ \varepsilon ^{(n)}_{1,i}-E\varepsilon ^{(n)}_{1,i}\right] \right| \ge Cn^{-\frac{1}{2r}}\log n\right) \nonumber \\&\quad =\sum \limits _{n=1}^\infty P\left( \left| \sum \limits _{i=1}^nW_{ni}(x)\left[ \varepsilon _{1,i}(n)-E\varepsilon _{1,i}(n)\right] \right| \ge C n^{-\frac{1}{2r}}\log n\right) \nonumber \\&\quad \le c_4\sum \limits _{n=1}^\infty \exp \left\{ -\frac{ C^2 n^{-\frac{1}{r}}\log ^2 n}{2\left( 2c_3n^{-\frac{1}{r}}+c_1 C n^{-\frac{1}{r}}\log n\right) }\right\} \nonumber \\&\quad \le c_5\sum \limits _{n=1}^\infty \frac{1}{n^{\frac{C}{3c_1}}}<\infty . \end{aligned}$$

(16)

Meanwhile, by $(H_2)$, Lemma B and $EZ^{2r+2}<\infty$, it yields

$$\begin{aligned}&\sum \limits _{n=2}^\infty P\left( |T_{n2}|\ge n^{-\frac{1}{2r}}\log n\right) \nonumber \\&\quad =\sum \limits _{n=2}^\infty P\left( \left| \sum \limits _{i=1}^n W_{ni}(x)\left[ \varepsilon ^{(n)}_{2,i}-E\varepsilon ^{(n)}_{2,i}\right] \right| \ge n^{-\frac{1}{2r}}\log n\right) \nonumber \\&\quad =\sum \limits _{n=2}^\infty P\left( \left| \sum \limits _{i=1}^n W_{ni}(x)\left[ \varepsilon _{2,i}(n)-E\varepsilon _{2,i}(n)\right] \right| \ge n^{-\frac{1}{2r}}\log n\right) \nonumber \\&\quad \le \sum \limits _{n=2}^\infty \frac{c_1}{\log n}n^{\frac{1}{2r}}\sum \limits _{i=1}^n |W_{ni}(x)|E\left( |\varepsilon _{i}|I\left( |\varepsilon _i|>n^{\frac{1}{2r}}\right) \right) \nonumber \\&\quad \le \sum \limits _{n=1}^\infty c_2n^{\frac{1}{2r}} E\left( ZI\left( Z>n^{\frac{1}{2r}}\right) \right) \nonumber \\&\quad =c_2\sum \limits _{n=1}^\infty n^{\frac{1}{2r}}\sum \limits _{k=n}^\infty E\left( ZI\left( k^{\frac{1}{2r}}<Z\le (k+1)^{\frac{1}{2r}}\right) \right) \nonumber \\&\quad =c_2\sum \limits _{k=1}^\infty E\left( ZI\left( k^{\frac{1}{2r}}<Z\le (k+1)^{\frac{1}{2r}}\right) \right) \sum \limits _{n=1}^kn^{\frac{1}{2r}}\nonumber \\&\quad \le c_3\sum \limits _{k=1}^\infty k^{\frac{1}{2r}+1} E\left( ZI\left( k^{\frac{1}{2r}}<Z\le (k+1)^{\frac{1}{2r}}\right) \right) \nonumber \\&\quad \le c_3\sum \limits _{k=1}^\infty E\left( Z^{2r+2}I\left( k^{\frac{1}{2r}}<Z\le (k+1)^{\frac{1}{2r}}\right) \right) \nonumber \\&\quad \le c_3EZ^{2r+2}<\infty . \end{aligned}$$

(17)

Consequently, by (15), (16), (17), (14) is completely proved. The desired result (6) follows from (13) and (14) immediately. $\square$

Proof of Corollary 2.1

It suffices to show that the conditions of Theorem 2.1 are satisfied. For every $x\in [0,1]$, we can argue by definition of $R_i(x)$ and choice of $x_{ni}$, $k_n=\lceil n^{1/r}\rceil$ and (7) that

$$\begin{aligned} \sum \limits _{i=1}^n \tilde{W}_{ni}(x)=\sum \limits _{i=1}^n W_{nR_i(x)}(x)=\sum \limits _{i=1}^{k_n} \frac{1}{k_n}=1, \end{aligned}$$

$$\begin{aligned} \max \limits _{1\le i\le n}\tilde{W}_{ni}(x)=\frac{1}{k_n}\le c_1n^{-\frac{1}{r}}. \end{aligned}$$

For $r>3/2$, we have by taking $l>1/(2r-3)$ that

$$\begin{aligned}&\sum \limits _{i=1}^n \tilde{W}_{ni}(x)I\left( |x_{ni}-x|>n^{-\frac{1}{2r}}\log n\right) \le \sum \limits _{i=1}^n \tilde{W}_{ni}(x)\frac{|x_{ni}-x|^l}{|n^{-\frac{1}{2r}}\log n|^l}\\&\quad =\sum \limits _{i=1}^{k_n} \frac{1}{k_n}\frac{\left| x^{(n)}_{R_i(x)}-x\right| ^l}{\log ^{l}n}n^{\frac{l}{2r}}\le \sum \limits _{i=1}^{k_n} \frac{1}{k_n}\frac{\big (\frac{i}{n}\big )^l}{\log ^{l}n}n^{\frac{l}{2r}}\\&\quad \le \frac{\big (\frac{k_n}{n}\big )^l}{\log ^{l}n}n^{\frac{l}{2r}}\le \frac{c_1}{n^{l(1-\frac{3}{2r})}\log ^{l}n} \le \frac{c_2}{n^{\frac{1}{2r}}\log ^{l} n}. \end{aligned}$$

Since that g(x) satisfies the Lipschitz condition for $x\in [0,1]$, it can be seen that the assumptions of $(H_1)$–$(H_3)$ are satisfied. Consequently, Corollary 2.1 follows from Theorem 2.1 immediately. $\square$

Proof of Theorem 2.2:

Obviously, it has $(b_{ni}^{(j)})\varepsilon _i=\big (b_{ni}^{(j)}\big )^{+}\varepsilon _i-\big (b_{ni}^{(j)}\big )^{-}\varepsilon _i$. Furthermore, $\left\{ (b_{ni}^{(j)})^{+}\varepsilon _i,1\le i\le n\right\}$ and $\left\{ (b_{ni}^{(j)})^{-}\varepsilon _i,1\le i\le n\right\}$ are also END random variables. Without loss of generality, we assume $b_{ni}^{(j)}\ge 0$ in this proof too. On the one hand, for $r\ge 2$ and $j\in \{1,2,\ldots p\}$, by $EZ^r<\infty$, (5), $b_n^{(j)}\rightarrow \infty$, Corollary 3.2 of Shen (2011) and Lemma B, we obtain that

$$\begin{aligned} E|\hat{\beta }_{nj}-\beta _{j}|^r= & {} E\left| \sum \limits _{i=1}^n\frac{b_{ni}^{(j)}\varepsilon _i}{b_n^{(j)}}\right| ^r =\frac{1}{(b_{n}^{(j)})^{r}}E\left| \sum \limits _{i=1}^nb_{ni}^{(j)}\varepsilon _i\right| ^r\nonumber \\\le & {} \frac{C_1}{(b_{n}^{(j)})^{r}}\left\{ \sum \limits _{i=1}^n|b_{ni}^{(j)}|^rE|\varepsilon _i|^r +\left( \sum \limits _{i=1}^n(b_{ni}^{(j)})^2E\varepsilon _i^2\right) ^{r/2}\right\} \nonumber \\\le & {} \frac{C_2EZ^r}{(b_{n}^{(j)})^{r}}\left\{ \sum \limits _{i=1}^n|b_{ni}^{(j)}|^r +\left( \sum \limits _{i=1}^n(b_{ni}^{(j)})^2\right) ^{r/2}\right\} \nonumber \\\le & {} \frac{C_3EZ^r}{(b_{n}^{(j)})^{r}}\left( \sum \limits _{i=1}^n(b_{ni}^{(j)})^2\right) ^{r/2} =\frac{C_3EZ^r}{(b_{n}^{(j)})^{r/2}}\rightarrow 0 \quad as \quad n\rightarrow \infty ,\qquad \end{aligned}$$

(18)

where the third inequality uses the fact that $\left( \sum _{i=1}^n a_i^\alpha \right) ^{1/\alpha }\ge \left( \sum _{i=1}^n a_i^\beta \right) ^{1/\beta }$ for any positive number sequence $\{a_i,1\le i\le n\}$ and $1\le \alpha \le \beta$. Therefore, (9) follows from (18).

On the other hand, by Markov’s inequality, $\sum \nolimits _{n=1}^{\infty }(b_n^{(j)})^{-r/2}<\infty$ and (18), one obtains that for any $\lambda >0$,

$$\begin{aligned} \sum \limits _{n=1}^\infty P(|\hat{\beta }_{nj}-\beta _{j}|>\lambda )\le \sum \limits _{n=1}^\infty \frac{E|\hat{\beta }_{nj}-\beta _{j}|^r}{\lambda ^r}\le \frac{C_1}{\lambda ^r}\sum \limits _{n=1}^\infty \frac{1}{(b_{n}^{(j)})^{r/2}}<\infty . \end{aligned}$$

Thus, (10) is completely proved. $\square$