Abstract
Consider the linear regression model
where \(e_{i}=g(\ldots,\varepsilon_{i-1},\varepsilon_{i})\) are general dependence errors. The Bahadur representations of M-estimators of the parameter β are given, by which asymptotically the theory of M-estimation in linear regression models is unified. As applications, the normal distributions and the rates of strong convergence are investigated, while \(\{\varepsilon_{i},i\in{Z}\}\) are m-dependent, and the martingale difference and \((\varepsilon,\psi)\)-weakly dependent.
Similar content being viewed by others
1 Introduction
Consider the following linear regression model:
where \(\beta=(\beta_{1},\ldots,\beta_{p})^{T}\in{R^{p}}\) is an unknown parametric vector, \(x_{i}^{T}\) denotes the ith row of an \(n\times{p}\) design matrix X, and \(\{e_{i}\}\) are stationary dependence errors with a common distribution.
An M-estimate of β is defined as any value of β minimizing
for a suitable choice of the function ρ, or any solution for β of the estimating equation
for a suitable choice of ψ.
There is a body of statistical literature dealing with linear regression models with independent and identically distributed (i.i.d.) random errors, see e.g. Babu [1], Bai et al. [2], Chen [7], Chen and Zhao [8], He and Shao [24], Gervini and Yohai [23], Huber and Ronchetti [28], Xiong and Joseph [50], Salibian-Barrera et al. [44]. Recently, linear regression models with serially correlated errors have attracted increasing attention from statisticians; see, for example, Li [33], Wu [49], Maller [38], Pere [41], Hu [25, 26]. Over the last 40 years, M-estimators in linear regression models have been investigated by many authors. Let \(\{\eta_{i}\}\) be i.i.d. random variables. Koul [30] discussed the asymptotic behavior of a class of M-estimators in the model (1.1) with long range dependence errors \(e_{i}=G(\eta_{i})\). Wu [49] and Zhou and Shao [52] discussed the model (1.1) with \(e_{i}=G(\ldots,\eta_{i-1},\eta_{i})\) and derived strong Bahadur representations of M-estimators and a central limit theorem. Zhou and Wu [53] considered the model (1.1) with \(e_{i}=\sum_{j=0}^{\infty}{a}_{j}\eta_{i-j}\), and obtained some asymptotic results including consistency of robust estimates. Fan et al. [20] investigated the model (1.1) with the errors \(e_{i}=f(e_{i-1})+\eta_{i}\) and established the moderate deviations and strong Bahadur representations for M-estimators. Wu [47] discussed strong consistency of an M-estimator in the model (1.1) for negatively associated samples. Fan [19] considered the model (1.1) with φ-mixing errors, and the moderate deviations for the M-estimators. In addition, Berlinet et al. [4], Boente and Fraiman [5], Chen et al. [6], Cheng et al. [9], Gannaz [22], Lô and Ronchetti [37], Valdora and Yohai [45] and Yang [51] have also studied some asymptotic properties of M-estimators in nonlinear models. However, no people have investigated a unified the theory of M-estimation in linear regression models with more general errors.
In this paper, we assume that
where \(g(\cdot)\) is a measurable function such that \(e_{i}\) is a proper random variable, and \(\{\varepsilon_{i},i\in{Z}\}\) (where Z is the set of integers) are very general random variables, including m-dependent, martingale difference, \((\varepsilon,\psi)\)-weakly dependent, and so on.
We try to investigate the unified the theory of M-estimation in the linear regression model. In the article, we use the idea of Wu [49] to study the Bahadur representative of M-estimator, and we extend some results to general errors. The paper is organized as follows. In Sect. 2, the weak and strong linear representation of an M-estimate of the vector regression parameter β in the model (1.1) are presented. Section 3 contains some applications of our results, including the m-dependent, \((\varepsilon,\psi)\)-weakly dependent, martingale difference. In Sect. 4, proofs of the main results are given.
2 Main results
In the section, we investigate the weak and strong linear representation of an M-estimate of the vector regression parameter β in the model (1.1). Without loss of generality, we assume that the true parameter \(\beta=0\). We start with some notation and assumptions.
For a vector \(v=(v_{1},\ldots,v_{p})\), let \(|v|=(\sum_{i=1}^{p}{v_{i}}^{2})^{\frac{1}{2}}\). A random vector V is said to be in \(L^{q},q>0\), if \(E(|V|^{q})<\infty\). Let \(\Vert V \Vert _{q}=E(|V|^{q})^{\frac{1}{q}}\), \(\Vert V \Vert = \Vert V \Vert _{2}\), \(\Sigma_{n}=\sum_{i=1}^{n}{x_{i}}x_{i}^{T}=X^{T}X\) and assume that \(\Sigma_{n}\) is positive definite for large enough n. Let \(x_{in}=\Sigma_{n}^{-\frac{1}{2}}x_{i},\beta_{n}=\Sigma_{n}^{-\frac{1}{2}}\beta \). Then the model (1.1) can be written as
with \(\sum_{i=1}^{n}{x_{in}}x_{in}^{T}=I_{p}\), where \(I_{p}\) is an identity matrix of order p. Assume that ρ has derivative ψ. For \(l\geq0\) and a function f, write \(f\in{C^{l}}\) if f has derivatives up to lth order and \(f^{(l)}\) is continuous. Define the function
where \(\mathcal{F}_{i}^{*}=(\ldots,\varepsilon_{-1},\varepsilon '_{0},\varepsilon_{1},\ldots,\varepsilon_{i-1},\varepsilon_{i}), \mathcal{F}_{i}=(\ldots,\varepsilon_{-1},\varepsilon_{0},\varepsilon _{1},\ldots,\varepsilon_{i-1},\varepsilon_{i})\), let \(\varepsilon'_{i}\) be an i.i.d. copy of \(\varepsilon_{i}\), and \(e_{k}^{*}=g(\mathcal{F}_{k}^{*})\).
Throughout the paper, we use the following assumptions.
-
(A1)
\(\rho(\cdot)\) is a convex function, \(E\psi(e_{i})=0,0< E\psi^{2}(e_{i})\).
-
(A2)
\(\varphi(t)\equiv{E}\psi(e_{i}+t)\) has a strictly positive derivative at \(t=0\).
-
(A3)
\(m(t)\equiv{E}|\psi(e_{i}+t)-\psi(e_{i})|^{2}\) is continuous at \(t=0\).
-
(A4)
\(r_{n}\equiv\max_{1\leq{i}\leq{n}}|x_{in}|=\max_{1\leq {i}\leq{n}}(x_{i}^{T}\Sigma_{n}^{-1}x_{i})^{\frac{1}{2}}=o(1)\).
-
(A5)
There exists a \(\delta_{0}>0\) such that
$$ L_{i}\equiv\sup_{ \vert s \vert , \vert t \vert \leq\delta_{0},s\neq{t}}\frac{ \vert \psi _{i+1}(s;\mathcal{F}_{i})-\psi_{i+1}(t;\mathcal{F}_{i}) \vert }{ \vert s-t \vert } \in{L^{1}}. $$(2.3) -
(A6)
Let \(\psi_{i}(\cdot;\mathcal{F}_{i-1})\in{C^{l}},l\geq0\). For some \(\delta_{0}>0,\max_{1\leq{i}\leq{n}}\sup_{|\delta|\leq\delta _{0}} \Vert \psi_{i}^{(l)}(\delta;\mathcal{F}_{i-1}) \Vert <\infty\) and
$$ \sum_{i=0}^{\infty}\sup_{ \vert \delta \vert < \delta_{0}} \bigl\Vert E\bigl(\psi _{i}^{(l)}(\delta; \mathcal{F}_{i-1})|\mathcal{F}_{0}\bigr)-E\bigl(\psi _{i}^{(l)}\bigl(\delta;\mathcal{F}_{i-1}^{*}\bigr)| \mathcal{F}_{0}^{*}\bigr) \bigr\Vert < \infty. $$(2.4) -
(A7)
$$\begin{aligned} &\sum_{i=0}^{\infty}\sup_{ \vert \delta \vert < \delta_{0}} \bigl\Vert E\bigl(\psi _{i}^{(l)}\bigl(\delta; \mathcal{F}_{i-1}^{*}\bigr)|\mathcal{F}_{0}\bigr)-E\bigl(\psi _{i}^{(l)}\bigl(\delta;\mathcal{F}_{i-1}^{*}\bigr)| \mathcal{F}_{-1}\bigr) \bigr\Vert < \infty, \end{aligned}$$(2.5)$$\begin{aligned} &\sum_{i=0}^{\infty}\sup_{ \vert \delta \vert < \delta_{0}} \bigl\vert E\psi ^{(l)}(e_{i}+\delta)-E\psi^{(l)} \bigl(e_{i}^{*}+\delta\bigr) \bigr\vert < \infty. \end{aligned}$$(2.6)
Remark 1
Conditions (A1)–(A5) and (A6) are imposed in the M-estimation considering the theory of linear regression models with dependent errors (Wu [49]; Zhou and Shao [52]). Condition (2.6) is similar to (7) of Wu [49]. \(\Vert E(\psi_{i}^{(l)}(\delta;\mathcal {F}_{i-1})|\mathcal{F}_{0})-E(\psi_{i}^{(l)}(\delta;\mathcal {F}_{i-1}^{*})|\mathcal{F}_{0}^{*}) \Vert \) measures the difference of the contribution of \(\varepsilon_{0}\) and its copy \(\varepsilon'_{0}\) in predicting \(\psi(e_{i}+\delta)\). However, \(\Vert E(\psi_{i}^{(l)}(\delta;\mathcal{F}_{i-1}^{*})|\mathcal{F}_{0})-E(\psi _{i}^{(l)}(\delta;\mathcal{F}_{i-1}^{*})|\mathcal{F}_{-1}) \Vert \) measures the contribution of \(\varepsilon_{0}\) in predicting \(\psi(e_{i}+\delta)\) under the given copy of \(\varepsilon_{0}\): \(\varepsilon'_{0}\).
If \(\{\varepsilon_{i}\}\) are i.i.d., then (A6) and (A7) hold. For the other settings, (A6) and (A7) are very easily satisfied. The following proposition provides some sufficient conditions for (A6) and (A7).
Proposition 2.1
Let \(F_{i}(u|\mathcal{F}_{0})=P(e_{i}\leq{u}|\mathcal{F}_{0})\) and \(f_{i}(u|\mathcal{F}_{0})\) be the conditional distribution and density function of \(e_{i}\) at u given \(\mathcal{F}_{0}\), respectively. Let \(f_{i}(u)\) and \(f_{i}^{*}(u)\) be the density function of \(e_{i}\) and \(e_{i}^{*}\), respectively.
-
(1)
Let \(f_{i}(\cdot|\mathcal{F}_{i})\in{C^{l}},l\geq0\), \(\omega(i)=\int_{R} \Vert f_{i}(u|\mathcal{F}_{0})-f_{i}(u|\mathcal{F}_{0}^{*}) \Vert \psi(u;\delta_{0})\,du\) and \(\psi(u;\delta_{0})=|\psi(u+\delta_{0})|+|\psi(u-\delta_{0})|\). If \(\sum_{i=1}^{\infty}\omega(i)<\infty\), then (A6) holds.
-
(2)
Let
$$\overline{\omega}(i)= \int_{R} \bigl\Vert f_{i}^{(l)}(u| \mathcal {F}_{0})-f_{i}^{(l)}\bigl(u| \mathcal{F}_{0}^{*}\bigr) \bigr\Vert \psi(u;\delta_{0})\,du $$and \(\tilde{\omega}(i)=\int_{R}|f_{i}(u)-f_{i}^{*}(u)|\psi^{(l)}(u;\delta_{0})\,du\). If \(\sum_{i=1}^{\infty}\overline{\omega}(i)<\infty\) and \(\sum_{i=1}^{\infty}\tilde{\omega}(i)<\infty\), then assumption (A7) holds.
Proof
(1) By the conditions of (1), we have
Namely (A6) holds.
(2) (A7) follows from
and
Hence, the proposition is proved. □
Define the M-processes
where
Theorem 2.1
Let \(\{\delta_{n},n\in{N}\}\) be a sequence of positive numbers such that \(\delta_{n}\rightarrow\infty\) and \(\delta _{n}{r_{n}}\rightarrow0\). If (A1)–(A5), and (A6) and (A7) with \(l=0,1,\ldots,p\) hold, then
where
Corollary 2.1
Assume that (A1)–(A5), and (A6) and (A7) with \(l=0,1,\ldots,p\) hold. If \(\varphi(t)=t\varphi'(0)+O(t^{2})\) as \(t\rightarrow0\), \(\Omega(\hat{\beta}_{n})=O_{p}(r_{n})\), then, for \(|\hat{\beta}_{n}|\leq\delta_{n}\),
Moreover, if, as \(t\rightarrow0\), \(m(t)=O(|t|^{\lambda})\) for some \(\lambda>0\), then
Remark 2
If \(\{e_{i}\}\) i.i.d., then \(|\hat{\beta}_{n}|\leq\delta_{n}\) follows from (3.2) of Rao and Zhao [42]. If \(\{\varepsilon_{i}\}\) i.i.d., then \(|\hat{\beta}_{n}|\leq\delta_{n}\) follows from Theorem 1 of Wu [49] and Zhou and Shao [52]. If \(e_{i}=f(e_{i-1})+\varepsilon_{i}\), where the function \(f:R\times{R}\rightarrow{R}\) satisfies some condition and \(\{\varepsilon_{i}\}\) i.i.d., then \(|\hat{\beta}_{n}|\leq\delta _{n}\) follows from Theorem 2.2 of Fan et al. [20]. If \(\{\varepsilon_{i}\} \) NA, then \(|\hat{\beta}_{n}|\leq\delta_{n}\) follows from Theorem 1 of Wu [47]. Therefore the condition \(|\hat{\beta}_{n}|\leq\delta_{n}\) is not strong. In the paper, we do not discuss it.
Theorem 2.2
Assume that (A1)–(A3), (A5), and (A6) and (A7) with \(l=0,1,\ldots,p\) hold. Let \(\lambda_{n}\) be the minimum eigenvalue of \(\Sigma_{n}\), \(b_{n}=n^{-\frac{1}{2}}(\log{n})^{{3}/{2}}(\log{\log {n}})^{{1}/{2}+\upsilon}\), \(\upsilon>0\), \(\tilde{n}=2^{\lceil{\log {n}}/{\log{2}}\rceil}\) and \(q>\frac{3}{2}\). If \(\liminf_{n\rightarrow\infty}\lambda_{n}/n>0, \sum_{i=1}^{n}|x_{i}|^{2}=O(n)\) and \(\tilde{r}_{n}=\max_{1\leq{i}\leq{n}}|x_{i}|=O(n^{1/2}(\log{n})^{-2})\), then
where \(L_{n}=\sqrt{\tilde{\tau}_{n}(2b_{n})}(\log_{n})^{q}\), \(B_{\tilde{n}}=b_{n}(\sum_{i=1}^{n}|x_{i}|^{4})^{{1}/{2}}(\log{n})^{3/2}(\log\log {n})^{(1+\upsilon)/2}\) and
Corollary 2.2
Assume that \(\varphi(t)=t\varphi'(0)+O(t^{2})\) and \(m(t)=O(\sqrt{t})\) as \(t\rightarrow0\), and \(\tilde{\Omega}_{n}=O_{a.s.}(\tilde{r}_{n})\). Under the conditions of Theorem 2.2, we have:
-
(1)
\(\tilde{\beta}_{n}=O_{a.s.}(b_{n})\);
-
(2)
\(\varphi'(0)\Sigma_{n}\tilde{\beta}_{n}-\sum_{i=1}^{n}\psi(e_{i})x_{i} =O_{a.s.}(L_{\tilde{n}}+B_{\tilde{n}}+b_{n}^{2}\sum_{i=1}^{n}|x_{i}|^{3}+\tilde{r}_{n})\),
where \(\tilde{\beta}_{n}\) is the minimizer of (1.2).
Remark 3
From the above results, we easily obtain the corresponding conclusions of Wu [49].
From the corollary below, we only derive convergence rates of \(\tilde{\beta}_{n}\). However, it is to be regretted that we cannot give laws of the iterated logarithm \(n^{1/2}(\log\log{n})^{1/2}\), which is still an open problem.
Corollary 2.3
Under the conditions of Corollary 2.2, we have
Proof
Note that \(\tilde{n}=2^{\lceil\log{n}/\log2\rceil}=O(n)\) and \(m(t)=O(\sqrt{t})\) as \(t\rightarrow0\); we have
and
By Corollary 2.2, we have
and
3 Applications
In the following three subsections, we shall investigate some applications of our results. In Sect. 3.1, we consider that \(\varepsilon _{i}\) is a m-dependent random variable sequence. We shall investigate that \(\{\varepsilon_{i}\}\) are \((\varepsilon,\psi)\)-weakly dependent in Sect. 3.2, and martingale difference errors \(\{\varepsilon_{i}\}\) in Sect. 3.3.
3.1 m-dependent process
In the subsection, we shall firstly show that the m-dependent sequence satisfies conditions (A6) and (A7) and secondly obtain the asymptotic normal distribution and strong convergence rates for M-estimators of the parameter. Koul [30] discussed the asymptotic behavior of a class of M-estimators in the model (1.1) with long range dependence errors \(e_{i}=g(\varepsilon_{i})\), where \(\varepsilon_{i}\) i.i.d. Here we assume that \(\varepsilon_{i}\) is a m-dependent sequence, of which the definition was given by Example 2.8.1 in Lehmann [32]. For m-dependent sequences or processes, there are some results (e.g., see Hu et al. [27], Romano and Wolf [43] and Valk [46]).
Proposition 3.1
Let \(\varepsilon_{i}\) in (1.4) be a m-dependent sequence. Then (A6) and (A7) hold.
Proof
Note that \(\varepsilon_{i}\) is a m-dependent sequence, we have
and
Therefore, (A6) and (A7) follow from (3.1), (3.2) and \(E\psi ^{(l)}(e_{i}+\delta)=E\psi^{(l)}(e_{i}^{*}+\delta)\). □
Corollary 3.1
Assume that (A1)–(A5) hold. If \(\varphi(t)=t\varphi'(0)+O(t^{2})\) and \(m(t)=O(|t|^{\lambda})\) for some \(\lambda>0\) as \(t\rightarrow0\), \(\Omega (\hat{\beta}_{n})=0\) and \(0<\sigma_{\psi}^{2}=E[\psi(e_{i})]^{2}<\infty\), then
In order to prove Corollary 3.1, we give the following lemmas.
Lemma 3.1
(Lehmann [32])
Let \(\{\xi_{i},i\geq1\}\) be a stationary m-dependent sequence of random variables with \(E\xi_{i}=0\) and \(0<\sigma ^{2}=\operatorname{Var}(\xi_{i})<\infty\), and \(T_{n}=\sum_{i=1}^{n}\xi_{i}\). Then
where \(\tau^{2}=\lim_{n\rightarrow\infty}\operatorname{Var}(n^{-1/2}T_{n})=\sigma ^{2}+2\sum_{i=2}^{m+1}\operatorname{Cov}(\xi_{1},\xi_{i})\).
Using the argument of Lemma 3.1, we easily obtain the following result. Here we omit the proof.
Lemma 3.2
Let \(\{\xi_{i},i\geq1\}\) be a stationary m-dependent sequence of random variables with \(E\xi_{i}=0\) and \(0<\sigma_{i}^{2}=\operatorname{Var}(\xi _{i})<\infty\), and \(T_{n}=\sum_{i=1}^{n}\xi_{i}\). Then
where \(\tau^{2}=\operatorname{Var}(n^{-1/2}T_{n})=n^{-1}\sum_{i=1}^{n}\sigma _{i}^{2}+2n^{-1}\sum_{i=2}^{m+1}(n-i)\operatorname{Cov}(\xi_{1},\xi_{i})\).
Proof of Corollary 3.1
By (2.10), we have
Since \(\{\xi_{i},i\geq1\}\) is a stationary m-dependent sequence, so is \(\{[\varphi'(0)]^{-1}\psi({e}_{i}){x}_{in},i\geq1\}\). Let \(u\in{R}^{p}\), \(|u|=1\). Then \(E (u^{T}[\varphi'(0)]^{-1}\psi({e}_{i}){x}_{in} )=0\) and
Therefore, by \(r_{n}=o(1)\) and \(0<\sigma_{\psi}^{2}={E}[\psi({e}_{i})]^{2}<\infty \), we have
Thus the corollary follows from Lemma 3.2, (3.3) and (3.4). □
Corollary 3.2
Assume that (A1)–(A5) hold. If \(\varphi(t)=t\varphi '(0)+O(t^{2})\) and \(m(t)=O(\sqrt{t})\) as \(t\rightarrow0\), and \(\tilde{\Omega}_{n}(\tilde{\beta}_{n})=O_{a.s.}(\tilde{r}_{n}),0<\sigma_{\psi}^{2}={E}[\psi ({e}_{i})]^{2}<\infty\), then
Proof
The corollary follows from Proposition 3.1 and Corollary 2.2. □
3.2 \((\varepsilon,\psi)\)-weakly dependent process
In the subsection, we assume that \(\{\varepsilon_{i}\}\) are \((\varepsilon ,\psi)\)-weakly dependent (Doukhan and Louhichi [14] and Dedecker et al. [11]) random variables. In 1999, Doukhan and Louhichi proposed a new idea of \((\varepsilon,\psi)\)-weakly dependence which focuses on covariance rather than the total variation distance between joint distributions and the product of the corresponding marginal. It has been shown that this concept is more general than mixing and includes, under natural conditions on the process parameters, essentially all classes of processes of interest in statistics. Therefore, many researchers are interested in the \((\varepsilon,\psi)\)-weakly dependent and related possesses, and one obtained lots of sharp results. For example, Doukhan and Louhichi [14], Dedecker and Doukhan [10], Dedecker and Prieur [12], Doukhan and Neumann [16], Doukhan and Wintenberger [17], Bardet et al. [3], Doukhan and Wintenberger [18], Doukhan et al. [13]. However, a few people (only Hwang and Shin [29], Nze et al. [40]) investigated regression models with \((\varepsilon,\psi)\)-weakly dependent errors. Nobody has investigated a robust estimate for the regression model with \((\varepsilon,\psi )\)-weakly dependent errors. To give the definition of the \((\varepsilon,\psi)\)-weakly dependence, let us consider a process \(\xi=\{\xi_{n},n\in{Z}\}\) with values in a Banach space \((\mathcal{E}, \Vert \cdot \Vert )\). For \(h:\mathcal{E}^{u}\rightarrow {R}\), \(u\in{N}\), we define the Lipschitz modulus of h,
where we have the \(l_{1}\)-norm, i.e., \(\Vert (y_{1},y_{2},\ldots,y_{u}) \Vert _{1}=\sum_{i=1}^{u}|y_{i}|\).
Definition 1
(Doukhan and Louhich [14])
A process \(\xi=\{\xi _{n},n\in{Z}\}\) with values in \(R^{d}\) is called a \((\varepsilon,\psi )\)-weakly dependent process if, for some classes of functions \(\mathcal {E}^{u},\mathcal{E}^{v}\rightarrow{R},F_{u},G_{v}\):
as \(r\rightarrow\infty\).
According to the definition, mixing sequences (\(\alpha,\rho,\beta ,\varphi\)-mixing), associated sequences (positively or negatively associated), Gaussian sequences, Bernoulli shifts and Markovian models or time series bootstrap processes with discrete innovations are \((\varepsilon,\psi)\)-weakly dependent (Doukhan et al. [15]).
From now on, assume that the classes of functions contain functions bounded by 1. Distinct functions Ψ yield \(\eta,\theta,\kappa\) and a λ weak dependence of the coefficients as follows (Doukhan et al. [15]):
In Corollary 3.3, we only consider λ and η-weakly dependence. Let \(\{\varepsilon_{i}\}\) be λ or η-weakly dependent, and assume that g satisfies: for each \(s\in{Z}\), if \(x,y\in {R^{Z}}\) satisfy \(x_{i}=y_{i}\) for each index \(i\neq{s}\)
Lemma 3.3
(Dedecker et al. [11])
Assume that g satisfies the condition (3.7) with \(l\geq0\) and some sequence \(b_{s}\geq0\) such that \(\sum_{s}|s|b_{s}<\infty\). Assume that \(E|\varepsilon_{0}|^{m'}<\infty \) with \(lm< m'\) for some \(m>2\). Then:
-
(1)
If the process \(\{\varepsilon_{i},i\in{Z}\}\) is λ-weakly dependent with coefficients \(\lambda_{\varepsilon}(r)\), then \(e_{n}\) is λ-weakly dependent with coefficients
$$ \lambda_{e}(k)=c\inf_{r\leq[k/2]}\biggl(\sum _{i\geq{r}}b_{i}\biggr)\vee \bigl[(2r+1)^{2} \lambda_{\varepsilon}(k-2r)^{\frac{m'-1-l}{m'-1+l}}\bigr]. $$(3.8) -
(2)
If the process \(\{\varepsilon_{i},i\in{Z}\}\) is η-weakly dependent with coefficients \(\eta_{\varepsilon}(r)\), then \(e_{n}\) is η-weakly dependent and there exists a constant \(c>0\) such that
$$ \eta_{e}(k)=c\inf_{r\leq[k/2]}\biggl(\sum _{i\geq{r}}b_{i}\biggr)\vee \bigl[(2r+1)^{1+\frac{1}{m'-1}} \eta_{\varepsilon}(k-2r)^{\frac{m'-2}{m'-1}}\bigr]. $$
Lemma 3.4
(Bardet et al. [3])
Let \(\{\xi_{n},n\in{Z}\}\) be a sequence of \(R^{k}\)-valued random variables. Assume that there exists some constant \(C>0\) such that \(\max_{1\leq{i}\leq{k}} \Vert \xi_{i} \Vert _{p}\leq{C},p\geq1\). Let h be a function from \(R^{k}\) to R such that \(h(0)=0\) and for \(x,y\in{R^{k}}\), there exist a in \([1,p]\) and \(c>0\) such that
Now we define the sequence \(\{\zeta_{n},n\in{Z}\}\) by \(\zeta_{n}=h(\xi_{n})\). Then:
(1) If the process \(\{\xi_{i},i\in{Z}\}\) is λ-weakly dependent with coefficients \(\lambda_{\xi}(r)\), then \(\{\zeta_{n},n\in{Z}\}\) is also with coefficients
(2) If the process \(\{\xi_{i},i\in{Z}\}\) is ζ-weakly dependent with coefficients \(\eta_{\xi}(r)\), so is \(\{\zeta_{n},n\in{Z}\}\) with coefficients \(\eta_{\zeta}(r)=O(\eta_{\xi}^{\frac{p-a}{p-1}}(r))\).
Lemma 3.5
(Dedecker et al. [11])
Let \(\{\xi_{i},i\in{Z}\}\) be a centered and stationary real-valued sequence with \(E|\xi_{0}|^{2+\varsigma }<\infty\), \(\varsigma>0\), \(\sigma^{2}=\sum_{k\in{Z}}\operatorname{Cov}(\xi_{0},\xi _{k})\) and \(S_{n}=\sum_{i=1}^{n}\xi_{i}\). If \(\lambda_{\xi}(r)=O(r^{-\lambda})\) for \(\lambda>4+2/\varsigma\), then \(n^{-1/2}S_{n}\rightarrow{N}(0,\sigma^{2})\) as \(n\rightarrow\infty\).
Corollary 3.3
Let \(\{\varepsilon_{i}\}\) be λ-weakly dependent with coefficients \(\lambda_{\varepsilon}(r)=O(\exp(-r\lambda))\) for some \(\lambda>0\), and \(b_{i}=O(\exp(-ib))\) for some \(b>0\). Assume that \(\psi(0)=0\), and, for \(x,y\in{R}\), there exists a constant \(c>0\) such that
Under the conditions of Corollary 2.1, we have
where \(\Sigma=\sum_{i=1}^{n}{x}_{1n}\operatorname{Cov}(\psi(e_{1}),\psi(e_{i}))x_{in}^{T}\).
Proof
Note that \(\{\varepsilon_{i}\}\) is λ-weakly dependent. By Lemma 3.3, we find that \(\{e_{i}\}\) is λ-weakly dependent with coefficients
from (3.8) and Proposition 3.1 in Chap. 3 (Dedecker et al. [11]).
Let \(u\in{R}^{p}\), \(|u|=1\), and \(\zeta_{i}=h(e_{i})=u\psi(0)x_{in}=0\). Then \(h(0)=0=u\psi(0)x_{in}=0\). Choose \(p=2,a=1\), in (3.9), and by (3.11), we have
for \(x, y\in{R}\) and \(c>0\). Therefore, by Lemma 3.4, \(\{\zeta_{i},i\in{N}\}\) is λ-weakly dependent with coefficients
By Corollary 2.1, we have
By (3.13) and (3.15), there exist \(b>0,a>0,l\geq0\) and \(m'>lm\) for some \(m>2\) such that
for enough large r and \(\lambda>4+2/\varsigma\) with \(\varsigma>0\).
By Lemma 3.5 and (3.16)–(3.17), we have
where \(\sigma^{2}=\sum_{i=1}^{n}u^{T}x_{1n}\operatorname{Cov}(\psi(e_{1}),\psi (e_{i}))x_{in}^{T}u\). Using the Cramer device, we complete the proof of Corollary 3.3. □
Lemma 3.6
(Dedecker et al. [11])
Suppose that \(\{\xi_{i},1\leq {i}\leq{n}\}\) are stationary real-valued random variables with \(E\xi _{i}=0\) and \(P(|\xi_{i}|\leq{M}<\infty)=1\) for all \(i=1,2,\ldots,n\). Let \(\Psi:N^{2}\rightarrow{N}\) be one of the following functions:
for some \(0<\alpha<1\). We assume that there exist constants \(K,L_{1},L_{2}<\infty,\mu\geq0\) and a nonincreasing sequence of real coefficients \(\{\rho(n),n\geq0\}\) such that, for all u-tuples \((s_{1},\ldots,s_{u})\) and all v-tuples \((t_{1},\ldots,t_{v})\) with \(1\leq {s_{1}}\leq\cdots\leq{s_{u}}\leq{t_{1}}\leq\cdots\leq{t_{v}}\leq{n,}\) the following inequality is fulfilled:
where
Let \(S_{n}=\sum_{i=1}^{n}\xi_{i}\) and \(\sigma_{n}^{2}=\operatorname{Var}(\sum_{i=1}^{n}\xi_{i})\). If \(\sigma^{2}=\lim_{n\rightarrow\infty}\sigma_{n}^{2}/n>0\), then
Corollary 3.4
Let \(\{\varepsilon_{i}\}\) be η-weakly dependent with coefficients \(\eta_{\varepsilon}(r)=O(\exp(-r\eta))\) for some \(\eta >0\), and \(b_{i}=O(\exp(-ib))\) for some \(b>0\). Assume that \(\psi(0)=0\) and (3.11) hold. Under the conditions of Corollary 2.2 with \(\tilde {r}_{n}=O(n^{1/2}(\log{n})^{-2})\) replaced by \(0<\min_{1\leq{i}\leq {n}}|x_{ij}|<\max_{1\leq{i}\leq{n}}|x_{ij}|<\infty\), and \(0<\sigma_{\psi}^{2}=E\psi^{2}(e_{i})<\infty\), we have:
-
(1)
for \(3/2< q\leq7/4, \Sigma_{n}\tilde{\beta}_{n}=O_{a.s.}(nb_{n})=O_{a.s.}(n^{1/2}(\log{n})^{3/2}(\log\log {n})^{1/2+\upsilon})\);
-
(2)
for \(q\geq7/4, \Sigma_{n}\tilde{\beta}_{n}=O_{a.s.}(n^{1/2}(\log {n})^{-1/4+q}(\log\log{n})^{1/4+2/\upsilon})\).
Proof
Let \(\xi_{i}=\psi(e_{i})x_{ij},j=1,\ldots,p\). Then for \(\forall \mu_{n}\rightarrow\infty\) as \(n\rightarrow\infty\)
Therefore, there exists some \(0< M<\infty\) such that
Similar to the proofs of (3.13) and (3.15), we easily obtain
where
Let \(\Psi(u,v)=u+v,K^{2}={r}_{n}^{uv}M^{1(u+v-2)}\) and
Thus (3.19) holds. Since \(\lim_{s\rightarrow\infty}\ln (s+1)/s=0\), there exist \(b>0,\eta>0,l\geq0\) and \(m'>lm\) for some \(m>2\) and \(m'>2\) such that
Thus
By Lemma 3.6 and Corollary 2.3, we have
Therefore, by Corollary 2.3, (3.23) and (3.31), we complete the proof of Corollary 3.4. □
3.3 Linear martingale difference processes
In the subsection, we will investigate martingale difference errors \(\{ \varepsilon_{i}\}\). We shall provide some sufficient conditions for (A6) and (A7) and give the central limit theorem and strong convergence rates.
Let \(\{\varepsilon_{i}\}\) be a martingale difference sequence, and \(a_{j}\) be real numbers such that \(e_{i}=\sum_{j=0}^{\infty}a_{j}\varepsilon _{i-j}\) exists. It is well known that the theory of martingales provides a natural unified method for dealing with limit theorems. Under its influence, there is great interest in the martingale difference. Liang and Jing [34] were concerned with the partial linear model under the linear com of martingale differences and obtained asymptotic normality of the least squares estimator of the parameter. Nelson [39] has given conditions for the pointwise consistency of weighted least squares estimators from multivariate regression models with martingale difference errors. Lai [31] investigated stochastic regression models with martingale difference sequence errors and obtained strong consistency and asymptotic normality of the least squares estimate of the parameter.
Let \(F_{\varepsilon}\) be the distribution function of \(\varepsilon_{0}\) and let \(f_{\varepsilon}\) be its density.
Proposition 3.2
Suppose that \(E\varepsilon_{0}=0,\varepsilon_{0}\in L^{4/(2-\gamma)}\), \(\kappa_{\gamma}=\int_{R}\psi^{2}(u)\omega_{-\gamma }(du)<\infty,1<\gamma<2\) and \(\sum_{k=0}^{p}\int_{R}|f_{\varepsilon}^{(k)}(v)|^{2}\omega_{\gamma}(dv)<\infty\), where \(\omega_{\gamma}(dv)=(1+|v|)^{\gamma}\). If \(\sum_{j=0}^{\infty}|a_{j}|<\infty\), then \(\sum_{i=0}^{\infty}\omega(i)<\infty,\sum_{i=0}^{\infty}\bar{\omega}(i)<\infty\) and \(\sum_{i=0}^{\infty}\tilde{\omega}(i)<\infty\).
Proof
Let \(Z_{n}=\sum_{j=0}^{\infty}a_{j}\varepsilon _{n-j},Z_{n}^{*}=Z_{n}-a_{n}\varepsilon_{0}-a_{n}\varepsilon'\), and
where \(U_{n}=Z_{n}-a_{n}\varepsilon_{0}\). By the Schwartz inequality, we have
Note that
and
Let \(I_{k}=\int_{R}[f^{(k)}_{\varepsilon}(v)]^{2}\omega_{\gamma}(dv)\). By the Schwartz inequality, we have
By \(\sup_{j} E\varepsilon_{j}^{2}<\infty\) and Chatterji’s inequality (Lin and Bai [35]), we have
By (3.33)–(3.37) and the Schwartz inequality, we have
Note that \(\sum_{j=0}^{\infty}|a_{j}|<\infty\) implies \(\sum_{j=0}^{\infty}a_{j}^{2}<\infty\) and \(\sum_{j=0}^{\infty}|a_{j}|^{1+\gamma/2}<\infty\), and by (3.33) and (3.39), we have
The general case \(k\geq1\) similarly follows. Similar to the proof of (3.39), we easily prove the other results. □
From Propositions 2.1 and 3.2, (A6) and (A7) hold. Hence, we can obtain the following two corollaries from Corollaries 2.1 and 2.2. In order to prove the following two corollaries, we first give some lemmas.
Lemma 3.7
(Liptser and Shiryayev [36])
Let \(\xi=(\xi_{k})_{-\infty < k<\infty}\) be a strictly stationary sequence on a probability space \((\Omega,\mathcal{F},P)\), and \(\mathcal{G}\) be a σ-algebra of invariant sets of the sequence ξ and \(\mathcal{F}_{k}=\sigma(\ldots ,\xi_{k-1},\xi_{k})\). For a certain \(p\geq2\), let \(E|\xi_{0}|^{p}<\infty\) and \(\sum_{k\geq1}\gamma_{k}(p)<\infty\), where \(\gamma_{k}(p)=\{E|E(\xi _{k}|F_{0})|^{\frac{p}{p-1}}\}^{\frac{p-1}{p}}\). Then
where the random variable Z has the characteristic function \(E\exp (-\frac{1}{2}\lambda^{2}\sigma^{2})\), and \(\sigma^{2}=E(\xi_{0}^{2}|\mathcal {G})+2\sum_{k\geq1}E(\xi_{0}\xi_{k}|\mathcal{G})\).
Corollary 3.5
Assume that (A1)–(A5) hold, \(\varphi(t)=t\varphi '(0)+O(t^{2})\) and \(m(t)=O(|t|^{\lambda})\) for some \(\lambda>0\) as \(t\rightarrow0\), \(\Omega(\hat{\beta}_{n})=O_{p}(r_{n})\). Under the conditions of Proposition 3.2, \(E|\psi(e_{k})|^{\frac{p}{p-1}}<\infty,p\geq2\) and \(\sum_{k=1}^{n}|x_{kn}|<\infty\), we have
where the random variable Z has the characteristic function \(E\exp (-\frac{1}{2}\lambda^{2}\sigma^{2})\), and \(\sigma^{2}=(\varphi '(0))^{-2}x_{1n}^{T}x_{1n}E(\psi^{2}(e_{1})|\mathcal{G})+2(\varphi ^{\prime}(0))^{-2}x_{1n}^{T}\sum_{k\geq2}x_{kn}E(\psi(e_{1})\psi (e_{k})|\mathcal{G})\).
Proof
By Proposition 2.1, Proposition 3.2 and Corollary 2.1, we have
By \(E|\psi(e_{k})|^{\frac{p}{p-1}}<\infty\) and \(\sum_{k=1}^{n}|x_{kn}|<\infty\), we have
and
By Proposition 2.1, Proposition 3.2 and Corollary 2.2, we easily obtain the following result. Here we omit the proof. □
Corollary 3.6
Assume that (A1)–(A5) hold, \(\varphi(t)=t\varphi '(0)+O(t^{2})\) and \(m(t)=O(\sqrt {t})\) as \(t\rightarrow0\), \(\tilde{\Omega}_{n}(\tilde{\beta}_{n})=O_{a.s.}(\tilde{r}_{n})\). Under the conditions of Proposition 3.2, we have
4 Proofs of the main results
For the proofs of Theorem 2.1 and Theorem 2.2, we need some lemmas as follows.
Lemma 4.1
(Freedman [21])
Let τ be a stopping time, and K a positive real number. Suppose that \(P\{|\xi_{i}|\leq K,i\leq\tau\}=1\), where \(\{\xi_{i}\}\) are measurable random variables and \(E(\xi_{i}|\mathcal {F}_{i-1})=0\). Then, for all positive real numbers a and b,
Lemma 4.2
Let
Assume that (A5) and (A6) hold. Then
Proof
Note that \(p=\sum_{i=1}^{n} x_{in}^{T}x_{in}\leq(\max_{1\leq i\leq n}|x_{in}|)^{2}n=nr_{n}^{2}\), and \(\delta_{n}r_{n}\rightarrow 0\), we have \(\delta_{n}=o(n^{1/2})\). For any positive sequence \(\mu_{n}\rightarrow\infty\), let
and
By the monotonicity of ψ and \(\delta\geq0\), we have
By (4.3), the \(c_{r}\)-inequality and (A3), we have
Thus
By the Chebyshev inequality,
Similarly,
Let \(x_{in}=(x_{i1n},\ldots,x_{ipn})^{T}=(x_{i1},\ldots ,x_{ip})^{T},D_{x}(i)=(2\times1_{x_{i1}\geq0}-1,\ldots,2\times1_{x_{ip}\geq 0}-1)\in\Pi_{p},\Pi_{p}=\{-1,1\}^{p}\). For \(d\in\Pi_{p},j=1,2,\ldots,p\), define
Since \(M_{n}(\beta_{n})=\sum_{d\in\Pi_{p}}(M_{n,1,d}(\beta_{n}),\ldots ,M_{n,p,d}(\beta_{n}))^{T}\), it suffices to prove that Lemma 4.2 holds with \(M_{n}(\beta_{n})\) replaced by \((M_{n,j,d}(\beta_{n})\).
Let \(|\beta_{n}|\leq\delta_{n},\eta_{i,j,d}(\beta_{n})=(\psi(e_{i}-x_{in}^{T}\beta _{n})-\psi(e_{i}))x_{ij}1_{D_{x}(i)=d}\) and
Note that
By (4.9), for large enough n, we have
Let the projections \(\mathcal{P}_{k}(\cdot)=E(\cdot|\mathcal{F}_{k})-E(\cdot |\mathcal{F}_{k-1})\). Since
Note that \(\{\mathcal{P}_{i}(\eta_{i,j,d}(\beta_{n})1_{|\eta_{i,j,d}(\beta _{n})|\leq t_{n}})\}\) are bound martingale differences. By Lemma 4.1 and (4.10), for \(|\beta_{n}|\leq t_{n}\), we have
Let \(l=n^{8}\) and \(K_{l}=\{(k_{1}/l,\ldots,k_{p}/l):k_{i}\in Z,|k_{i}|\leq n^{9}\}\). Then \(\# K_{l}=(2n^{9}+1)^{p}\), where the symbol # denotes the number of elements of the set \(K_{l}\). It is easy to show
By (4.12) and (4.13), for \(\forall\varsigma>1\), we have
By (4.5), (4.6) and (4.14), we have
For a, let \(\langle a\rangle_{l,-1}=\lceil a\rceil_{l}=\lceil al\rceil /l\) and \(\langle a\rangle_{l,1}=\lfloor a\rfloor_{l}=\lfloor al\rfloor /l\). For a vector \(\beta_{n}=(\beta_{1n},\ldots,\beta_{pn})^{T}\), let \(\langle\beta_{n}\rangle_{l,d}=(\langle\beta_{1n}\rangle_{l,d_{1}},\ldots ,\langle\beta_{pn}\rangle_{l,d_{p}})\).
By (A5), for \(|s|,|t|\leq r_{n}\delta_{n}\) and large n, we have
Let \(V_{n}=\sum_{i=1}^{n}L_{i-1}\). By condition (A5), the Markov inequality and \(L_{i}\in L^{1}\), we have
Note that \(|\beta_{n}-\langle\beta_{n}\rangle_{l,d}|\leq Cl^{-1}\), which implies \(\max_{1\leq i\leq n}|x_{in}^{T}(\beta_{n}-\langle\beta _{n}\rangle_{l,d})|=o(l^{-1})\). Thus
Without loss of generality, assume that \(j=1\) in the following proof.
Let \(d=(1,-1,1,\ldots,1)\). Then \(\langle\beta_{n}\rangle_{l,d}=(\lfloor \beta_{1n}\rfloor_{l},\lceil\beta_{2n}\rceil_{l},\lfloor\beta_{3n}\rfloor _{l},\ldots,\lfloor\beta_{pn}\rfloor_{l})\) and \(\langle\beta_{n}\rangle _{l,-d}=(\lceil\beta_{1n}\rceil_{l},\lfloor\beta_{2n}\rfloor_{l},\lceil \beta_{3n}\rceil_{l},\ldots,\lceil\beta_{pn}\rceil_{l})\). Since ψ is nondecreasing,
Note that
Namely
Therefore
Note that \(l^{-1}V_{n}=O_{p}(n^{-8}n^{4})=O_{p}(n^{-4})\), (4.2) immediately follows from (4.15) and (4.19). □
Lemma 4.3
Assume that the processes \(X_{t}=g(\mathcal{F}_{t})\in L^{2}\). Let \(g_{n}(\mathcal{F}_{0})=E(g(\mathcal{F}_{n})|\mathcal{F}_{0}),n\geq 0\). Then
where \(R= \Vert E[g_{n}(\mathcal{F}_{0}^{*})|\mathcal{F}_{-1}]-E[g_{n}(\mathcal {F}_{0}^{*})|\mathcal{F}_{0}] \Vert \).
Proof
Since
we have
By the Jensen inequality, we have
That is,
Note that
and
By (4.24), (4.25) and the Jensen inequality, we have
□
Remark 4
If \(\{\varepsilon_{i}\}\) i.i.d., then \(R=0\). In this case, the above lemma becomes Theorem 1 of Wu [48].
Lemma 4.4
Let \(\{\delta_{n},n\in N\}\) be a sequence of positive numbers such that \(\delta_{n}\rightarrow\infty\) and \(\delta_{n} r_{n}\rightarrow0\). If (A6)–(A7) hold, then
where
Proof
Let \(I=\{n_{1},\ldots,n_{q}\}\in\{1,2,\ldots,p\}\) be a nonempty set and \(1\leq n_{1}<\cdots<n_{q}\), and \(u_{I}=(u_{1}1_{1\in I},\ldots ,u_{p}1_{p\in I})\), with vector \(u=(u_{1},\ldots,u_{p})\). Write
In the following, we will prove that
uniformly over \(|u|\leq p\delta_{n}\).
In fact, let
and
Then \(T_{n}=\sum_{k=0}^{\infty}J_{k}\), and \(J_{k}\) are martingale differences. By the orthogonality of martingale differences and the stationarity of \(\{e_{i}\}\), and Lemma 4.3, we have
By Lemma 4.3, \(\psi_{i}(\cdot;\mathcal{F}_{i-1})\in C^{l},l\geq0\) and the \(c_{r}\)-inequality, for \(k\geq0\), we have
where
Note that \(E\psi^{(q)}(e_{i}+\delta)=\frac{d^{q}E\psi (e_{i}+t)}{dt^{q}}|_{t=\delta}\), we have
By the conditions (A6), (A7) and (4.29)–(4.31), we have
Let \(|u|\leq p\delta_{n}\). By \(\max_{1\leq i\leq n}|x_{in}u|\leq p\delta_{n} r_{n}\rightarrow0\). Note that \(\delta_{n}\rightarrow\infty\) and \(\delta_{n} r_{n}\rightarrow0\). By (4.28), we have
Since
the result (4.27) follows from (4.32) and (4.33). □
Lemma 4.5
Let \(\pi_{i},i\geq1\) be a sequence of bounded positive numbers, and let there exist a constant \(c_{0}\geq1\) such that \(\max_{1\leq i\leq2^{d}}\pi_{i}\leq c_{0}\min_{1\leq i\leq2^{d}}\pi _{i}\) holds for all large n. And let \(\omega_{d}=2c_{0}\pi_{2^{d}}\) and \(q>3/2\). Assume that (A5) and \(\tilde{r}_{n}=O(\sqrt{n})\) hold. Then as \(d\rightarrow\infty\)
where \(\tilde{M}_{n}(\beta)=\sum_{i=1}^{n}\{\psi(e_{i}-x_{i}^{T}\beta )-E(\psi(e_{i}-x_{i}^{T}\beta)|\mathcal{F}_{i-1})\}x_{i}\).
Proof
Let
and
Since \(q>3/2\) and \(2(q-1)>1,\sum_{d=2}^{\infty}(\mu_{2^{d}}^{-1}\log \mu_{2^{d}})^{2}<\infty\). By the argument of Lemma 4.2 and the Borel–Cantelli lemma, we have
Similar to the proof of (4.12), we have
Let \(l=n^{8d}\) and \(K_{l}=\{(k_{1}/l,\ldots,k_{p}/l):k_{i}\in Z,|k_{i}|\leq n^{9d}\}\). Then \(\#K_{l}=(2n^{9d}+1)^{p}\). By (4.34) and (4.35), for \(\forall\varsigma>1\), we have
Therefore,
Since \(\tilde{r}_{n}=O(\sqrt{n})\) and \(\max_{1\leq i\leq 2^{d}}|x_{i}^{T}(\beta-\langle\beta\rangle_{l,d})|=O(2^{2^{d}}l^{-1}),Cl^{-1}V\) in (4.17) can be replaced by \(Cl^{-1}2^{2^{d}}V\), and the lemma follows from \(P(V_{2^{d}}\geq2^{5d},\mathrm{i.o}.)=0\). □
Lemma 4.6
Let \(\pi_{i},i\geq1\) be a sequence of bounded positive numbers, and let there exist a constant \(c_{0}\geq1\) such that \(\max_{1\leq i\leq2^{d}}\pi\leq c_{0}\min_{1\leq i\leq2^{d}}\pi _{i}\) and \(\pi_{n}=o(n^{-1/2}(\log n)^{2})\) hold for all large n. And let \(\omega_{d}=2c_{0}\pi_{2^{d}}\). Assume that (A6), (A7) and \(\tilde{r}_{n}=O(\sqrt {n}(\log n)^{-2})\) hold. Then
and, as \(d\rightarrow\infty\), for any \(\upsilon>0\),
where \(\tilde{N}_{n}(\beta)=\sum_{i=1}^{n}\{\psi_{1}(-x_{i}^{T}\beta ;\mathcal{F}_{i-1})-\varphi(-x_{i}^{T}\beta)\}x_{n}\).
Proof
Let \(Q_{n,j}(\beta)=\sum_{i=1}^{n}\psi_{1}(-x_{i}^{T}\beta ;F_{i-1})x_{ij},i\leq j\leq p\), and
Note that
It is easy to see that the argument in the proof of Lemma 4.4 implies that there exists a positive constant \(C<\infty\) such that
holds uniformly over \(1\leq n'< n\leq2^{d}\). Therefore (4.38) holds.
Let \(\Lambda=\sum_{r=0}^{d}\mu_{r}^{-1}\), where
For a positive integer \(k\leq2^{d}\), write its dyadic expansion \(k=2^{r_{1}}+\cdots+2^{r_{j}}\), where \(0\leq r_{j}<\cdots<r_{1}\leq d\), and \(k(i)=2^{r_{1}}+\cdots+2^{r_{i}}\). By the Schwartz inequality, we have
Thus
Since \(\upsilon>0\) and \(\omega_{d}^{2q}\sum_{i=1}^{2^{d}}|x_{i}|^{2+2q}=O(\omega_{d}^{2}\sum_{i=1}^{2^{d}}|x_{i}|^{4})\), (4.42) implies that
By the Borel–Cantelli lemma, (4.39) follows from (4.46). □
Lemma 4.7
Under the conditions of Theorem 2.2, we have:
-
(1)
\(\sup_{|\beta|\leq b_{n}}|\tilde{K}_{n}(\beta)-\tilde{K}_{n}(0)|=O_{a.s.}(L_{\tilde{n}}+B_{\tilde{n}})\);
-
(2)
for and \(\upsilon>0,\tilde{K}_{n}(0)=O_{a.s.}(h_{n})\), where \(h_{n}=n^{1/2}(\log n)^{3/2}(\log\log n)^{1/2+\upsilon/4}\).
Proof
Observe that \(\tilde{K}_{n}(\beta)=\tilde{M}_{n}(\beta)+\tilde{N}_{n}(\beta)\). Since \(n^{-5/2}=o(B_{\tilde{n}})\), (1) follows from Lemma 4.5 and 4.6. □
As with the argument in (4.29), we have \(\tilde{K}_{n}(0)=O(\sqrt{n})\).
Proof of Theorem 2.1
Observe that
By (4.47), Lemma 4.2 and Lemma 4.4, we have
This completes the proof of Theorem 2.1. □
Proof of Corollary 2.1
Take an arbitrary sequence \(\delta _{n}\rightarrow\infty\), which satisfies the assumption of Theorem 2.1. Note that
and
for \(|\hat{\beta}_{n}|\leq\delta_{n}\). By Theorem 2.1 and (4.49), we have
By (4.52), \(\varphi(t)=t\varphi'(0)+O(t^{2})\) as \(t\rightarrow0\), and \(\sum_{i=1}^{n}x_{in}x_{in}^{T}=I_{p}\), we have
and
Namely
By \(m(t)=O(|t|^{\lambda})\ (t\rightarrow0)\) for some \(\lambda>0\), we have
Then it follows from (4.53) and (4.54) that
for any \(\delta_{n}\rightarrow\infty\), which implies
□
Proof of Theorem 2.2
By Lemma 4.7, we have Theorem 2.2. □
Proof of Corollary 2.2
(1) By Lemma 4.7, we have
where \(b_{n}=n^{-1/2}(\log n)^{3/2}(\log\log n)^{1/2+\upsilon}\). Let
and
Note that
It is easy to show that \(b_{n}^{3}\sum_{i=1}^{n}|x_{i}|^{3}=O(n\tilde{r}_{n})b_{n}^{3}=o(nb_{n}^{2})\). By \(\varphi(t)=t\varphi'(0)+O(t^{2})\), we have
By \(m(t)=O(\sqrt{n})\) as \(t\rightarrow0\), we have \((L_{\tilde{n}}+B_{\tilde{n}}+h_{n})b_{n}=o(nb_{n}^{2})\). Thus
By the convexity of the function \(\Theta_{n}(\cdot)\), we have
Therefore the minimizer \(\hat{\beta}_{n}\) satisfies \(\hat{\beta}_{n}=O_{a.s.}(b_{n})\).
(2) Let \(|\hat{\beta}_{n}|\leq b_{n}\). By a Taylor expansion, we have
Therefore (2) follows from Theorem 2.2 and (1). □
References
Babu, G.J.: Strong representations for LAD estimators in linear models. Probab. Theory Relat. Fields 83, 547–558 (1989)
Bai, Z.D., Rao, C.R., Wu, Y.: M-estimation of multivariate linear regression parameters under a convex discrepancy function. Stat. Sin. 2, 237–254 (1992)
Bardet, J., Doukhan, P., Lang, G., Ragache, N.: Dependent Lindeberg central limit theorem and some applications. ESAIM Probab. Stat. 12, 154–172 (2008)
Berlinet, A., Liese, F., Vaida, I.: Necessary and sufficient conditions for consistency of M-estimates in regression models with general errors. J. Stat. Plan. Inference 89, 243–267 (2000)
Boente, G., Fraiman, R.: Robust nonparametric regression estimation for dependent observations. Ann. Stat. 17(3), 1242–1256 (1989)
Chen, J., Li, D.G., Zhang, L.X.: Bahadur representation of nonparametric M-estimators for spatial processes. Acta Math. Sin. Engl. Ser. 24(11), 1871–1882 (2008)
Chen, X.: Linear representation of parametric M-estimators in linear models. Sci. China Ser. A 23(12), 1264–1275 (1993)
Chen, X., Zhao, L.: M-methods in Linear Model. Shanghai Scientific & Technical Publishers, Shanghai (1996)
Cheng, C.L., Van Ness, J.W.: Generalized m-estimators for errors-in-variables regression. Ann. Stat. 20(1), 385–397 (1992)
Dedecker, J., Doukhan, P.: A new covariance inequality and applications. Stoch. Process. Appl. 106, 63–80 (2003)
Dedecker, J., Doukhan, P., Lang, G., Leon, J.R., Louhichi, S., Prieur, C.: Weak Dependence: With Examples and Applications. Springer, New York (2007)
Dedecker, J., Prieur, C.: New dependence coefficients, examples and applications to statistics. Probab. Theory Relat. Fields 132, 203–236 (2005)
Doukhan, P., Klesov, O., Lang, G.: Rates of convergence in some SLLN under weak dependence conditions. Acta Sci. Math. (Szeged) 76, 683–695 (2010)
Doukhan, P., Louhichi, S.: A new weak dependence condition and applications to moment inequalities. Stoch. Process. Appl. 84, 313–342 (1999)
Doukhan, P., Mayo, N., Truquet, L.: Weak dependence, models and some applications. Metrika 69, 199–225 (2009)
Doukhan, P., Neumann, M.H.: Probability and moment inequalities for sums of weakly dependent random variables with applications. Stoch. Process. Appl. 117, 878–903 (2007)
Doukhan, P., Wintenberger, O.: An invariance principle for weakly dependent stationary general models. Probab. Math. Stat. 27(1) (2007)
Doukhan, P., Wintenberger, O.: Weakly dependent chains with infinite memory. Stoch. Process. Appl. 118, 1997–2013 (2008)
Fan, J.: Moderate deviations for M-estimators in linear models with ϕ-mixing errors. Acta Math. Sin. Engl. Ser. 28(6), 1275–1294 (2012)
Fan, J., Yan, A., Xiu, N.: Asymptotic properties for M-estimators in linear models with dependent random errors. J. Stat. Plan. Inference 148, 49–66 (2014)
Freedman, D.A.: On tail probabilities for martingales. Ann. Probab. 3(1), 100–118 (1975)
Gannaz, I.: Robust estimation and wavelet thresholding in partially linear models. Stat. Comput. 17, 293–310 (2007)
Gervini, D., Yohai, V.J.: A class of robust and fully efficient regression estimators. Ann. Stat. 30(2), 583–616 (2002)
He, X., Shao, Q.: A general Bahadur representation of M-estimators and its application to linear regression with nonstochastic designs. Ann. Stat. 24(8), 2608–2630 (1996)
Hu, H.C.: QML estimators in linear regression models with functional coefficient autoregressive processes. Math. Probl. Eng. 2010, Article ID 956907 (2010) https://doi.org/10.1155/2010/956907
Hu, H.C.: Asymptotic normality of Huber–Dutter estimators in a linear model with AR(1) processes. J. Stat. Plan. Inference 143(3), 548–562 (2013)
Hu, Y., Ming, R., Yang, W.: Large deviations and moderate deviations for m-negatively associated random variables. Acta Math. Sci. 27B(4), 886–896 (2007)
Huber, P.J., Ronchetti, E.M.: Robust Statistics, 2nd edn. John Wiley & Sons, New Jersey (2009)
Hwang, E., Shin, D.: Semiparametric estimation for partially linear regression models with ψ-weak dependent errors. J. Korean Stat. Soc. 40, 411–424 (2011)
Koul, H.L.: M-estimators in linear regression models with long range dependent errors. Stat. Probab. Lett. 14, 153–164 (1992)
Lai, T.L.: Asymptotic properties of nonlinear least squares estimates in stochastic regression models. Ann. Stat. 22(4), 1917–1930 (1994)
Lehmann, E.L.: Elements of Large-Sample Theory. Springer, New York (1998)
Li, I.: On Koul’s minimum distance estimators in the regression models with long memory moving averages. Stoch. Process. Appl. 105, 257–269 (2003)
Liang, H., Jing, B.: Asymptotic normality in partial linear models based on dependent errors. J. Stat. Plan. Inference 139, 1357–1371 (2009)
Lin, Z., Bai, Z.: Probability Inequalities. Science Press, Beijing (2010)
Liptser, R.S., Shiryayev, A.N.: Theory of Martingale. Kluwer Academic Publishers, London (1989)
Lô, S.N., Ronchetti, E.: Robust and accurate inference for generalized linear models. J. Multivar. Anal. 100, 2126–2136 (2009)
Maller, R.A.: Asymptotics of regressions with stationary and nonstationary residuals. Stoch. Process. Appl. 105, 33–67 (2003)
Nelson, P.I.: A note on strong consistency of least squares estimators in regression models with martingale difference errors. Ann. Stat. 8(5), 1057–1064 (1980)
Nze, P.A., Bühlmann, P., Doukhan, P.: Weak dependence beyond mixing and asymptotics for nonparametric regression. Ann. Stat. 30(2), 397–430 (2002)
Pere, P.: Adjusted estimates and Wald statistics for the AR(1) model with constant. J. Econom. 98, 335–363 (2000)
Rao, C.R., Zhao, L.C.: Linear representation of M-estimates in linear models. Can. J. Stat. 20(4), 359–368 (1992)
Romano, J.P., Wolf, M.: A more general central limit theorem for m-dependent random variables with unbounded m. Stat. Probab. Lett. 47, 115–124 (2000)
Salibian-Barrera, M., Aelst, S.V., Yohai, V.J.: Robust tests for linear regression models based on τ-estimates. Comput. Stat. Data Anal. 93, 436–455 (2016) https://doi.org/10.1016/j.csda
Valdora, M., Yohai, V.J.: Robust estimators for generalized linear models. J. Stat. Plan. Inference 146, 31–48 (2014)
Valk, V.D.: Hilbert space representations of m-dependent processes. Ann. Probab. 21(3), 1550–1570 (1993)
Wu, Q.: Strong consistency of M estimator in linear model for negatively associated samples. J. Syst. Sci. Complex. 19, 592–600 (2006)
Wu, W.B.: Nonlinear system theory: another look at dependence. Proc. Natl. Acad. Sci. USA 102(40), 14150–14154 (2005)
Wu, W.B.: M-estimation of linear models with dependent errors. Ann. Stat. 35(2), 495–521 (2007)
Xiong, S., Joseph, V.R.: Regression with outlier shrinkage. J. Stat. Plan. Inference 143, 1988–2001 (2013)
Yang, Y.: Asymptotics of M-estimation in non-linear regression. Acta Math. Sin. Engl. Ser. 20(4), 749–760 (2004)
Zhou, Z., Shao, X.: Inference for linear models with dependent errors. J. R. Stat. Soc. Ser. B 75(2), 323–343 (2013)
Zhou, Z., Wu, W.B.: On linear models with long memory and heavy-tailed errors. J. Multivar. Anal. 102, 349–362 (2011)
Acknowledgements
The author’s work was supported by the National Natural Science Foundation of China (No. 11471105, 11471223), and the Natural Science Foundation of Hubei Province (No. 2016CFB526).
Author information
Authors and Affiliations
Contributions
The author organized and wrote this paper. Further he examined all the steps of the proofs in this paper. The author read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The author declares to have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Hu, H. Bahadur representations of M-estimators and their applications in general linear models. J Inequal Appl 2018, 123 (2018). https://doi.org/10.1186/s13660-018-1715-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13660-018-1715-x