Large deviations for randomly weighted least squares estimator in a nonlinear regression model

Abstract

In this work, we introduce the random weighting method to the nonlinear regression model and study the asymptotic properties for the randomly weighted least squares estimator with dependent errors. The results reveal that this new estimator is consistent. Moreover, some simulations are also carried out to show the performance of the proposed estimator.

Appendix

Lemma A.1

Let $(W_{1},\cdots ,W_{n})\sim Dir(\alpha _{1},\cdots ,\alpha _{n})$ for each $n\ge 1$ and $\{\xi _{n},n\ge 1\}$ is a sequence of nonnegative END random variables. Then $\{W_{n}\xi _{n},n\ge 1\}$ is still a sequence of END random variables.

Proof

It follows from Example 5.4 of Block et al. (1982) that $\{W_{n},n\ge 1\}$ is a sequence of nonnegative NOD random variables, which is independent of $\{\xi _{n},n\ge 1\}$. Hence, for any real numbers $z_1,\cdots ,z_n$, we have by Definition 1.2 and the properties of NOD random variables (see in Bozorgnia et al. (1996) or Lemmas 1.1 and 1.2 of Wang et al. (2011) for example) that

$$\begin{aligned}{} & {} P\left( W_1\xi _1 \le z_1, \cdots ,W_n\xi _n\le z_n\right) \\{} & {} \quad =\int \cdots \int I\left( w_1 y_1 \le z_1, \cdots ,w_n y_n\le z_n\right) dF_{W_1,\cdots ,W_n,\xi _1,\cdots ,\xi _n}\left( w_1,\cdots ,w_n,y_1,\cdots ,y_n\right) \\{} & {} \quad =\int \cdots \int I\left( w_1 y_1 \le z_1, \cdots ,w_n y_n\le z_n\right) dF_{W_1,\cdots ,W_n}\left( w_1,\cdots ,w_n\right) dF_{\xi _1,\cdots ,\xi _n}\left( y_1,\cdots ,y_n\right) \\{} & {} \quad =\int \cdots \int P\left( w_1 \xi _1 \le z_1, \cdots ,w_n \xi _n\le z_n\right) dF_{W_1,\cdots ,W_n}\left( w_1,\cdots ,w_n\right) \\{} & {} \quad \le M\int \cdots \int P(w_1 \xi _1 \le z_1)\cdots P(w_n \xi _n\le z_n)dF_{W_1,\cdots ,W_n}\left( w_1,\cdots ,w_n\right) \\{} & {} \quad = ME\left[ F_{\xi _1}\left( \frac{z_1}{W_1}\right) \cdots F_{\xi _n}\left( \frac{z_n}{W_n}\right) \right] \le ME\left[ F_{\xi _1}\left( \frac{z_1}{W_1}\right) \right] \cdots E\left[ F_{\xi _n}\left( \frac{z_n}{W_n}\right) \right] \\{} & {} \quad =M\int \int I(w_1 y_1 \le z_1)dF_{W_1}(w_1)dF_{\xi _1}(y_1)\cdots \int \int I(w_n y_n \le z_n)dF_{W_n}(w_n)dF_{\xi _n}(y_n)\\{} & {} \quad =M\int \int I(w_1 y_1 \le z_1)dF_{W_1,\xi _1}(w_1,y_1)\cdots \int \int I(w_n y_n \le z_n)dF_{W_n,\xi _n}(w_n,y_n)\\{} & {} \quad =MP\left( W_1\xi _1 \le z_1)\cdots P(W_n\xi _n\le z_n\right) . \end{aligned}$$

Similarly, we also have that

$$\begin{aligned} P\left( W_1\xi _1>z_1, \cdots ,W_n\xi _n> z_n\right) \le M P\left( W_1\xi _1>z_1)\cdots P(W_n\xi _n>z_n\right) . \end{aligned}$$

Therefore, $\{W_{n}\xi _{n},n\ge 1\}$ is still a sequence of END random variables by Definition 1.2 again. $\square$

Lemma A.2

Let $\{a_{ni},1\le i\le n,n\ge 1\}$ be an array of real numbers and $\{X_{n},n\ge 1\}$ be a sequence of END random variables with $EX_{n}=0$ and $E|X_{n}|^{p}<\infty$ for each $n\ge 1$ and some $p>1$. Then there exist positive constants $C_{p}$ and $C_{p}'$ depending only on p such that

$$\begin{aligned} E\left| \sum _{i=1}^{n}a_{ni}X_{i}\right| ^{p}\le & {} C_{p}\left[ \sum _{i=1}^{n}E|a_{ni}X_{i}|^{p}+\left( \sum \limits _{i=1}^{n}Ea_{ni}X_{i}^{2}\right) ^{p/2}\right] ,\text { if }p\ge 2,\\ E\left| \sum _{i=1}^{n}a_{ni}X_{i}\right| ^{p}\le & {} C_{p}'\sum _{i=1}^{n}E|a_{ni}X_{i}|^{p},\text { if }1<p<2.\end{aligned}$$

Proof

Noting that $a_{ni}=a_{ni}^{+}-a_{ni}^{-}$ for each $1\le i\le n$, $n\ge 1$, the Rosenthal inequality above is a direct consequence of Corollary 3.2 in Shen (2011) by using $C_{r}$-inequality. The second inequality, i.e., Marcinkiewicz-Zygmund inequality can be obtained by the first one and the method used in the proof of Theorem 2.1 in Chen et al. (2014). The details are omitted. $\square$

Lemma A.3

Let $(Y_{1},\cdots ,Y_{n})\sim Dir(\alpha _{1},\cdots ,\alpha _{n})$ and $\alpha _0=\sum _{i=1}^{n}\alpha _{i}$. Then for any $p>0$ and each $1\le i\le n$,

$$\begin{aligned} EY_{i}^{p}=\frac{\Gamma (\alpha _{0})\Gamma (\alpha _{i}+p)}{\Gamma (\alpha _{0}+p)\Gamma (\alpha _{i})}. \end{aligned}$$

Proof

Without loss of generality, we only need to show $EY_{1}^{p}=\frac{\Gamma (\alpha _{0})\Gamma (\alpha _{1}+p)}{\Gamma (\alpha _{0}+p)\Gamma (\alpha _{1})}$. By Definition 1.1 and some standard calculation, we have that

$$\begin{aligned}{} & {} EY_{1}^{p}=\frac{\Gamma (\alpha _{0})}{\prod _{i=1}^{n}\Gamma (\alpha _{i})}\int _{0}^{1}y_{1}^{\alpha _{1}+p-1}dy_{1}\int _{0}^{1-y_{1}}y_{2}^{\alpha _{1}-1}dy_{2}\cdots \\{} & {} \qquad \quad \times \int _{0}^{1-\sum _{i=1}^{n-2}y_{i}}y_{n-1}^{\alpha _{n-1}-1}\left( 1-\sum _{i=1}^{n-1}y_{i}\right) ^{\alpha _{n}-1}dy_{n-1}\\{} & {} \quad =\frac{\Gamma (\alpha _{0})}{\prod _{i=1}^{n}\Gamma (\alpha _{i})}\int _{0}^{1}y_{1}^{\alpha _{1}+p-1}dy_{1}\int _{0}^{1-y_{1}}y_{2}^{\alpha _{1}-1}dy_{2} \cdots \\{} & {} \qquad \times \int _{0}^{1-\sum _{i=1}^{n-3}y_{i}}y_{n-2}^{\alpha _{n-2}-1}\left( 1-\sum _{i=1}^{n-2}y_{i}\right) ^{\alpha _{n-1}+\alpha _{n}-1}dy_{n-2}\int _{0}^{1} \eta _{n-1}^{\alpha _{n-1}-1}(1-\eta _{n-1})^{\alpha _{n}-1}d\eta _{n-1}\\{} & {} \quad =\frac{\Gamma (\alpha _{0})}{\prod _{i=1}^{n}\Gamma (\alpha _{i})}\int _{0}^{1}y_{1}^{\alpha _{1}+p-1}dy_{1}\int _{0}^{1-y_{1}}y_{2}^{\alpha _{1}-1}dy_{2} \cdots \\{} & {} \qquad \times \int _{0}^{1-\sum _{i=1}^{n-3}y_{i}}y_{n-2}^{\alpha _{n-2}-1}\left( 1-\sum _{i=1}^{n-2}y_{i}\right) ^{\alpha _{n-1}+\alpha _{n}-1}dy_{n-2}\cdot \frac{\Gamma (\alpha _{n-1}) \Gamma (\alpha _{n})}{\Gamma (\alpha _{n-1}+\alpha _{n})}\\{} & {} \quad =\cdots =\frac{\Gamma (\alpha _{0})}{\prod _{i=1}^{n}\Gamma (\alpha _{i})}\cdot \frac{\Gamma (\alpha _{1}+p)\prod _{i=2}^{n}\Gamma (\alpha _{i})}{\Gamma (\alpha _{0}+p)} =\frac{\Gamma (\alpha _{0})\Gamma (\alpha _{1}+p)}{\Gamma (\alpha _{0}+p)\Gamma (\alpha _{1})}, \end{aligned}$$

where the second equality above follows by letting $\eta _{n-1}=y_{n-1}/\left( 1-\sum _{i=1}^{n-2}y_{i}\right)$. $\square$

Lemma A.4

(cf. Hu 2004) Let $(\Omega ,\mathscr {F},P)$ be a probability space, $[T_{1},T_{2}]$ be a closed interval on the real line. Assume that $V(\theta )=V(\omega ,\theta )$ $(\theta \in [T_{1},T_{2}],\omega \in \Omega )$ is a stochastic process such that $V(\omega ,\theta )$ is continuous for all $\omega \in \Omega$. If there exist numbers $\alpha >0$, $r>0$ and $C=C(T_{1},T_{2})<\infty$ such that

$$\begin{aligned} E|V(\theta _{1})-V(\theta _{2})|^{r}\le C|\theta _{1}-\theta _{2}|^{1+\alpha },~\text { for any }\theta _{1},\theta _{2}\in [T_{1},T_{2}],\end{aligned}$$

then for any $\varepsilon >0$, $a>0$, $\theta _{0},\theta _{0}+\varepsilon \in [T_{1},T_{2}]$, and $\gamma \in (2,2+\alpha )$, it has

$$\begin{aligned} P\left( \sup _{\theta _{0}\le \theta _{1},\theta _{2}\le \theta _{0}+\varepsilon }|V(\theta _{1})-V(\theta _{2})|\ge a\right) \le \frac{8C}{(\alpha -\gamma +2)(\alpha -\gamma +3)}\left( \frac{8\gamma }{\gamma -2}\right) ^{r}\frac{\varepsilon ^{\alpha +1}}{a^{r}}. \end{aligned}$$