Robust estimation and variable selection for varying-coefficient partially nonlinear models based on modal regression

Abstract

In this paper, we propose a robust two-stage estimation and variable selection procedure for varying-coefficient partially nonlinear model based on modal regression. In the first stage, each coefficient function is approximated by B-spline basis functions and then QR decomposition is employed to remove the nonparametric component from the original model. For the simple parametric model, an estimation and variable selection procedure for parameter is proposed based on modal regression. In the second stage, similar procedure for coefficient function is developed. The proposed procedure is not only flexible and easy to implement, but also is robust and efficient. Under some mild conditions, certain asymptotic properties of the resulting estimators are established. Moreover, the bandwidth selection and estimation algorithm for the proposed method is discussed. Furthermore, we conduct some simulations and a real example to evaluate the performances of the proposed estimation and variable selection procedure in finite samples.

Appendix A. Proofs of theorems

Proof of of Theorem 1

Let $\delta _{n} =n^{-r/(2r+1)} + a_{n_1}$ and ${\mathbf {v}}=(v_1,\ldots ,v_q)^T$. Define ${\varvec{\beta }}={\varvec{\beta }}_0+\delta _{n}{\mathbf {v}}$.

We show that, for any given $\varepsilon > 0$, there exists a large enough constant C such that

$$\begin{aligned} P\left\{ \underset{{||{\mathbf {v}}||=C}}{\mathrm {\sup }}L({\varvec{\beta }})< L({\varvec{\beta }}_0)\right\} \le 1 - \varepsilon , \end{aligned}$$

(A.1)

where $L({\varvec{\beta }})$ is defined in (2.8). Let $\pi ({\varvec{\beta }})=L({\varvec{\beta }})- L({\varvec{\beta }}_0)$, by Taylor expansion, we have

$$\begin{aligned} \pi ({\varvec{\beta }})= & {} \sum _{i=1}^n\phi _h(\mathbf{Q }_{2i}^T\mathbf{Y }-\mathbf{Q }_{2i}^{T}g(\mathbf{Z },{\varvec{\beta }}_0+\delta _{n}{\mathbf {v}})) -\phi _h(\mathbf{Q }_{2i}^{T}\mathbf{Y }-\mathbf{Q }_{2i}^{T}g(\mathbf{Z },{\varvec{\beta }}_0))\nonumber \\&-n\sum _{j=1}^q\{p_{\lambda _{j}}(|\beta _{0j}+\delta _{n}{v_{j}}|)-p_{\lambda _{j}}(|\beta _{0j}|)\}\nonumber \\= & {} {J}_1+{J}_2 \end{aligned}$$

(A.2)

Using Taylor expanding $g(\mathbf{Z },{\varvec{\beta }})$ around ${\varvec{\beta }}_0$, we have

$$\begin{aligned} \mathbf{Q }_{2i}^{T}g(\mathbf{Z },{\varvec{\beta }}_0+\delta _{n}{\mathbf {v}})=\mathbf{Q }_{2i}^{T}g(\mathbf{Z },{\varvec{\beta }}_0)+\mathbf{Q }_{2i}^{T}g'(\mathbf{Z },{\varvec{\beta }}_0)\delta _{n}{\mathbf {v}}(1+O_p(1)) \end{aligned}$$

(A.3)

Then, for ${J}_1$, using Taylor expansion and (A.3), we obtain that

$$\begin{aligned} {J}_{1}= & {} \sum _{i=1}^n\delta _{n}{\phi '_h}(\varepsilon _{i})\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}_0){\mathbf {v}}\nonumber \\&+\sum _{i=1}^n\delta _{n}^{2}{\phi ''_h}(\varepsilon _{i})[\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}_0){\mathbf {v}}]^{2}\nonumber \\&+\sum _{i=1}^n\delta _{n}^{3}{\phi '''_h}(\xi _{i})[\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}_0){\mathbf {v}}]^{3}\nonumber \\= & {} {J}_{11}+{J}_{12}+{J}_{13} \end{aligned}$$

(A.4)

where $\xi _{i}$ lies in $\varepsilon _{i}$ and $\varepsilon _{i}-\delta _{n}\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}_0){\mathbf {v}}$.

By calculating the mean and variance of ${J}_{11}$, we have ${J}_{11}=O_{p}(n\delta _{n}^{2}||{\mathbf {v}}||)$. Similarly, we also have ${J}_{13}=O_{p}(n\delta _{n}^{3}||{\mathbf {v}}||^{3})$.

By the Assumption A5 and A7,

$$\begin{aligned} {J}_{12}=\delta _{n}^{2}nF(x,h){\mathbf {v}}^{T}E[\mathbf{Q }_{2}^{T} g{'}(\mathbf{Z },{\varvec{\beta }}_0)(\mathbf{Q }_{2}^{T}g^{'}(\mathbf{Z },{\varvec{\beta }}_0))^{T}]{\mathbf {v}}(1+O_p(1)) \end{aligned}$$

(A.5)

We have ${J}_{12}=O_{p}(n\delta _{n}^{2}||{\mathbf {v}}||^{2}).$ Hence, by choosing a sufficiently large C, ${J}_{12}$ dominates both ${J}_{11}$ and ${J}_{13}$ in $||{\mathbf {v}}||=C.$

We next consider ${J}_{2}$, by invoking $p_{\lambda }(0)=0$ and $p_\lambda (\beta )>0$ for any $\beta$, then by the argument of the Taylor expansion, we obtain that

$$\begin{aligned} {J}_{2}\le & {} n\delta _{n}\sum _{j=1}^{s_1}\left\{ p'_{\lambda _{j}}(|\beta _{0j}|)\text {sign}|\beta _{j}|v_{j}+\frac{1}{2}\delta _{n}p''_{\lambda j}(|\beta _{0j}|)v_{j}^{2} \right\} \nonumber \\\le & {} n\sqrt{s_1}\delta _{n}a_{n_1}||{\mathbf {v}}||+n\delta _{n}^{2}b_{n_1}||{\mathbf {v}}||^{2} \end{aligned}$$

(A.6)

Then by $b_{n_1}\rightarrow 0$, ${J}_{2}$ is also dominated by ${J}_{12}$ in $||{\mathbf {v}}||=C$. Hence, by choosing a sufficiently large C, (A.1) holds. This completes the Theorem 1. $\square$

Proof of of Theorem 2

It is sufficient to show that, for any ${\varvec{\beta }}$ that satisfies $||{\varvec{\beta }} -{\varvec{\beta }}_{0}|| = O_{p}(n^{-r/(2r+1)})$ and for some given $\varepsilon = C n^{-r/(2r+1)}$, when $n\rightarrow \infty$ with probability tending to 1, we have,

$$\begin{aligned} \frac{\partial L({{\varvec{\beta }}})}{\partial {\beta }_j}< 0, ~~\mathrm {for} ~~ 0<\beta _j < \varepsilon , j = s_1+1,\ldots ,q, \end{aligned}$$

(A.7)

and

$$\begin{aligned} \frac{\partial L({{\varvec{\beta }}})}{\partial {\beta }_j} > 0, ~~\mathrm {for} ~~ -\varepsilon<\beta _j < 0, j = s_1+1,\ldots ,q,\nonumber \\ \end{aligned}$$

(A.8)

By similar proof of (A.3) and (A.4) in Theorem 1, we can obtain that

$$\begin{aligned} \frac{\partial L({{\varvec{\beta }}})}{\partial {\beta }_j}= & {} \sum _{i=1}^n\frac{\partial \mathbf{Q }_{2i}^{T}g(\mathbf{Z },{\varvec{\beta }})}{\partial \beta _j}{\phi '_h}(\varepsilon _{i}-\delta _{n}\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}){\mathbf {v}}) -n{p'_{\lambda _j}}(|\beta _j|)\text {sign}(\beta _j)\nonumber \\= & {} \sum _{i=1}^n\frac{\partial \mathbf{Q }_{2i}^{T}g(\mathbf{Z },{\varvec{\beta }})}{\partial \beta _j} \big \{{\phi '_h}(\varepsilon _{i})+\delta _{n}{\phi ''_h}(\varepsilon _{i})\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}){\mathbf {v}}\nonumber \\&+\delta _{n}^{2}{\phi '''_h}(\xi _{i})[\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}){\mathbf {v}}]^{2}\big \}\nonumber \\&-n{p'_{\lambda _j}}(|\beta _j|)\text {sign}(\beta _j)\nonumber \\= & {} n\lambda _j\left\{ \frac{{p'_{\lambda _j}}(|\beta _j|)\text {sign}(\beta _j)}{\lambda _j}+O_p\left( \frac{n^{-r/(2r+1)}}{\lambda _j}\right) \right\} \end{aligned}$$

(A.9)

where $\xi _{i}$ is between $\varepsilon _{i}$ and $\varepsilon _{i}-\delta _{n}\mathbf{Q }_{2i}^{T}g{'}(\mathbf{Z },{\varvec{\beta }}_0){\mathbf {v}}$.

By the Assumption A9 and $\lambda _j n^{r/(2r+1)}>\lambda _{min} n^{r/(2r+1)}\longrightarrow \infty$, the sign of derivation is determined by that of ${\beta }_j$. Then (A.7) and (A.8) hold, which imply that ${\hat{\beta }}_j=0,j=s_1+1,\cdots ,q$ with probability tending to 1. $\square$

Proof of of Theorem 3

Let $\delta _{n} =n^{-r/(2r+1)} + a_{n_2}$ and ${\mathbf {v}}=({\mathbf {v}}_1^T,\ldots ,{\mathbf {v}}_p^T)^T$ be a pL-dimensional vector. Define ${\varvec{\gamma }}={\varvec{\gamma }}_0+\delta _{n}{\mathbf {v}}$, where ${\varvec{\gamma }}_0=({\varvec{\gamma }}_{01}^T,\ldots ,{\varvec{\gamma }}_{0p}^T)^T$ is the best approximation of ${\varvec{\alpha }}(u)$ in the B-spline space.

We first show that for any given $\varepsilon > 0$, there exists a large enough constant C such that

$$\begin{aligned} P\left\{ \underset{{||{\mathbf {v}}||=C}}{\mathrm {\sup }}L({\varvec{\gamma }})< L({\varvec{\gamma }}_0)\right\} \le 1 - \varepsilon \end{aligned}$$

(A.10)

where $L({\varvec{\gamma }})$ is defined in (2.12).

Let $\pi ({\varvec{\gamma }})=L({\varvec{\gamma }})- L({\varvec{\gamma }}_0)$. By Taylor expansion with simple calculation, we have

$$\begin{aligned} \pi ({\varvec{\gamma }})= & {} \delta _n\sum _{i=1}^n \phi _h'(\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i))\mathbf{W }_i^T {\mathbf {v}}\nonumber \\&+\delta _n^2\sum _{i=1}^n \phi _h''(\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i))(\mathbf{W }_i^T {\mathbf {v}})^2\nonumber \\&+\delta _n^3\sum _{i=1}^n \phi _h'''(\xi _i)(\mathbf{W }_i^T {\mathbf {v}})^3\nonumber \\&+n \sum _{j=1}^p\{p_{\lambda _j}(||{\varvec{\gamma }}_j||_\mathbf{H })-p_{\lambda _j}(||{\varvec{\gamma }}_{0j}||_\mathbf{H })\}\nonumber \\= & {} I_1+I_2+I_3+I_4 \end{aligned}$$

(A.11)

where $\mathbf{R }(u)=(R_1(u),\ldots ,R_p(u))^T$ with $R_j(u)=\alpha _j(u)-{\varvec{B}}(u)^T {\varvec{\gamma }}_{0j},j=1,\ldots ,p$, $\xi _i$ is between $\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i)$ and $\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i)-\delta _n \mathbf{W }_i^T {\mathbf {v}}$.

By the Assumption A1, A2 and Corollary 6.21 in Schumaker (1981), we have

$$\begin{aligned} ||R_j(u)||=O(K^{-r}) \end{aligned}$$

Then, by Taylor expansion, we have

$$\begin{aligned} I_1= & {} \delta _n\sum _{i=1}^n \phi _h'(\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i))\mathbf{W }_i^T {\mathbf {v}}\nonumber \\= & {} \delta _n\sum _{i=1}^n \{\phi _h'(\varepsilon _i)+\phi _h''(\varepsilon _i)\mathbf{X }_i^T \mathbf{R }(U_i)+ \phi _h'''(\xi _i^*)(\mathbf{X }_i^T \mathbf{R }(U_i))^2\}\mathbf{W }_i^T {\mathbf {v}} \end{aligned}$$

(A.12)

where $\xi _i^*$ is between $\varepsilon _i$ and $\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i)$.

By the Assumption A6, A8, and some calculations, we get

$$\begin{aligned} I_1=O_p(n K^{-r} \delta _n || {\mathbf {v}}||)=O_p(n \delta _n^2 || {\mathbf {v}}||) \end{aligned}$$

(A.13)

For $I_2$, we can prove that

$$\begin{aligned} I_2= & {} \delta _n^2\sum _{i=1}^n \phi _h''(\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i))(\mathbf{W }_i^T {\mathbf {v}})^2\nonumber \\= & {} \delta _n^2\sum _{i=1}^n[\phi _h''(\varepsilon _i)+\phi _h'''(\xi _i)(\mathbf{X }_i^T \mathbf{R }(U_i))](\mathbf{W }_i^T {\mathbf {v}})^2\nonumber \\= & {} O_p(n \delta _n^2|| {\mathbf {v}}||^2) \end{aligned}$$

(A.14)

Therefore, by choosing a sufficient large C, $I_2$ dominates $I_1$ uniformly $|| {\mathbf {v}}||=C$. Similar to $I_2$, we can prove that

$$\begin{aligned} I_3=O_p(n \delta _n^3|| {\mathbf {v}}||^3) \end{aligned}$$

(A.15)

By the condition $a_{n_2}\longrightarrow 0$, hence $\delta _n \longrightarrow 0$, then $I_3=O_p(I_2)$ is obtained by the fact $\delta _n|| {\mathbf {v}}||\longrightarrow 0$ with $||{\mathbf {v}}||=C$. Therefore, $I_3$ is also dominated by $I_2$ uniformly in $|| {\mathbf {v}}||=C$.

For $I_4$, invoking $p_\lambda (0)=0$ and some argument of Taylor expansion, we get that

$$\begin{aligned} I_4= & {} n\sum _{j=1}^p p_{\lambda _j}(||{\varvec{\gamma }}_j||_\mathbf{H })-p_{\lambda _j}(||{\varvec{\gamma }}_{j0}||_\mathbf{H })\nonumber \\= & {} n\sum _{j=1}^p p_{\lambda _j}(||{\varvec{\gamma }}_{0j}+\delta _n v_j||_\mathbf{H })-p_{\lambda _j}(||{\varvec{\gamma }}_{0j}||_\mathbf{H })\nonumber \\= & {} n\sum _{j=1}^p p'_{\lambda _j}(||{\varvec{\gamma }}_{0j}||_\mathbf{H }) (||{\varvec{\gamma }}_{j0}+\delta _n v_j||_\mathbf{H }-||{\varvec{\gamma }}_{0j}||_\mathbf{H })\nonumber \\&+\frac{1}{2}p''_{\lambda _j}(||{\varvec{\gamma }}_{j0}||_\mathbf{H })(||{\varvec{\gamma }}_{j0}+\delta _n v_j||_\mathbf{H }-||{\varvec{\gamma }}_{0j}||_\mathbf{H })^2(1+o_p(1))\nonumber \\\le & {} \sum _{j=1}^s n \delta _n p'_{\lambda _j}(||{\varvec{\gamma }}_{0j}||_\mathbf{H }) ||v_j||+\frac{1}{2} n \delta _n^2 p''_{\lambda _j}(||{\varvec{\gamma }}_{0j}||_\mathbf{H }) ||v_j||^2(1+o_p(1))\nonumber \\\le & {} \sqrt{s} (n \delta _n a_n ||{\mathbf {v}}||+ n \delta _n^2 b_n ||{\mathbf {v}}||^2) \end{aligned}$$

(A.16)

Then, with the condition $b_{n_2}\longrightarrow 0$, it is easy to show that $I_4$ is also dominated by $I_2$ uniformly in $||{\mathbf {v}}||=C$. Hence, by choosing a sufficient large C, (A.10) holds. So, there exits a local maximizer such that

$$\begin{aligned} ||\hat{{\varvec{\gamma }}}-{\varvec{\gamma }}_0||=O_p(\delta _n)=O_p(n^{-\frac{r}{2r+1}}+a_{n_2}) \end{aligned}$$

(A.17)

Note that

$$\begin{aligned} ||{\hat{\alpha }}_k(.)-\alpha _{0k}(.)||^2= & {} \int _0^1 |{\hat{\alpha }}_k(u)-\alpha _{0k}(u)|^2 du\nonumber \\= & {} 2 ({\hat{{\varvec{\gamma }}}}_k-{{\varvec{\gamma }}}_k)^T \mathbf{H } ({\hat{{\varvec{\gamma }}}}_k-{{\varvec{\gamma }}}_k) +2\int _0^1 R_k(u)^2 du \end{aligned}$$

(A.18)

This invoking $||H||=O(1)$ and (A.17), we have

$$\begin{aligned} ({\hat{{\varvec{\gamma }}}}_k-{{\varvec{\gamma }}}_k)^T \mathbf{H } ({\hat{{\varvec{\gamma }}}}_k-{{\varvec{\gamma }}}_k)=O_p(n^{-\frac{2r}{2r+1}}+a_{n_2}^2) \end{aligned}$$

(A.19)

Moreover, we can get that

$$\begin{aligned} \int _0^1 R_k(u)^2 du=O_p(n^{-\frac{2r}{2r+1}}) \end{aligned}$$

(A.20)

Invoking (A.19), (A.20) and (A.18),

$$\begin{aligned} ||{\hat{\alpha }}_k(.)-\alpha _{0k}(.)||=O_p\left( n^{-\frac{r}{2r+1}}+a_{n_2}\right) ,k=1,\ldots ,p. \end{aligned}$$

(A.21)

This completes the Theorem 3. $\square$

Proof of of Theorem 4

It is sufficient to show that, for any ${\varvec{\gamma }}$ that satisfies $||{\varvec{\gamma }} -{\varvec{\gamma }}_{0}|| = O_{p}(n^{-r/(2r+1)})$ and for some given small $\varepsilon = C n^{-r/(2r+1)}$,

$$\begin{aligned} \frac{\partial L({{\varvec{\gamma }}})}{\partial ||{{\varvec{\gamma }}}_k||_\mathbf{H }}< 0, ~~\mathrm {for} ~~ 0<||{\varvec{\gamma }}_k||_H < \varepsilon , k = s_2+1,\ldots ,p\nonumber \\ \end{aligned}$$

(A.22)

and

$$\begin{aligned} \frac{\partial L({{\varvec{\gamma }}})}{\partial ||{{\varvec{\gamma }}}_k||_\mathbf{H }} > 0, ~~\mathrm {for} ~~-\varepsilon< ||{\varvec{\gamma }}_k||_H <0,k = s_2+1,\ldots ,p. \end{aligned}$$

(A.23)

By a similar proof of (A.11) in Theorem 3, we can show that,

$$\begin{aligned} \frac{\partial L({\varvec{\gamma }})}{\partial ||{{\varvec{\gamma }}}_k||_\mathbf{H }}= & {} \frac{\partial Q({\varvec{\gamma }})}{\partial ||{{\varvec{\gamma }}}_k||_\mathbf{H }} - n p'_{\lambda _k}(||{{\varvec{\gamma }}}_k||_\mathbf{H })\mathrm {sign}(||{{\varvec{\gamma }}}_k||_\mathbf{H })\nonumber \\= & {} \sum _{i=1}^{n}\big \{W_{ik}\phi _h'(\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i))+ W_{ik} \phi _h''(\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i)) W_{ik}^T({\varvec{\gamma }}_{k}-{\varvec{\gamma }}_{0k})\nonumber \\&+W_{ik}\phi _h'''(\eta _i) [{W}_{ik}^T({\varvec{\gamma }}_{k}-{\varvec{\gamma }}_{0k})]^2\big \}-n p'_{\lambda _k}(||{{\varvec{\gamma }}}_k||_\mathbf{H })\mathrm {sign}(||{{\varvec{\gamma }}}_k||_\mathbf{H })\nonumber \\= & {} n\lambda _k\left\{ \frac{p'_{\lambda _k}(||{{\varvec{\gamma }}}_k||_\mathbf{H })\mathrm {sign}(||{{\varvec{\gamma }}}_k||_\mathbf{H })}{{{\varvec{\gamma }}}_k} +O_p\left( \frac{n^{-r/(2r+1)}}{\lambda _k}\right) \right\} \end{aligned}$$

(A.24)

where $W_{ik}$ denote the kth element of $\mathbf{W }_i$, $\eta _i$ is between $Y_i-\mathbf{W }_i^T{\varvec{\gamma }}$ and $\varepsilon _i+\mathbf{X }_i^T \mathbf{R }(U_i)$.

By the Assumption A9 and $\lambda _k n^{r/(2r+1)}>\lambda _{min} n^{r/(2r+1)}\longrightarrow \infty$, the sign of derivation is completely determined by that of $||{\varvec{\gamma }}_k||_\mathbf{H }$. Then (A.22) and (A.23) hold, which imply that $\hat{{\varvec{\gamma }}}_j=0,j=s_2+1,\ldots ,p$ with probability tending to 1. Invoking $\mathrm {sup}_u||\mathbf{B }(u)||=O(1)$, the result of Theorem 4 is obtained by ${\hat{\alpha }}_j(u)=\mathbf{B }(u)^T\hat{{\varvec{\gamma }}}_j$.

$\square$