Structure identification for varying coefficient models with measurement errors based on kernel smoothing

Abstract

Measurement error data are often encountered in a broad spectrum of scientific fields, including engineering, economics, biomedical sciences and epidemiology. Simply ignoring the measurement errors would result in biased estimators. Combining the local kernel smoothing and the SCAD approach, this paper proposes a bias-corrected penalized method to capture the underlying structure of varying coefficient models with measurement errors. We show that, under the proper choice of tuning parameters and some regular conditions, the proposed method can consistently remove all the unimportant variables and separate the constant effects and varying effects. The corresponding algorithm is also developed to compute the estimates using the local quadratic approximation. Simulation studies are conducted to assess the finite sample performance of the proposed method.

Appendix

Proof of Theorem 3.1

The proof of Theorem 3.1 is similar to that of Theorem  3.2 and thus is not given in detail. $\square$

Proof of Theorem 3.2

For any matrix $A=(a_{ij}),$$\Vert A\Vert ^2=\sum _{i,j}a_{ij}^2.$ Denote $\varvec{M}=(m_{ij})\in R^{n\times p}$ with rows $\varvec{m}_1^\top ,\ldots , \varvec{m}_n^\top$ and columns $\varvec{m}_{(1)},\ldots , \varvec{m}_{(p)}.$ Let $\alpha _n=(nh)^{-1/2}.$ It suffices to show that for any given $\epsilon >0,$ there exists a large constant C such that

$$\begin{aligned} \Pr \left\{ \inf \limits _{n^{-1}\Vert \varvec{M}\Vert ^2=C^2}Q_{\lambda _1,\lambda _2}\left( \varvec{\Phi }_0+\alpha _n \varvec{M}\right) > Q_{\lambda _1,\lambda _2}\left( \varvec{\Phi }_0\right) \right\} \ge 1 - \epsilon . \end{aligned}$$

Based on the definition of $Q_{\lambda _1,\lambda _2}(\cdot ),$ we have

$$\begin{aligned}&Q_{\lambda _1,\lambda _2}\left( \varvec{\Phi }_0+\alpha _n \varvec{M}\right) - Q_{\lambda _1,\lambda _2}\left( \varvec{\Phi }_0\right) \nonumber \\&\quad =\sum _{j=1}^{n}\left[ Q_u^c\left( \varvec{\phi }_0\left( U_j\right) +\alpha _n \varvec{m}_j\right) -Q_u^c\left( \varvec{\phi }_0\left( U_j\right) \right) \right] \nonumber \\&\qquad + \sum _{k=1}^p\rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_k\right\| \right) \left( \left\| \varvec{a}_{0k}+\alpha _n \varvec{m}_{(k)}\right\| -\left\| \varvec{a}_{0k}\right\| \right) \nonumber \\&\qquad + \sum _{k=1}^p\rho '_{\lambda _{2}}\left( \left\| \tilde{\varvec{b}}_k\right\| \right) \left( \left\| P_e\left( \varvec{a}_{0k}+\alpha _n \varvec{m}_{(k)}\right) \right\| -\left\| P_e\varvec{a}_{0k}\right\| \right) \nonumber \\&\quad \ge \sum _{j=1}^{n}\left[ Q_u^c\left( \varvec{\phi }_0\left( U_j\right) +\alpha _n \varvec{m}_j\right) -Q_u^c\left( \varvec{\phi }_0\left( U_j\right) \right) \right] \nonumber \\&\qquad -\, \alpha _n\sum _{k\notin S_z}\rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_k\right\| \right) \left\| \varvec{m}_{(k)}\right\| -\,\alpha _n\sum _{k\in S_v}\rho '_{\lambda _{2}}\left( \left\| \tilde{\varvec{b}}_k\right\| \right) \left\| P_e \varvec{m}_{(k)}\right\| \nonumber \\&\quad =T_1+T_2+T_3. \end{aligned}$$

(6.1)

Let $\widehat{\Sigma }(U_j)=n^{-1}\sum _{i=1}^n\mathbf {Z}_i\mathbf {Z}_i^{\top }K_h(U_j-U_i),$$\Omega (U_j)=n^{-1}\sum _{i=1}^n\Sigma \otimes K_h(U_j-U_i)$ and $\hat{ \varvec{e}}_j=\sum _{i=1}^n\mathbf {Z}_iK_h(U_j-U_i)(Y_i-\mathbf {Z}_i^\top \varvec{\phi }_0(U_j)),$ we have

$$\begin{aligned} T_1= & {} \sum _{j=1}^{n}\alpha _n^2 \varvec{m}_j^{\top }\left[ \widehat{\Sigma }\left( U_j\right) -\Omega \left( U_j\right) \right] \varvec{m}_j-2\sum _{j=1}^{n}\alpha _n \varvec{m}_j^{\top }\left[ \hat{ \varvec{e}}_j+\Omega \left( U_j\right) \varvec{\phi }_0\left( U_j\right) \right] \\\ge & {} n\alpha _n^2\sum _{j=1}^{n}\lambda _j^{\min }\left\| \varvec{m}_j\right\| ^2-2\alpha _n\left( \sum _{j=1}^{n}\left\| \varvec{m}_j\right\| ^2\right) ^{1/2} \left( \sum _{j=1}^{n}\left\| \hat{ \varvec{e}}_j+\Omega \left( U_j\right) \varvec{\phi }_0\left( U_j\right) \right\| ^2\right) ^{1/2}\\\ge & {} n^2\alpha _n^2\lambda ^{\min }C^2 -2n^{1/2}\alpha _n C \left( \sum _{j=1}^{n}\left\| T_{1j}\right\| ^2\right) ^{1/2}, \end{aligned}$$

where $\lambda _j^{\min }$ indicates the smallest eigenvalue of $\widehat{\Sigma }(U_j)-\Omega (U_j),$$\lambda ^{\min }=\min \{\lambda _j^{\min },\,j=1,\ldots ,n\}$ and $T_{1j}=\hat{ \varvec{e}}_j+\Omega (U_j)\varvec{\phi }_0(U_j).$

Using the formula (A1) in You et al. (2006),

$$\begin{aligned} \widehat{\Sigma }(u)-\Omega (u)=f(u)\Gamma (u)+O_P\left( \left( \frac{\log n}{nh}\right) ^{1/2}\right) , \end{aligned}$$

we have $\Pr (\lambda ^{\min }\rightarrow \lambda ^{\min }_0)\rightarrow 1,$ where $\lambda ^{\min }_0=\inf _{u\in [0,1]}\lambda _{\min }(f(u)\Gamma (u)),$$\lambda _{\min }(A)$ stands for the minimal eigenvalue of an arbitrary positive definite matrix A. By conditions (C1)–(C2), it is easy to see $\lambda ^{\min }_0>0.$

We now decompose $T_{1j}$ as

$$\begin{aligned} T_{1j}= & {} \sum _{i=1}^{n}\left[ \left( \mathbf {X}_i+\mathbf {V}_i\right) K_h\left( U_j-U_i\right) \left( \mathbf {X}_i^{\top }\varvec{\phi }_0\left( U_i\right) +\varepsilon _i-\mathbf {X}_i^{\top }\varvec{\phi }_0\left( U_j\right) -\mathbf {V}_i^{\top }\varvec{\phi }_0\left( U_j\right) \right) \right. \\&\left. \quad +\Sigma \otimes K_h\left( U_j-U_i\right) \varvec{\phi }_0\left( U_j\right) \right] \\= & {} \sum _{i=1}^{n}\mathbf {X}_i\mathbf {X}_i^{\top }K_h\left( U_j-U_i\right) \left( \varvec{\phi }_0\left( U_i\right) -\varvec{\phi }_0\left( U_j\right) \right) \\&\quad +\sum _{i=1}^{n}\mathbf {V}_i\mathbf {X}_i^{\top }K_h\left( U_j-U_i\right) \left( \varvec{\phi }_0\left( U_i\right) -\varvec{\phi }_0\left( U_j\right) \right) \\&\quad +\sum _{i=1}^{n}\mathbf {X}_iK_h\left( U_j-U_i\right) \varepsilon _i+\sum _{i=1}^{n}\mathbf {V}_iK_h\left( U_j-U_i\right) \varepsilon _i\\&\quad -\sum _{i=1}^{n}\left[ \mathbf {V}_i\mathbf {V}_i^{\top }K_h\left( U_j-U_i\right) -\Sigma \otimes K_h\left( U_j-U_i\right) \right] \varvec{\phi }_0\left( U_j\right) \\&\quad -\sum _{i=1}^{n}\mathbf {X}_i\mathbf {V}_i^{\top }K_h\left( U_j-U_i\right) \varvec{\phi }_0\left( U_j\right) \\\triangleq & {} T_{21}+T_{22}+T_{23}+T_{24}+T_{25}+T_{26}. \end{aligned}$$

Based on the proof of Wang and Xia (2009), we have $\Vert T_{21}\Vert ^2=O_P(nh^{-1})$ and $\Vert T_{23}\Vert ^2=O_P(nh^{-1}).$

For $T_{22},$

$$\begin{aligned} E\left\| T_{22}\right\| ^2= & {} \sum _{i=1}^{n}E\left\| \mathbf {V}_i\right\| ^2\left[ \mathbf {X}_i^{\top }K_h\left( U_j-U_i\right) \left( \varvec{\phi }_0\left( U_i\right) -\varvec{\phi }_0\left( U_j\right) \right) \right] ^2\\= & {} \sum _{i=1}^{n}E\left\| \mathbf {V}_i\right\| ^2E\left[ \mathbf {X}_i^{\top }K_h\left( U_j-U_i\right) \left( \varvec{\phi }_0\left( U_i\right) -\varvec{\phi }_0\left( U_j\right) \right) \right] ^2\\= & {} (n-1)tr(\Sigma ) E\left[ \mathbf {X}_1^{\top }K_h\left( U_2-U_1\right) \left( \varvec{\phi }_0\left( U_1\right) -\varvec{\phi }_0\left( U_2\right) \right) \right] ^2. \end{aligned}$$

By Taylor expression, we have

$$\begin{aligned}&E\left[ \mathbf {X}_1^{\top }K_h\left( U_2-U_1\right) \left( \varvec{\phi }_0\left( U_1\right) -\varvec{\phi }_0\left( U_2\right) \right) \right] ^2\\&\quad \le 2E\left[ \mathbf {X}_1^{\top }K_h\left( U_2-U_1\right) \varvec{\phi }'_0\left( U_2\right) \left( U_1-U_2\right) \right] ^2\\&\qquad +\frac{1}{2} E\left[ \mathbf {X}_1^{\top }\varvec{\phi }''_0(U^*)K_h\left( U_2-U_1\right) \left( U_1-U_2\right) ^2\right] ^2\\&\quad \triangleq T_{221}+T_{222}, \end{aligned}$$

where $\varvec{\phi }'_0(\cdot )=(\phi '_{01}(\cdot ),\ldots ,\phi '_{0p}(\cdot ))^{\top },$$\varvec{\phi }''_0(\cdot )=(\phi ''_{01}(\cdot ),\ldots ,\phi ''_{0p}(\cdot ))^{\top }$ and $U^*$ is between $U_1$ and $U_2.$

Based on some simple calculations,

$$\begin{aligned} T_{221}/2&=E\left\{ E\left\{ \left[ \mathbf {X}_1^{\top }\varvec{\phi }'_0\left( U_2\right) \right] ^2 K_h^2\left( U_2-U_1\right) \left( U_1-U_2\right) ^2|U_1,\,U_2\right\} \right\} \\&= E\left[ \varvec{\phi }'_0\left( U_2\right) ^{\top }\Gamma \left( U_1\right) \varvec{\phi }'_0\left( U_2\right) K_h^2\left( U_2-U_1\right) \left( U_1-U_2\right) ^2 \right] \\&= \int \int \varvec{\phi }'_0\left( u_2\right) ^{\top }\Gamma \left( u_1\right) \varvec{\phi }'_0\left( u_2\right) K_h^2\left( u_2-u_1\right) \left( u_1-u_2\right) ^2f\left( u_1\right) f\left( u_2\right) du_2du_1\\&{\mathop {=}\limits ^{(u_2-u_1)/h=s}} h\int \int \varvec{\phi }'_0\left( u_2\right) ^{\top }\Gamma \left( u_2+hs\right) \varvec{\phi }'_0\left( u_2\right) K^2(s)s^2f\left( u_2+hs\right) f\left( u_2\right) ds du_2. \end{aligned}$$

Let $\widetilde{\Gamma }(u)=\Gamma (u)f(u),$ by Taylor expansion

$$\begin{aligned} \widetilde{\Gamma }\left( u_2+hs\right) =\widetilde{\Gamma }\left( u_2\right) +\widetilde{\Gamma }'\left( u_2\right) hs+ch^2s^2, \end{aligned}$$

where c is a constant. Consequently, we have

$$\begin{aligned} T_{221}/2= & {} h\int \int \varvec{\phi }'_0\left( u_2\right) ^{\top }\widetilde{\Gamma }\left( u_2\right) \varvec{\phi }'_0\left( u_2\right) K^2(s)s^2 f\left( u_2\right) ds du_2\\&\quad +h^2\int \int \varvec{\phi }'_0\left( u_2\right) ^{\top }\widetilde{\Gamma }'\left( u_2\right) \varvec{\phi }'_0\left( u_2\right) K^2(s) s^3 f\left( u_2\right) ds du_2\\&\quad +ch^3\int \int \varvec{\phi }'_0\left( u_2\right) ^{\top } \varvec{\phi }'_0\left( u_2\right) K^2(s)s^4 f\left( u_2\right) ds du_2\\= & {} O(h). \end{aligned}$$

Similarly, $T_{222}=O(h).$ As a result, we have $\Vert T_{22}\Vert ^2=O_P(nh)=O_P(nh^{-1})$ due to $h\rightarrow 0.$

For $T_{24},$

$$\begin{aligned} E\left\| T_{24}\right\| ^2= & {} \sum _{i=1}^{n}E\left\| \mathbf {V}_i\right\| ^2E\left[ \varepsilon _i^2K_h^2\left( U_j-U_i\right) \right] \\= & {} tr(\Sigma )\left[ E\left( \varepsilon _1^2K_h^2(0)\right) +(n-1)E\varepsilon _2^2K_h^2\left( U_1-U_2\right) \right] . \end{aligned}$$

Let $\theta (u)=E(\varepsilon _2^2|U_2=u),$ we have

$$\begin{aligned}&E\varepsilon _2^2K_h^2\left( U_1-U_2\right) \nonumber \\&\quad = \int \int \theta \left( u_2\right) K_h^2\left( u_1-u_2\right) f\left( u_1\right) f\left( u_2\right) du_1du_2\nonumber \\&\quad {\mathop {=}\limits ^{(u_1-u_2)/h=s}}h^{-1}\int \int \theta \left( u_2\right) K^2(s) f\left( u_2+hs\right) f\left( u_2\right) dsdu_2\nonumber \\&\quad =O(1)h^{-1}\int K^2(s)ds\int \theta \left( u_2\right) f\left( u_2\right) du_2\nonumber \\&\quad =O(1)h^{-1}\int K^2(s)ds E \varepsilon _2^2=O\left( h^{-1}\right) . \end{aligned}$$

(6.2)

It follows that $E\Vert T_{24}\Vert ^2=O(nh^{-1}).$

For $T_{25},$ using the Cauchy–Schwartz inequality, we obtain

$$\begin{aligned} E\left\| T_{25}\right\| ^2\le & {} E\left\| \sum _{i=1}^n\left[ \mathbf {V}_i\mathbf {V}_i^{\top }K_h\left( U_j-U_i\right) -\Sigma \otimes K_h\left( U_j-U_i\right) \right] \right\| ^2 E\left\| \varvec{\phi }_0\left( U_j\right) \right\| ^2. \end{aligned}$$

Denote $\Sigma =(\sigma _{kl}),$ then

$$\begin{aligned}&E\left\| \sum _{i=1}^n\left[ \mathbf {V}_i\mathbf {V}_i^{\top }K_h\left( U_j-U_i\right) -\Sigma \otimes K_h\left( U_j-U_i\right) \right] \right\| ^2\\&\quad =\sum _{k,l}E\left( \sum _{i=1}^n\left( \sigma _{kl}-V_{ik}V_{il}\right) K_h\left( U_j-U_i\right) \right) ^2\\&\quad =\sum _{k,l}\left\{ E[\left( \sigma _{kl}-V_{ik}V_{il}\right) ^2K_h^2(0)+(n-1)E\left[ \left( \sigma _{kl}-V_{1k}V_{1l}\right) ^2K_h^2\left( U_1-U_2\right) \right] \right\} . \end{aligned}$$

Similar to the proof of $T_{24},$ we have $E\Vert T_{25}\Vert ^2=O(nh^{-1}).$

For $T_{26},$

$$\begin{aligned} E\left\| T_{26}\right\| ^2\le & {} E\left\| \sum _{i=1}^{n}\mathbf {X}_i\mathbf {V}_i^{\top }K_h\left( U_j-U_i\right) \right\| ^2 E\left\| \varvec{\phi }_0\left( U_j\right) \right\| ^2\\= & {} \sum _{k,l}\left[ E\left( X_{1k}V_{1l}K_h(0)\right) ^2+(n-1)E\left( X_{1k}V_{1l}K_h\left( U_2-U_1\right) \right) ^2\right] E\left\| \varvec{\phi }_0\left( U_j\right) \right\| ^2\\= & {} \sum _{k,l}\left[ EV_{1l}^2E\left( X^2_{1k}K^2_h(0)\right) +(n-1)EV_{1l}^2E\left( X_{1k}^2 K_h^2\left( U_2-U_1\right) \right) \right] E\left\| \varvec{\phi }_0\left( U_j\right) \right\| ^2. \end{aligned}$$

Similar to the proof of (6.2), we have

$$\begin{aligned} E\left( X_{1k}^2 K_h^2\left( U_2-U_1\right) \right) =O\left( h^{-1}\right) . \end{aligned}$$

It follows that $\Vert T_{26}\Vert ^2=O(nh^{-1}).$

Now, we consider $T_2.$

$$\begin{aligned} \left| T_2\right| ^2\le p\alpha _n^2\max \left\{ \rho '^2_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_k\right\| \right) : k\notin S_z\right\} \Vert \varvec{M}\Vert ^2. \end{aligned}$$

Note that

$$\begin{aligned} \rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_k\right\| \right) =\rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_k\right\| \right) I\left( \left\| \tilde{\varvec{a}}_k\right\| \le a\lambda _{1}\right) . \end{aligned}$$

For $k\notin S_z,$$n^{-1}\Vert \tilde{\varvec{a}}_k\Vert ^2>0,$ with probability tending to 1, and $\lambda _{1}/\sqrt{n}\rightarrow 0,$ thus for any $\xi >0,$

$$\begin{aligned} \Pr \left( \rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_k\right\| \right) >n^{3/2}\alpha _n \xi \right) \le \Pr \left( \left\| \tilde{\varvec{a}}_k\right\| \le a\lambda _{1}\right) \rightarrow 0, \end{aligned}$$

(6.3)

which implies that $\rho '_{\lambda _{1}}(\Vert \tilde{\varvec{a}}_k\Vert )=o_P(n^{3/2}\alpha _n).$ So we have $T_2=o_P(n^2\alpha _n^2)C.$ Similarly, for $k\in S_v,$$n^{-1}\Vert \tilde{\varvec{b}}_k\Vert ^2>0,$ with probability tending to 1, and $\lambda _{2}/\sqrt{n}\rightarrow 0,$ so we have $T_3=o_P(n^2\alpha _n^2)C.$

By choosing a large C,  (6.1) is positive with probability close to 1. This completes the proof of Theorem 3.2. $\square$

Proof of Theorem 3.3

We first prove the part (1). We only need to prove that $\Pr (\Vert \hat{\varvec{a}}_k\Vert =0)\rightarrow 1$ with $k=p,$ where $\hat{\varvec{a}}_k$ is the kth column of $\hat{\varvec{\Phi }}.$ The proof for $p_0+p_1<k<p$ is similar. If $\Vert \hat{\varvec{a}}_p\Vert \ne 0,$ then

$$\begin{aligned} \left. \frac{\partial Q_{\lambda _1,\lambda _2}(\varvec{\Phi })}{\partial \varvec{a}_p}\right| _{\varvec{\Phi }=\hat{\varvec{\Phi }}} = J_1+J_2+J_3, \end{aligned}$$

(6.4)

where $J_1$ is an $n \times 1$ vector with its lth component given by

$$\begin{aligned}&J_{1l}= -2\sum _{i=1}^n \left[ \left( Y_i - \mathbf {Z}_i^\top \hat{\varvec{\phi }}\left( U_l\right) \right) K_h\left( U_l-U_i\right) Z_{ip}+\Sigma _{p\cdot }^{\top }\otimes K_h\left( U_l-U_i\right) \hat{\varvec{\phi }}\left( U_l\right) \right] ,\\&J_2 = \rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_p\right\| \right) \hat{\varvec{a}}_p/\left\| \hat{\varvec{a}}_p\right\| , \end{aligned}$$

and

$$\begin{aligned} J_3= \rho '_{\lambda _{2}}\left( \left\| \tilde{\varvec{b}}_p\right\| \right) \hat{\varvec{b}}_p/\left\| \hat{\varvec{b}}_p\right\| , \end{aligned}$$

where $\Sigma _{p\cdot }^{\top }$ is the pth row of $\Sigma .$ From Theorem 3.1, we have $n^{-1}\Vert \tilde{\varvec{a}}_p\Vert ^2=O_P(n^{-4/5})$ and $n^{-1}\Vert \tilde{\varvec{b}}_p\Vert ^2=O_P(n^{-4/5}).$ According to the definition of the SCAD and $n^{1/10}/\max (\lambda _{1},\,\lambda _{2})\rightarrow 0,$ we obtain

$$\begin{aligned} \left\| J_2\right\| /\max \left( \lambda _{1},\,\lambda _{2}\right) \le \rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_p\right\| \right) /\lambda _{1}{\mathop {\rightarrow }\limits ^{P}} 1. \end{aligned}$$

In fact, for any $\eta > 0,$

$$\begin{aligned} \Pr \left( \left| \frac{\rho '_{\lambda _{1}}(\Vert \tilde{\varvec{a}}_p\Vert )}{\lambda _{1}} - 1\right|> \eta \right)\le & {} \Pr \left( \left\| \tilde{\varvec{a}}_p\right\|> \lambda _{1}\right) \\= & {} \Pr \left( \left\| \tilde{\varvec{a}}_p\right\| /\sqrt{n} > \lambda _{1}/\sqrt{n}\right) \\\rightarrow & {} 0. \end{aligned}$$

Similarly,

$$\begin{aligned} \left\| J_3\right\| /\max \left( \lambda _{1},\,\lambda _{2}\right) \le \rho '_{\lambda _{2}}\left( \left\| \tilde{\varvec{b}}_p\right\| \right) /\lambda _{2}{\mathop {\rightarrow }\limits ^{P}} 1. \end{aligned}$$

By standard arguments of kernel smoothing, we have $\Vert J_1\Vert =O_P(nh^{-1/2}).$ Consequently, with probability tending to 1, the normal equation (6.4) cannot hold, which implies $\Pr (\Vert \hat{\varvec{a}}_p\Vert =0)\rightarrow 1.$

Now we prove the part (2). Similarly, we only need to prove that $\Pr (\Vert \hat{\varvec{b}}_k\Vert =0)\rightarrow 1$ with $k=p_0+p_1.$ The proof for $p_0<k<p_0+p_1$ is similar. If $\Vert \hat{\varvec{b}}_{p_0+p_1}\Vert \ne 0,$ then

$$\begin{aligned} \left. \frac{\partial Q_{\lambda _1,\lambda _2}(\varvec{\Phi })}{\partial \varvec{a}_{p_0+p_1}}\right| _{\varvec{\Phi }=\hat{\varvec{\Phi }}} = J_1^*+J_2^*+J_3^*, \end{aligned}$$

(6.5)

where $J_1^*$ is an $n \times 1$ vector with its lth component given by

$$\begin{aligned} J_{1l}^*= & {} -2\sum _{i=1}^n \bigg [\left( Y_i - \mathbf {Z}_i^\top \hat{\varvec{\phi }}\left( U_l\right) \right) K_h\left( U_l-U_i\right) Z_{i(p_0+p_1)}\\&+\Sigma _{(p_0+p_1)\cdot }^{\top }\otimes K_h\left( U_l-U_i\right) \hat{\varvec{\phi }}\left( U_l\right) \bigg ],\\ J_2^*= & {} \rho '_{\lambda _{1}}\left( \left\| \tilde{\varvec{a}}_{p_0+p_1}\right\| \right) \hat{\varvec{a}}_{p_0+p_1}/\left\| \hat{\varvec{a}}_{p_0+p_1}\right\| , \end{aligned}$$

and

$$\begin{aligned} J_3^*= \rho '_{\lambda _{2}}\left( \left\| \tilde{\varvec{b}}_{p_0+p_1}\right\| \right) \hat{\varvec{b}}_{p_0+p_1}/\left\| \hat{\varvec{b}}_{p_0+p_1}\right\| . \end{aligned}$$

By Theorem 3.1, we have $n^{-1}\Vert \tilde{\varvec{a}}_{p_0+p_1}\Vert ^2>0$ with probability tending to 1 and $n^{-1}\Vert \tilde{\varvec{b}}_{p_0+p_1}\Vert ^2=O_P(n^{-4/5}).$ From (6.3), we have

$$\begin{aligned} \left\| J_2^*\right\| =o_P\left( nh^{-1/2}\right) . \end{aligned}$$

Based on the definition of the SCAD, together with $n^{1/10}/\max (\lambda _{1},\,\lambda _{2})\rightarrow 0,$ we can obtain

$$\begin{aligned} \left\| J_3^*\right\| /\max \left( \lambda _{1},\,\lambda _{2}\right) \le \rho '_{\lambda _{2}}\left( \left\| \tilde{\varvec{b}}_{p_0+p_1}\right\| \right) /\lambda _{2}\rightarrow 1. \end{aligned}$$

By standard arguments of kernel smoothing, we have $\Vert J_1^*\Vert =O_P(nh^{-1/2}).$ Consequently, with probability tending to one, the normal equation (6.5) cannot hold, which implies $\Pr (\Vert \hat{\varvec{b}}_{p_0+p_1}\Vert =0)\rightarrow 1.$$\square$