Estimation and variable selection for partial functional linear regression

Abstract

We propose a new estimation procedure for estimating the unknown parameters and function in partial functional linear regression. The asymptotic distribution of the estimator of the vector of slope parameters is derived, and the global convergence rate of the estimator of unknown slope function is established under suitable norm. The convergence rate of the mean squared prediction error for the proposed estimators is also established. Based on the proposed estimation procedure, we further construct the penalized regression estimators and establish their variable selection consistency and oracle properties. Finite sample properties of our procedures are studied through Monte Carlo simulations. A real data example about the real estate data is used to illustrate our proposed methodology.

Appendix: Proofs

In this section, let $C>0$ denote a generic constant of which the value may change from line to line. For a matrix $A=(a_{ij})$, set $\Vert A\Vert _{\infty }=\max _{i}\sum _{j}|a_{ij}|$ and $|A|_{\infty }=\max _{i,j}|a_{ij}|$. For a vector $v=(v_{1},\ldots ,v_{k})^{T}$, set $\Vert v\Vert _{\infty }=\sum _{j=1}^{k}|v_{j}|$ and $|v|_{\infty }=\max _{1\le j\le k}|v_{j}|$. Denote $W_{l}=\sum _{j=1}^{\infty }\gamma _{j}\xi _{lj}$, $\tilde{W}_{i}=W_{i}-\frac{1}{n}\sum _{l=1}^{n}W_{l}\tilde{\xi }_{li}$, $\tilde{\varepsilon }_{i}=\varepsilon _{i}-\frac{1}{n}\sum _{l=1}^{n}\varepsilon _{l}\tilde{\xi }_{li}$ and $\tilde{W}=(\tilde{W}_{1},\ldots ,\tilde{W}_{n})^{T}$, $\tilde{\varepsilon }=(\tilde{\varepsilon }_{1},\ldots ,\tilde{\varepsilon }_{n})^{T}$. Then

$$\begin{aligned} \hat{\pmb {\beta }}-\pmb {\beta }_{0}=(\tilde{Z}^{T}\tilde{Z})^{-1}\tilde{Z}^{T}(\tilde{W}+\tilde{\varepsilon }). \end{aligned}$$

(A.1)

Lemma A.1

Suppose that Assumptions 1, 2, 4 and 5 hold, then it holds that

$$\begin{aligned} \frac{1}{n}\tilde{Z}^{T}\tilde{Z}=\Omega +o_{p}(1). \end{aligned}$$

Proof

Let $\tilde{Z}_{i}=(\tilde{Z}_{i1},\ldots ,\tilde{Z}_{id})^{T}$. Set $\vec {\xi }_{li}=\sum _{j=1}^{m}\frac{\xi _{lj} \xi _{ij}}{\lambda _{j}}$, $\vec {Z}_{ir1}=Z_{ir}-\frac{1}{n}\sum _{l=1}^{n}Z_{lr}\vec {\xi }_{li}$ and $\vec {Z}_{ir2}=\frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\tilde{\xi }_{li}-\vec {\xi }_{li}).$ Then $\tilde{Z}_{ir}=\vec {Z}_{ir1}-\vec {Z}_{ir2}$ and

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\tilde{Z}_{ir}\tilde{Z}_{iq}= & {} \frac{1}{n}\sum _{i=1}^{n}(\vec {Z}_{ir1}\vec {Z}_{iq1}-\vec {Z}_{ir1}\vec {Z}_{iq2}-\vec {Z}_{ir2}\vec {Z}_{iq1}\nonumber \\&+\vec {Z}_{ir2}\vec {Z}_{iq2}), \quad r,q=1,\ldots ,d. \end{aligned}$$

(A.2)

Let $\vec {Z}_{ir21}=\sum _{j=1}^{m}\frac{1}{\lambda _{j}} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\hat{\xi }_{lj}-\xi _{lj})\right] \xi _{ij}$, $\vec {Z}_{ir22}=\sum _{j=1}^{m}\left( \frac{1}{\hat{\lambda }_{j}} -\frac{1}{\lambda _{j}}\right) \left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj}\right) \xi _{ij}$ and $\vec {Z}_{ir23}=\sum _{j=1}^{m}\frac{1}{\hat{\lambda }_{j}} \left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj}\right) (\hat{\xi }_{ij}-\xi _{ij}).$ We then have

$$\begin{aligned} |\vec {Z}_{ir2}\vec {Z}_{iq2}|\le \frac{3}{2}\left( \vec {Z}_{ir21}^{2}+\vec {Z}_{ir22}^{2}+\vec {Z}_{ir23}^{2}+\vec {Z}_{iq21}^{2}+\vec {Z}_{iq22}^{2}+\vec {Z}_{iq23}^{2}\right) . \end{aligned}$$

(A.3)

Lemma 5.1 of Hall and Horowitz (2007) implies that

$$\begin{aligned} \hat{\xi }_{lj}-\xi _{lj}= \sum _{k\ne j}\frac{\xi _{lk}}{\hat{\lambda }_{j}-\lambda _{k}}\int \Delta \hat{\phi }_{j}\phi _{k}+\xi _{lj}\int (\hat{\phi }_{j}-\phi _{j})\phi _{j}, \end{aligned}$$

(A.4)

where $\Delta =\hat{K}-K$. We then obtain that

$$\begin{aligned} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\hat{\xi }_{lj}-\xi _{lj})\right] ^{2}\le & {} 2\left( \sum _{k\ne j}\frac{\vec {\xi }_{rk}}{\hat{\lambda }_{j}-\lambda _{k}} \int \Delta \hat{\phi }_{j}\phi _{k}\right) ^{2} +2\left( \vec {\xi }_{rj}\int (\hat{\phi }_{j}-\phi _{j})\phi _{j}\right) ^{2} \\\le & {} 2\left[ \sum _{k\ne j}\frac{\vec {\xi }_{rk}^{2}}{\left( \hat{\lambda }_{j}-\lambda _{k}\right) ^{2}}\right] \left[ \sum _{k=1}^{\infty }\left( \int \Delta \hat{\phi }_{j}\phi _{k}\right) ^{2}\right] \\&+\,2\vec {\xi }_{rj}^{2}\left( \int (\hat{\phi }_{j}-\phi _{j})\phi _{j}\right) ^{2}, \end{aligned}$$

where $\vec {\xi }_{rj}=\frac{1}{n}\sum _{l=1}^{n}Z_{lr}\xi _{lj}$. Lemma 1 of Cardot et al. (2007) implies that

$$\begin{aligned} |\lambda _{j}-\lambda _{k}|\ge \lambda _{j}-\lambda _{j+1}\ge \lambda _{m}-\lambda _{m+1}\ge \lambda _{m}/(m+1)\ge \lambda _{m}/(2m) \end{aligned}$$

uniformly for $1\le j\le m$. By (5.2) of Hall and Horowitz (2007), it holds that $\sup _{j\ge 1}|\hat{\lambda }_{j}-\lambda _{j}|\le |\Vert \Delta \Vert |=O_{p}(n^{-1/2})$ and

$$\begin{aligned} \begin{array}{ll} \left( \int (\hat{\phi }_{j}-\phi _{j})\phi _{j}\right) ^{2} \le \Vert \hat{\phi }_{j}-\phi _{j}\Vert ^{2}\le C\frac{|\Vert \Delta \Vert |^{2}}{(\lambda _{j}-\lambda _{j+1})^{2}}\le C|\Vert \Delta \Vert |^{2}\lambda _{j}^{-2}j^{2}, \end{array} \end{aligned}$$

(A.5)

where $|\Vert \Delta \Vert |=(\int _{{\mathcal {T}}}\int _{{\mathcal {T}}}\Delta ^{2}(s,t)\mathrm{d}s\mathrm{d}t)^{1/2}$. Using Parseval’s identity, we get that

$$\begin{aligned} \sum _{k=1}^{\infty } \left( \int \Delta \hat{\phi }_{j}\phi _{k}\right) ^{2} =\int \left( \int \Delta \hat{\phi }_{j}\right) ^{2}\le |\Vert \Delta \Vert |^{2}=O_{p}(n^{-1}). \end{aligned}$$

Assumption 4 implies that $|\hat{\lambda }_{j}-\lambda _{j}|=o_{p}(\lambda _{m}/m)$. Consequently, $\sum _{k\ne j}\frac{\vec {\xi }_{rk}^{2}}{(\hat{\lambda }_{j}-\lambda _{k})^{2}}=\sum _{k\ne j}\frac{\vec {\xi }_{rk}^{2}}{(\lambda _{j}-\lambda _{k})^{2}}[1+o_{p}(1)]$, where $o_{p}(1)$ holds uniformly for $1\le j\le m$. By arguments similar to those used in the proof of Lemma 2 of Cardot et al. (2007) and use the fact that $(\lambda _{j}-\lambda _{k})^{2}\ge (\lambda _{k}-\lambda _{k+1})^{2}$, we deduce that

$$\begin{aligned} \sum _{k\ne j}\frac{1}{(\lambda _{j}-\lambda _{k})^{2}}E(\vec {\xi }_{rk}^{2})&\le C\sum _{k\ne j}\frac{1}{(\lambda _{j}-\lambda _{k})^{2}}\left( n^{-1}\lambda _{k}+g_{rk}^{2}\lambda _{k}^{2}\right) \\&\le C\left( n^{-1}\lambda _{j}^{-1}j^{2}\log j+1\right) . \end{aligned}$$

Lemma 1 of Cardot et al. (2007) yields that

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-2}j^{2}\log j\le m^{-2}\lambda _{m}^{-2}\sum _{j=1}^{m}j^{4}\log j\le \lambda _{m}^{-2}m^{3}\log m \end{aligned}$$

and $\sum _{j=1}^{m}\lambda _{j}^{-1}\le \lambda _{m}^{-1}m$. Therefore,

$$\begin{aligned} \begin{array}{ll} \sum _{j=1}^{m}\frac{1}{\lambda _{j}} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\hat{\xi }_{lj}-\xi _{lj})\right] ^{2} =O_{p}\left( n^{-2}\lambda _{m}^{-2}m^{3}\log m +n^{-1}\lambda _{m}^{-1}m\right) \end{array}\nonumber \\ \end{aligned}$$

(A.6)

and

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\vec {Z}_{ir21}^{2}\le & {} \left( \sum _{j=1}^{m}\frac{1}{\lambda _{j}} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\hat{\xi }_{lj}-\xi _{lj})\right] ^{2}\right) \left( \sum _{j=1}^{m}\frac{1}{n\lambda _{j}}\sum _{i=1}^{n}\xi _{ij}^{2}\right) \nonumber \\= & {} O_{p}\left( n^{-2}\lambda _{m}^{-2}m^{4}\log m +n^{-1}\lambda _{m}^{-1}m^{2}\right) . \end{aligned}$$

(A.7)

Decomposing $\frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj}=\vec {\xi }_{rj}+\frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\hat{\xi }_{lj} -\xi _{lj})$ and using (A.6), we get

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\vec {Z}_{ir22}^{2}\le & {} C\sum _{j=1}^{m} \frac{(\hat{\lambda }_{j}-\lambda _{j})^{2}}{\lambda _{j}^{3}} \left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj}\right) ^{2} [1+o_{p}(1)] \left( \sum _{j=1}^{m}\frac{1}{n\lambda _{j}}\sum _{i=1}^{n}\xi _{ij}^{2}\right) \nonumber \\= & {} O_{p}\left( n^{-1}\lambda _{m}^{-1}m+n^{-3}\lambda _{m}^{-4}m^{4}\log m +n^{-2}\lambda _{m}^{-3}m^{2}\right) . \end{aligned}$$

(A.8)

By (A.10) of Tang (2015), it holds that

$$\begin{aligned} n\Vert \hat{\phi }_{j}-\phi _{j}\Vert ^{2}/(j^{2}\log j)=O_{p}(1), \end{aligned}$$

(A.9)

where $O_{p}(\cdot )$ holds uniformly for $1\le j\le m$. Using (A.8) and (A.9), we obtain

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\vec {Z}_{ir23}^{2}\le & {} \left( \sum _{j=1}^{m}\frac{1}{\hat{\lambda }^{2}} \left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj}\right) ^{2}\right) \left( \frac{1}{n}\sum _{i=1}^{n}\Vert X_{i}\Vert ^{2}\right) \left( \sum _{j=1}^{m}\Vert \hat{\phi }_{j}-\phi _{j}\Vert ^{2}\right) \nonumber \\= & {} O_{p}\left( \left( n^{-2}\lambda _{m}^{-1}m^{4}+n^{-1}m^{3}+n^{-3}\lambda _{m}^{-3}m^{6}\log m +n^{-2}\lambda _{m}^{-2}m^{4}\right) \log m\right) .\nonumber \\ \end{aligned}$$

(A.10)

Hence, by (A.3), (A.7), (A.8), and (A.10) and Assumption 4, we conclude that

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}|\vec {Z}_{ir2}\vec {Z}_{iq2}|=O_{p}\left( n^{-2} \lambda _{m}^{-2}m^{4}\log m +n^{-1}\lambda _{m}^{-1}m^{2}\right) =o_{p}(1). \end{aligned}$$

(A.11)

Define $\check{\xi }_{jr}=\frac{1}{n}\sum _{l=1}^{n}\lambda _{j}^{-1/2}\xi _{lj}Z_{lr}$. Since $E[\max _{1\le j\le m}(\check{\xi }_{jr}-E(\check{\xi }_{jr}))^{2}]\le \frac{1}{n}\sum _{j=1}^{m}\lambda _{j}^{-1}E(\xi _{j}Z_{r})^{2}\le Cn^{-1}$, we then have $\max _{1\le j\le m}|\check{\xi }_{jr}-E(\check{\xi }_{jr})|=O_{p}(n^{-1/2})$. Hence

$$\begin{aligned}&\frac{1}{n}\sum _{i=1}^{n}\vec {Z}_{ir1}\vec {Z}_{iq1}\nonumber \\&\quad =\frac{1}{n}\sum _{i=1}^{n}Z_{ir}Z_{iq} -2\sum _{j=1}^{m}\check{\xi }_{jr}\check{\xi }_{jq}+ \sum _{j=1}^{m}\frac{\check{\xi }_{jr}\check{\xi }_{jq}}{n\lambda _{j}}\left( \sum _{i=1}^{n}\xi _{ij}^{2}\right) +\sum _{j\ne j'}\check{\xi }_{jr}\check{\xi }_{j'q}\bar{\xi }_{jj'} \nonumber \\&\quad =\sum _{j=1}^{\infty }g_{rj}g_{qj}\lambda _{j} +E(H_{r}H_{q})-2\sum _{j=1}^{m}g_{rj}g_{qj}\lambda _{j} +\sum _{j=1}^{m}g_{rj}g_{qj}\lambda _{j}+o_{p}(1) \nonumber \\&\quad =E(H_{r}H_{q})+o_{p}(1), \end{aligned}$$

(A.12)

where $\bar{\xi }_{jj'}=\frac{1}{n(\lambda _{j}\lambda _{j'})^{1/2}}\sum _{i=1}^{n}\xi _{ij}\xi _{ij'}$. Now Lemma A.1 follows from (A.2), (A.11), (A.12) and the fact that $\frac{1}{n}|\sum _{i=1}^{n}\vec {Z}_{ir1}\vec {Z}_{iq2}| \le \left( \frac{1}{n}\sum _{i=1}^{n}\vec {Z}_{ir1}^{2}\right) ^{1/2} \left( \frac{1}{n}\sum _{i=1}^{n}\vec {Z}_{iq2}^{2}\right) ^{1/2}$. $\square$

Lemma A.2

Under Assumptions 1–4, it holds that

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j} \left[ \gamma _{j}-\frac{1}{\hat{\lambda }_{j}} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) \right] ^{2} =O_{p}\left( n^{-1}\lambda _{m}^{-1}m\right) . \end{aligned}$$

Proof

Set $S_{1}=\sum _{j=1}^{m}\lambda _{j}\left[ \gamma _{j}-\frac{1}{\lambda _{j}} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\xi _{lj}\right) \right] ^{2}$, $S_{2}=\sum _{j=1}^{m}\frac{1}{\lambda _{j}} \left[ \frac{1}{n}\sum _{l=1}^{n}W_{l}(\hat{\xi }_{lj}-\xi _{lj})\right] ^{2}$ and $S_{3}=\sum _{j=1}^{m}\lambda _{j} \left( \frac{1}{\hat{\lambda }_{j}}-\frac{1}{\lambda _{j}}\right) ^{2} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) ^{2}$. We have

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j} \left[ \gamma _{j}-\frac{1}{\hat{\lambda }_{j}} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) \right] ^{2}\le 3(S_{1}+S_{2}+S_{3}). \end{aligned}$$

(A.13)

Since $E\left[ \gamma _{j}-\frac{1}{\lambda _{j}}\left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\xi _{lj}\right) \right] =0$, then by Assumptions 1–3, we obtain that

$$\begin{aligned} E(S_{1})=\sum _{j=1}^{m}\frac{1}{\lambda _{j}} Var\left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\xi _{lj}\right) \le \sum _{j=1}^{m}\frac{1}{n^{2}\lambda _{j}}\sum _{l=1}^{n} E\left( W_{l}^{2}\xi _{lj}^{2}\right) \le Cm/n.\nonumber \\ \end{aligned}$$

(A.14)

Similar to the proof of (A.6) and (A.8) and using Assumption 4, we deduce that

$$\begin{aligned} S_{2}=O_{p}\left( n^{-2}\lambda _{m}^{-2}m^{3}\log m +n^{-1}\lambda _{m}^{-1}m\right) =O_{p}(n^{-1}\lambda _{m}^{-1}m) \end{aligned}$$

(A.15)

and

$$\begin{aligned} S_{3}\le & {} C\sum _{j=1}^{m}\frac{(\hat{\lambda }_{j}-\lambda _{j})^{2}}{\lambda _{j}^{3}}\left( \bar{\zeta }_{j}^{2} +\left[ \frac{1}{n}\sum _{l=1}^{n}\zeta _{l}(\hat{\xi }_{lj}-\xi _{lj})\right] ^{2}\right) [1+o_{p}(1)] \nonumber \\= & {} O_{p}\left( n^{-1}\lambda _{m}^{-1}+n^{-3}\lambda _{m}^{-4}m^{3}\log m +n^{-2}\lambda _{m}^{-3}m)=O_{p}(n^{-1}\lambda _{m}^{-1}\right) .\qquad \end{aligned}$$

(A.16)

Now Lemma A.2 follows from (A.13)–(A.16). $\square$

Lemma A.3

Under Assumptions 1, 2, 4 and 5, it holds that

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-1} \left( \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\right) ^{2} =O_{p}(nm+\lambda _{m}^{-2}m^{4}\log m). \end{aligned}$$

Proof

Let $Z_{ir}^{*}=Z_{ir}-\sum _{j'=1}^{m}\frac{1}{\lambda _{j'}} \left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\xi _{lj'}\right) \xi _{ij'}$. Observe that

$$\begin{aligned} \left( \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\right) ^{2}\le & {} 4\left( \sum _{i=1}^{n}\xi _{ij}Z_{ir}^{*}\right) ^{2} \nonumber \\&+\,4\left( \sum _{i=1}^{n}\xi _{ij}\sum _{j'=1}^{m}\frac{1}{\lambda _{j'}} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\left( \hat{\xi }_{lj'}-\xi _{lj'}\right) \right] \xi _{ij'}\right) ^{2} \nonumber \\&+\, 4\left( \sum _{i=1}^{n}\xi _{ij}\sum _{j'=1}^{m} \left( \frac{1}{\hat{\lambda }_{j'}}-\frac{1}{\lambda _{j'}}\right) \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj'}\right] \xi _{ij'}\right) ^{2} \nonumber \\&+\,4\left( \sum _{i=1}^{n}\xi _{ij}\sum _{j'=1}^{m} \frac{1}{\hat{\lambda }_{j'}} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj'}\right] (\hat{\xi }_{ij'}-\xi _{ij'})\right) ^{2} \nonumber \\=: & {} 4(T_{j1}+T_{j2}+T_{j3}+T_{j4}). \end{aligned}$$

(A.17)

By direct computation and using Assumption 1, we get

$$\begin{aligned} E\left( \xi _{ij}^{2}{Z_{ir}^{*}}^{2}\right)\le & {} 2E\left( \xi _{ij}^{2}Z_{ir}^{2}\right) +2E \left[ \xi _{ij}^{2}\left( \sum _{j'=1}^{m}\frac{1}{\lambda _{j'}} \left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\xi _{lj'}\right) \xi _{ij'}\right) ^{2}\right] \\\le & {} C\left( \lambda _{j}+m\lambda _{j}/n^{2} +(n-1)m\lambda _{j}/n^{2}+m^{2}\lambda _{j}/n^{2}\right) \le C\lambda _{j} \end{aligned}$$

and

$$\begin{aligned} \left| \sum _{i_{1}\ne i_{2}}E\left( \xi _{i_{1}j}\xi _{i_{2}j}{Z_{i_{1}r}^{*}}{Z_{i_{2}r}^{*}}\right) \right| \le C[(n-1)(n+2)\lambda _{j}/n+(n-1)m\lambda _{j}/n]\le Cn\lambda _{j}. \end{aligned}$$

Hence

$$\begin{aligned} E(T_{j1})=\sum _{i=1}^{n}E\left( \xi _{ij}^{2}{Z_{ir}^{*}}^{2}\right) +\sum _{i_{1}\ne i_{2}}E\left( \xi _{i_{1}j}\xi _{i_{2}j}{Z_{i_{1}r}^{*}}{Z_{i_{2}r}^{*}}\right) \le Cn\lambda _{j}. \end{aligned}$$

(A.18)

Since $\sum _{j'=1}^{m}\frac{1}{\lambda _{j'}}E\left( \sum _{i=1}^{n}\xi _{ij}\xi _{ij'}\right) ^{2}\le Cn^{2}\lambda _{j}$, then by (A.6), we have

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-1}T_{j2}\le & {} \left( \sum _{j'=1}^{m}\frac{1}{\lambda _{j'}}\left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\hat{\xi }_{lj'}- \xi _{lj'})\right] ^{2}\right) \nonumber \\&\times \left( \sum _{j=1}^{m}\lambda _{j}^{-1}\sum _{j'=1}^{m}\frac{1}{\lambda _{j'}}\left( \sum _{i=1}^{n}\xi _{ij}\xi _{ij'}\right) ^{2}\right) \nonumber \\= & {} O_{p}(n^{-2}\lambda _{m}^{-2}m^{3}\log m)O_{p}(n^{2}m)=O_{p}(\lambda _{m}^{-2}m^{4}\log m).\qquad \end{aligned}$$

(A.19)

Similar to the proof (A.8) and using Assumption 4, we deduce that

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-1}T_{j3}\le & {} \left( \sum _{j'=1}^{m}\lambda _{j'} \left( \frac{1}{\hat{\lambda }_{j'}}-\frac{1}{\lambda _{j'}}\right) ^{2} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj'}\right] ^{2}\right) \nonumber \\&\times \left( \sum _{j=1}^{m}\lambda _{j}^{-1}\sum _{j'=1}^{m} \frac{1}{\lambda _{j'}}\left( \sum _{i=1}^{n}\xi _{ij}\xi _{ij'}\right) ^{2}\right) \nonumber \\= & {} O_{p}\left( \lambda _{m}^{-2}m^{2}+n^{-1}\lambda _{m}^{-4}m^{4}\log m\right) =o_{p}\left( \lambda _{m}^{-2}m^{2}\log m\right) .\qquad \end{aligned}$$

(A.20)

and

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-1}T_{j4}\le & {} \left( \sum _{j'=1}^{m}\frac{1}{\lambda _{j'}^{2}} \left[ \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\hat{\xi }_{lj'}\right] ^{2}\right) [1+o_{p}(1)] \nonumber \\&\quad \times \left( \sum _{j=1}^{m}\frac{1}{\lambda _{j}} \sum _{i=1}^{n}\xi _{ij}^{2}\right) \left( \sum _{j'=1}^{m}\sum _{i=1}^{n}(\hat{\xi }_{ij'}-\xi _{ij'})^{2}\right) \nonumber \\= & {} O_{p}\left( n^{-1}\lambda _{m}^{-1}m^{5}\log m+n^{-2}\lambda _{m}^{-3}m^{7}(\log m)^{2}\right) =o_{p}\left( \lambda _{m}^{-2}m^{4}\log m\right) .\nonumber \\ \end{aligned}$$

(A.21)

Now Lemma A.3 follows from (A.17)–(A.21) and Assumption 4. $\square$

Lemma A.4

Under Assumptions 1–5, it holds that

$$\begin{aligned} n^{-1/2}\left| \sum _{j=1}^{m}\frac{1}{\hat{\lambda }_{j}} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) \sum _{i=1}^{n}(\hat{\xi }_{ij}-\xi _{ij})\tilde{Z}_{ir}\right| =o_{p}(1). \end{aligned}$$

Proof

Let $\breve{W}_{j}=\frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}$. Applying the Cauchy–Schwarz inequality, we get

$$\begin{aligned} \left( \sum _{j=1}^{m}\frac{1}{\hat{\lambda }_{j}}\breve{W}_{j}\sum _{i=1}^{n}(\hat{\xi }_{ij}-\xi _{ij})\tilde{Z}_{ir}\right) ^{2}\le \left( \sum _{j=1}^{m}\frac{1}{\hat{\lambda }_{j}^{2}}\breve{W}_{j}^{2}\right) \left( \sum _{j=1}^{m}\left( \sum _{i=1}^{n}(\hat{\xi }_{ij}-\xi _{ij}) \tilde{Z}_{ir}\right) ^{2}\right) . \end{aligned}$$

Using (A.4), (A.5), Assumption 4 and Parseval’s identity and the arguments similar to those used to prove Lemma A.3, we deduce that

$$\begin{aligned}&\sum _{j=1}^{m} \left( \sum _{i=1}^{n}(\hat{\xi }_{ij}-\xi _{ij})\tilde{Z}_{ir}\right) ^{2} \\&\quad \le 2\sum _{j=1}^{m}\left[ \left( \sum _{k\ne j}(\hat{\lambda }_{j}-\lambda _{k})^{-1}\int \Delta \hat{\phi }_{j}\phi _{k} \sum _{i=1}^{n}\xi _{ik}\tilde{Z}_{ir}\right) ^{2}\right. \\&\qquad \left. +\left( \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\right) ^{2} \left( \int (\hat{\phi }_{j}-\phi _{j})\phi _{j}\right) ^{2}\right] \\&\le C|\Vert \Delta \Vert |^{2}\sum _{j=1}^{m}\left[ \sum _{k\ne j}\left( \hat{\lambda }_{j}-\lambda _{k}\right) ^{-2} \left( \sum _{i=1}^{n}\xi _{ik}\tilde{Z}_{ir}\right) ^{2} +\lambda _{j}^{-2}j^{2}\left( \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\right) ^{2}\right] \\&=O_{p} \left( \lambda _{m}^{-1}m^{3}\log m+n^{-1}\lambda _{m}^{-3}m^{6}\log m\right) =o_{p}(n). \end{aligned}$$

Let $\vec {W}_{j}=\frac{1}{n}\sum _{l=1}^{n}W_{l}\xi _{lj}$. Decomposing $\frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}=\vec {W}_{j}+\frac{1}{n}\sum _{l=1}^{n}W_{l}(\hat{\xi }_{lj} -\xi _{lj})$ and using arguments similar to those used in the proof of (A.6) and using Assumption 4 , we obtain that

$$\begin{aligned} \sum _{j=1}^{m}\frac{1}{\hat{\lambda }_{j}^{2}}\breve{W}_{j}^{2}=O_{p}(n^{-1}\lambda _{m}^{-1}m+1+n^{-2}\lambda _{m}^{-3}m^{3}\log m+n^{-1}\lambda _{m}^{-2}m)=O_{p}(1). \end{aligned}$$

This finishes the proof of Lemma A.4. $\square$

Lemma A.5

Under Assumptions 1–5, it holds that

$$\begin{aligned} n^{-1/2}\left| \sum _{i=1}^{n}\tilde{W}_{i}\tilde{Z}_{ir}\right| =o_{p}(1). \end{aligned}$$

Proof

Observe that

$$\begin{aligned} \sum _{i=1}^{n}\tilde{W}_{i}\tilde{Z}_{ir}= & {} \sum _{j=1}^{m}\left[ \gamma _{j}-\frac{1}{\hat{\lambda }_{j}} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) \right] \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\nonumber \\&-\sum _{j=1}^{m}\frac{1}{\hat{\lambda }_{j}} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) \sum _{i=1}^{n}\left( \hat{\xi }_{ij}-\xi _{ij}\right) \tilde{Z}_{ir}\nonumber \\&+\sum _{j=m+1}^{\infty }\gamma _{j} \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir} \end{aligned}$$

(A.22)

Lemmas A.2 and A.3 and Assumption 4 imply that

$$\begin{aligned}&n^{-\frac{1}{2}}\left| \sum _{j=1}^{m}\left[ \gamma _{j}-\frac{1}{\hat{\lambda }_{j}}\left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) \right] \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\right| \nonumber \\&\quad \le n^{-\frac{1}{2}}\left( \sum _{j=1}^{m}\lambda _{j}\left[ \gamma _{j}-\frac{1}{\hat{\lambda }_{j}} \left( \frac{1}{n}\sum _{l=1}^{n}W_{l}\hat{\xi }_{lj}\right) \right] ^{2}\right) ^{\frac{1}{2}} \left( \sum _{j=1}^{m}\lambda _{j}^{-1}(\sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir})^{2}\right) ^{\frac{1}{2}} \nonumber \\&\quad =O_{p}\left( n^{-1/2}\lambda _{m}^{-1/2}m+n^{-1}\lambda _{m}^{-3/2}m^{5/2}(\log m)^{1/2}\right) =o_{p}(1). \end{aligned}$$

(A.23)

By arguments similar to those used in the proof of Lemma A.3, we obtain that

$$\begin{aligned}&\left( \sum _{j=m+1}^{\infty }\gamma _{j} \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\right) ^{2}\nonumber \\&\quad \le \left( \sum _{j=m+1}^{\infty }\gamma _{j}^{2}\right) \left( \sum _{j=m+1}^{\infty } \left( \sum _{i=1}^{n}\xi _{ij}\tilde{Z}_{ir}\right) ^{2}\right) \nonumber \\&\quad =O_{p}\left( nm^{-2\gamma +1}+\lambda _{m}^{-2}m^{-2\gamma +4}\log m\right) \sum _{j=m+1}^{\infty }\lambda _{j}\nonumber \\&\quad =o_{p}(n). \end{aligned}$$

(A.24)

Now Lemma A.5 follows from (A.22)–(A.24) and Lemma A.4. $\square$

Proof of Theorem 2.1

By arguments similar to those used to prove Lemmas A.4 and A.5, we deduce that $n^{-1/2}\sum _{i=1}^{n}\left( \frac{1}{n}\sum _{l=1}^{n}\varepsilon _{l}\tilde{\xi }_{li}\right) \tilde{Z}_{ir}=o_{p}(1)$. Hence

$$\begin{aligned} n^{-\frac{1}{2}}\sum _{i=1}^{n}\tilde{Z}_{ir}\tilde{\varepsilon }_{i}=n^{-\frac{1}{2}}\sum _{i=1}^{n}\tilde{Z}_{ir}\varepsilon _{i}+o_{p}(1). \end{aligned}$$

We decompose $\sum _{i=1}^{n}\tilde{Z}_{ir}\varepsilon _{i}$ into three terms as

$$\begin{aligned} \sum _{i=1}^{n}\tilde{Z}_{ir}\varepsilon _{i}= & {} \sum _{i=1}^{n}\varepsilon _{i}\left( Z_{ir}-\sum _{j=1}^{m}\frac{E(Z_{lr}\xi _{j})}{\lambda _{j}}\xi _{ij}\right) -\sum _{i=1}^{n}\varepsilon _{i}\sum _{j=1}^{m}\frac{\xi _{ij}}{\lambda _{j}}\nonumber \\&\left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\xi _{lj} -E(Z_{lr}\xi _{j})\right) -\sum _{i=1}^{n}\varepsilon _{i}\frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\tilde{\xi }_{li}-\vec {\xi }_{li}). \end{aligned}$$

Similar to the proof of Lemma A.4, we have $\sum _{i=1}^{n}\varepsilon _{i} \frac{1}{n}\sum _{l=1}^{n}Z_{lr}(\tilde{\xi }_{li}-\vec {\xi }_{li})=o_{p}(n)$. Since

$$\begin{aligned} \sum _{i=1}^{n}\varepsilon _{i}\left( Z_{ir}-\sum _{j=1}^{m}\frac{E(Z_{lr}\xi _{j} )}{\lambda _{j}}\xi _{ij}\right) =\sum _{i=1}^{n}\varepsilon _{i}H_{ir}+\sum _{i=1} ^{n}\varepsilon _{i}\sum _{j=m+1}^{\infty }g_{rj}\xi _{ij}, \end{aligned}$$

$\sum _{i=1}^{n}\varepsilon _{i}\sum _{j=1}^{m}\frac{\xi _{ij}}{\lambda _{j} }\left( \frac{1}{n}\sum _{l=1}^{n}Z_{lr}\xi _{lj}-E(Z_{lr}\xi _{j})\right) =o_{p}(n)$ and $\sum _{i=1}^{n}\varepsilon _{i}\sum _{j=m+1}^{\infty }g_{kj}\xi _{ij} =o_{p}(n)$, it follows that

$$\begin{aligned} n^{-\frac{1}{2}}\sum _{i=1}^{n}\tilde{Z}_{ir}\tilde{\varepsilon }_{i}=n^{-\frac{1}{2}}\sum _{i=1}^{n}H_{ir}\varepsilon _{i}+o_{p}(1). \end{aligned}$$

(A.25)

Now (2.9) follows from (A.1), Lemmas A.1 and A.5, (A.25) and the central limit theorem. The proof of Theorem 2.1 is finished. $\square$

Lemma A.6

Define $\check{\gamma }_{j}=\frac{1}{\hat{\lambda }_{j} }E[(Y-Z^{T}\pmb {\beta }_{0})\xi _{j}]$. Under the assumptions of Theorem 3.2, it holds that

$$\begin{aligned} \sum _{j=1}^{m}(\hat{\gamma }_{j}-\check{\gamma }_{j})^{2} =O_{p}\left( n^{-1}m\lambda _{m}^{-1} +n^{-2}m\lambda _{m}^{-2}\sum _{j=1}^{m}\gamma _{j}^{2}\lambda _{j}^{-2}j^{3}\right) . \end{aligned}$$

Proof

Define $I_{1}=\frac{1}{n}\sum _{i=1}^{n} (Y_{i}-Z_{i}^{T}\pmb {\beta }_{0})\xi _{ij} -\gamma _{j}\lambda _{j}$, $I_{2}=\frac{1}{n}\sum _{i=1}^{n}(Y_{i}-Z_{i}^{T}\pmb {\beta }_{0})(\hat{\xi }_{ij}-\xi _{ij})$ and $I_{3}=\frac{1}{n}\sum _{i=1}^{n}Z_{i}^{T}(\hat{\pmb {\beta }}-\pmb {\beta }_{0})\hat{\xi }_{ij}$. Noting that $E[(Y-Z^{T}\pmb {\beta }_{0})\xi _{j}]=\gamma _{j}\lambda _{j}$, we have

$$\begin{aligned} \sum _{j=1}^{m}(\hat{\gamma }_{j}-\check{\gamma }_{j})^{2}\le 3\sum _{j=1} ^{m}\lambda _{j}^{-2}\left( I_{1}^{2}+I_{2}^{2}+I_{3}^{2}\right) [1+o_{p} (1)], \end{aligned}$$

(A.26)

where $o_{p}(1)$ holds uniformly for $j=1,\ldots ,m$. Since $E(I_{1})=0$ and $E(I_{1}^{2})\le \frac{1}{n}[\sum _{k=1}^{\infty }\gamma _{k} ^{2}E(\xi _{k}^{2}\xi _{j}^{2})+\sigma ^{2}\lambda _{j}]\le C\lambda _{j}/n$, we obtain that

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-2}I_{1}^{2}=O_{p}\left( n^{-1}\sum _{j=1} ^{m}\lambda _{j}^{-1}\right) =O_{p}(n^{-1}m\lambda _{m} ^{-1}). \end{aligned}$$

(A.27)

Let $M(t)=E[(Y_{i}-Z_{i}^{T}\pmb {\beta }_{0})X_{i}(t)] =\sum _{k=1}^{\infty }\gamma _{k}\lambda _{k}\phi _{k}(t)$. Then

$$\begin{aligned} I_{2}^{2}\le & {} 2\int _{{\mathcal {T}}}\left( \frac{1}{n}\sum _{i=1}^{n}(Y_{i} -Z_{i}^{T}\pmb {\beta }_{0})X_{i}(t)-M(t)\right) ^{2} \mathrm{d}t\Vert \hat{\phi }_{j}-\phi _{j}\Vert ^{2}\\&+\,2\left( \int _{{\mathcal {T}}}M(t)(\hat{\phi }_{j}(t)-\phi _{j}(t))\mathrm{d}t\right) ^{2}. \end{aligned}$$

Applying Assumption 1, it holds that

$$\begin{aligned}&E\left( \int _{{\mathcal {T}}}\left( \frac{1}{n}\sum _{i=1}^{n}(Y_{i}-Z_{i}^{T}\pmb {\beta }_{0})X_{i}(t)-M(t)\right) ^{2} \mathrm{d}t\right) \\&\quad \le \frac{1}{n}\int _{{\mathcal {T}}}E[(Y_{i}-Z_{i}^{T}\pmb {\beta }_{0})^{2}X_{i}^{2}(t)]\mathrm{d}t=O(n^{-1}). \end{aligned}$$

From (A.9), we obtain $\sum _{j=1}^{m}\lambda _{j}^{-2}\Vert \hat{\phi }_{j}-\phi _{j}\Vert ^{2}=O_{p}(n^{-1}m^{3}\lambda _{m}^{-2} \log m)$. By arguments similar to those used in the proof of (5.15) of Hall and Horowitz (2007), it follows that

$$\begin{aligned}&\sum _{j=1}^{m}\lambda _{j}^{-2}\left( \int _{{\mathcal {T}}}M(t)(\hat{\phi }_{j}(t)-\phi _{j}(t))\mathrm{d}t\right) ^{2}\\&\quad =O_{p}\left( \frac{m}{n\lambda _{m} }+\frac{m}{n^{2}\lambda _{m}^{2}}\sum _{j=1}^{m} \gamma _{j}^{2}\lambda _{j}^{-2}j^{3}+\frac{m^{3}\log m}{n^{2} \lambda _{m}^{2}}\right) . \end{aligned}$$

Hence, using the assumption that $n^{-1}m^{2}\lambda _{m} ^{-1}\log m\rightarrow 0$, we obtain

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-2}I_{2}^{2}=O_{p}\left( n^{-1}m \lambda _{m}^{-1}+n^{-2}m\lambda _{m}^{-2}\sum _{j=1}^{m}\gamma _{j}^{2}\lambda _{j}^{-2}j^{3}\right) . \end{aligned}$$

(A.28)

Using Theorem 3.1, it holds that

$$\begin{aligned} \sum _{j=1}^{m}\lambda _{j}^{-2}I_{3}^{2}\le & {} \left( \sum _{j=1} ^{m}\frac{1}{n\lambda _{j}^{2}}\sum _{i=1}^{n}\hat{\xi }_{ij} ^{2}\right) \left( \frac{1}{n}\sum _{i=1}^{n}[Z_{i}^{T}(\hat{\pmb {\beta }}-\pmb {\beta }_{0})]^{2}\right) \nonumber \\= & {} O_{p}\left( m\lambda _{m}^{-1}+n^{-1}m^{3}\lambda _{m}^{-2}\log m\right) O_{p}(n^{-1})=O_{p}\left( n^{-1}m\lambda _{m}^{-1}\right) .\nonumber \\ \end{aligned}$$

(A.29)

Now Lemma A.6 follows from combining (A.26)–(A.29). $\square$

Proof of Theorem 2.2

Note that

$$\begin{aligned} \int _{{\mathcal {T}}}[\hat{\gamma }(t)-\gamma (t)]^{2}\mathrm{d}t\le & {} C\left( \sum _{j=1}^{m}(\hat{\gamma }_{j}-\check{\gamma }_{j})^{2}\right. \nonumber \\&\left. +\sum _{j=1}^{m}(\check{\gamma }_{j}-\gamma _{j})^{2} +m\sum _{j=1}^{m}\gamma _{j}^{2}\Vert \hat{\phi }_{j}-\phi _{j}\Vert ^{2} +\sum _{j=m+1}^{\infty }\gamma _{j}^{2}\right) \nonumber \\ \end{aligned}$$

(A.30)

and

$$\begin{aligned} \sum _{j=1}^{m}(\check{\gamma }_{j}-\gamma _{j})^{2} =\sum _{j=1}^{m}\frac{(\hat{\lambda }_{j}-\lambda _{j})^{2}}{\lambda _{j}^{2}}\gamma _{j}^{2}[1+o_{p}(1)] =O_{p}\left( n^{-1}\lambda _{m}^{-1}\sum _{j=1}^{m}\gamma _{j}^{2}\lambda _{j}^{-1}\right) .\nonumber \\ \end{aligned}$$

(A.31)

Assumption 3 implies that $m\sum _{j=1}^{m}\gamma _{j}^{2}\Vert \hat{\phi }_{j}-\phi _{j}\Vert ^{2}=O_{p}(mn^{-1}\sum _{j=1}^{m}\gamma _{j}^{2}j^{2}\log j)=o_{p}(m/n)$ and $\sum _{j=m+1}^{\infty }\gamma _{j}^{2}=O(m^{-2\gamma +1})$. Now (2.10) follows from Lemma A.6, (A.30) and (A.31). The proof of Theorem 2.2 is finished. $\square$

Proof of Theorem 2.3

Observe that

$$\begin{aligned} \begin{array} [c]{l} \text{ MSPE }\le 2\{\Vert \hat{\gamma }-\gamma \Vert _{K}^{2}+(\hat{\pmb {\beta }} -\pmb {\beta }_{0})^{T}E(ZZ^{T})(\hat{\pmb {\beta }}-\pmb {\beta }_{0})\}, \end{array} \end{aligned}$$

(A.32)

where $\Vert \hat{\gamma }-\gamma \Vert _{K}^{2}=\int _{{\mathcal {T}}}\int _{{\mathcal {T}} }K(s,t)[\hat{\gamma }(s)-\gamma (s)][\hat{\gamma }(t)-\gamma (t)]\mathrm{d}s\mathrm{d}t$. Under the assumptions of Theorem 2.3, using arguments similar to those used in the proof of Theorem 2 of Tang (2015), we deduce that $\Vert \hat{\gamma }-\gamma \Vert _{K}^{2}=O_{p}(n^{-(\tau +2\delta -1)/(\tau +2\delta )})$. Now (2.12) follows from (A.32) and Theorem 2.1. The proof of Theorem 2.3 is finished. $\square$

Lemma A.7

Under the assumptions of Theorem 3.1, there exists a local minimizer $\hat{\pmb {\beta }}$ of (3.1) such that $\Vert \hat{ \pmb {\beta }}-\pmb {\beta }_{0}\Vert =O_{p}(n^{-1/2})$.

Proof

Let

$$\begin{aligned} P_{n}(\pmb {\beta })=n\sum _{k=1}^{d}p_{\nu _{n}}^{\prime }\left( \left| \beta _{k}^{(0)} \right| \right) |\beta _{k}|, \quad P_{n1}(\pmb {\beta })=n\sum _{k=1}^{d_{1}}p_{\nu _{n}}^{\prime }\left( \left| \beta _{k}^{(0)} \right| \right) |\beta _{k}| \end{aligned}$$

and $D_{n}(\pmb {\beta })=(\tilde{Y}-\tilde{Z}\pmb {\beta })^{T}(\tilde{Y}-\tilde{Z}\pmb {\beta })+P_{n}(\pmb {\beta })$. It suffices to prove that for any given $\varepsilon >0$, there exists a constant C such that

$$\begin{aligned} P\left\{ \sup _{\Vert u\Vert =C}D_{n}(\pmb {\beta }_{0}+n^{-1/2}u)>D_{n}(\pmb {\beta }_{0})\right\} \ge 1-\varepsilon . \end{aligned}$$

(A.33)

Note that

$$\begin{aligned} D_{n}(\pmb {\beta }_{0}+n^{-1/2}u)-D_{n}(\pmb {\beta }_{0})\ge & {} -2n^{-1/2}(\tilde{Y}-\tilde{Z}\pmb {\beta }_{0})^{T}\tilde{Z}u+n^{-1}u^{T}\tilde{Z}^{T}\tilde{Z}u \nonumber \\&+\,\left[ P_{n1}\left( \pmb {\beta }_{01}+n^{-1/2}u_{1}\right) -P_{n1}(\pmb {\beta }_{01})\right] \qquad \qquad \end{aligned}$$

(A.34)

and

$$\begin{aligned} (\tilde{Y}-\tilde{Z}\pmb {\beta }_{0})^{T}\tilde{Z}=(\tilde{W}+\tilde{\varepsilon })^{T}\tilde{Z}. \end{aligned}$$

By Lemma A.5, we have that $n^{-1/2}\tilde{W}^{T}\tilde{Z}=o_{p}(1)$. By (A.25), it follows that $n^{-1/2}\tilde{\varepsilon }^{T}\tilde{Z}=O_{p}(1)$. By Theorem 2.1, it holds that $\pmb {\beta }^{(0)}\rightarrow _{P}\pmb {\beta }_{0}$, and we then have $P\{P_{n1}( \pmb {\beta }_{01}+n^{-1/2}u_{1})-P_{n1}(\pmb {\beta }_{01})=0\}\rightarrow 1$ as $n\rightarrow \infty$. Hence, for sufficiently large C, (A.33) follows from (A.34) and Lemma A.1 and the fact that $\Omega$ is positive definite. The proof of Lemma A.7 is complete. $\square$

Proof of Theorem 3.1

We first prove that for any $\pmb {\beta }=(\pmb {\beta }_{1}^{T},\pmb {\beta }_{2}^{T})^{T}$ in the neighborhood $\Vert \pmb {\beta }-\pmb {\beta }_{0}\Vert =O(n^{-1/2})$ for sufficiently large n and $\pmb {\beta } _{2}\ne \pmb {0}$, with probability tending to 1, we have

$$\begin{aligned} D_{n}((\pmb {\beta }_{1},\pmb {\beta }_{2}))>D_{n}((\pmb {\beta }_{1},\pmb {0})). \end{aligned}$$

(A.35)

Observe that

$$\begin{aligned} D_{n}\left( \left( \pmb {\beta }_{1},\pmb {\beta }_{2}\right) \right) -D_{n}\left( \left( \pmb {\beta }_{1},\pmb {0}\right) \right)= & {} -2\left( \tilde{W}-\tilde{Z}\left( \left( \pmb {\beta }_{1}-\pmb {\beta }_{01}\right) ^{T},\pmb {0}^{T}\right) ^{T}+\tilde{\varepsilon }\right) ^{T}\tilde{Z}\left( \pmb {0}^{T},\pmb {\beta }_{2}^{T}\right) ^{T} \\&+\left( \pmb {0}^{T},\pmb {\beta }_{2}^{T}\right) \tilde{Z}^{T}\tilde{Z}\left( \pmb {0}^{T},\pmb {\beta }_{2}^{T}\right) ^{T} +n\sum _{k=d_{1}}^{d}p_{\nu _{n}}^{\prime }\left( \left| \beta _{k}^{(0)} \right| \right) |\beta _{k}| \end{aligned}$$

By Lemma A.5, we have that $n^{-1/2}\tilde{W}^{T}\tilde{Z}=o_{p}(1)$. By (A.25), it follows that $n^{-1/2}\tilde{\varepsilon }^{T}\tilde{Z}=O_{p}(1)$. Hence, using Lemma A.1 and the fact that $\Vert \pmb {\beta }_{2}\Vert =O(n^{-1/2})$ and $n^{1/2}\nu _{n}\rightarrow +\,\infty$ and the result of Theorem 2.1, we deduce that with probability tending to 1, it holds that

$$\begin{aligned}&D_{n}\left( \left( \pmb {\beta }_{1},\pmb {\beta }_{2}\right) \right) -D_{n}\left( \left( \pmb {\beta }_{1},\pmb {0}\right) \right) \\&\quad =O_{p}\left( n^{1/2}\right) \sum _{k=d_{1}}^{d}|\beta _{k}|+n\sum _{k=d_{1}}^{d}p_{\nu _{n}}^{\prime }\left( \left| \beta _{k}^{\left( 0\right) } \right| \right) |\beta _{k}| \\&\quad =n\nu _{n}\sum _{k=d_{1}}^{d}\left[ O_{p}\left( \left( n^{1/2} \nu _{n}\right) ^{-1}\right) +\nu _{n}^{-1}p_{\nu _{n}}^{ \prime }\left( \left| \beta ^{\left( 0\right) }_{k}\right| \right) \right] |\beta _{k}|>0. \end{aligned}$$

By Lemma A.7 and (A.35), there exists a $\sqrt{n}$-consistent local minimizer $\check{\pmb {\beta }}=(\check{ \pmb {\beta }}_{1},\pmb {0}^{T})^{T}$ of (3.1). Note that

$$\begin{aligned} D_{n}((\hat{\pmb {\beta }}_{\mathrm{PLS}1},\hat{\pmb {\beta }}_{\mathrm{PLS}2}))= & {} D_{n}((\check{\pmb {\beta }}_{1},\pmb {0}))-2\sqrt{n}\left[ n^{-1/2}(\tilde{Y}-\tilde{Z}\check{\pmb {\beta }})^{T}\tilde{Z}(\hat{\pmb {\theta }}_\mathrm{PLS}-\check{ \pmb {\beta }}) \right. \nonumber \\&+\,n^{-1/2}(\hat{\pmb {\theta }}_\mathrm{PLS}-\check{\pmb {\beta }})^{T}\tilde{Z}^{T}\tilde{Z}(\hat{\pmb {\theta }}_\mathrm{PLS}-\check{ \pmb {\beta }})\nonumber \\&\left. +\,\sqrt{n}\sum _{k=d_{1}+1}^{d}p_{\nu _{n}}^{\prime }\left( \left| \beta _{k}^{(0)}\right| \right) |\hat{\beta }_{PLSk}|\right] , \end{aligned}$$

(A.36)

where $\hat{\pmb {\beta }}_\mathrm{PLS}=(\hat{\beta }_{\mathrm{PLS}1},\ldots ,\hat{\beta }_{PLSd})^{T}$. Write $\tilde{Z}=(\tilde{\pmb {Z}}_{1},\tilde{\pmb {Z}}_{2})$. Since $\hat{\pmb {\beta }}_\mathrm{PLS}$ is a minimizers of (3.1) and $\check{\pmb {\beta }}$ is a local minimizer of (3.1), we then have that

$$\begin{aligned} (\tilde{Y}-\tilde{Z}\check{\pmb {\beta }})^{T}\tilde{Z}(\hat{\pmb {\theta }}_\mathrm{PLS}-\check{ \pmb {\beta }})=(\tilde{W}+\tilde{\varepsilon })^{T}\tilde{\pmb {Z}}_{2}\hat{\pmb {\theta }}_{\mathrm{PLS}2}+(\pmb {\beta }_{0}-\check{\pmb {\beta }})\tilde{Z}^{T}\tilde{\pmb {Z}}_{2}\hat{\pmb {\theta }}_{\mathrm{PLS}2}. \end{aligned}$$

(A.37)

By Lemma A.5, we have that $n^{-1/2}\tilde{W}^{T}\tilde{\pmb {Z}}_{2}=o_{p}(1)$. By (A.25), it follows that $n^{-1/2}\tilde{\varepsilon }^{T}\tilde{\pmb {Z}}_{2}=O_{p}(1)$. The fact that $\pmb {\beta }_{0}-\check{\pmb {\beta }}=O_{p}(n^{-1/2})$ and Lemma A.1 imply that $n^{-1/2}(\pmb {\beta }_{0}-\check{\pmb {\beta }})\tilde{Z}^{T}\tilde{\pmb {Z}}_{2}=O_{p}(1)$. If $\hat{\pmb {\beta }}_\mathrm{PLS}\ne \check{ \pmb {\beta }}$, under the assumptions of Theorem 3.1, then by (A.36) and (A.37), we have $D_{n}((\hat{\pmb {\beta }}_{\mathrm{PLS}1},\hat{\pmb {\beta }}_{\mathrm{PLS}2}))> D_{n}((\check{\pmb {\beta }}_{1},\pmb {0}))$. This is a contradiction to the fact that $\hat{\pmb {\beta }}_\mathrm{PLS}$ is a minimizer of (3.1). So $\hat{\pmb {\beta }}_{\mathrm{PLS}2}=0$ and $\hat{\pmb {\beta }}_{\mathrm{PLS}1}=\check{\pmb {\beta }}_{1}$.

We now prove asymptotic normality part. Consider $D_{n}((\pmb {\beta }_{1}, \pmb {0}))$ as a function of $\pmb {\beta }_{1}$. Noting that with probability tending 1, $\hat{\pmb {\beta }}_{\mathrm{PLS}1}$ is the $\sqrt{n}$-consistent minimizer of $D_{n}((\pmb {\beta }_{1},\pmb {0}))$ and satisfies

$$\begin{aligned} \frac{\partial D_{n}((\pmb {\beta }_{1},\pmb {0}))}{\partial \pmb {\beta }_{1}} \left| _{\pmb {\beta }_{1}=\hat{\pmb {\beta }}_{\mathrm{PLS}1}}=-2\tilde{\pmb {Z}}_{1}^{T}(\tilde{Y}-\tilde{Z}\hat{\pmb {\beta }}_\mathrm{PLS})=0 \right. \end{aligned}$$

Hence

$$\begin{aligned} \hat{\pmb {\beta }}_{\mathrm{PLS}1}-\pmb {\beta }_{01}= \left( \tilde{\pmb {Z}}_{1}^{T}\tilde{\pmb {Z}}_{1}\right) ^{-1}\tilde{\pmb {Z}}_{1}^{T}\tilde{Y}. \end{aligned}$$

By arguments similar to those used in the proof of (2.9), we can prove (3.2). The proof of Theorem 3.1 is finished. $\square$

Proof of Theorem 3.2

Similar to the proofs of Theorems 2.2 and 2.3, we can complete the proof of Theorem 3.2. $\square$