Joint feature screening for ultra-high-dimensional sparse additive hazards model by the sparsity-restricted pseudo-score estimator

Abstract

Due to the coexistence of ultra-high dimensionality and right censoring, it is very challenging to develop feature screening procedure for ultra-high-dimensional survival data. In this paper, we propose a joint screening approach for the sparse additive hazards model with ultra-high-dimensional features. Our proposed screening is based on a sparsity-restricted pseudo-score estimator which could be obtained effectively through the iterative hard-thresholding algorithm. We establish the sure screening property of the proposed procedure theoretically under rather mild assumptions. Extensive simulation studies verify its improvements over the main existing screening approaches for ultra-high-dimensional survival data. Finally, the proposed screening method is illustrated by dataset from a breast cancer study.

Appendix: Proofs of the theorems

To prove Theorem 1, we firstly give a large deviation result for martingale under the additive hazards model. This result could be proved along the similar line as those for Theorem 3.1 in Bradic et al. (2011). So we omit the proof here for simplicity.

Lemma 1

Under Assumptions 1–3, for any positive sequence $\{u_{n}\}$ bounded away from zero, if $\max \limits _{1 \le j\le p}\sigma _{j}^{2}=O(u_{n})$, there exist positive constants $c_{7}$ and $c_{8}$ such that

$$\begin{aligned} \mathrm {pr}(|\sqrt{n}U_{n,j}(\varvec{\beta }^{*})|>u_{n})\le c_{7}\mathrm {exp}(-c_{8}u_{n}) \end{aligned}$$

uniformly over j, where $U_{n,j}(\varvec{\beta }^{*})$ is the jth component of $\varvec{U}_{n}(\varvec{\beta }^{*})$.

This large deviation result represents a uniform, nonasymptotic exponential inequality for martingales under the additive hazards model and is independent of dimensionality p. So it will be very useful for the high-dimensional additive hazards model.

Proof of Theorem 1

Denote $\hat{\varvec{\beta }}_{M}$ to be the (unrestricted) pseudo-score estimator of $\varvec{\beta }$ based on model M. In order to establish the sure screening property, we just need to prove

$$\begin{aligned} \mathrm {pr}\big ({\hat{M}}\in {\mathbf {M}}_{+}^{k}\big )\longrightarrow 1, \end{aligned}$$

as n goes to $\infty$. It suffices to show

$$\begin{aligned} \mathrm {pr} \left( \max _{M\in {\mathbf {M}}_{-}^{k}}L_{n}(\hat{\varvec{\beta }}_{M}) \ge \min _{M\in {\mathbf {M}}_{+}^{k}}L_{n}(\hat{\varvec{\beta }}_{M})\right) \longrightarrow 0, \end{aligned}$$

as n goes to $\infty$.

For any $M\in {\mathbf {M}}_{-}^{k}$, let $M^{\prime }=M\cup M_{0}\in {\mathbf {M}}_{+}^{2k}$.

Firstly, consider $\varvec{\beta }_{M^{\prime }}$ being close to $\varvec{\beta }_{M^{\prime }}^{*}$ such that $\Vert \varvec{\beta }_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\Vert _{2}=c_{2}n^{-\tau _{1}}$. After some algebraic manipulations, we have

$$\begin{aligned}&L_{n}\big (\varvec{\beta }_{M^{\prime }}\big )-L_{n}\big (\varvec{\beta }_{M^{\prime }}^{*}\big ) \nonumber \\&\quad =\big (\varvec{\beta }_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\big )^{T} \varvec{U}_{n,M^{\prime }}\big (\varvec{\beta }_{M^{\prime }}^{*}\big ) -\frac{1}{2}\big (\varvec{\beta }_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\big )^{T} \varvec{V}_{n,M^{\prime }} \big (\varvec{\beta }_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\big ). \end{aligned}$$

Then, by the Cauchy–Schwartz inequality and Assumption 5, we can conclude that

$$\begin{aligned}&L_{n}\big (\varvec{\beta }_{M^{\prime }}\big )-L_{n}\big (\varvec{\beta }_{M^{\prime }}^{*}\big ) \nonumber \\&\quad \le \Vert \varvec{\beta }_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\Vert _{2} \Vert \varvec{U}_{n,M^{\prime }}\big (\varvec{\beta }_{M^{\prime }}^{*}\big )\Vert _{2} -\frac{c_{4}}{2}\Vert \varvec{\beta }_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\Vert _{2}^{2} \nonumber \\&\quad \le \,c_{2}n^{-\tau _{1}} \Vert \varvec{U}_{n,M^{\prime }}\big (\varvec{\beta }_{M^{\prime }}^{*}\big )\Vert _{2} -\frac{1}{2}c_{4}c_{2}^{2}n^{-2\tau _{1}}. \end{aligned}$$

Thus, we have

$$\begin{aligned}&\mathrm {pr}\left( L_{n}\left( \varvec{\beta }_{M^{\prime }}\right) -L_{n}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) \ge 0\right) \nonumber \\&\quad \le \,\mathrm {pr}\left( \Vert \varvec{U}_{n,M^{\prime }}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) \Vert _{2}\ge \frac{1}{2}c_{2}c_{4}n^{-\tau _{1}}\right) \nonumber \\&\quad \le \,\sum \limits _{j\in M^{\prime }} \mathrm {pr}\left( U_{n,j}^{2}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) \ge \frac{1}{2k}\left( \frac{1}{2}c_{2}c_{4}n^{-\tau _{1}}\right) ^{2}\right) \nonumber \\&\quad =\sum \limits _{j\in M^{\prime }} \mathrm {pr}\left( |U_{n,j}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) |\ge \left( \frac{1}{2k}\right) ^{1/2}\frac{1}{2}c_{2}c_{4}n^{-\tau _{1}}\right) , \end{aligned}$$

where the second inequality is obtained by Bonferroni inequality.

Because $M_{0}\subset M'$, we can get that $U_{n,j}(\varvec{\beta }_{M^{\prime }}^{*})=U_{n,j}(\varvec{\beta }^{*})$. Then under the conditions in Theorem 1 and by Lemma 1, we have

$$\begin{aligned}&\mathrm {pr}\left( |U_{n,j}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) |\ge \left( \frac{1}{2k}\right) ^{1/2}\frac{1}{2}c_{2}c_{4}n^{-\tau _{1}}\right) \nonumber \\&\quad =\mathrm {pr}\left( |\sqrt{n}U_{n,j}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) |\ge \left( \frac{n}{2k}\right) ^{1/2}\frac{1}{2}c_{2}c_{4}n^{-\tau _{1}}\right) \nonumber \\&\quad =\mathrm {pr}\left( |\sqrt{n}U_{n,j}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) |\ge \frac{1}{2\sqrt{2}}c_{2}c_{4}c_{3}^{-\frac{1}{2}}n^{\frac{1}{2}-\tau _{1}-\frac{\tau _{2}}{2}}\right) \nonumber \\&\quad \le \,c_{7}\mathrm {exp}(-c_{8}n^{\frac{1}{2}-\tau _{1}-\frac{\tau _{2}}{2}}). \end{aligned}$$

Then

$$\begin{aligned} \mathrm {pr}(L_{n}\big (\varvec{\beta }_{M^{\prime }}\big )-L_{n}\big (\varvec{\beta }_{M^{\prime }}^{*}\big )\ge 0) \le 2kc_{7}\mathrm {exp}\left( -c_{8}n^{\frac{1}{2}-\tau _{1}-\frac{\tau _{2}}{2}}\right) . \end{aligned}$$

Then, by the Bonferroni inequality and assumptions in Theorem 1, we can arrive at

$$\begin{aligned}&\mathrm {pr} \left( \max _{M\in {\mathbf {M}}_{-}^{k}}L_{n}\left( \varvec{\beta }_{M^{\prime }}\right) \ge L_{n}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) \right) \nonumber \\&\quad \le \,\sum \limits _{M\in {\mathbf {M}}_{-}^{k}} \mathrm {pr}\left( L_{n}\left( \varvec{\beta }_{M^{\prime }}\right) \ge L_{n}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) \right) \nonumber \\&\quad \le \,p^{k}2kc_{7}\mathrm {exp}\left( -c_{8}n^{\frac{1}{2}-\tau _{1}-\frac{\tau _{2}}{2}}\right) \nonumber \\&\quad \le \,2c_{7} \mathrm {exp}\left( \mathrm {log}(c_{3})+\tau _{2}\mathrm {log}(n)+c_{9}n^{m+\tau _{2}} -c_{8}n^{\frac{1}{2}-\tau _{1}-\frac{\tau _{2}}{2}}\right) \nonumber \\&\quad =o(1), \end{aligned}$$

where $c_{9}$ is a positive constant.

By the concavity of $L_{n}(\varvec{\beta }_{M^{\prime }})$, we can conclude that the above result holds for any $\varvec{\beta }_{M^{\prime }}$ that $\Vert \varvec{\beta }_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\Vert \ge c_{2}n^{-\tau _{1}}$.

For any $M\in {\mathbf {M}}_{-}^{k}$, let $\check{\varvec{\beta }}_{M^{\prime }}$ being $\hat{\varvec{\beta }}_{M}$ augmented with zeros corresponding to the elements in $M^{\prime }/M_{0}$. It is easy to see that $\Vert \check{\varvec{\beta }}_{M^{\prime }}-\varvec{\beta }_{M^{\prime }}^{*}\Vert \ge \Vert \varvec{\beta }_{M_{0}/M}^{*}\Vert \ge c_{2}n^{-\tau _{1}}$. So we have

$$\begin{aligned}&\mathrm {pr} \left( \max _{M\in {\mathbf {M}}_{-}^{k}}L_{n}(\hat{\varvec{\beta }}_{M}) \ge \min _{M\in {\mathbf {M}}_{+}^{k}}L_{n}(\hat{\varvec{\beta }}_{M})\right) \nonumber \\&\quad =\mathrm {pr} \left( \max _{M\in {\mathbf {M}}_{-}^{k}}L_{n}\left( \check{\varvec{\beta }}_{M^{\prime }}\right) \ge \min _{M\in {\mathbf {M}}_{+}^{k}}L_{n}(\hat{\varvec{\beta }}_{M})\right) \nonumber \\&\quad \le \,\mathrm {pr} \left( \max _{M\in {\mathbf {M}}_{-}^{k}}L_{n}\left( \check{\varvec{\beta }}_{M^{\prime }}\right) \ge L_{n}\left( \varvec{\beta }_{M^{\prime }}^{*}\right) \right) \nonumber \\&\quad =o(1). \end{aligned}$$

Then the proof is finished. $\square$

Proof of Theorem 2

Denote $Q_{n}(\varvec{\beta }\mid \varvec{{\hat{\beta }}}^{(t)})=L_{n}(\varvec{{\hat{\beta }}}^{(t)})+ (\varvec{\beta }-\varvec{{\hat{\beta }}}^{(t)})^{T}\varvec{U}_{n}(\varvec{{\hat{\beta }}}^{(t)}) -\frac{u}{2}\Vert \varvec{\beta }-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2}$. Then $\varvec{{\hat{\beta }}}^{(t+1)}=\mathop {\mathrm {argmin}}_{\varvec{\beta }\in {\mathcal {B}}(k)} \{-Q_{n}(\varvec{\beta }\mid \varvec{{\hat{\beta }}}^{(t)})\}.$

After some algebraic manipulations, it is easy to see that

$$\begin{aligned}&L_{n}\left( \varvec{{\hat{\beta }}}^{(t)}\right) \nonumber \\&\quad = Q_{n}\left( \varvec{{\hat{\beta }}}^{(t)}|\varvec{{\hat{\beta }}}^{(t)}\right) \nonumber \\&\quad \le \,Q_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}|\varvec{{\hat{\beta }}}^{(t)}\right) \nonumber \\&\quad =L_{n}\left( \varvec{{\hat{\beta }}}^{(t)}\right) +\left( \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\right) ^{T}\varvec{U}_{n}(\varvec{{\hat{\beta }}}^{(t)}) -\frac{u}{2}\Vert \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2} \nonumber \\&\quad =L_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}\right) -\frac{u}{2}\Vert \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2} +\left( \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\right) ^{T}\varvec{U}_{n}\left( \varvec{{\hat{\beta }}}^{(t)}\right) \nonumber \\&\qquad +\,L_{n}\left( \varvec{{\hat{\beta }}}^{(t)}\right) -L_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}\right) \nonumber \\&\quad =L_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}\right) -\frac{u}{2}\Vert \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2}+ \frac{1}{2}\left( \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\right) ^{T}\varvec{V}_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\right) . \end{aligned}$$

It is easy to see that

$$\begin{aligned} \left( \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\right) ^{T}\varvec{V}_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\right) \le \rho _{\mathrm {max}}(\varvec{V}_{n})\Vert \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2}. \end{aligned}$$

So under the assumptions in Theorem 2, we have

$$\begin{aligned}&L_{n}\left( \varvec{{\hat{\beta }}}^{(t)}\right) \nonumber \\&\quad \le \,L_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}\right) -\frac{u}{2}\Vert \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2}+\frac{1}{2}\rho _{\mathrm {max}}(\varvec{V}_{n})\Vert \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2}\nonumber \\&\quad = L_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}\right) -\frac{1}{2}(u-\rho _{\mathrm {max}}(\varvec{V}_{n}))\Vert \varvec{{\hat{\beta }}}^{(t+1)}-\varvec{{\hat{\beta }}}^{(t)}\Vert _{2}^{2}\nonumber \\&\quad \le \,L_{n}\left( \varvec{{\hat{\beta }}}^{(t+1)}\right) . \end{aligned}$$

This ends up the proof. $\square$

Before presenting the proof of Theorem 2, let’s introduce a lemma firstly.

Lemma 2

Define $\varvec{{\hat{\beta }}}^{(0)}=\mathrm {argmax}_{\varvec{\beta }}\{L_{n}(\varvec{\beta })-\lambda \Vert \varvec{\beta }\Vert _{1}\}$, where $\lambda$ satisfies $\lambda n^{\frac{1}{2}-m}\rightarrow \infty$, $\lambda n^{\tau _{1}+\tau _{2}}\rightarrow 0$. Under Assumptions 1–3 and 6, if $\max \limits _{1 \le j\le p}\sigma _j^2=O(\lambda n^{\frac{1}{2}})$, we have

$$\begin{aligned} \mathrm {pr}\left( \Vert \varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*}\Vert _{1}\le 8c_{5}^{-1}\lambda q\right) \rightarrow 1, \end{aligned}$$

where $c_{5}$ is defined in Assumption 6.

Proof

It is easy to see that

$$\begin{aligned} L_{n}\left( \varvec{{\hat{\beta }}}^{(0)}\right) -\lambda \Vert \varvec{{\hat{\beta }}}^{(0)}\Vert _{1}- \left( L_{n}(\varvec{\beta }^{*})-\lambda \Vert \varvec{\beta }^{*}\Vert _{1}\right) \ge 0, \end{aligned}$$

or equivalently

$$\begin{aligned} L_{n}(\varvec{\beta }^{*})-L_{n}(\varvec{{\hat{\beta }}}^{(0)})\le \lambda \Vert \varvec{\beta }^{*}\Vert _{1}-\lambda \Vert \varvec{{\hat{\beta }}}^{(0)}\Vert _{1}. \end{aligned}$$

Define $\varvec{\delta }=(\varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*})=(\delta _{1},\ldots ,\delta _{p})^{T}$. By some algebraic manipulations, we have

$$\begin{aligned}&L_{n}(\varvec{{\hat{\beta }}}^{(0)})-L_{n}(\varvec{\beta }^{*}) \nonumber \\&\quad =\varvec{b}_{n}^{T}\varvec{{\hat{\beta }}}^{(0)}-\frac{1}{2}\varvec{{\hat{\beta }}}^{(0) T}\varvec{V}_{n}\varvec{{\hat{\beta }}}^{(0)} -\left\{ \varvec{b}_{n}^{T}\varvec{\beta }^{*}-\frac{1}{2}\varvec{\beta }^{*T}\varvec{V}_{n}\varvec{\beta }^{*}\right\} \nonumber \\&\quad =(\varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*})^{T}\left\{ \varvec{b}_{n}-\frac{1}{2}\varvec{V}_{n}\varvec{\beta }^{*}\right\} -\frac{1}{2}(\varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*})^{T}\varvec{V}_{n}(\varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*}) \nonumber \\&\quad =\varvec{\delta }^{T}U_{n}(\varvec{\beta }^{*})-\frac{1}{2}\varvec{\delta }^{T}\varvec{V}_{n}\varvec{\delta }. \end{aligned}$$

Then we have

$$\begin{aligned}&\varvec{\delta }^{T}\varvec{V}_{n}\varvec{\delta } \nonumber \\&\quad =2\varvec{\delta }^{T}\varvec{U}_{n}(\varvec{\beta }^{*}) +L_{n}(\varvec{\beta }^{*})-L_{n}(\varvec{{\hat{\beta }}}^{(0)}) \nonumber \\&\quad \le \,2\varvec{\delta }^{T}\varvec{U}_{n}(\varvec{\beta }^{*}) +\lambda \Vert \varvec{\beta }^{*}\Vert _{1}-\lambda \Vert \varvec{{\hat{\beta }}}^{(0)}\Vert _{1}. \end{aligned}$$

Denote ${\mathcal {A}}=\{\max \nolimits _{1 \le j\le p}|U_{n,j}(\varvec{\beta }^{*})|\le \frac{\lambda }{4}\}$. Because $\max \limits _{1 \le j\le p}\sigma _{j}^{2}=O(\lambda n^{\frac{1}{2}})$, then by Lemma 1, we have

$$\begin{aligned}&\mathrm {pr}({\mathcal {A}}^{c})\nonumber \\&\quad \le \,\sum _{j=1}^{p}\mathrm {pr}\left( |U_{n,j}(\varvec{\beta }^{*})|>\frac{\lambda }{4}\right) \nonumber \\&\quad = \sum _{j=1}^{p} \mathrm {pr}\left( |\sqrt{n}U_{n,j}(\varvec{\beta }^{*})|>\frac{\sqrt{n}\lambda }{4}\right) \nonumber \\&\quad \le \,p c_{7}\mathrm {exp}\left( -c_{8}\frac{\sqrt{n}\lambda }{4}\right) \nonumber \\&\quad \le \,c_{7}\mathrm {exp}\left( c_{10}n^{m}-c_{8}\frac{\sqrt{n}\lambda }{2}\right) \nonumber \\&\quad \rightarrow 0, \end{aligned}$$

where $c_{10}$ is a positive constant. So we obtain that $\mathrm {pr}({\mathcal {A}})\rightarrow 1$ and $\Vert \varvec{U}_{n}(\varvec{\beta }^{*})\Vert _{\infty }=O_{p}(\lambda )$. Under the event ${\mathcal {A}}$, it is easy to see that

$$\begin{aligned}&\varvec{\delta }^{T}\varvec{V}_{n}\varvec{\delta } \nonumber \\&\quad \le \frac{1}{2}\lambda \Vert \varvec{\delta }\Vert _{1}+\lambda \Vert \varvec{\beta }^{*}\Vert _{1}-\lambda \Vert \varvec{{\hat{\beta }}}^{(0)}\Vert _{1}. \end{aligned}$$

Thus

$$\begin{aligned}&\varvec{\delta }^{T}\varvec{V}_{n}\varvec{\delta }+\frac{1}{2}\lambda \Vert \varvec{\delta }\Vert _{1} \nonumber \\&\quad \le \,\lambda \Vert \varvec{\delta }\Vert _{1}+\lambda \Vert \varvec{\beta }^{*}\Vert _{1}-\lambda \Vert \varvec{{\hat{\beta }}}^{(0)}\Vert _{1} \nonumber \\&\quad \le \,\lambda \sum _{j=1}^{p} \left( |{\hat{\beta }}_{j}^{(0)}-\beta _{j}^{*}|+|\beta _{j}^{*}|-|{\hat{\beta }}_{j}^{(0)}|\right) \nonumber \\&\quad = \lambda \sum _{j\in M_{0}} \left( |{\hat{\beta }}_{j}^{(0)}-\beta _{j}^{*}|+|\beta _{j}^{*}|-|{\hat{\beta }}_{j}^{(0)}|\right) \nonumber \\&\quad \le \,2\lambda \sum _{j\in M_{0}}|\delta _{j}| \nonumber \\&\quad \le \,2\lambda \Vert \varvec{\delta }_{M_{0}}\Vert _{1}. \end{aligned}$$

It is easy to see that $\varvec{V}_{n}$ is semipositive definite. Thus $\Vert \varvec{\delta }\Vert _{1}\le 4\Vert \varvec{\delta }_{M_{0}}\Vert _{1}$, and furthermore $\Vert \varvec{\delta }_{M_{0}^{c}}\Vert _{1}\le 3\Vert \varvec{\delta }_{M_{0}}\Vert _{1}$. By the Cauchy–Schwarz inequality and Assumption 6,

$$\begin{aligned} \Vert \varvec{\delta }_{M_{0}}\Vert _{1}^{2}\le q \Vert \varvec{\delta }_{M_{0}}\Vert _{2}^{2} \le q c_{5}^{-1}\varvec{\delta }^{T}\varvec{V}_{n}\varvec{\delta } \le 2 c_{5}^{-1}\lambda q \Vert \varvec{\delta }_{M_{0}}\Vert _{1}. \end{aligned}$$

So $\Vert \varvec{\delta }_{M_{0}}\Vert _{1}\le 2 c_{5}^{-1}\lambda q$. Then finally we arrive at

$$\begin{aligned} \Vert \varvec{\delta }\Vert _{1}=\Vert \varvec{\delta }_{M_{0}^{c}}\Vert _{1}+\Vert \varvec{\delta }_{M_{0}}\Vert _{1} \le 4 \Vert \varvec{\delta }_{M_{0}}\Vert _{1} \le 8 c_{5}^{-1}\lambda q. \end{aligned}$$

This finishes the proof. $\square$

Proof of Theorem 3

Recall that $w=\mathrm {min}_{j\in M_{0}}\Vert \beta _{j}^{*}\Vert$. We just need to show $\mathrm {pr}(\Vert \varvec{{\hat{\beta }}}^{(t)}-\varvec{\beta }^{*}\Vert _{\infty }<\frac{w}{2})\rightarrow 1$. It suffices to prove $\Vert \varvec{{\hat{\beta }}}^{(t)}-\varvec{\beta }^{*}\Vert _{\infty }=o_{p}(w)$. As in Xu and Chen (2014), we use the method of mathematical induction to get this result.

When $t=0$, by Lemma 2, we have

$$\begin{aligned} \mathrm {pr}(\Vert \varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*}\Vert _{1}\le 8c_{5}^{-1}\lambda q)\rightarrow 1. \end{aligned}$$

Because $\lambda =o(n^{-(\tau _{1}+\tau _{2})})$, $q=O(n^{\tau _{2}})$, $w^{-1}=O(n^{\tau _{1}})$, $\lambda qw^{-1}=o(n^{-(\tau _{1}+\tau _{2})})O(n^{\tau _{2}})O(n^{\tau _{1}})=o(1)$. Thus $\lambda q=o(w)$. So we have $\Vert \varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*}\Vert _{1}=o_{p}(w)$. It is noted that $\Vert \varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*}\Vert _{\infty }\le \Vert \varvec{{\hat{\beta }}}^{(0)}-\varvec{\beta }^{*}\Vert _{1}$. Then the desired result is obtained for $t=0$.

Suppose that $\Vert \varvec{{\hat{\beta }}}^{(t-1)}-\varvec{\beta }^{*}\Vert _{\infty }=o_{p}(w)$. In the following, we will show that $\Vert \varvec{{\hat{\beta }}}^{(t)}-\varvec{\beta }^{*}\Vert _{\infty }=o_{p}(w)$ is also true. From the adaptive iterative hard-thresholding algorithm, it is noted that $\varvec{{\hat{\beta }}}^{(t)}={\mathbf {H}}(\varvec{{\tilde{\beta }}}^{(t-1)};k)$, where $\varvec{{\tilde{\beta }}}^{(t-1)}=\varvec{{\hat{\beta }}}^{(t-1)}+u^{-1}{\dot{L}}_{n}(\varvec{{\hat{\beta }}}^{(t-1)})$. If $\Vert \varvec{{\tilde{\beta }}}^{(t-1)}-\varvec{\beta }^{*}\Vert _{\infty }=o_{p}(w)$ holds, it can be seen that elements of $\varvec{{\tilde{\beta }}}^{(t-1)}_{M_{0}}$ are among the ones with top k largest absolute values in probability. Thus $\Vert \varvec{{\hat{\beta }}}^{(t)}-\varvec{\beta }^{*}\Vert _{\infty }\le \Vert \varvec{{\tilde{\beta }}}^{(t-1)}-\varvec{\beta }^{*}\Vert _{\infty }=o_{p}(w)$. So what remains is to prove $\Vert \varvec{{\tilde{\beta }}}^{(t-1)}-\varvec{\beta }^{*}\Vert _{\infty }=o_{p}(w)$. Note that $\Vert \varvec{{\tilde{\beta }}}^{(t-1)}-\varvec{\beta }^{*}\Vert _{\infty }\le \Vert \varvec{{\hat{\beta }}}^{(t-1)}-\varvec{\beta }^{*}\Vert _{\infty }+ u^{-1}\Vert \varvec{U}_{n}(\varvec{{\hat{\beta }}}^{(t-1)})\Vert _{\infty }$. By some algebraic manipulations, we could obtain that

$$\begin{aligned}&\Vert \varvec{U}_{n}(\varvec{{\hat{\beta }}}^{(t-1)})\Vert _{\infty } \nonumber \\&\quad =\Vert \varvec{U}_{n}(\varvec{\beta }^{*})+\varvec{U}_{n}(\varvec{{\hat{\beta }}}^{(t-1)})-\varvec{U}_{n}(\varvec{\beta }^{*})\Vert _{\infty } \nonumber \\&\quad \le \,\Vert \varvec{U}_{n}(\varvec{\beta }^{*})\Vert _{\infty }+ \Vert \varvec{U}_{n}(\varvec{{\hat{\beta }}}^{(t-1)})-\varvec{U}_{n}(\varvec{\beta }^{*})\Vert _{\infty } \nonumber \\&\quad = \Vert \varvec{U}_{n}(\varvec{\beta }^{*})\Vert _{\infty }+ \Vert \varvec{V}_{n}(\varvec{\beta }^{*}-\varvec{{\hat{\beta }}}^{(t-1)})\Vert _{\infty } \nonumber \\&\quad \le \,\Vert \varvec{U}_{n}(\varvec{\beta }^{*})\Vert _{\infty }+ \Vert \varvec{V}_{n}\Vert _{\infty }\Vert \varvec{\beta }^{*}-\varvec{{\hat{\beta }}}^{(t-1)}\Vert _{\infty }. \end{aligned}$$

Thus

$$\begin{aligned}&u^{-1}\Vert \varvec{U}_{n}(\varvec{{\hat{\beta }}}^{(t-1)})\Vert _{\infty } \nonumber \\&\quad \le \,u^{-1}O_{p}(\lambda )+u^{-1}\Vert \varvec{V}_{n}\Vert _{\infty }o_{p}(w) \nonumber \\&\quad \le \,(c_{6}r)^{-1}\lambda O_{p}(1)+(c_{6}r)^{-1}n^{\tau _{3}}o_{p}(w)O_{p}(1)\nonumber \\&\quad = c_{6}^{-1}O(n^{-\tau _{3}})o(n^{-(\tau _{1}+\tau _{2})}) O_{p}(1) +c_{6}^{-1}O_{p}(n^{-\tau _{3}})n^{\tau _{3}}o_{p}(w)\nonumber \\&\quad =o_{p}(w). \end{aligned}$$

This ends up the proof. $\square$