Robust MAVE for single-index varying-coefficient models

Abstract

In this paper, a robust, efficient and easily implemented estimation procedure for single-index varying-coefficient models is proposed by combining minimum average variance estimation (MAVE) with exponential squared loss. The merit of the proposed method is robust against outliers or heavy-tailed error distributions while asymptotically efficient as the original MAVE under the normal error case. A practical minorization–maximization algorithm is proposed for implementation. Under some regularity conditions, asymptotic distributions of the resulting estimators are derived. Simulation studies and a real data example are conducted to examine the finite sample performance of the proposed method. Both theoretical and empirical findings confirm that our proposed method works very well.

Appendices

Appendix: Proofs of main results

For convenience of presentation, we denote $\varphi _{\gamma }(t)=\exp (-t^2/\gamma )$, $\varvec{\mu }_{\varvec{\theta }}({\varvec{x}})=E({\varvec{X}} \vert \varvec{\theta }^{\mathrm{T}}{\varvec{X}}=\varvec{\theta }^{\mathrm{T}}{\varvec{x}})$, $\delta _{\varvec{\theta }}=\Vert \varvec{\theta }-\varvec{\theta }_{0} \Vert$ and $\delta _{n}=\{ \log n /(nh)\}^{1/2}$.

Proof of Theorem 1

Proof

From (7), we directly consider the maximization of the following objective function

$$\begin{aligned} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi _{\gamma } \big (Y_{i}- {\varvec{a}}^{\mathrm{T}}{\varvec{Z}}_{i}-{\varvec{d}}^\mathrm{T}{\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0} \big ). \end{aligned}$$

(13)

Let $(\tilde{\varvec{a}},\tilde{\varvec{d}})$ be the maximizer of (13), and it satisfies the following equation

$$\begin{aligned} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi '_{\gamma } \big (Y_{i}-\tilde{\varvec{a}}^{\mathrm{T}}{\varvec{Z}}_{i}-\tilde{\varvec{d}}^{\mathrm{T}}{\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0} \big ) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} =0. \end{aligned}$$

(14)

Denote $r_{i}={\varvec{g}}_{0}^{\mathrm{T}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}){\varvec{Z}}_{i} -{\varvec{g}}_{0}^{\mathrm{T}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{Z}}_{i}-{\varvec{g}}_{0}'^{\mathrm{T}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}$, we have

$$\begin{aligned} Y_{i}-\tilde{\varvec{a}}^{\mathrm{T}}{\varvec{Z}}_{i}-\tilde{\varvec{d}}^{\mathrm{T}}{\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}&=\epsilon _{i} +r_{i} -\big \{ \tilde{\varvec{a}}-{\varvec{g}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}) \big \}^{\mathrm{T}}{\varvec{Z}}_{i} \\&~~~~ -\big \{ \tilde{\varvec{d}}-{\varvec{g}}_{0}'(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}) \big \}^{\mathrm{T}} {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}. \end{aligned}$$

By Taylor expansion and (14), it follows that

$$\begin{aligned} \begin{pmatrix} \tilde{\varvec{a}} \\ \tilde{\varvec{d}}h \end{pmatrix}&= \begin{pmatrix} {\varvec{g}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}) \\ {\varvec{g}}'_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}})h \end{pmatrix} + \mathbf{S}_{n}^{-1}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi '_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} \nonumber \\&~~~~ +\mathbf{S}_{n}^{-1}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi ''_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} r_{i}, \end{aligned}$$

(15)

where

$$\begin{aligned} \mathbf{S}_{n}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})=\frac{1}{n}\sum \limits _{i=1}^{n}&K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi ''_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix}^{\mathrm{T}} \\&~ \times \big \{1+O_{p}(h^{2}+\delta _{n}) \big \}. \end{aligned}$$

By some direct calculations, we obtain

$$\begin{aligned} \mathbf{S}_{n}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})&= \begin{pmatrix} f_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})\mathbf{D}_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) &{} h\big \{ f_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})\mathbf{D}_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \big \}' \\ h\big \{ f_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})\mathbf{D}_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \big \}' &{} f_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})\mathbf{D}_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \end{pmatrix} E\big \{ \varphi _{\gamma }''(\epsilon ) \big \} \nonumber \\&~~~~ + O_{p}(h^{2}+\delta _{n}), \nonumber \\ r_{i}&={\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{Z}}_{i}(\varvec{\theta }_{0}-\varvec{\theta })^{\mathrm{T}}{\varvec{X}}_{i0} +\frac{1}{2}{\varvec{g}}''^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{Z}}_{i} \big \{ \varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i0} \big \}^{2} \nonumber \\&~~~~ +R(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i},\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}},{\varvec{Z}}_{i}), \end{aligned}$$

(16)

where $R(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i},\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}},{\varvec{Z}}_{i})$ is defined as remainder.

Using the fact $(\mathbf{A}+\mathbf{B}h)^{-1}=\mathbf{A}^{-1}-h\mathbf{A}^{-1}{} \mathbf{B}{} \mathbf{A}^{-1}+O(h^2)$, for the third term on the right-hand side of (15), we have

$$\begin{aligned}&\mathbf{S}_{n}^{-1}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi ''_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} {\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{Z}}_{i}{\varvec{X}}_{i0}^{\mathrm{T}}(\varvec{\theta }_{0}-\varvec{\theta }) \\&~~ =\begin{pmatrix} \big \{ \mathbf{D}_{\varvec{\theta }}^{-1}(\varvec{\theta }^\mathrm{T}{\varvec{x}})\mathbf{C}_{\varvec{\theta }}^{\mathrm{T}}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) -{\varvec{g}}'_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{x}}^{\mathrm{T}} \big \}(\varvec{\theta }_{0}-\varvec{\theta }) + O_{p}(h\delta _{\varvec{\theta }}+\delta _{n}\delta _{\varvec{\theta }}) \\ O_{p}(h\delta _{\varvec{\theta }}+\delta _{n}\delta _{\varvec{\theta }}) \end{pmatrix}, \\&\mathbf{S}_{n}^{-1}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi ''_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} {\varvec{g}}''^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{Z}}_{i}\{\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}\}^{2} \\&~~ =\begin{pmatrix} {\varvec{g}}''_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}})h^{2}\mu _{2} +O_{p}(h^{3}+h^{2}\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}) \\ O_{p}(h^{3}+h^{2}\delta _{\varvec{\theta }}+h\delta _{\varvec{\theta }}) \end{pmatrix}, \\&\mathbf{S}_{n}^{-1}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi ''_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} R(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i},\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}},{\varvec{Z}}_{i}) \\&~~ =\begin{pmatrix} O_{p}(h^{2}\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}) \\ O_{p}(h^{3}+h^{2}\delta _{\varvec{\theta }}+h\delta _{\varvec{\theta }}^{2}+\delta _{n}\delta _{\varvec{\theta }}^{2}) \end{pmatrix}. \end{aligned}$$

For the second term on the right-hand side of (15), we can write

$$\begin{aligned} \mathbf{S}_{n}^{-1}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi '_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}/h \end{pmatrix} =\begin{pmatrix} {\varvec{r}}_{n,1}({\varvec{x}}) \\ {\varvec{r}}_{n,2}({\varvec{x}}) \end{pmatrix} +O_{p}(h\delta _{n}), \end{aligned}$$

where ${\varvec{r}}_{n,1}({\varvec{x}})=\big [n f_{\varvec{\theta }}(\varvec{\theta }^\mathrm{T}{\varvec{x}}) \mathbf{D}_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) E\{ \varphi _{\gamma }''(\epsilon ) \} \big ]^{-1} \sum _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi '_{\gamma }(\epsilon _{i}) {\varvec{Z}}_{i},$ ${\varvec{r}}_{n,2}({\varvec{x}})=\big [n f_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) \mathbf{D}_{\varvec{\theta }}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}}) E\{ \varphi _{\gamma }''(\epsilon ) \} \big ]^{-1} \sum _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0}) \varphi '_{\gamma }(\epsilon _{i}) \big ( \varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ix}/h \big ) {\varvec{Z}}_{i}.$

Combining the above equations, we have

$$\begin{aligned} \tilde{\varvec{a}}&={\varvec{g}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}) +\big \{ \mathbf{D}_{\varvec{\theta }}^{-1}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})\mathbf{C}_\mathbf{\theta }^{\mathrm{T}}(\varvec{\theta }^{\mathrm{T}}{\varvec{x}})-{\varvec{g}}'_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}}){\varvec{x}}^{\mathrm{T}} \big \}(\varvec{\theta }_{0}-\varvec{\theta }) +\frac{1}{2}{\varvec{g}}''_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}})h^{2}\mu _{2} \\&~~~~ +{\varvec{r}}_{n,1}({\varvec{x}})+O_{p}(h^{3}+h^{2}\delta _{n}+h\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}), \\ \tilde{\varvec{d}}&={\varvec{g}}'_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{x}})+h^{-1}{\varvec{r}}_{n,2}({\varvec{x}}) +O_{p}(\delta _{n}+\delta _{\varvec{\theta }}+h^{-1}\delta _{n}\delta _{\varvec{\theta }}). \end{aligned}$$

Let $\tilde{\varvec{a}}_{j}$ and $\tilde{\varvec{d}}_{j}$ be, respectively, the values of $\tilde{\varvec{a}}$ and $\tilde{\varvec{d}}$ with ${\varvec{x}}$ replaced by ${\varvec{X}}_{j}$. Replacing $({\varvec{a}}_{j},{\varvec{d}}_{j})$ in (6) with $(\tilde{\varvec{a}}_{j}$, $\tilde{\varvec{d}}_{j})$, and denoting $\Delta _{ij}={\varvec{g}}_{0}^{\mathrm{T}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}){\varvec{Z}}_{i}-\tilde{\varvec{a}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i}-\tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i}\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{ij}$, we obtain

$$\begin{aligned} \frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi '_{\gamma }\big ( \epsilon _{i}+\Delta _{ij}-\tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i}{\varvec{X}}_{ij}^{\mathrm{T}} (\varvec{\theta }-\varvec{\theta }_{0}) \big ) \tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i}{\varvec{X}}_{ij} /\varsigma _{\varvec{\theta }}({\varvec{X}}_{j}) =0, \end{aligned}$$

(17)

where $\varsigma _{\varvec{\theta }}({\varvec{x}})=n^{-1}\sum _{i=1}^{n}K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{i0})$.

Let

$$\begin{aligned}&\mathbf{M}_{n}=\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi ''_{\gamma }(\epsilon _{i}) \big \{\tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i} \big \}^{2} {\varvec{X}}_{ij}{\varvec{X}}_{ij}^{\mathrm{T}} \big / \varsigma _{\varvec{\theta }}({\varvec{X}}_{j}), \\&{\varvec{N}}_{n1}=\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi '_{\gamma }(\epsilon _{i}) \tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i} {\varvec{X}}_{ij} \big / \varsigma _{\varvec{\theta }}({\varvec{X}}_{j}), \\&{\varvec{N}}_{n2}=\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi ''_{\gamma }(\epsilon _{i}) \Delta _{ij} \tilde{\varvec{d}}_{j}^\mathrm{T}{\varvec{Z}}_{i} {\varvec{X}}_{ij} \big / \varsigma _{\varvec{\theta }}({\varvec{X}}_{j}). \end{aligned}$$

Using the condition $\varvec{\theta }\in {\varvec{\Theta }}_{n}$ and exchanging the order of the summation, we have

$$\begin{aligned} \mathbf{M}_{n}&=\frac{1}{n}\sum \limits _{i=1}^{n}\varphi ''_{\gamma }(\epsilon _{i}) \big \{ {\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}} {\varvec{X}}_{i}){\varvec{Z}}_{i} \big \}^{2} \big \{ {\varvec{X}}_{i}-\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}_{i}) \big \} \big \{ {\varvec{X}}_{i}-\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}_{i}) \big \}^{\mathrm{T}} \\&~~~~ +\frac{1}{n}\sum \limits _{i=1}^{n}\varphi ''_{\gamma }(\epsilon _{i}) \big \{ {\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}){\varvec{Z}}_{i} \big \}^{2} E\big [ \big \{ {\varvec{X}}_{i}-\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}_{i}) \big \} \big \{ {\varvec{X}}_{i}-\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}_{i}) \big \}^{\mathrm{T}} \big ] \\&~~~~ +O_{p}(h^{-1}\delta _{n}+h+\delta _{\theta }) \\&= \mathbf{W}_{0} E\big \{ \varphi ''_{\gamma }(\epsilon ) \big \}+O_{p}(h^{-1}\delta _{n}+h+\delta _{\varvec{\theta }}), \\ {\varvec{N}}_{n1}&=\frac{1}{n}\sum \limits _{i=1}^{n} \big \{ {\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}){\varvec{Z}}_{i} \big \} \big \{ {\varvec{X}}_{i}-\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}_{i}) \big \} \varphi '_{\gamma }(\epsilon _{i}) \\&~~~~ +O_{p}(h^{3}+h\delta _{\varvec{\theta }}+h^{-1}\delta _{n}^{2}+h^{-1}\delta _{n}\delta _{\varvec{\theta }}), \end{aligned}$$

where

$$\begin{aligned} \mathbf{W}_{0}=E\big [\big \{ {\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^\mathrm{T}{\varvec{X}}){\varvec{Z}} \big \}^{2} \big \{ {\varvec{X}}{\varvec{X}}^{\mathrm{T}} -{\varvec{X}}\varvec{\mu }_{\varvec{\theta }_{0}}^{\mathrm{T}}({\varvec{X}}) -\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}){\varvec{X}}^{\mathrm{T}} +E({\varvec{X}}{\varvec{X}}^{\mathrm{T}} \vert \varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}) \big \}\big ]. \end{aligned}$$

By the expansions of $\tilde{\varvec{a}}$ and $\tilde{\varvec{d}}$, we have

$$\begin{aligned} \Delta _{ij}&=-{\varvec{Z}}_{i}^{\mathrm{T}}{\varvec{r}}_{n,1}({\varvec{X}}_{j}) +\big \{ {\varvec{g}}'_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{j})-\tilde{\varvec{d}}_{j} \big \}^{\mathrm{T}} \varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{ij} \nonumber \\&\quad +\frac{1}{2}{\varvec{g}}''^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{j}){\varvec{Z}}_{i} \big \{(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{ij})^{2}-h^{2}\mu _{2} \big \} \nonumber \\&\quad +{\varvec{Z}}_{i}^{\mathrm{T}} \big \{\mathbf{D}_{\varvec{\theta }}^{-1}(\varvec{\theta }^{T}{\varvec{X}}_{j}) \mathbf{C}_{\varvec{\theta }}^{\mathrm{T}}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{j}) -{\varvec{g}}'_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{j}){\varvec{X}}_{j}^{\mathrm{T}} \big \}(\varvec{\theta }_{0}-\varvec{\theta }) \nonumber \\&\quad +\big \{ O_{p}(h^{3}+h^{2}\delta _{n}+h\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}) \vert {\varvec{Z}}_{i} \vert +R(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i},\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{j},{\varvec{Z}}_{i}) \big \} \nonumber \\&\equiv -\Delta _{ij}^{1}+\Delta _{ij}^{2}+\Delta _{ij}^{3}+\Delta _{ij}^{4}+\Delta _{ij}^{5}, \end{aligned}$$

(18)

where $R(\varvec{\theta }_{0}^{\mathrm{T}}\mathrm{X}_{i},\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{j},{\varvec{Z}}_{i})$ is defined in (16). For (18), we obtain

$$\begin{aligned}&\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi ''_{\gamma }(\epsilon _{i}) \Delta _{ij}^{1} \tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i} {\varvec{X}}_{ij} \big / \varsigma _{\varvec{\theta }}({\varvec{X}}_{j}) \\&\quad =O_{p}(h^{3}+h\delta _{\varvec{\theta }}+h^{-1}\delta _{n}^{2}+h^{-1}\delta _{n}\delta _{\varvec{\theta }}), \\&\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi ''_{\gamma }(\epsilon _{i}) \Delta _{ij}^{2} \tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i} {\varvec{X}}_{ij} \big / \varsigma _{\varvec{\theta }}({\varvec{X}}_{j}) \\&\quad =O_{p}(h^{3}+h\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}+h^{-1}\delta _{n}\delta _{\varvec{\theta }}), \\&\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi ''_{\gamma }(\epsilon _{i}) \Delta _{ij}^{3} \tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i} {\varvec{X}}_{ij} \big / \varsigma _{\varvec{\theta }}({\varvec{X}}_{j}) \\&\quad =O_{p}(h^{3}+h\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}), \\&\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi ''_{\gamma }(\epsilon _{i}) \Delta _{ij}^{4} \tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i} {\varvec{X}}_{ij} \big / \varsigma _{\mathrm{\theta }}({\varvec{X}}_{j}) \\&~~ =\mathbf{W}_{1}(\varvec{\theta }-\varvec{\theta }_{0})E\{\varphi ''_{\gamma }(\epsilon )\} +O_{p}(h^{-1}\delta _{n}\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}), \\&\frac{1}{n^{2}}\sum \limits _{j=1}^{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }^{\mathrm{T}}{\varvec{X}}_{ij}) \varphi ''_{\gamma }(\epsilon _{i}) \Delta _{ij}^{5} \tilde{\varvec{d}}_{j}^{\mathrm{T}}{\varvec{Z}}_{i} {\varvec{X}}_{ij} \big / \varsigma _{\varvec{\theta }}({\varvec{X}}_{j}) \\&~~ =O_{p}(h^{3}+h\delta _{\varvec{\theta }}+h\delta _{n}+\delta _{\varvec{\theta }}^{2}), \end{aligned}$$

where $\mathbf{W}_{1}=E\big [ \mathbf{T}_{\varvec{\theta }_{0}}(\varvec{\theta }_{0}^\mathrm{T}{\varvec{X}}) \mathbf{D}_{\varvec{\theta }_{0}}^{-1}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}) \mathbf{T}_{\varvec{\theta }_{0}}^{\mathrm{T}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}) \big ]$ and $\mathbf{T}_{\varvec{\theta }_{0}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}) =\mathbf{C}_{\varvec{\theta }_{0}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}) -{\varvec{X}}{\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}) \mathbf{D}_{\varvec{\theta }_{0}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}})$. Then, we have

$$\begin{aligned} {\varvec{N}}_{n2}=\mathbf{W}_{1} E\big \{ \varphi ''_{\gamma }(\epsilon ) \big \} (\varvec{\theta }-\varvec{\theta }_{0}) +O_{p}(h^{3}+h\delta _{\varvec{\theta }}+h^{-1}\delta _{n}^{2}+h^{-1}\delta _{n}\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}). \end{aligned}$$

From the above equations and Taylor expansion for (17), we have

$$\begin{aligned} \mathbf{W}_{0} E\big \{ \varphi ''_{\gamma }(\epsilon ) \big \} (\varvec{\theta }-\varvec{\theta }_{0})&=\frac{1}{n}\sum \limits _{i=1}^{n} \big \{{\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}){\varvec{Z}}_{i} \big \} \big \{{\varvec{X}}_{i}-\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}_{i}) \big \} \varphi '_{\gamma }(\epsilon _{i}) \\&\quad +\mathbf{W}_{1} E\big \{ \varphi ''_{\gamma }(\epsilon ) \big \} (\varvec{\theta }-\varvec{\theta }_{0}) \\&\quad +O_{p}(h^{3}+h\delta _{\varvec{\theta }}+h^{-1}\delta _{n}^{2} +h^{-1}\delta _{n}\delta _{\varvec{\theta }}+\delta _{\varvec{\theta }}^{2}). \end{aligned}$$

Following the proof of Theorem 4.2 in Xia (2006), we obtain

$$\begin{aligned} \hat{\varvec{\theta }}-\varvec{\theta }_{0}&={\varvec{\Sigma }}_{0}^{-} E^{-1}\big \{ \varphi ''_{\gamma }(\epsilon ) \big \} \frac{1}{n}\sum \limits _{i=1}^{n} \big \{ {\varvec{g}}'^{\mathrm{T}}_{0}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}){\varvec{Z}}_{i} \big \} \big \{ {\varvec{X}}_{i}-\varvec{\mu }_{\varvec{\theta }_{0}}({\varvec{X}}_{i}) \big \} \varphi '_{\gamma }(\epsilon _{i}) \\&\quad +o_{p}(n^{-1/2}), \end{aligned}$$

where ${\varvec{\Sigma }}_{0}=\mathbf{W}_{0}-\mathbf{W}_{1}$. Under the assumptions of Theorem 1, it then follows from the above equation and the central limit theorem. $\square$

Proof of Theorem 2

Proof

From (15) in Theorem 1, we have

$$\begin{aligned} \begin{pmatrix} \hat{\varvec{g}}(u) \\ h\hat{\varvec{g}}'(u) \end{pmatrix}&=\begin{pmatrix} {\varvec{g}}_{0}(u) \\ {\varvec{g}}'_{0}(u)h \end{pmatrix} + \hat{\mathbf{S}}_{n}^{-1}(u) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\hat{\varvec{\theta }}^{\mathrm{T}}{\varvec{X}}_{i}-u) \varphi '_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}(\hat{\varvec{\theta }}^{\mathrm{T}}{\varvec{X}}_{i}-u)/h \end{pmatrix} \\&\quad +\hat{\mathbf{S}}_{n}^{-1}(u) \frac{1}{n} \sum \limits _{i=1}^{n} K_{h}(\hat{\varvec{\theta }}^{\mathrm{T}}{\varvec{X}}_{i}-u)\varphi ''_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}(\hat{\varvec{\theta }}^{\mathrm{T}}{\varvec{X}}_{i}-u)/h \end{pmatrix} r_{i}(u), \end{aligned}$$

where $r_{i}(u)={\varvec{g}}_{0}^{\mathrm{T}}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}){\varvec{Z}}_{i} -{\varvec{g}}_{0}^{\mathrm{T}}(u){\varvec{Z}}_{i}-{\varvec{g}}_{0}'^{\mathrm{T}}(u){\varvec{Z}}_{i}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}-u)$, and

$$\begin{aligned} \hat{\mathbf{S}}_{n}(u)&=\frac{1}{n}\sum \limits _{i=1}^{n} K_{h}(\hat{\varvec{\theta }}^{\mathrm{T}}{\varvec{X}}_{i}-u) \varphi ''_{\gamma }(\epsilon _{i}) \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}(\hat{\varvec{\theta }}^{\mathrm{T}}{\varvec{X}}_{i}-u)/h \end{pmatrix} \begin{pmatrix} {\varvec{Z}}_{i} \\ {\varvec{Z}}_{i}(\hat{\varvec{\theta }}^{\mathrm{T}}{\varvec{X}}_{i}-u)/h \end{pmatrix} ^{\mathrm{T}} \\&\quad \times \big \{ 1+O_{p}(h^{2}+\delta _{n}) \big \}. \end{aligned}$$

Note that Theorem 1 implies $\Vert \hat{\varvec{\theta }}-\varvec{\theta }_{0} \Vert =O_{p}(n^{-1/2})$. Therefore, we have

$$\begin{aligned} \hat{\varvec{g}}(u)&={\varvec{g}}_{0}(u)+f_{\varvec{\theta }_{0}}^{-1}(u)\mathbf{D}_{\varvec{\theta }_{0}}^{-1}(u) E^{-1}\big \{ \varphi ''_{\gamma }(\epsilon ) \big \} \frac{1}{n}\sum \limits _{i=1}^{n} K_{h}(\varvec{\theta }_{0}^{\mathrm{T}}{\varvec{X}}_{i}-u) {\varvec{Z}}_{i} \varphi '_{\gamma }(\epsilon _{i}) \\&\quad +\frac{h^{2}}{2}\mu _{2}{\varvec{g}}_{0}''(u) +o_{p}(h^{2}). \end{aligned}$$

The proof of Theorem 2 is completed by applying Slutsky’s theorem and the central limit theorem. $\square$