Semi-parametric estimation and forecasting for exogenous log-GARCH models

Abstract

Advanced computing and processing techniques have yielded abundant information for financial time series forecasting. It is, therefore, natural to ask for possible extensions of time series models to accommodate the wealth of information. In this article, we develop a new model for financial volatility estimation and forecasting by incorporating exogenous covariates in a semi-parametric log-GARCH model. With additional information, we gain an increased prediction power. We propose a quasi-maximum likelihood procedure via spline smoothing technique. Consistent estimators and asymptotic normality are obtained under mild regularity conditions. Simulation experiments provide strong evidence that corroborates the asymptotic theories. Additionally, an application to SPY index data demonstrates strong competitive advantage of our model comparing with GARCH(1,1) and log-GARCH(1,1) models.

Appendix: Proof of the theorems

Throughout the section, let $\Vert \cdot \Vert$ be the Euclidean norm and supremum norm $||\phi ||_{\infty }=\,$ sup$_{x\in [0,1]}|\phi (x)|$. For any matrix ${\mathbf {A}}$, denote its spectral norm as $\left\| {\mathbf {A}}\right\| _{2}$  $=$  $\mathrm{sup}_{\mathbf {x}\ne \mathbf{0}}\frac{ \left\| {\mathbf {Ax}}\right\| }{\left\| {\mathbf {x}}\right\| }$. For any vector ${\mathbf {V}}$, define $\Vert {\mathbf {V}} \Vert _{\infty }=\,$ max$(\vert v_{1}\vert ,\ldots , \vert v_{n}\vert )$, and for any measurable function $\phi$, define $||\phi ||_{2}^{2}=E\phi ^2({\mathbf {X}}).$ Throughout the section, $\kappa , \kappa _1$ and $\kappa _2$ will be used as constants in a generic way.

Proof of Lemma 1

Proof

Recursively using the expression in (3), we have

$$\begin{aligned} \log \sigma _{t}^{2}= & {} c\left[ 1+\sum _{j=0}^{\infty }\prod _{k=0}^{j}\left\{ m({\mathbf {Z}}_{t-1-k})+a\right\} \right] \\&+\sum _{j=1}^{\infty }\left[ \prod _{k=0}^{j-1}\{m({\mathbf {Z}}_{t-1-k})+a\} \right] m({\mathbf {Z}}_{t-1-j})\log \epsilon _{t-j-1}^{2}+m({\mathbf {Z}}_{t-1})\log \epsilon _{t-1}^{2} . \end{aligned}$$

Under Assumption (A2), the first term on the right-hand side converges almost surely since

$$\begin{aligned} P\left( \lim _{t\rightarrow \infty }\left| \sum _{j=t}^{\infty }\prod _{k=0}^{j}\left\{ m\left( {\mathbf {Z}}_{t-1-k}\right) +a\right\} c\right| \le \lim _{t\rightarrow \infty }\left| \sum _{j=t}^{\infty }\delta ^{j+1}c\right| =0\right) =1. \end{aligned}$$

Similarly, the second term converges almost surely under the same assumption. Therefore, $\log \sigma _{t}^{2}$ converges almost surely. ${\mathbf {Z}}_{t}$ is stationary and ergodic, then $\log \sigma _{t}^{2}$, as a function of an ergodic process is stationary and ergodic.

In the following we list Lemmas 2, 3 and 4 to verify the regularity conditions for maximum likelihood function in (5).

Lemma 2

Under Assumptions (A2) and (A4), $E_{{\varvec{\gamma }}_{0}}\left\| \frac{\partial l_{t}({\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }}}\right\| <\infty$ , $E_{{\varvec{\gamma }}_{0}}\left( \frac{\partial l_{t}({\varvec{\gamma }}_{0})}{ \partial {\varvec{\gamma }}}\right) =0, E_{{\varvec{\gamma }}_{0}}\left\| \frac{\partial ^{2}l_{t}({\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }} \partial {\varvec{\gamma }}}\right\| <\infty$.

The proof of 2 is provided in the supplementary material to this article. The proofs of Lemmas 3 to 5, 7 to 10 below are also provided in the supplementary material.

Lemma 3

Let ${\mathbf {J}}=E_{{\varvec{\gamma }}_{0}}\left( \frac{\partial ^{2}l_{t}( {\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }}\partial {\varvec{\gamma }}} \right)$, then ${\mathbf {J}}$ is invertible.

Lemma 4

Under Assumptions (A2) and (A4), we have

$$\begin{aligned} E_{{\varvec{\gamma }}_{0}}\sup _{{\varvec{\gamma }}\in V({\varvec{\gamma }} _{0})}\left| \frac{\partial ^{3}l_{t}({\varvec{\gamma }})}{\partial {\varvec{\gamma }}_{i}\partial {\varvec{\gamma }}_{j}\partial {\varvec{\gamma }} _{k}}\right| <\infty , \end{aligned}$$

here $V({\varvec{\gamma }} _{0})=\left\{ {\varvec{\gamma }}:\exists \ \xi > 0, \left\| {\varvec{\gamma }}-{\varvec{\gamma }} _{0}\right\| _{\infty }<\xi ,{\varvec{\gamma }}\in {\varTheta }_1 \right\}$.

Next we consider an approximation to the likelihood function in (5). Define $\breve{l}_{t}=\log \breve{\sigma }_{t}^{2}+y_{t}^{2}\breve{\sigma }_{t}^{-2}$, where

$$\begin{aligned} \log \breve{\sigma }_{t}^{2}=\frac{1-a^{t}}{1-a}c+\sum _{j=1}^{t}a^{j-1}\left( {\mathbf {Z}}_{t-j}^{1{\mathrm{\tiny T}}}{\varvec{\beta }}+\eta \left( {\mathbf { Z}}_{t-j}^{2}\right) \right) \log y_{t-j}^{2}. \end{aligned}$$

(10)

Lemma 5

Under Assumptions (A2) and (A4),

• 1. $\lim _{T\rightarrow \infty }\sup _{{\varvec{\gamma }}\in {\varTheta }_{1}}1/T\sum _{t=1}^{\infty }\left| l_{t}({\varvec{\gamma }})-\breve{l}_{t}({\varvec{\gamma }})\right| =0,a.s$, and $E_{{\varvec{\gamma }}_{0}}\left| l_{t}({\varvec{\gamma }}_{0})\right| <\infty .$

• 2. If there exists $t \in {\mathcal {Z}}$ such that $\log \sigma ^{2}_{t}({\varvec{\gamma }})=\log {\sigma }_{t}^{2}({\varvec{\gamma }}_{0}),a.s.$, then we have ${\varvec{\gamma }}={\varvec{\gamma }}_{0}$.

Following Theorem 7.1 in Francq and Zakoïan (2010) and applying Lemma 5, we immediately obtain the following lemma.

Lemma 6

Under Assumptions (A2) and (A4), as $T$ goes to infinity, ${\varvec{\breve{\gamma }}}\rightarrow {\varvec{\gamma }}_{0}, \ a.s.$

Lemma 7

Under Assumptions (A2) and (A4),

$$\begin{aligned} \left\| T^{-1/2}\sum _{t=1}^{T}\left\{ \frac{\partial l_{t}({\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }}}-\frac{\partial \breve{l}_{t}({\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }}} \right\} \right\| _{\infty }= & {} o_{p}(1) \end{aligned}$$

(11)

$$\begin{aligned} \sup _{{\varvec{\gamma }}\in V({\varvec{\gamma }}_0)}\left\| T^{-1}\sum _{t=1}^{T}\left\{ \frac{\partial ^{2}l_{t}({\varvec{\gamma }})}{\partial \mathbf { \gamma }\partial {\varvec{\gamma }}^{T}}-\frac{\partial ^{2}\breve{l}_{t}({\varvec{\gamma }})}{ \partial {\varvec{\gamma }}\partial {\varvec{\gamma }}^{T}}\right\} \right\| _{\infty }= & {} o_{p}(1). \end{aligned}$$

(12)

Lemma 8

Under Assumptions (A2) and (A4), as $T\rightarrow \infty$,

$$\begin{aligned}&T^{-1/2}\sum _{t=1}^{T}\frac{\partial l_{t}({\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }}} \overset{D}{\rightarrow }N(0,\alpha {\mathbf {J}}),\\&T^{-1}\sum _{t=1}^{T}\frac{\partial ^{2}l_{t}(\breve{{\varvec{\gamma }}})(i,j)}{\partial {\varvec{\gamma }}_{i}\partial {\varvec{\gamma }}_{j}}\overset{P}{ \rightarrow }{\mathbf {J}}(i,j), \end{aligned}$$

where $\alpha$ is $E_{{\varvec{\gamma }}_{0}}\left( 1-\epsilon _{t}^{2}\right) ^2$ and $\mathbf {J} =E_{{\varvec{\gamma }}_{0}}\left( \frac{\partial ^{2}l_{1}( {\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }}\partial {\varvec{\gamma }}^{\mathrm{\tiny T}}} \right)$.

Now we consider the likelihood function with $\eta$ approximated by $\tilde{ \eta }$ in (10). Define $\tilde{l}_{t}=\log \tilde{\sigma } _{t}^{2}+\frac{y_{t}^{2}}{\tilde{\sigma }_{t}^{2}}$, where $\log \tilde{\sigma }_{t}^{2}=\frac{1-a^{t}}{1-a}c+\sum _{j=1}^{t}a^{j-1}\left( {\mathbf {Z}}_{t-j}^{1{\mathrm{\tiny T}}}{\varvec{\beta }}+\tilde{\eta }\left( {\mathbf {Z}}_{t-j}^{2}\right) \right) \log y_{t-j}^{2}.$ Then define

$$\begin{aligned} \varvec{\tilde{\gamma }}=\underset{{\varvec{\gamma }}\in {\varTheta }_1 }{\text {argmin}} \ \frac{1}{T}\sum _{t=1}^{T}\tilde{l}_{t}. \end{aligned}$$

(13)

Lemma 9

Under Assumptions (A1)–(A5), as $T$ goes to infinity, $\tilde{{\varvec{\gamma }}}\rightarrow {\varvec{\gamma }}_{0},\ a.s.$

Lemma 10

Under Assumptions (A1)–(A5),

$$\begin{aligned} \left\| T^{-1/2}\sum _{t=1}^{T}\left\{ \frac{\partial \breve{l_{t}}({\varvec{\gamma }}_{0})}{ \partial {\varvec{\gamma }}}-\frac{\partial \tilde{l}_{t}({\varvec{\gamma }}_{0})}{\partial \mathbf { \gamma }}\right\} \right\| _{\infty }= & {} o_{p}(1),\\ \sup _{{\varvec{\gamma }}\in V({\varvec{\gamma }}_{0})}\left\| T^{-1}\sum _{t=1}^{n}\left\{ \frac{\partial ^{2}\breve{l}_{t}({\varvec{\gamma }})}{\partial {\varvec{\gamma }}\partial {\varvec{\gamma }}^{T}}-\frac{\partial ^{2}\tilde{l}({\varvec{\gamma }}) _{t}}{\partial {\varvec{\gamma }}\partial {\varvec{\gamma }}^{T}}\right\} \right\| _{\infty }= & {} o_{p}(1). \end{aligned}$$

Combining Lemmas 6, 7, 8, 9, 10 with Slutsky’s Lemma, we immediately have

Lemma 11

Under Assumptions (A1)–(A5),

$$\begin{aligned} \sqrt{T}\left( {\varvec{{\tilde{\gamma }}}}-{\varvec{\gamma }}_{0}\right) \rightarrow N\left( 0,\alpha \mathbf {J}^{-1}\right) , \end{aligned}$$

where $\alpha$ is $E_{{\varvec{\gamma }}_{0}}\left( 1-\epsilon _{t}^{2}\right) ^2$ and $\mathbf {J} =E_{{\varvec{\gamma }}_{0}}\left( \frac{\partial ^{2}l_{1}( {\varvec{\gamma }}_{0})}{\partial {\varvec{\gamma }}\partial {\varvec{\gamma }}^{\mathrm{\tiny T}}} \right)$.

Next, define $\hat{l}_{t}=\log \hat{\sigma }_{t}^{2}+y^{2}\hat{\sigma }_{t}^{-2}$ with

$$\begin{aligned} \log \hat{\sigma }_{t}^{2}=\frac{1-a^{t}}{1-a}+\sum _{j=1}^{t}a^{j-1}\left( {\mathbf {Z}}_{t-j}^{1{\mathrm{\tiny T}}}{\varvec{\beta }}+{\mathbf {B}}({\mathbf {Z}}_{t-j}^2) {\varvec{\lambda }}\right) \log y_{t-j}^{2}. \end{aligned}$$

Lemma 12

Under Assumptions (A1)–(A5),

$$\begin{aligned} \sup _{{\varvec{\theta }}\in {\varTheta }}\left( \frac{\partial ^{2}\hat{\mathcal {L}}({\varvec{\theta }})}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}}\right) ^{-1}=O(1). \end{aligned}$$

Proof

Define $\log \breve{\hat{\sigma }}^{2}_{t}({\varvec{\gamma }},{\varvec{\lambda }})= \frac{c}{1-a}+\sum _{j=1}^{\infty }a^{j-1}\left( {\mathbf {Z}}_{t-j}^{1} \mathbf {\beta }+\mathbf {B}({\mathbf {Z}}_{t-j}^{2}){\varvec{\lambda }}\right) \log y_{t-j}^{2},$ and $\breve{\hat{l}}_{t}= \log \breve{\hat{\sigma }}^{2}_{t} + y^{2}_{t}e^{-\log \breve{\hat{\sigma }}^{2}_{t}}$. It is easy to show that

$$\begin{aligned} \frac{\partial ^{2}\breve{\hat{l}}_{t}({\varvec{\theta }})}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}} = (1-y_{t}^{2}e^{-\log \breve{\hat{\sigma }}^{2}_{t}}) \frac{\partial ^{2}\log \breve{\hat{\sigma }}^{2}_{t}}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}}+y_{t}^{2}e^{-\log \breve{\hat{\sigma }}_{t}^{2}}\frac{\partial \log \breve{\hat{\sigma }}^{2}_{t}}{\partial {\varvec{\theta }}}\frac{\partial \log \breve{\hat{\sigma }}^{2}_{t}}{\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}}. \end{aligned}$$

Similarly as in Lemma 3, we have $\frac{\partial ^{2}\breve{\hat{l}}_{t} ({\varvec{\theta }})}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}}$ is invertible. Notice that

$$\begin{aligned} \sup _{\left( {\varvec{\gamma }},{\varvec{\lambda }}\right) \in {\varTheta }_{1}\times {\varTheta }_{2}}\left| \breve{\hat{l}}_t({\varvec{\gamma }}, {\varvec{\lambda }})\!-\!\hat{l}_t\left( {\varvec{\gamma }},{\varvec{\lambda }}\right) \right| \!=\! \sum _{j=t+1}^{\infty }a^{j-1}\left( {\mathbf {Z}}_{t-j}^{1}{\varvec{\beta }}\!+\!\mathbf {B} ({\mathbf {Z}}_{t-j}^{2}){\varvec{\lambda }}\right) \log y_{t-j}^{2} \!=\! O(\delta ^{t}), \end{aligned}$$

which is of $o_{a.s}(1)$. We have,

$$\begin{aligned} \sup _{\left( {\varvec{\gamma }},{\varvec{\lambda }}\right) \in {\varTheta }_{1}\times {\varTheta }_{2}}\frac{\partial ^{2} \hat{l}_{t}({\varvec{\gamma }},{\varvec{\lambda }})}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}}\overset{a.s}{\rightarrow }\sup _{\left( {\varvec{\gamma }},{\varvec{\lambda }}\right) \in {\varTheta }_{1}\times {\varTheta }_{2}}\frac{\partial ^{2}\breve{\hat{l}}_{t}({\varvec{\gamma }},{\varvec{\lambda }})}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}}, \end{aligned}$$

which is invertible. Then $\frac{\partial ^{2}\hat{\mathcal {L}}({\varvec{\theta }})}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{{\mathrm{\tiny T}}}}$ is invertible as desired.

Recall that ${\varvec{\theta }}\!=\!\left( c,a,{\varvec{\beta }}^{\mathrm{\tiny T}},{\varvec{\lambda }}^{\mathrm{\tiny T}}\right) ^{\mathrm{\tiny T}}\!=\!\left( {\varvec{\gamma }}^{\mathrm{\tiny T}},{\varvec{\lambda }}^{\mathrm{\tiny T}}\right) ^{\mathrm{\tiny T}}, \hat{{\varvec{\theta }}}=\left( {\varvec{\hat{\gamma }}}^{{\mathrm{\tiny T}}}, \hat{{\varvec{\lambda }}}^{{\mathrm{\tiny T}}} \right) ^{{\mathrm{\tiny T}}} ={\text {argmin}}_{{\varvec{\theta }} \in {\varTheta }}\frac{1}{T}$ $\sum _{t=1}^{T}\hat{l}_{t}$ with ${\varTheta }={\varTheta }_1\times {\varTheta }_2$ and $\tilde{{\varvec{\theta }}}=\left( {\varvec{\tilde{\gamma }}}^{{\mathrm{\tiny T}}}, \tilde{{\varvec{\lambda }}}^{{\mathrm{\tiny T}}} \right) ^{{\mathrm{\tiny T}}}$ with ${\varvec{\tilde{\gamma }}}$ in (13) and $\tilde{\eta }(\cdot )=\mathbf {B}\tilde{{\varvec{\lambda }}}$.

Lemma 13

Under Assumptions (A1)–(A5),

$$\begin{aligned} \left\| \hat{{\varvec{\theta }}}-\tilde{{\varvec{\theta }} }\right\| =O\left\{ N^{1/2}\left( h^{p}+T^{-1}\right) \right\} ,a.s. \end{aligned}$$

Proof

Let $\hat{\mathcal {L}}_T = T^{-1}\sum _{t=1}^T \hat{l}_t$. By Taylor expansion,

$$\begin{aligned} \frac{\partial \hat{\mathcal {L}}_T(\hat{{\varvec{\theta }}})}{\partial {\varvec{\theta }}} -\frac{\partial \hat{\mathcal {L}} _T(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\theta }}} = \frac{\partial ^2 \hat{\mathcal {L}}_T({\varvec{\zeta }})}{\partial {\varvec{\theta }} \partial {\varvec{\theta }}^T}\left( \hat{{\varvec{\theta }}}-\tilde{{\varvec{\theta }}}\right) , \end{aligned}$$

where ${\varvec{\zeta }}=\mathbf {t}\hat{{\varvec{\theta }}}+\left( \mathbf {I}-\mathbf {t} \right) \tilde{\varvec{\theta }}$. Therefore, $\hat{{\varvec{{\theta }}}}-\tilde{{\varvec{\theta }}}=-\left( \frac{\partial ^2 \hat{\mathcal {L}}_T({\varvec{\zeta }})}{\partial {\varvec{\theta }} \partial {\varvec{\theta }}^T}\right) ^{-1} \frac{\partial \hat{\mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\theta }}}$.

First, $\frac{\partial \hat{\mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\theta }}} = \left\{ \left( \frac{\partial \hat{\mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\gamma }}} \right) ^{^{{\mathrm{\tiny T}}}},\left( \frac{\partial \hat{\mathcal {L}}_{T} (\tilde{{\varvec{\theta }}})}{\partial {\varvec{\lambda }}}\right) ^{^{{\mathrm{\tiny T}}}}\right\} ^{^{{ \mathrm{\tiny T}}}}$, where

$$\begin{aligned} \frac{\partial \hat{\mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\gamma }}} =\frac{1}{T} \sum _{t=1}^{T}\left\{ \left( 1-\frac{y_{t}^{2}}{\tilde{\sigma }_{t}^{2}} \right) \frac{\partial \log \tilde{\sigma }_{t}^{2}}{\partial {\varvec{\gamma }}}\right\} . \end{aligned}$$

(14)

Simple computations give,

(15)

Under Assumption (A2), the preceding terms are all bounded. To be specific,

$$\begin{aligned} |H_{1,t}|_{\infty }=|H_{3,t}|_{\infty }=O_{a.s}\left( \frac{1}{1-\delta }\right) , \quad {\text {and}} \quad |H_{2,t}|_{\infty }=O_{a.s}\left( \frac{1}{\left( 1-\delta \right) ^{2}} \right) . \end{aligned}$$

Since $\left| 1-y_{t}^{2}\tilde{\sigma }_{t}^{-2} \right| \le \left| 1-{y_{t}^{2}}{{\sigma }_{t}^{-2}}\right| +{y_{t}^{2}}O(h^{p})$, we obtain

$$\begin{aligned} \left| 1-y_{t}^{2}\tilde{\sigma }_{t}^{-2}\right| =O\left( \delta ^{t}+h^{p}\right) . \end{aligned}$$

(16)

Combining (14), (15) and (16), we have $\left\| \frac{\partial \hat{\mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\gamma }}}\right\| _{\infty }=O\left\{ \frac{1}{T(1-\delta )^{3}}+\frac{h^{p}}{(1-\delta )^{2}}\right\} =O(T^{-1}+h^{p}).$ We also have

$$\begin{aligned} \left\| \frac{\partial \hat{\mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{ \partial {\varvec{\lambda }}}\right\| _{\infty }=\left\| T^{-1}\sum _{t=1}^{T}\left\{ \left( 1-\frac{y_{t}^{2}}{\tilde{\sigma }_{t}^{2}} \right) \sum _{j=1}^{t}a^{j-1}\log y_{t-j}^{2}\mathbf {B}^{^{{ \mathrm{\tiny T}}}}\right\} \right\| _{\infty }=O(T^{-1}). \end{aligned}$$

Thus, $\left\| \frac{\partial \hat{\mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\theta }} }\right\| _{\infty }=O(T^{-1}+h^{p}).$

Second, let

$$\begin{aligned} V_{T}=\frac{\partial ^{2}\hat{\mathcal {L}}_{T}({\varvec{\zeta }})}{\partial {\varvec{\theta }}\partial {\varvec{\theta }}^{T}}=\left. \left( \begin{array}{cc} \frac{\partial ^{2}\hat{\mathcal {L}}_{T}}{\partial {\varvec{\gamma }}\partial {\varvec{\gamma }}^{T}} &{}\quad \frac{\partial ^{2}\hat{\mathcal {L}}_{T}}{\partial {\varvec{\gamma }}\partial {\varvec{\lambda }}^{T}} \\ \frac{\partial ^{2}\hat{\mathcal {L}}_{T}}{\partial {\varvec{\lambda }}\partial {\varvec{\gamma }}^{T}} &{}\quad \frac{\partial ^{2}\hat{\mathcal {L}}_{T}}{\partial {\varvec{\lambda }}\partial {\varvec{\lambda }}^{T}} \end{array} \right) \right| _{{\varvec{\theta }}={\varvec{\zeta }}}. \end{aligned}$$

According to Lemma 12, we have $V_{T}^{-1}=O(1), a.s.$, then

$$\begin{aligned} \left\| \hat{{\varvec{\theta }}}-\tilde{{\varvec{\theta }}}\right\| _{\infty } \le \left\| V_{T}^{-1}\right\| _{\infty } \left\| \frac{\partial \hat{ \mathcal {L}}_{T}(\tilde{{\varvec{\theta }}})}{\partial {\varvec{\theta }}} \right\| _{\infty } =O\left\{ N^{1/2}\left( h^{p}+T^{-1}\right) \right\} ,a.s. \end{aligned}$$

Proof of Theorem 1

We have

$$\begin{aligned} \left\| \hat{\eta }-\tilde{\eta }\right\| _{2}^{2}= & {} \left\| \left( \hat{{\varvec{\lambda }}}-\tilde{{\varvec{\lambda }}}\right) ^{^{{ \mathrm{\tiny T}}}}\mathbf {B}\right\| _{2}^{2}=\left( \hat{{\varvec{\lambda }}}-\tilde{ {\varvec{\lambda }}}\right) ^{{\mathrm{\tiny T}}}E\left\{ T^{-1}\sum _{j=1}^{T} \mathbf {B}({\mathbf {Z}}_{j}^{2})\mathbf {B}({\mathbf {Z}}_{j}^{2})\right\} \left( \hat{{\varvec{\lambda }}}-\tilde{{\varvec{\lambda }}}\right) \\\le & {} \kappa \left\| \hat{{\varvec{\lambda }}}-\tilde{{\varvec{\lambda }}} \right\| _{2}^{2}. \end{aligned}$$

According to Lemma 13, $\left\| \hat{\eta }-\tilde{\eta }\right\| _{2}=O_{p}\left\{ N^{1/2}\left( h^{p}+T^{-1}\right) \right\}$. Therefore,

$$\begin{aligned} {\left\| \hat{\eta }-{\eta _0 }\right\| }_{2}\le & {} {\left\| \hat{\eta }- \tilde{\eta }\right\| }_{2}+{\left\| \tilde{\eta }-\eta _0 \right\| }_{2} \\= & {} O_{p}\left\{ N^{1/2}\left( h^{p}+T^{-1}\right) \right\} +O_{p}\left( h^{p}\right) , \end{aligned}$$

which is of $O_{p}\left\{ N^{1/2}\left( h^{p}+T^{-1}\right) \right\}$.

For the additive components, by Lemma 1 of Stone (1985), $\left\| \hat{\eta }_{s_{2}}-\eta _{s_{2},0}\right\| _{2,s_2}=O_{p}\left\{ N^{1/2}\left( h^{p}+T^{-1}\right) \right\}$, for $1\le s_{2}\le d_{2}$. And, similar to Lemma A.8 in Wang and Yang (2007), we obtain the same rate for empirical norms, i.e., $\left\| \hat{\eta }_{s_{2}}-\eta _{s_{2},0}\right\| _{n,s_2}=O_{p}\left\{ N^{1/2}\left( h^{p}+T^{-1}\right) \right\}$, for $1\le s_{2}\le d_{2}.$

Proof of Theorem 2

“The” result follows immediately from Lemmas 11,  13 and Slutsky’s Lemma.