Optimal model averaging estimator for semi-functional partially linear models

Abstract

There have been many papers on frequentist model averaging over the past decade, but very little attention has been paid to how to conduct frequentist model averaging in functional data analysis. The present paper considers an optimal model averaging estimator for a semi-functional partially linear model with heteroscedasticity. Mallows-type and generalized cross-validation weight choice criteria are developed to assign model averaging weights. Under some regular assumptions, the resulting model averaging estimators are proved to be asymptotically optimal. Simulation results demonstrate the finite-sample performance of the proposed methods, and an empirical application with $\hbox {PM}_{2.5}$ data illustrates the proposed estimates.

Appendix

Lemma 1

If condition 1 hold, then for $1\le s \le S_n$, we have

$$\begin{aligned} \lambda _{\max }(\varvec{H}_{(s)}\varvec{H}_{(s)}^T) = O(\phi (h_s)^{-1}). \end{aligned}$$

Proof of Lemma 1

Denote the largest singualr value of a matrix A by $\lambda _{max}(A)$, and by the Reisz inequality (see Speckman Speckman (1988)), we have that

$$\begin{aligned} \lambda ^2_{\max }(\varvec{K}_{(s)})\le \max \limits _i \sum \limits _{j = 1}^{n}| {K}_{(s),ij} | \max \limits _j \sum \limits _{i = 1}^{n}| {K}_{(s),ij} |. \end{aligned}$$

Note that ${K}_{(s),ij}$ is non-negative, then $\max \nolimits _i \sum \nolimits _{j = 1}^{n}| {K}_{(s),ij} |$ is 1. From condition C.1, we have that $\max \nolimits _{1 \le i,j \le n}|{K}_{(s),ij} |=O((n\phi (h_s))^{-1})$, which implies that $\max \nolimits _j \sum \nolimits _{i = 1}^{n}|{K}_{(s),ij} |=O(\phi (h_s)^{-1})$. Therefore, we have that $\lambda ^2_{\max }(\varvec{K}_{(s)})=O(\phi (h_s)^{-1})$.

Because $\tilde{\varvec{H}}_{(s)}$ is an idempotent matrix, $\lambda _{\max }(\tilde{\varvec{H}}_{(s)})$ = 1. Then, for $1\le s \le S_n$, we have

$$\begin{aligned}&\lambda _{\max }(\varvec{H}_{(s)}\varvec{H}_{(s)}^T) \\&\quad \le \lambda _{\max }^2(\varvec{H}_{(s)})\\&\quad =\lambda _{\max }^2\{\tilde{\varvec{H}}_{(s)}(\varvec{I}-\varvec{K}_{(s)})+\varvec{K}_{(s)} \}\\&\quad \le [\lambda _{\max }(\tilde{\varvec{H}}_{(s)})\{1+\lambda _{\max }(\varvec{K}_{(s)})\}+\lambda _{\max }(\varvec{K}_{(s)})]^2\\&\quad =[\{1+\lambda _{\max }(\varvec{K}_{(s)})\}+\lambda _{\max }(\varvec{K}_{(s)})]^2\\&\quad = O(\phi (h_s)^{-1}). \end{aligned}$$

Proof of Theorem 1

Denote $||\varvec{Z}||^2 = \sum \nolimits _{i=1}^{d}Z_i^2$, and $\langle \varvec{Z}_1,\varvec{Z}_2\rangle =\sum \nolimits _{i=1}^{d}Z_{1,i}Z_{2,i}$, where $\varvec{Z}, \varvec{Z}_1, \varvec{Z}_2$ are d-dimensional vectors composed of $Z_{i}, Z_{1,i},Z_{2,i}$, respectively. The proof is similar to that of Theorem 1 of Wan et al. (2010). Let $A(\varvec{\omega })=\varvec{I}-\varvec{H}(\varvec{\omega })$,then

$$\begin{aligned} C_n(\varvec{\omega })=L_n(\varvec{\omega })+||\varvec{\varepsilon }||^2+2\langle \varvec{\varepsilon },A(\varvec{\omega })\varvec{\mu }\rangle +2({{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle ). \end{aligned}$$

Theorem 1 is valid if the following hold: as n $\rightarrow \infty$.

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },A(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.1)

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.2)

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |L_n(\varvec{\omega })/R_n(\varvec{\omega })-1|\rightarrow 0. \end{aligned}$$

(A.3)

From the first part of condition C.2, we have

$$\begin{aligned} \lambda _{\max }{(\varvec{\Omega })}=O(1). \end{aligned}$$

(A.4)

Using the triangle inequality, Chebyshevs inequality, Theorem 2 of Whittle (1960), condition C.2, we obtain

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },A(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })>\delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \sum \limits _{s = 1}^{S_n} \omega _s |\varvec{\varepsilon }^{T}(I-\varvec{H}_{(s)})\varvec{\mu }|>\delta \xi _n\right\} \\&\quad \le P\left\{ \max \limits _{1\le s \le S_n} |\varvec{\varepsilon }^{T}(I-\varvec{H}_{(s)})\varvec{\mu }|>\delta \xi _n\right\} \\&\quad = P \left\{ \{|\langle \varvec{\varepsilon },A(\omega _1^0)\varvec{\mu }\rangle |>\delta \xi _n\} \bigcup \{|\langle \varvec{\varepsilon },A(\omega _2^0)\varvec{\mu }\rangle |>\delta \xi _n\}\right. \\&\qquad \left. \bigcup \ldots \bigcup \{|\langle \varvec{\varepsilon },A(\omega _{S_n}^0)\varvec{\mu }\rangle |>\delta \xi _n\}\right\} \\&\quad \le \sum \limits _{s = 1}^{S_n} P \left\{ |\langle \varvec{\varepsilon },A(\omega _s^0)\varvec{\mu }\rangle |>\delta \xi _n\} \right\} \\&\quad \le \sum \limits _{s = 1}^{S_n} E \left\{ \langle \varvec{\varepsilon },A(\omega _s^0)\varvec{\mu }\rangle ^{2N} / (\delta \xi _n)^{2N} \right\} \\&\quad \le C (\delta \xi _n)^{-2N}\sum \limits _{s = 1}^{S_n} || \varvec{\Omega } ^{1/2} A(\varvec{\omega }_s^0)\varvec{\mu }||^{2N} \\&\quad \le C (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega })\sum \limits _{s = 1}^{S_n} || A(\varvec{\omega }_s^0)\varvec{\mu }||^{2N} \\&\quad \le C (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega })\sum \limits _{s = 1}^{S_n} R_n(\omega _s^0)^N, \end{aligned}$$

Then, by combining (A.4) and second part of condition C.2, we have (A.1).

Similarly, we have

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })> \delta \right\} \\&\quad \le \sum \limits _{s = 1}^{S_n} E \left\{ ({{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle )^{2N}/ (\delta \xi _n)^{2N} \right\} \\&\quad \le C (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega }) \sum \limits _{s = 1}^{S_n} \{{{\,\mathrm{\text {trace}}\,}}[\varvec{\Omega } \varvec{H}(\omega _s^0)^{T}\varvec{H}(\omega _s^0) ]\} ^{N} \\&\quad \le C^{'} (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega }) \sum \limits _{s = 1}^{S_n} R_n(\omega _s^0)^N. \end{aligned}$$

Then by combining (A.4) and second part of condition C.2, we obtain (A.2).

Note that (A.3) is equivalent to

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{||\varvec{H}(\varvec{\omega })\varvec{\varepsilon }||^{2}-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T} \varvec{H}(\varvec{\omega }))-2\langle A(\varvec{\omega })\varvec{\mu } ,\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle }{R_n(\varvec{\omega })}\right| . \end{aligned}$$

In order to prove (A.3), we need to show, as $n\rightarrow \infty$,

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{||\varvec{H}(\varvec{\omega })\varvec{\varepsilon }||^{2}-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T} \varvec{H}(\varvec{\omega }))}{R_n(\varvec{\omega })}\right| \rightarrow 0, \end{aligned}$$

(A.5)

and

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{\langle A(\varvec{\omega })\varvec{\mu } ,\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle }{R_n(\varvec{\omega })}\right| \rightarrow 0. \end{aligned}$$

(A.6)

Lemma 1 implies that $\max \nolimits _{s = 1,\ldots , S_n}[\lambda _{\max }(\varvec{H}_{(s)})]=O(\phi (h)^{-1/2})$ and $\max \nolimits _{s = 1,\ldots , S_n}[\lambda _{\max }(\varvec{H}_{(s)}$ $\varvec{H}_{(s)}^T)] = O(\phi (h)^{-1})$. Then, we obtain

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \left| \frac{||\varvec{H}(\varvec{\omega })\varvec{\varepsilon }||^{2}-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T} \varvec{H}(\varvec{\omega }))}{R_n(\varvec{\omega })}\right|> \delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} \omega _t \omega _s \left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}^{T}\varvec{H}_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}_{(s)}^{T} \varvec{H}_{(t)})\right|> \delta \xi _n \right\} \\&\quad \le P\left\{ \max \limits _{1\le t\le S_n} \max \limits _{1\le s \le S_n}\left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}^{T}\varvec{H}_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}_{(s)}^{T} \varvec{H}_{(t)})\right| > \delta \xi _n \right\} \\&\quad \le \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} E\left\{ [\langle \varvec{\Omega }^{-1/2}\varvec{\varepsilon },\varvec{\Omega }^{1/2}H (\varvec{\omega }_t^0)H(\varvec{\omega }_s^0)\varvec{\Omega }^{1/2}\varvec{\Omega }^{-1/2} \varvec{\varepsilon }\rangle \right. \\&\qquad \left. -{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega }_s^0)^{T} \varvec{H}(\varvec{\omega }_t^0))]^{2N}/(\delta \xi _n)^{2N} \right\} \\&\quad \le C \lambda _{\max }^{N}({\varvec{\Omega }})(\delta \xi _n)^{-2N} \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} {{\,\mathrm{\text {trace}}\,}}( \varvec{H}(\varvec{\omega }_t^0)^{T}\varvec{H}(\varvec{\omega }_s^0) \varvec{\Omega } \varvec{H}(\varvec{\omega }_s^0)^{T} \varvec{H}(\varvec{\omega }_t^0))^{N}\\&\quad \le C S_n \lambda _{\max }^{N}({\varvec{\Omega }})(\delta \xi _n)^{-2N} \lambda _{\max }^{2N}({\varvec{H}(\varvec{\omega }_t^0)}) \sum \limits _{s = 1}^{S_n} {{\,\mathrm{\text {trace}}\,}}( \varvec{\Omega } \varvec{H}(\varvec{\omega }_s^0)^{T}\varvec{H}(\varvec{\omega }_s^0))^{N}\\&\quad \le C S_n\lambda _{\max }^{N}({\varvec{\Omega }})(\delta \xi _n)^{-2N} \phi (h)^{-N} \sum \limits _{s = 1}^{S_n} R_n({\omega _s^0})^{N}. \end{aligned}$$

And, by combining (A.4) and second part of condition C.2, we obtain (A.5). Then, from Lemma 1, we have

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{\langle A(\varvec{\omega })\varvec{\mu } ,\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle }{R_n(\varvec{\omega })}\right|> \delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} \omega _t \omega _s \left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}(I-\varvec{H}_{(s)})\varvec{\mu }\right|> \delta \xi _n \right\} \\&\quad \le P\left\{ \max \limits _{1\le t\le S_n} \max \limits _{1\le s \le S_n} \left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}(I-\varvec{H}_{(s)})\varvec{\mu }\right| > \delta \xi _n \right\} \\&\quad \le \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} E\left\{ \langle \varvec{H}(\varvec{\omega }_t^0) \varvec{\varepsilon },A(\varvec{\omega }_s^0)\varvec{\mu }\rangle ^{2N}/(\delta \xi _n)^{2N} \right\} \\&\quad \le C(\delta \xi _n)^{-2N}\sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} || \varvec{H}(\varvec{\omega }_t^0) \varvec{\Omega }^{1/2} A(\varvec{\omega }_s^0)\varvec{\mu }||^{2N}\\&\quad \le C S_n \lambda _{\max }^{N}({\varvec{\Omega }})\lambda _{\max }^{2N}(\varvec{H}(\varvec{\omega }_t^0) ) (\delta \xi _n)^{-2N} \sum \limits _{s = 1}^{S_n} R_n(\varvec{\omega }_s^0)^{N}\\&\quad \le C S_n \lambda _{\max }^{N}({\varvec{\Omega }}) \phi (h)^{-N} (\delta \xi _n)^{-2N} \sum \limits _{s = 1}^{S_n} R_n(\varvec{\omega }_s^0)^{N}.\\ \end{aligned}$$

And, by combining (A.4) and second part of condition C.2, we have (A.6). This completes the proof of Theorem 1.

Proof of Theorem 2

Theorem 2 holds, if we have

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }}(\varvec{\omega }))-{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })=o_p(1). \end{aligned}$$

Let $Q_{(s)}=\text {diag}({\widetilde{h}}_{(s),11},{\widetilde{h}}_{(s),22}, \ldots ,{\widetilde{h}}_{(s),nn})$ and $Q(\varvec{\omega }) =\sum \nolimits _{s = 1}^{S_n} w_s Q_{(s)}$, where ${\widetilde{h}}_{(s),ii}$ is the ith diagonal element of $\varvec{H}_{(s)}$. Then, we have

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }} (\varvec{\omega }))-{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{Y}-\varvec{H}(\varvec{\omega })\varvec{Y}]^{T} Q(\varvec{\omega })[\varvec{Y}-\varvec{H}(\varvec{\omega })\varvec{Y}] -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega }) |/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{\varepsilon }+\varvec{U}-\varvec{H} (\varvec{\omega })\varvec{Y}]^{T}Q(\varvec{\omega })[\varvec{\varepsilon }+\varvec{U}-\varvec{H}(\varvec{\omega }) \varvec{Y}] -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad \le \sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}Q(\varvec{\omega })\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\qquad +\sup \limits _{\varvec{\omega } \in H_n}|(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})^{T} Q(\varvec{\omega })(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})|/R_n(\varvec{\omega }) \\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T} Q(\varvec{\omega }) [\varvec{H}(\varvec{\omega })\varvec{Y}-\varvec{U}]|/R_n(\varvec{\omega })\\&\quad \le \sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}Q(\varvec{\omega }) \varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\qquad + \sup \limits _{\varvec{\omega } \in H_n}|(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})^{T} Q(\varvec{\omega })(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})|/R_n(\varvec{\omega }) \\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T} Q(\varvec{\omega }) [\varvec{H}(\varvec{\omega })\varvec{U}-\varvec{U}]|/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T} Q(\varvec{\omega }) \varvec{H}(\varvec{\omega })\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{H} (\varvec{\omega })\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{H} (\varvec{\omega })\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\quad = I_1+I_2+I_3+I_4+I_5. \end{aligned}$$

Define $\rho =\max \limits _s\max \limits _i |{\widetilde{h}}_{(s),ii}|$. From Lemma 1, condition C.4 and condition C.5, we have

$$\begin{aligned} \rho&\le c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|\}\nonumber \\&\le c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)})-{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)})|\}+c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&\le c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)})|\}+c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)})|\}\nonumber \\&\quad +c n^{-1}\max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})| \nonumber \\&= c n^{-1}{\tilde{p}}+ c n^{-1} 2^{-1}\max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)}+\varvec{K}^T_{(s)}) \tilde{\varvec{H}}_{(s)})|\}\nonumber \\&\quad +c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&\le c n^{-1}{\tilde{p}}+ c n^{-1} 2^{-1}\max \limits _s\{\lambda _{\max }(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)}+\varvec{K}^T_{(s)} \tilde{\varvec{H}}_{(s)})\text {rank}\nonumber \\&\qquad (\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)} +\varvec{K}^T_{(s)}\tilde{\varvec{H}}_{(s)})\}\nonumber \\&\quad +c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&\le c n^{-1}{\tilde{p}}+ c n^{-1} 2\max \limits _s\{p_{s}\lambda _{\max }(\tilde{\varvec{H}}_{(s)})\lambda _{\max }(\varvec{K}_{(s)})\} +c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&=O(n^{-1}{\tilde{p}}+ n^{-1}{\tilde{p}} \phi (h)^{-1/2}+n^{-1}\phi (h)^{-1}). \end{aligned}$$

(A.7)

From the second part of condition C.2, we have

$$\begin{aligned} \xi _n^{-1}\phi (h)^{-1}=o(1) \end{aligned}$$

(A.8)

and

$$\begin{aligned} \xi _n^{-N}\phi (h)^{-N}S_n=o(1). \end{aligned}$$

(A.9)

Using (A.4), Lemma 1 and (A.7), Chebyshev’s inequality and Theorem 2 of Whittle (1960), we obtain, for any $\delta >0$,

$$\begin{aligned} P(I_1>\delta )&\le \sum \limits _{s = 1}^{S_n}P\{|\varvec{\varepsilon }^{T}Q_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{\Omega })|>\delta \xi _n\}\\&\le ({\delta \xi _n})^{-2N}\sum \limits _{i = 1}^{S_n} E\{ \varvec{\varepsilon }^{T}Q_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{\Omega })\}^{2N}\\&\le c ({\delta \xi _n})^{-2N}\sum \limits _{i = 1}^{S_n} {{\,\mathrm{\text {trace}}\,}}^{N} \{ \varvec{\Omega }^{1/2} Q_{(s)}\varvec{\Omega } Q_{(s)} \varvec{\Omega }^{1/2}\}\\&\le c ({\delta \xi _n})^{-2N}S_n \lambda _{\max }^{2N}(\varvec{\Omega }) \rho ^{2N} n^{N}\\&= c ({\delta \xi _n})^{-2N} S_n \lambda _{\max }^{2N}(\varvec{\Omega }) O(n^{-1}{\tilde{p}}^2+ n^{-1}{\tilde{p}}^2 \phi (h)^{-1}+n^{-1}\phi (h)^{-2})^N,\\ I_2&\le \rho \sup \limits _{\varvec{\omega } \in H_n} \{(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})^{T}(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})\}/R_n(\varvec{\omega })\\&=\rho \sup \limits _{\varvec{\omega } \in H_n}[L_n(\varvec{\omega })/R_n(\varvec{\omega })]=O(n^{-1}{\tilde{p}}+ n^{-1}{\tilde{p}} \phi (h)^{-1/2}+n^{-1}\phi (h)^{-1}),\\ I_3&\le 2\sup \limits _{\varvec{\omega } \in H_n} \{ ||\varvec{\varepsilon }||^2 \rho ^2 (\varvec{U}-H(\varvec{\omega })\varvec{U})^{T}(\varvec{U}-H(\varvec{\omega })\varvec{U})\}/R_n^{2}(\varvec{\omega })\}^{1/2} \\&\le 2||\varvec{\varepsilon }|| \rho \xi _n^{-1/2} = 2\xi _n^{-1/2} O(n^{-1/2}{\tilde{p}}+ n^{-1/2}{\tilde{p}} \phi (h)^{-1/2}+n^{-1/2}\phi (h)^{-1}),\\ P(I_4/2>\delta )&\le \sum \limits _{s = 1}^{S_n}P\{|\varvec{\varepsilon }^{T} Q_{(s)}\varvec{H}_{(s)}\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{H}_{(s)}\varvec{\Omega } )|>\delta \xi _n\}\\&\le (\delta \xi _n)^{-2N}\sum \limits _{s = 1}^{S_n}E\{\varvec{\varepsilon }^{T} Q_{(s)}\varvec{H}_{(s)}\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{H}_{(s)}\varvec{\Omega }) \}^{2N}\\&\le C (\delta \xi _n)^{-2N}\sum \limits _{s = 1}^{S_n}{{\,\mathrm{\text {trace}}\,}}^{N}\{\varvec{\Omega }^{1/2} Q_{(s)}\varvec{H}_{(s)}\varvec{\Omega } \varvec{H}_{(s)}^{T} Q_{(s)}\varvec{\Omega }^{1/2} \}\\&\le C (\delta \xi _n)^{-2N}S_n \lambda _{\max }^{2N}{(\varvec{\Omega })} n^{N} \rho ^{2N} \max \limits _s [\lambda _{\max }^N(\varvec{H}_{(s)}\varvec{H}_{(s)}^{T})] \\&= \xi _n^{-2N}\phi (h)^{-N} S_n O(n^{-1}{\tilde{p}}^2+ n^{-1}{\tilde{p}}^2 \phi (h)^{-1}+n^{-1}\phi (h)^{-2})^{N}, \\ I_5&\le 2\xi _n^{-1} \rho \lambda _{\max }(\varvec{\Omega }) \sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega }))|\\&\le 2\xi _n^{-1} \rho \lambda _{\max }(\varvec{\Omega }) \max \limits _s |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|\\&= 2\xi _n^{-1} \rho \lambda _{\max }(\varvec{\Omega }) [{\tilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}]\\&= 2\xi _n^{-1}O(n^{-1}{\tilde{p}}^2\phi (h)^{-1} + n^{-1}{\tilde{p}} \phi (h)^{-3/2}+n^{-1}\phi (h)^{-2}).\\ \end{aligned}$$

Thus, by combining (A.8) and (A.9) and condition C.6, we have $I_1+I_2+I_3+I_4+I_5=o_p(1)$. This completes the proof.

Proof of Corollary 1

Corollary 1 holds, if we can show

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }}_{(s^{*})}) -{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega }){\varvec{\Omega }})|/R_n(\varvec{\omega })=o_p(1). \end{aligned}$$

Note that

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }}_{(s^{*})})-{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{Y}-\varvec{H}_{(s^{*})}\varvec{Y}]^{T}Q(\varvec{\omega })[\varvec{Y}-\varvec{H}_{(s^{*})}\varvec{Y}] -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{\varepsilon }+\varvec{U}-\varvec{H}_{(s^{*})}\varvec{U}-\varvec{H}_{(s^{*})} \varvec{\varepsilon }]^{T}Q(\varvec{\omega })[\varvec{\varepsilon }+\varvec{U} -\varvec{H}_{(s^{*})}\varvec{U}-\varvec{H}_{(s^{*})}\varvec{\varepsilon }]\\&\qquad -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad \le \sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}(\varvec{I}_n-\varvec{H}_{(s^{*})})^T Q(\varvec{\omega })(\varvec{I}_n-\varvec{H}_{(s^{*})})\varepsilon \\&\qquad -{{\,\mathrm{\text {trace}}\,}}((\varvec{I}_n-\varvec{H}_{(s^{*})})^{T} Q(\varvec{\omega }) (\varvec{I}_n-\varvec{H}_{(s^{*})}) \varvec{\Omega }) |/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}(\varvec{I}_n-\varvec{H}_{(s^{*})})^{T} Q(\varvec{\omega })(\varvec{I}_n-\varvec{H}_{(s^{*})})\varvec{U} |/R_n(\varvec{\omega })\\&\qquad +\sup \limits _{\varvec{\omega } \in H_n}|\varvec{U}^{T}(\varvec{I}_n-\varvec{H}_{(s^{*})})^{T} Q(\varvec{\omega })(\varvec{I}_n-\varvec{H}_{(s^{*})})\varvec{U} |/R_n(\varvec{\omega })\\&\qquad +\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}^{T}_{(s^{*})}Q(\varvec{\omega })\varvec{H}_{(s^{*})}\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}^{T}_{(s^{*})}Q(\varvec{\omega })\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\quad = J_1+J_2+J_3+J_4+J_5. \end{aligned}$$

Following the proof (A.7) in Zhang and Wang (2019), we have $J_1+J_2+J_3+J_4+J_5=o_p(1)$. This completes the proof.

Proof of Theorem 3

Note that

$$\begin{aligned} GCV(\varvec{\omega })&= \varvec{Y}^T\varvec{A}(\varvec{\omega })^T\varvec{A}(\varvec{\omega })\varvec{Y}+ \varvec{Y}^T \varvec{M}(\varvec{\omega })\varvec{Y}\\&= L_n(\varvec{\omega })+||\varvec{\varepsilon }||^2+ 2\varvec{\varepsilon }^T\varvec{A}(\varvec{\omega })\varvec{\mu }-2\varvec{\varepsilon }^T\varvec{H} (\varvec{\omega })\varvec{\varepsilon }\\&\quad +\varvec{\mu }^T\varvec{M}(\varvec{\omega })\varvec{\mu }+\varvec{\varepsilon }^T\varvec{M}(\varvec{\omega }) \varvec{\varepsilon }+2\varvec{\varepsilon }^T\varvec{M}(\varvec{\omega })\varvec{\mu }, \end{aligned}$$

where $\varvec{M}(\varvec{\omega }) = \varvec{T}(\varvec{\omega })^T\varvec{A}(\varvec{\omega })+\varvec{T}(\varvec{\omega })^T\varvec{T} (\varvec{\omega })+\varvec{A}(\varvec{\omega })^T\varvec{T}(\varvec{\omega })$ and $\varvec{T}(\varvec{\omega }) = \sum \nolimits _{s=1}^{S_n}\omega _s\varvec{D}_{(s)}\varvec{A}_{(s)}$. Theorem 3 holds if we can show that, as $n \rightarrow \infty$,

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} | \langle \varvec{\varepsilon }, \varvec{A}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.10)

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} | \langle \varvec{\varepsilon }, \varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.11)

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\mu },\varvec{M}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.12)

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{M}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.13)

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{M}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.14)

and

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |L_n(\varvec{\omega })/R_n(\varvec{\omega })-1|\rightarrow 0. \end{aligned}$$

(A.15)

From (A.1) and (A.3), we can obtain (A.10) and (A.15), respectively. Note that (A.11) is valid, if

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )- \langle \varvec{\varepsilon }, \varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.16)

and

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )|/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

(A.17)

From (A.2), we obtain (A.16). Then, from condition C.5, we have

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })&\le \xi _n^{-1} \max \limits _s |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)}\varvec{\Omega })|\nonumber \\&= \xi _n^{-1} \max \limits _s |{{\,\mathrm{\text {trace}}\,}}({Q}_{(s)}\varvec{\Omega })|\nonumber \\&\le \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} \max \limits _s \sum \limits _{i}^{n}|{\tilde{h}}_{(s),ii}|.\nonumber \\&\le \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} \max \limits _s [\Lambda |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|]\nonumber \\ \end{aligned}$$

(A.18)

From (A.7) and Lemma 1, we have

$$\begin{aligned} \max \limits _s |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})| = O({\widetilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}), \end{aligned}$$

(A.19)

and

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} \lambda _{\max }(\varvec{A}(\varvec{\omega }))\le 1+\max \limits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{H}_{(s)})] = O(\phi (h)^{-1/2}). \end{aligned}$$

(A.20)

From (A.19), condition C.7, (A.4), and (A.8), we have

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad \le \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} \max \limits _s [\Lambda |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|]\\&\quad = \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} O({\widetilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1})\\&\quad = o(1). \end{aligned}$$

Thus, equation (A.11) is true.

From (A.19) and condition C.6, we obtain $\max \nolimits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)})]= {\bar{d}}/(1-{\bar{d}}) = O(\frac{ {\tilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}}{n})$, where ${\bar{d}}=\max \nolimits _{s\in \{1,\ldots ,S_n\}}[d_{(s)}]$. Then, we have

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} \lambda _{\max }(\varvec{T}(\varvec{\omega }))&\le \max \limits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)}\varvec{A}_{(s)})]\nonumber \\&\le \max \limits _{s\in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)})] \max _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{A}_{(s)})]\nonumber \\&= O(\frac{{\tilde{p}}\phi (h)^{-1}+\phi (h)^{-3/2} }{n}). \end{aligned}$$

(A.21)

Equation (A.12) is valid if

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.22)

and

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

(A.23)

From conditions C.3, C.6, C.7 and (A.8), we have

$$\begin{aligned}&|\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\nonumber \\&\quad =| \sum \limits _{t=1}^{S_n}\sum \limits _{m=1}^{S_n} \omega _t\omega _m \varvec{\mu }^{T}\varvec{A}_{(t)}^{T}\varvec{D}_{(t)} \varvec{A}_{(m)}\varvec{\mu }|/R_n(\varvec{\omega })\nonumber \\&\quad =| \varvec{\mu }^{T}\sum \limits _{t=1}^{S_n} \omega _t \varvec{A}_{(t)}^{T}\varvec{D}_{(t)} \varvec{A}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\nonumber \\&\quad \le \left( \varvec{\mu }^{T}\sum \limits _{t=1}^{S_n} \omega _t \varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\sum \limits _{t=1}^{S_n} \omega _t \varvec{D}_{(t)}\varvec{A}_{(t)}\varvec{\mu } \varvec{\mu }^{T}\varvec{A}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }/R^2_n(\varvec{\omega })\right) ^{1/2}\nonumber \\&\quad \le \left( \varvec{\mu }^{T}\sum \limits _{t=1}^{S_n} \omega _t \varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\sum \limits _{t=1}^{S_n} \omega _t \varvec{D}_{(t)}\varvec{A}_{(t)}\varvec{\mu } /R_n(\varvec{\omega })\right) ^{1/2}\nonumber \\&\quad =\left( \sum \limits _{t=1}^{S_n}\sum \limits _{m=1}^{S_n}\omega _t \omega _m \varvec{\mu }^{T}\varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\varvec{D}_{(m)} \varvec{A}_{(m)}\varvec{\mu } /R_n(\varvec{\omega })\right) ^{1/2}\nonumber \\&\quad \le \xi _n^{-1}\varvec{\mu }^{T}\varvec{\mu }\lambda _{\max }(\varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\varvec{D}_{(m)} \varvec{A}_{(m)})\nonumber \\&\quad = \frac{{\tilde{p}}^2\phi (h)^{-2}+\phi (h)^{-3}+{\tilde{p}}\phi (h)^{-5/2} }{n\xi _n}\nonumber \\&\quad = o(1), \end{aligned}$$

(A.24)

and from conditions C.3, C.6, C.7, (A.8) and (A.21), we have

$$\begin{aligned}&|\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\nonumber \\&\quad \le \varvec{\mu }^T\varvec{\mu }\lambda _{\max }(\varvec{T}(\varvec{\omega }))\lambda _{\max }(\varvec{T}(\varvec{\omega }))/\xi _n\nonumber \\&\quad = \frac{{\tilde{p}}^2\phi (h)^{-2}+\phi (h)^{-3}+{\tilde{p}}\phi (h)^{-5/2} }{n\xi _n}\nonumber \\&\quad = o(1). \end{aligned}$$

(A.25)

Then, equation (A.12) is obtained. Equation (A.13) is valid if

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.26)

and

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

(A.27)

Note that $\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }\rangle \le \lambda ^2_{\max }(\varvec{T}(\varvec{\omega }))\varvec{\varepsilon }^{T}\varvec{\varepsilon }$, and from conditions C.6, C.7, (A.4), (A.8) and (A.21), we have

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \lambda ^2_{\max }(\varvec{T}(\varvec{\omega }))\varvec{\varepsilon }^{T}\varvec{\varepsilon }/R_n(\varvec{\omega })>\delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \varvec{\varepsilon }^{T}\varvec{\varepsilon }> \lambda ^{-2}_{\max }(\varvec{T}(\varvec{\omega }))\xi _n\delta \right\} \\&\quad \le E(\varvec{\varepsilon }^{T}\varvec{\varepsilon } )\lambda ^2_{\max }(\varvec{T}(\varvec{\omega })) \xi _n^{-1}\delta ^{-1}\\&\quad \le {{\,\mathrm{\text {trace}}\,}}(\Omega )\lambda ^2_{\max }(\varvec{T}(\varvec{\omega })) \xi _n^{-1}\delta ^{-1}\\&\quad \le \lambda _{\max }(\Omega )n\lambda ^2_{\max }(\varvec{T}(\varvec{\omega })) \xi _n^{-1}\delta ^{-1}\\&\quad =o(1), \end{aligned}$$

which states that (A.26) is true. Note that

$$\begin{aligned}&|\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\varepsilon }\rangle |\nonumber \nonumber \\&\quad =|\varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{\varepsilon } - \varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{H}(\varvec{\omega })\varvec{\varepsilon }|\nonumber \nonumber \\&\quad =|\varvec{\varepsilon }^{T}\sum \limits _{s=1}^{S_n}\omega _s\varvec{D}_{(s)} \varvec{\varepsilon }-\varvec{\varepsilon }^{T}\sum \limits _{s=1}^{S_n}\omega _s\varvec{D}_{(s)}\varvec{H}_{(s)} \varvec{\varepsilon }-\varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{H}(\varvec{\omega })\varvec{\varepsilon }|\nonumber \nonumber \\&\quad \le \left| \max \limits _{s \in \{1,\ldots ,S_n\}}\lambda _{\max }(\varvec{D}_{(s)})\varvec{\varepsilon }^{T}\varvec{\varepsilon }\right| + \left| {\bar{d}}/(1-{\bar{d}})\right| \max \limits _{s \in \{1,\ldots ,S_n\}}\left| \varvec{\varepsilon }^{T}\varvec{H}_{(s)}\varvec{\varepsilon }\right| + \nonumber \nonumber \\&\qquad \qquad [(\varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }) (\varvec{\varepsilon }^{T}\varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\varvec{\varepsilon })]^{1/2}\nonumber \nonumber \\&\quad =L_1+L_2+L_3. \end{aligned}$$

(A.28)

From condition C.7, $\max \nolimits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)})] = O_p(\frac{ {\tilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}}{n})$, (A.4) and (A.8), we have

$$\begin{aligned} \begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} L_1/R_n(\varvec{\omega })\rightarrow 0. \end{aligned} \end{aligned}$$

Note that $L_2 \le \left| {\bar{d}}/(1-{\bar{d}})\right| \max \nolimits _{s \in \{1,\ldots ,S_n\}}\left| \varvec{\varepsilon }^{T}\varvec{H}_{(s)}\varvec{\varepsilon } - {{\,\mathrm{\text {trace}}\,}}\{ \varvec{H}_{(s)}\varvec{\Omega }\}\right| + \left| {\bar{d}}/(1-{\bar{d}})\right| \max \nolimits _{s \in \{1,\ldots ,S_n\}}\left| {{\,\mathrm{\text {trace}}\,}}\{ \varvec{H}_{(s)}\varvec{\Omega }\}\right|$. Then, similar to (A.16) and (A.17), we have

$$\begin{aligned} \begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} L_2/R_n(\varvec{\omega })\rightarrow 0. \end{aligned} \end{aligned}$$

From (A.5) and the fact that $R_n(\varvec{\omega })=E[L_n(\varvec{\omega })|\varvec{X},\varvec{T}]=||\varvec{H}(\varvec{\omega })\varvec{\mu }-\varvec{\mu }||^2+{{\,\mathrm{\text {trace}}\,}}\{\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\}>{{\,\mathrm{\text {trace}}\,}}\{\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\},$ we obtain

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega }) \varvec{\varepsilon }|/R_n(\varvec{\omega })=O_p(1). \end{aligned}$$

(A.29)

Combining with (A.26) and (A.29), we can show that $\sup \nolimits _{\varvec{\omega } \in H_n} L_3/R_n(\varvec{\omega })\rightarrow 0$. Thus, equation (A.27) is true. Then, equation (A.13) is true.

Equation (A.14) is true, if

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{A}(\varvec{\omega }) \varvec{\mu }|/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.30)

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega }) \varvec{\mu }|/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$

(A.31)

and

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{A}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega }) \varvec{\mu }|/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

(A.32)

From (A.26), the fact that $R_n(\varvec{\omega })=E[L_n(\varvec{\omega })|\varvec{X},\varvec{T}]=||\varvec{H} (\varvec{\omega })\varvec{\mu }-\varvec{\mu }||^2+{{\,\mathrm{\text {trace}}\,}}\{\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\}> \varvec{\mu }^T\varvec{A}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }$ and

$$\begin{aligned}&|\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le [(\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }/R_n(\varvec{\omega }))(\varvec{\mu }^T\varvec{A}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu })/R_n(\varvec{\omega })]^{1/2},\\ \end{aligned}$$

we obtain (A.30). From (A.25), (A.26) and

$$\begin{aligned}&|\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le [(\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }/R_n(\varvec{\omega }))(\varvec{\mu }^T\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu })/R_n(\varvec{\omega })]^{1/2},\\ \end{aligned}$$

we obtain (A.31).

Similar to (A.1), we have $\max \nolimits _{s\in \{ 1,\ldots , S_n\}}|\varvec{\varepsilon }^{T}\varvec{A}_{(s)}\varvec{\mu }|/R_n(\varvec{\omega }) = o_p(1)$, which along with condition C.6, (A.8), (A.25), (A.29) and

$$\begin{aligned}&|\varvec{\varepsilon }^{T}\varvec{A}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad = |\varvec{\varepsilon }^{T}\varvec{T}(\varvec{\omega })\varvec{\mu }- \varvec{\varepsilon }^{T} \varvec{H}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le |\varvec{\varepsilon }^{T}\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega }) + |\varvec{\varepsilon }^{T}\varvec{H}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le |{\bar{d}}/(1-{\bar{d}})|\max _{s\in \{ 1,\ldots , S_n\}}|\varvec{\varepsilon }^{T} \varvec{A}_{(s)}\varvec{\mu }|/R_n(\varvec{\omega })\\&\qquad + [(\varvec{\varepsilon }^{T}\varvec{H}^{T}(\varvec{\omega }) \varvec{H}(\varvec{\omega })\varvec{\varepsilon }/R_n(\varvec{\omega })) (\varvec{\mu }^T \varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }/R_n(\varvec{\omega }))]^{1/2},\\ \end{aligned}$$

we obtain (A.32). Then, equation (A.14) is valid. This completes the proof.