Empirical likelihood for high-dimensional linear regression models

Abstract

High-dimensional data are becoming prevalent, and many new methodologies and accompanying theories for high-dimensional data analysis have emerged in response. Empirical likelihood, as a classical nonparametric method of statistical inference, has proved to possess many good features. In this paper, our focus is to investigate the asymptotic behavior of empirical likelihood for regression coefficients in high-dimensional linear models. We give regularity conditions under which the standard normal calibration of empirical likelihood is valid in high dimensions. Both random and fixed designs are considered. Simulation studies are conducted to check the finite sample performance.

Appendix: Proofs of Propositions 1, 2 and 3

Before proving Propositions 1, 2 and 3, we present some auxiliary lemmas. Let $L_{n}\equiv \max _{1\le i,j\le n}\mid \widetilde{S}_{n,i,j}-V_{n,i,j}\mid$ be the largest absolute element of $\widetilde{\varvec{S}}_{n}-\varvec{V}_{n}$ with $\widetilde{\varvec{S}}_{n}=\frac{1}{n}\sum _{i=1}^{n}\varvec{Z}_{i}\varvec{Z}_{i}^\mathsf{T}$ and $\varvec{V}_{n}=\frac{1}{n}\sum _{i=1}^{n}\text{ Var }(\varvec{Z}_{i})=\frac{\sigma ^2}{n}\varvec{XX}^\mathsf{T}$. Let $\mathcal {M}(\varvec{A})$ denote the largest absolute element of matrix $\varvec{A}$. For convenience, define $(\varvec{A})_{ij}$ and $(\varvec{a})_{i}$ as the $(i,j)$ element of a matrix $\varvec{A}$ and the $i$th component of a vector $\varvec{a}$, respectively. To avoid confusion, we state here that the auxiliary variable $\varvec{Z}_{i}$ appeared in the following arguments and formulas is that for fixed design case.

Lemma 1

Assume Conditions $1^{'}$ and $2^{'}$. Then, for any given $\varepsilon >0$ and some positive constant $C_{q}$ only depending on $q$,

$$\begin{aligned} \hbox {{P}}(L_{n}\ge \varepsilon )<C_{q}K_{1}^{2}K_{2}\frac{p^2}{n^{q/2}\varepsilon ^{q}}. \end{aligned}$$

Proof

Let $\varvec{R}_{n}=\widetilde{\varvec{S}}_{n}-\varvec{V}_{n}\mathop {=}\limits ^{\wedge }(R_{jk})_{p\times p}$ with $R_{jk}=\frac{1}{n}\sum _{i=1}^{n}x_{ij}x_{ik}(\epsilon _{i}^{2}-\sigma ^{2}),\, j,k=1,\ldots ,p$. We have

$$\begin{aligned} \text{ P }(|R_{jk}|\ge \varepsilon )\le \frac{\text{ E }|\sqrt{n}R_{jk}|^{q}}{(\sqrt{n}\varepsilon )^{q}} =\frac{n^{-\frac{q}{2}}\text{ E }|\sum _{i=1}^{n}x_{ij}x_{ik}(\epsilon _{i}^{2}-\sigma ^{2})|^{q}}{n^{q/2}\varepsilon ^{q}}. \end{aligned}$$

Let $W_{ijk}=x_{ij}x_{ik}(\epsilon _{i}^{2}-\sigma ^{2})$, $1\le i\le n$ and $j, k=1, 2, \ldots , p$. Obviously, for any fixed $j, k$, the $W_{ijk}$’s are independent but not identically distributed random variables with mean zero. By the Theorem 2 of Dharmadhikari and Jogdeo (1969), we have

$$\begin{aligned} \text{ E }\left| \sum _{i=1}^{n}W_{ijk}\right| ^{q}\le C_{q}n^{q/2-1}\sum _{i=1}^{n}\text{ E }|W_{ijk}|^{q} \le C_{q}\text{ E }|\epsilon _{1}|^{2q}n^{q/2-1}\sum _{i=1}^{n}|x_{ij}|^{q}|x_{ik}|^{q}, \end{aligned}$$

where $C_{q}$ is a finite positive constant only depending on $q$. This leads to

$$\begin{aligned} \text{ P }(|R_{jk}|\ge \varepsilon ) \le C_{q}\text{ E }|\epsilon _{1}|^{2q}\frac{\frac{1}{n}\sum _{i=1}^{n}|x_{ij}|^{q}|x_{ik}|^{q}}{n^{q/2}\varepsilon ^{q}}. \end{aligned}$$

Therefore, by Conditions $1^{'}$ and $2^{'}$, we obtain

$$\begin{aligned} \text{ P }(L_{n}\ge \varepsilon )&\le \sum _{j,k=1}^{p}\text{ P }(|R_{jk}|\ge \varepsilon )\\&\le C_{q}\text{ E }|\epsilon _{1}|^{2q}p^{2}\frac{\frac{1}{n}\sum _{i=1}^{n}(p^{-1}\sum _{j=1}^{p}|x_{ij}|^{q})(p^{-1} \sum _{j=1}^{p}|x_{ik}|^{q})}{n^{q/2}\varepsilon ^{q}}\\&< C_{q}K_{1}^{2}K_{2}\frac{p^{2}}{n^{q/2}\varepsilon ^{q}}\,. \end{aligned}$$

$\square$

Lemma 2

$$\begin{aligned} \max _{1\le i\le p}|\gamma _{i}(\varvec{V}_{n})-\gamma _{i}(\widetilde{\varvec{S}}_{n})|\le pL_{n} \end{aligned}$$

Proof

The proof follows a similar line to that of Lemma 2 in Appendix A of Liu et al. (2013) and we refer to that for details. $\square$

According to Lemma 1 and Conditions $1^{'}$ and $2^{'}$, we have

$$\begin{aligned} \text{ P }(pL_{n}\ge \varepsilon )\le C_{q}K_{1}^{2}K_{2}\frac{p^{2+q}}{n^{q/2}\varepsilon ^{q}}, \end{aligned}$$

which, together with Condition 4, leads to $pL_{n}=o_{p}(1)$. This means that the eigenvalues of $\widetilde{\varvec{S}}_{n}$ are close to those of $\varvec{V}_{n}$.

Summarizing the results of Lemmas 1 and 2, we have the following Corollary:

Corollary 2

Suppose Conditions $1^{'}$, $2^{'}$ and $4$, then $c_{3}<\gamma _{p}(\widetilde{\varvec{S}}_{n})\le \gamma _{1}(\widetilde{\varvec{S}}_{n})<c_{4}$ holds with probability tending to one as $n\rightarrow \infty$.

In order to prove $\sup _{1\le i\le n}|\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}|=o_{p}(1)$ for the Taylor expansions of (4) and (5), we need the following lemma. Let $\rho _{n}\equiv \max _{1\le i\le n}\Vert \varvec{Z}_{i}\Vert$.

Lemma 3

Assume Conditions $1^{'}$ and $2^{'}$, then

$$\begin{aligned} \rho _{n}=o_{p}(n^{1/q}p^{1/2}). \end{aligned}$$

Proof

It is easy to see that

$$\begin{aligned} \rho _{n}&= \max _{1\le i\le n}\big \{\Vert \varvec{Z}_{i}\Vert ^{q/2}-\text{ E }\Vert \varvec{Z}_{i}\Vert ^{q/2}+\text{ E }\Vert \varvec{Z}_{i}\Vert ^{q/2}\big \}^{2/q}\\&\le \left\{ \max _{1\le i\le n}\big |\Vert \varvec{Z}_{i}\Vert ^{q/2}-\text{ E }\Vert \varvec{Z}_{i}\Vert ^{q/2}\big |+\text{ E }|\epsilon _{1}|^{q/2}\max _{1\le i\le n}\Vert \varvec{x}_{i}\Vert ^{q/2}\right\} ^{2/q}. \end{aligned}$$

Applying $C_{p}$ inequality twice, we have $\Vert \varvec{x}_{i}\Vert ^{q/2}\le p^{q/4}\big (\sum _{j=1}^{p}p^{-1}|x_{ij}|^{q}\big )^{1/2}\le p^{q/4}K_{1}^{1/2}$. Thus, $\max _{1\le i\le n}\Vert \varvec{x}_{i}\Vert ^{q/2}=O(p^{q/4})$.

On the other hand, according to Condition $2^{'}$ and the Lemma 3 of Owen (1990), which continues to hold for independent but not identically distributed random variables, we get

$$\begin{aligned}&\max _{1\le i\le n}\big |\Vert \varvec{Z}_{i}\Vert ^{q/2}-\text{ E }\Vert \varvec{Z}_{i}\Vert ^{q/2}\big |\\&\quad \le \max _{1\le i\le n}\left\{ \frac{\big |\Vert \varvec{Z}_{i}\Vert ^{q/2}-\text{ E }\Vert \varvec{Z}_{i}\Vert ^{q/2}\big |}{\big [\text{ Var }(\Vert \varvec{Z}_{i}\Vert ^{q/2})\big ]^{1/2}}\right\} \max _{1\le i\le n}\big \{\text{ Var }(\Vert \varvec{Z}_{i}\Vert ^{q/2})\big \}^{1/2}\\&\quad \le \max _{1\le i\le n}\left\{ \frac{\big |\Vert \varvec{Z}_{i}\Vert ^{q/2}-\text{ E }\Vert \varvec{Z}_{i}\Vert ^{q/2}\big |}{\big [\text{ Var }(\Vert \varvec{Z}_{i}\Vert ^{q/2})\big ]^{1/2}}\right\} \max _{1\le i\le n}\big \{\Vert \varvec{x}_{i}\Vert ^{q/2}\big \}(\text{ E }|\epsilon _{1}|^{q})^{1/2}\\&\quad =o_{p}(n^{1/2})O(p^{q/4})=o_{p}(n^{1/2}p^{q/4}). \end{aligned}$$

The last second equality is due to Lemma 3 of Owen (1990) and Condition $2^{'}$.

Summarizing the above, we obtain $\rho _{n}=\big \{o_{p}(n^{1/2}p^{q/4})+O(p^{q/4})\big \}^{2/q}=o_{p}(n^{1/q}p^{1/2})$. $\square$

Lemma 4

If Conditions $1^{'}$, $2^{'}$ and $3^{'}$ hold, then

$$\begin{aligned} \sup _{1\le i\le n}|\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}|=o_{p}(1),\quad \lambda =\widetilde{\varvec{S}}_{n}^{-1}(\bar{\varvec{Z}}_{n}+\varvec{\zeta }_{n}), \end{aligned}$$

where $\bar{\varvec{Z}}_{n}=O_{p}(p^{1/2}n^{-1/2})$ and $\varvec{\zeta }_{n}=o_{p}(n^{\frac{1-q}{q}}p^{3/2})$.

Proof

Let $\varvec{\lambda }=\Vert \varvec{\lambda }\Vert \varvec{\vartheta }$, where $\varvec{\vartheta }$ is a unit vector. Then we get from (5) that

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\frac{\varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}}{1+\Vert \varvec{\lambda }\Vert \varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}}=0. \end{aligned}$$

It follows that

$$\begin{aligned} 0=\frac{1}{n}\sum _{i=1}^{n}\frac{\varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}}{1+\Vert \varvec{\lambda }\Vert \varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}} =\frac{1}{n}\sum _{i=1}^{n}\varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}-\frac{\Vert \varvec{\lambda }\Vert \frac{1}{n}\sum _{i=1}^{n}\varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}\varvec{Z}_{i}^\mathsf{T}\varvec{\vartheta }}{1+\Vert \varvec{\lambda }\Vert \varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}}. \end{aligned}$$

Since

$$\begin{aligned} 0<1+\Vert \varvec{\lambda }\Vert \varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}\le 1+\Vert \varvec{\lambda }\Vert \max _{1\le i\le n}\Vert \varvec{Z}_{i}\Vert =1+\Vert \varvec{\lambda }\Vert \rho _{n}, \end{aligned}$$

hence

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}-\frac{\Vert \varvec{\lambda }\Vert \frac{1}{n}\sum _{i=1}^{n}\varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}\varvec{Z}_{i}^\mathsf{T}\varvec{\vartheta }}{1+\Vert \varvec{\lambda }\Vert \varvec{\vartheta }^\mathsf{T}\varvec{Z}_{i}} \le \varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}-\frac{\Vert \varvec{\lambda }\Vert }{1+\Vert \varvec{\lambda }\Vert \rho _{n}}\varvec{\vartheta }^\mathsf{T}\widetilde{\varvec{S}}_{n}\varvec{\vartheta }, \end{aligned}$$

which leads to $\varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}(1+\Vert \varvec{\lambda }\Vert \rho _{n})\ge \varvec{\vartheta }^\mathsf{T}\widetilde{\varvec{S}}_{n}\varvec{\vartheta }\Vert \varvec{\lambda }\Vert$ and then $\Vert \varvec{\lambda }\Vert (\varvec{\vartheta }^\mathsf{T}\widetilde{\varvec{S}}_{n}\varvec{\vartheta }-\varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}\rho _{n})\le \varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}$ follows.

Furthermore, we note that

$$\begin{aligned} \text{ E }(\bar{\varvec{Z}}_{n}^\mathsf{T}\bar{\varvec{Z}}_{n})&= \frac{1}{n^{2}}\sum _{i,j=1}^{n}\varvec{x}_{i}^\mathsf{T}\varvec{x}_{j}\text{ E }(\epsilon _{i}\epsilon _{j})\\&= \frac{\sigma ^{2}}{n}\text{ tr }\left( \frac{1}{n}\varvec{X}^\mathsf{T}\varvec{X}\right) =\frac{\sigma ^{2}}{n}\text{ tr }\Big (\frac{1}{n}\varvec{XX}^\mathsf{T}\Big )\\&\le \frac{p}{n^{2}}\sum _{i=1}^{n}\gamma _{1}(\varvec{x}_{i}\varvec{x}_{i}^\mathsf{T}\sigma ^{2}) <\frac{p}{n}c_{4}\\&= O\Big (\frac{p}{n}\Big ), \end{aligned}$$

which implies that $\bar{\varvec{Z}}_{n}=O_{p}(p^{1/2}n^{-1/2})$.

Since $\Vert \varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}\Vert \le \Vert \bar{\varvec{Z}}_{n}\Vert$, hence $\varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}=O_{p}(p^{1/2}n^{-1/2})$ and $\varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}\rho _{n}=o_{p}(1)$. Notice that $\varvec{\vartheta }^\mathsf{T}\widetilde{\varvec{S}}_{n}\varvec{\vartheta }\ge \gamma _{p}(\widetilde{\varvec{S}}_{n})>c_{3}$ holds with probability tending to one as $n\rightarrow \infty$. Therefore, we have $\Vert \varvec{\lambda }\Vert =O_{p}(|\varvec{\vartheta }^\mathsf{T}\bar{\varvec{Z}}_{n}|/c_{3})=O_{p}(\Vert \bar{\varvec{Z}}_{n}\Vert )=O_{p}(p^{1/2}n^{-1/2})$ and $\sup _{1\le i\le n}|\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}|=O_{p}(pn^{\frac{2-q}{2q}})=o_{p}(1)$. This enables us to expand (5) to get $\bar{\varvec{Z}}_{n}-\widetilde{\varvec{S}}_{n}\varvec{\lambda }+\varvec{\zeta }_{n}=\varvec{0}$ with $\varvec{\zeta }_{n}=\frac{1}{n}\sum _{i=1}^{n}\varvec{Z}_{i}(\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i})^{2}(1+o_{p}(1))$. Then we derive $\varvec{\lambda }=\widetilde{\varvec{S}}_{n}^{-1}(\bar{\varvec{Z}}_{n}+\varvec{\zeta }_{n})$. In addition, it is easy to see that

$$\begin{aligned} \Vert \varvec{\zeta }_{n}\Vert&\le \frac{2}{n}\sum _{i=1}^{n}\Vert \varvec{Z}_{i}\Vert |\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}|^{2} \le \frac{2}{n}\sum _{i=1}^{n}\rho _{n}\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}\varvec{Z}_{i}^\mathsf{T}\varvec{\lambda } \le 2\rho _{n}\Vert \varvec{\lambda }\Vert ^{2}\gamma _{1}(\widetilde{\varvec{S}}_{n})\\&= \rho _{n}\Vert \varvec{\lambda }\Vert ^{2}O_{p}(\gamma _{1}(\varvec{V}_{n})) =O_{p}(\rho _{n}\Vert \varvec{\lambda }\Vert ^{2}) =O_{p}(o_{p}(n^{1/q}p^{1/2})O_{p}(p/n))\\&= o_{p}(n^{\frac{1-q}{q}}p^{3/2}). \end{aligned}$$

Furthermore, we get from the last second equality that

$$\begin{aligned} O_{p}(o_{p}(n^{1/q}p^{1/2})O_{p}(p/n))=O_{p}(o_{p}(n^{1/q}p^{1/2})\bar{\varvec{Z}}_{n}p^{1/2}n^{-1/2})=\bar{\varvec{Z}}_{n}o_{p}(1). \end{aligned}$$

It follows that $\varvec{\zeta }_{n}=\bar{\varvec{Z}}_{n}o_{p}(1)$. $\square$

Lemma 5

Suppose Conditions $1^{'}$, $2^{'}$ and 4, then

$$\begin{aligned} (\widetilde{\varvec{S}}_{n}-\varvec{V}_{n}^{-1})\bar{\varvec{Z}}_{n}=(\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n})o_{p}(1). \end{aligned}$$

Proof

Note firstly that $\gamma _{1}(\widetilde{\varvec{S}}_{n}^{-1})<1/c_{3}$ with probability tending to one as $n\rightarrow \infty$. Applying Lemma 2 we have

$$\begin{aligned} \Vert \varvec{V}_{n}(\widetilde{\varvec{S}}_{n}^{-1}-\varvec{V}_{n}^{-1})\bar{\varvec{Z}}_{n}\Vert&= \Vert (\varvec{V}_{n}-\widetilde{\varvec{S}}_{n})\widetilde{\varvec{S}}_{n}^{-1}\bar{\varvec{Z}}_{n}\Vert =\left[ \bar{\varvec{Z}}_{n}\widetilde{\varvec{S}}_{n}^{-1}(\varvec{V}_{n}-\widetilde{\varvec{S}}_{n})^{2}\widetilde{\varvec{S}}_{n}^{-1}\bar{\varvec{Z}}_{n}\right] ^{1/2}\\&\le \Vert \bar{\varvec{Z}}_{n}\Vert \gamma _{1}(\widetilde{\varvec{S}}_{n}^{-1})\max _{1\le i\le n}|\gamma _{i}(\varvec{V}_{n}-\widetilde{\varvec{S}}_{n})| \le \Vert \bar{\varvec{Z}}_{n}\Vert \gamma _{1}(\widetilde{\varvec{S}}_{n}^{-1})pL_{n}\\&= \Vert \bar{\varvec{Z}}_{n}\Vert O_{p}(pL_{n}) =\Vert \bar{\varvec{Z}}_{n}\Vert o_{p}(1), \end{aligned}$$

which implies that $(\widetilde{\varvec{S}}_{n}^{-1}-\varvec{V}_{n}^{-1})\bar{\varvec{Z}}_{n}=(\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n})o_{p}(1)$. $\square$

The following result is a direct consequence of Lemmas 4 and 5.

Corollary 3

Under Conditions $1^{'}$, $2^{'}$, $3^{'}$ and 4, we have

$$\begin{aligned} \varvec{\lambda }=\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n}(1+o_{p}(1)). \end{aligned}$$

1.1 Proof of Proposition 1

Review that we have shown previously $\sup _{1\le i\le n}\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}=o_{p}(1)$, hence we can expand (4) as follows:

$$\begin{aligned} l_{n}(\varvec{\beta }_{0})&= 2\sum _{i=1}^{n}\left\{ \varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}-\frac{1}{2}\varvec{\lambda }^\mathsf{T}\varvec{Z}_{i}\varvec{Z}_{i}^\mathsf{T}\varvec{\lambda }+\frac{1}{3}(\varvec{Z}_{i}^\mathsf{T}\varvec{\lambda })^{3}(1+o_{p}(1))\right\} \\&= n\left\{ 2\bar{\varvec{Z}}_{n}^\mathsf{T}\varvec{\lambda }-\varvec{\lambda }^\mathsf{T}\widetilde{\varvec{S}}_{n}\varvec{\lambda }\right\} +\varGamma _{1}, \end{aligned}$$

where $\varGamma _{1}=\frac{2}{3}\sum _{i=1}^{n}(\varvec{Z}_{i}^\mathsf{T}\varvec{\lambda })^{3}(1+o_{p}(1))$. By substituting $\varvec{\lambda }$ for $\widetilde{\varvec{S}}_{n}^{-1}(\bar{\varvec{Z}}_{n}+\varvec{\zeta }_{n})$ in the expression above, we get

$$\begin{aligned} l_{n}(\varvec{\beta }_{0})=n\bar{\varvec{Z}}_{n}^\mathsf{T}\widetilde{\varvec{S}}_{n}^{-1}\bar{\varvec{Z}}_{n}+\varGamma _{1}-\varGamma _{2}, \end{aligned}$$

where $\varGamma _{2}=n\varvec{\zeta }_{n}^\mathsf{T}\widetilde{\varvec{S}}_{n}^{-1}\varvec{\zeta }_{n}$. It is clear that the result of Proposition 1 is followed if we can prove that $\varGamma _{1}=o_{p}(\sqrt{p})$ and $\varGamma _{2}=o_{p}(\sqrt{p})$.

Indeed, according to Corollary 2 we have

$$\begin{aligned} \varvec{Z}_{i}^\mathsf{T}\varvec{\lambda }=\varvec{Z}_{i}^\mathsf{T}\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n}(1+o_{p}(1)). \end{aligned}$$

This, together with the proof of Lemma 4, leads to $\varGamma _{1}=\frac{2}{3}\sum _{i=1}^{n}(\varvec{Z}_{i}^\mathsf{T}\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n})^{3}(1+o_{p}(1))$ and $\varvec{\zeta }_{n}=\frac{1}{n}\sum _{i=1}^{n}\varvec{Z}_{i}(\varvec{Z}_{i}^\mathsf{T}\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n})^{2}(1+o_{p}(1))$. On the other hand, we note that $\varGamma _{2}=n\varvec{\zeta }_{n}^\mathsf{T}(\widetilde{\varvec{S}}_{n}^{-1}-\varvec{V}_{n}^{-1})\varvec{\zeta }_{n}+n\varvec{\zeta }_{n}^\mathsf{T}\varvec{V}_{n}^{-1}\varvec{\zeta }_{n}$. Similar to Lemma 5, we can prove that $(\widetilde{\varvec{S}}_{n}^{-1}-\varvec{V}_{n}^{-1})\varvec{\zeta }_{n}=\varvec{V}_{n}^{-1}\varvec{\zeta }_{n}o_{p}(1)$. Therefore we get $\varGamma _{2}=n\varvec{\zeta }_{n}^\mathsf{T}\varvec{V}_{n}^{-1}\varvec{\zeta }_{n}(1+o_{p}(1))$.

It is not difficult to see that we can rewrite $\varGamma _{1}$ and $\varGamma _{2}$ as

$$\begin{aligned} \varGamma _{1}&= \frac{2n}{3}\sum _{i,j,k=1}^{p}\bar{\alpha }^{ijk}\bar{A}^{i}\bar{A}^{j}\bar{A}^{k}(1+o_{p}(1)),\\ \varGamma _{2}&= n\sum _{i,j,k,r,s=1}^{p}\bar{\alpha }^{ijk}\bar{\alpha }^{irs}\bar{A}^{j}\bar{A}^{k}\bar{A}^{r}\bar{A}^{s}(1+o_{p}(1)). \end{aligned}$$

With tedious calculation we further obtain

$$\begin{aligned} \text{ E }\bigg (\frac{2n}{3}\sum _{i,j,k=1}^{p}\bar{\alpha }^{ijk}\bar{A}^{i}\bar{A}^{j}\bar{A}^{k}\bigg )&= \frac{4}{3n}\sum _{i,j,k=1}^{p}(3\bar{\alpha }^{ijj}\bar{\alpha }^{ikk}+2\bar{\alpha }^{ijk}\bar{\alpha }^{ijk})(1+o(1)),\\ \text{ E }\bigg (n\sum _{i,j,k,r,s=1}^{p}\bar{\alpha }^{ijk}\bar{\alpha }^{irs}\bar{A}^{j}\bar{A}^{k}\bar{A}^{r}\bar{A}^{s}\bigg )&= \frac{1}{n}\sum _{i,j,k=1}^{p}(2\bar{\alpha }^{ijk}\bar{\alpha }^{ijk}+\bar{\alpha }^{ijj}\bar{\alpha }^{ikk})(1+o(1)). \end{aligned}$$

Recalling that $(\varvec{A})_{ij}$ denotes the $(i,j)$ element of a matrix $\varvec{A}$ and $(\varvec{a})_{i}$ the $i$th component of a vector $\varvec{a}$. Since

$$\begin{aligned} \text{ E }\big |\xi _{rk}|^{2} =\text{ E }|(\varvec{V}_{n}^{-1/2}\varvec{Z}_{r})_{k}\big |^{2} =\big (\varvec{V}_{n}^{-1/2}\varvec{x}_{r}\varvec{x}_{r}^\mathsf{T}\varvec{V}_{n}^{-1/2}\big )_{kk}\sigma ^{2}. \end{aligned}$$

Hence, Using $C_{p}$ inequality, Cauchy-Schwartz inequality, Young inequality and Lemma 4 of Liu et al. (2013), we can verify that

$$\begin{aligned} \bigg |\sum _{i,j,k=1}^{p}\bar{\alpha }^{ijk}\bar{\alpha }^{ijk}\bigg |\le \frac{c_{4}}{c_{3}}p\sum _{i,j=1}^{p}\bar{\alpha }^{iijj}\quad \text{ and }\quad \bigg |\sum _{i,j,k=1}^{p}\bar{\alpha }^{ijj}\bar{\alpha }^{ikk}\bigg |\le \frac{c_{4}}{c_{3}}p\sum _{i,j=1}^{p}\bar{\alpha }^{iijj}. \end{aligned}$$

Summarizing the arguments above, together with Condition $6^{'}$, we get

$$\begin{aligned} \text{ E }\bigg (\frac{2n}{3}\sum _{i,j,k=1}^{p}\bar{\alpha }^{ijk}\bar{A}^{i}\bar{A}^{j}\bar{A}^{k}\bigg )^{2} \le \frac{c_{4}}{c_{3}}\frac{20p}{3n}\sum _{i,j}^{p}\bar{\alpha }^{iijj}(1+o(1)) =O(p^{3}/n), \end{aligned}$$

and

$$\begin{aligned} \text{ E }\bigg (n\sum _{i,j,k,r,s=1}^{p}\bar{\alpha }^{ijk}\bar{\alpha }^{irs}\bar{A}^{j}\bar{A}^{k}\bar{A}^{r}\bar{A}^{s}\bigg ) \le \frac{3p}{n}\frac{c_{4}}{c_{3}}\bar{\alpha }^{iijj}(1+o(1)) =O(p^{3}/n), \end{aligned}$$

which means that $\varGamma _{1}=O_{p}(p^{3/2}n^{-1/2})=o_{p}(\sqrt{p})$ and $\Gamma _{2}=O_{p}(p^{3}/n)$ by Markov’s inequality. Further, $p=o(n^{2/5})$ is sufficient for $\varGamma _{2}=o_{p}(\sqrt{p})$, and under the Condition $7^{'}$, the order of $p$ can achieve $p=o(n^{1/2})$. $\square$

The following Lemma is needed for the proof of Proposition 2.

Lemma 6

Let $\varvec{G}_{n}=\mathbf I _{p}-\varvec{N}_{n}$ with $\varvec{N}_{n}=\varvec{V}_{n}^{-1/2}\widetilde{\varvec{S}}_{n}\varvec{V}_{n}^{-1/2}$. Suppose Condition $6^{'}$, then

$$\begin{aligned} \text{ tr }(\varvec{G}_{n}^{2})=O_{p}(p^{2}/n). \end{aligned}$$

Proof

Note that

$$\begin{aligned} \varvec{G}_{n}^{2} =\mathbf I _{p}-\frac{2}{n}\sum _{i=1}^{n}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}+\frac{1}{n^{2}}\sum _{i,j=1}^{n}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}\varvec{\xi }_{j}\varvec{\xi }_{j}^\mathsf{T}. \end{aligned}$$

Thus,

$$\begin{aligned} \text{ E }\big [\text{ tr }(\varvec{G}_{n}^{2})\big ]&= \text{ E }\bigg [\,p-\frac{2}{n}\sum _{r=1}^{p}\sum _{i=1}^{n}\xi _{ir}^{2}+\frac{1}{n^{2}}\sum _{i,j=1}^{n}\sum _{r,s=1}^{p}\xi _{ir}\xi _{is}\xi _{jr}\xi _{js}\bigg ]\\&= p-2\sum _{r=1}^{p}\bar{\alpha }^{rr}+\frac{1}{n^{2}}\sum _{r,s=1}^{p}\Big (\sum _{i=j}+\sum _{i\ne j}\Big )\text{ E }(\xi _{ir}\xi _{is}\xi _{jr}\xi _{js})\\&\le -p+\frac{1}{n}\sum _{r,s=1}^{p}\bar{\alpha }^{rrss}+\frac{1}{n^{2}}\sum _{r,s=1}^{p}\sum _{i,j=1}^{n}\text{ E }(\xi _{ir}\xi _{is})\text{ E }(\xi _{jr}\xi _{js})\\&= \frac{1}{n}\sum _{r,s=1}^{p}\bar{\alpha }^{rrss} =O(p^{2}/n). \end{aligned}$$

This implies that $\text{ tr }(\varvec{G}_{n}^{2})=O_{p}(p^{2}/n)$. $\square$

1.2 Proof of Proposition 2

Define $e_{n}=n\bar{\varvec{Z}}_{n}^\mathsf{T}(\varvec{V}_{n}^{-1}-\widetilde{\varvec{S}}_{n}^{-1})\bar{\varvec{Z}}_{n}$. It is easy to see that

$$\begin{aligned} e_{n}=n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\bar{\varvec{\xi }}_{n}-n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{V}_{n}^{1/2}\widetilde{\varvec{S}}_{n}^{-1}\varvec{V}_{n}^{1/2}\bar{\varvec{\xi }}_{n}=n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}(\mathbf I _{p}-\varvec{N}_{n}^{-1})\bar{\varvec{\xi }}_{n}. \end{aligned}$$

Note that $\mathbf I _{p}-\varvec{N}_{n}^{-1}=-\varvec{G}_{n}-\varvec{G}_{n}^{2}-\cdots -\varvec{G}_{n}^{k}+\varvec{G}_{n}^{k}(\mathbf I _{p}-\varvec{N}_{n}^{-1})$. Hence,

$$\begin{aligned} e_{n}=n[-\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}\bar{\varvec{\xi }}_{n}-\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{2}\bar{\varvec{\xi }}_{n}-\cdots -\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\bar{\varvec{\xi }}_{n}] +n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}(\mathbf I _{p}-\varvec{N}_{n}^{-1})\bar{\varvec{\xi }}_{n}.\qquad \quad \end{aligned}$$

(13)

Now we consider the convergence of the right-hand side of (13). First, we focus on the first term. By Lemmas 4 and 5, we have

$$\begin{aligned} \Vert \bar{\varvec{\xi }}_{n}\Vert ^{2} \le \frac{1}{c_{3}}\Vert \bar{\varvec{Z}}_{n}\Vert ^{2}=O_{p}(p/n), \end{aligned}$$

and $|\gamma _{i}(\varvec{G}_{n})| \le [\text{ tr }(\varvec{G}_{n}^{2})]^{1/2} =O_{p}(p/\sqrt{n})$. Therefore,

$$\begin{aligned} |\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\bar{\varvec{\xi }}_{n}|\le \Vert \bar{\varvec{\xi }}_{n}\Vert ^{2}\max _{1\le i\le p}|\gamma _{i}(\varvec{G}_{n}^{k})| \le \Vert \bar{\varvec{\xi }}_{n}\Vert ^{2}[\text{ tr }(\varvec{G}_{n}^{2})]^{k/2}=O_{p}(p^{k+1}/n^{k/2+1}), \end{aligned}$$

which implies that the series $n\sum _{k=1}^{\infty }(-\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\bar{\varvec{\xi }}_{n})$ is convergent, as long as $n$ is fixed and $p=o(n^{1/2})$. On the other hand, with lengthy algebra we have

$$\begin{aligned} \text{ E }\left| n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}\bar{\varvec{\xi }}_{n}\right| ^{2}=n^{2}\text{ E }\left| \bar{\varvec{Z}}_{n}^{\,\mathsf T}(\varvec{V}_{n}-\widetilde{\varvec{S}}_{n})\bar{\varvec{Z}}_{n}\right| ^{2}=O(p^{3}/n), \end{aligned}$$

which means that $n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}\bar{\varvec{\xi }}_{n}=O_{p}(p^{3/2}/n^{1/2})=o_{p}(\sqrt{p})$. Therefore we conclude from above that $n\sum _{k=1}^{\infty }(-\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\bar{\varvec{\xi }}_{n})=o_{p}(\sqrt{p})$.

For the second term in (13), we will next show in the following arguments that the remainder term $n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}(\mathbf I _{p}-\varvec{N}_{n}^{-1})\bar{\varvec{\xi }}_{n}$ is negligible as $k\rightarrow \infty$. For this, we firstly note that

$$\begin{aligned} \big |n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}(\mathbf I _{p}-\varvec{N}_{n}^{-1})\bar{\varvec{\xi }}_{n}\big | \le \big |n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\bar{\varvec{\xi }}_{n}\big |+\big |n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\varvec{N}_{n}^{-1}\bar{\varvec{\xi }}_{n}\big | \end{aligned}$$

and

$$\begin{aligned} \big |n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\bar{\varvec{\xi }}_{n}\big |=nO_{p}(p^{k+1}/n^{k/2+1})=O_{p}(p^{k+1}/n^{k/2}). \end{aligned}$$

Furthermore, applying the Lemma 4 of Liu et al. (2013), which remains valid for the case of any $n\times n$ symmetric matrix $\varvec{A}=(a_{ij})$, that is, $\mathcal {M}(\varvec{A})\le \max _{1\le i\le n}|\gamma _{i}(\varvec{A})|$, we have

$$\begin{aligned} \big |n\bar{\varvec{\xi }}_{n}^{\,\mathsf T}\varvec{G}_{n}^{k}\varvec{N}_{n}^{-1}\bar{\varvec{\xi }}_{n}\big |&\le n\Vert \bar{\varvec{\xi }}_{n}\Vert ^{2}\max _{1\le i\le n}\big \{|\gamma _{i}(\varvec{G}_{n}^{k}\varvec{N}_{n}^{-1})|\big \} \le np\Vert \bar{\varvec{\xi }}_{n}\Vert ^{2}\mathcal {M}(\varvec{G}_{n}^{k}\varvec{N}_{n}^{-1})\\&\le np^{2}\Vert \bar{\varvec{\xi }}_{n}\Vert ^{2}\max _{1\le i\le p}|\gamma _{i}(\varvec{G}_{n}^{k})|\gamma _{1}(\varvec{N}_{n}^{-1})\\&\le np^{2}\Vert \bar{\varvec{\xi }}_{n}\Vert ^{2}\big [\text{ tr }(\varvec{G}_{n}^{2})\big ]^{k/2}\gamma _{1}(\varvec{V}_{n})\gamma _{1}(\widetilde{\varvec{S}}_{n})\\&= O_{p}(p^{k+3}/n^{k+2}) =o_{p}(1). \end{aligned}$$

Under Condition 4, the last equality holds as $k\rightarrow \infty$. Therefore, the consequence of Proposition 2 follows. $\square$

We give the following Lemma for the proof of Proposition 3.

Lemma 7

Under model (7), suppose that $\text{ E }(\xi _{ij_{1}}^{\alpha _{1}}\xi _{ij_{2}}^{\alpha _{2}}\ldots \xi _{ij_{l}}^{\alpha _{l}})\le B$ for some finite positive constant $B<\infty$ and any $1\le i\le n$, whenever $\sum _{i=1}^{l}\alpha _{i}\le 6$, then for some finite positive constant $c$,

$$\begin{aligned} \text{ E }\left( \varvec{\xi }_{n}^\mathsf{T}\sum _{i=1}^{n-1}\varvec{\xi }_{i}\right) ^{6}\le cp^{6}(n^{3}+n^{2}+n). \end{aligned}$$

Proof

We state that, here and in the sequel, $c$ stands for constant which may be different from line to line and even from formula to formula and whose value is not of interest.

Indeed, a direct calculation gives that

$$\begin{aligned} \text{ E }\bigg (\varvec{\xi }_{n}^\mathsf{T}\sum _{i=1}^{n-1}\varvec{\xi }_{i}\bigg )^{6}&= \text{ E }\bigg [\bigg (\varvec{\xi }_{n}^\mathsf{T}\sum _{i_{1}=1}^{n-1}\varvec{\xi }_{i_{1}}\bigg )\cdots \bigg (\varvec{\xi }_{n}^\mathsf{T}\sum _{i_{6}=1}^{n-1}\varvec{\xi }_{i_{6}}\bigg )\bigg ]\\&= \sum _{i_{1},\ldots ,i_{6}=1}^{n-1}\sum _{j_{1},\ldots ,j_{6}=1}^{p}\text{ E }(\xi _{nj_{1}}\ldots \xi _{nj_{6}})\text{ E }(\xi _{i_{1}j_{1}}\ldots \xi _{i_{6}j_{6}})\\&\le B\big [5(n-1)(n-2)p^{6}B^{2}+10(n-1)(n-2)p^{6}B^{2}\\&+\,\,45(n-1)(n-2)(n-3)p^{6}B^{2}+(n-1)p^{6}B\big ]\\&\le cp^{6}(n^{3}+n^{2}+n) \end{aligned}$$

$\square$

1.3 Proof of Proposition 3

Let $\varvec{J}_{n}=\sum _{i=1}^{n}\varvec{\xi }_{i}$, it is clear that

$$\begin{aligned} \frac{n\bar{\varvec{Z}}_{n}^\mathsf{T}\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n}-p}{w_{n}/n}=\frac{n\Vert \bar{\varvec{\xi }}_{n}\Vert ^{2}-p}{w_{n}/n}=\frac{\Vert \varvec{J}_{n}\Vert ^{2}-np}{w_{n}} \end{aligned}$$

To prove Proposition 3, the strategy is to construct a martingale and then to apply the martingale central limit theorem based on it. Hence, it is technically convenient to define $M_{n}=\Vert \varvec{J}_{n}\Vert ^{2}-np$, $n\ge 1$, $M_{0}=0$, and $H_{n}=M_{n}-M_{n-1},n\ge 1$. It is easy to see that

$$\begin{aligned} M_{n}=\Vert \varvec{J}_{n-1}\Vert ^{2}-(n-1)p+2\varvec{\xi }_{n}^\mathsf{T}\varvec{J}_{n-1}+\Vert \varvec{\xi }_{n}\Vert ^{2}-p, \end{aligned}$$

and then

$$\begin{aligned} H_{n}=2\varvec{\xi }_{n}^\mathsf{T}\varvec{J}_{n-1}+\Vert \varvec{\xi }_{n}\Vert ^{2}-p,\quad \text{ E }H_{n}=\text{ E }\Vert \varvec{\xi }_{n}\Vert ^{2}-p. \end{aligned}$$

In addition, denote by $\varvec{\mathcal {F}}_{i}=\sigma (\varvec{\xi }_{1},\ldots ,\varvec{\xi }_{i})=\sigma (\varvec{J}_{1},\ldots ,\varvec{J}_{i})$, for $i=1, 2, \ldots , n$, the $\sigma$-fileds generated by $\varvec{\xi }_{1},\ldots ,\varvec{\xi }_{i}$. It can be seen that $\{M_{n}, \varvec{\mathcal {F}}_{n}, n\ge 1\}$ is not a martingale, although it is true in the situation of Portnoy (1988). In order to construct a martingale based on $M_{n}$ for martingale limit theorem, we define some other notation.

Let $\phi _{n}=H_{n}-\text{ E }H_{n}$, $n\ge 1$, and $W_{n}=\sum _{i=1}^{n}\phi _{i}=M_{n}-\sum _{i=1}^{n}\text{ E }H_{i}$. It is easy to show that, for each $s<t$,

$$\begin{aligned} \text{ E }[W_{t}|\varvec{\mathcal {F}}_{s}] =\Vert \varvec{J}_{s}\Vert ^{2}-s\,p =W_{s}, \end{aligned}$$

which indicates that $\{W_{n},\varvec{\mathcal {F}}_{n},n\ge 1\}$ is a martingale.

Recalling the definitions of $\sigma _{i}$’s and $w_{n}$, it is easy to see that $\sigma _{i}^{2}=\text{ E }\phi _{i}^{2}$ and $w_{n}^{2}=\sum _{i=1}^{n}\sigma _{i}^{2}$. To apply the martingale central limit theorem of Chow and Teicher (1997, Theorem 1, $\text{ P }_{336}$), it suffices to show that if $p^{3}/n\rightarrow 0$, then both

$$\begin{aligned} \sum _{i=1}^{n}\text{ E }|\phi _{i}|^{3}/w_{n}^{3}\rightarrow 0 \end{aligned}$$

(14)

and

$$\begin{aligned} \sum _{i=1}^{n}\text{ E }\big |\text{ E }(\phi _{i}^{2}|\varvec{\mathcal {F}}_{i-1})-\sigma _{i}^{2}\big |/w_{n}^{2}\rightarrow 0 \end{aligned}$$

(15)

hold.

Hence, in the following arguments, our task is to prove (14) and (15). For this, we calculate firstly $\sigma _{i}$’s, for $i\ge 1$. It is easy to see that, for $i\ge 1$,

$$\begin{aligned} \sigma _{i}^{2} =4\text{ E }[\varvec{J}_{i-1}^\mathsf{T}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1}]+\text{ E }(\Vert \varvec{\xi }_{i}\Vert ^{2}-p)^{2}-(\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2}-p)^{2}. \end{aligned}$$

For the first term of the expression above, we can verify that

$$\begin{aligned} \text{ E }[\varvec{J}_{i-1}^\mathsf{T}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1}]&= \text{ E }\big [\varvec{J}_{i-1}^\mathsf{T}\text{ E }(\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T})\varvec{J}_{i-1}\big ] =\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1}\varvec{V}^{(i)}\varvec{V}_{n}^{-1}\varvec{V}^{(k)})\\&\ge \sum _{k=1}^{i-1}p\frac{\gamma _{p}(\varvec{V}^{(i)})\gamma _{p}(\varvec{V}^{(k)})}{\gamma _{1}^{2}(\varvec{V}_{n})} \ge p(i-1)\left( \frac{c_{3}}{c_{4}}\right) ^{2}. \end{aligned}$$

Recall that $\text{ E }(\xi _{ij_{1}}^{\alpha _{1}}\xi _{ij_{2}}^{\alpha _{2}}\ldots \xi _{ij_{l}}^{\alpha _{l}})\le B$ for any $1\le i\le n$ and all $j_{1}, j_{2},\ldots , j_{l}=1, \ldots , p$, whenever $\sum _{i=1}^{k}\alpha _{i}\le 6$. Therefore we have, for $k\le 3$,

$$\begin{aligned} \text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2k}=\text{ E }\left[ \left( \sum _{j=1}^{p}\xi _{ij}^{2}\right) ^{k}\right] \le p^{k-1}\sum _{j=1}^{p}\text{ E }\xi _{ij}^{2k}\le p^{k}B. \end{aligned}$$

Then, it follows that $\text{ E }(\Vert \varvec{\xi }_{i}\Vert ^{2}-p)^{2}=O(p^{2})$, and $(\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2}-p)^{2}=O(p^{2})$. These lead to $\sigma _{i}^{2}\ge 4(\frac{c_{3}}{c_{4}})^{2}p(i-1)+O(p^{2})$ and then

$$\begin{aligned} w_{n}^{2} =2\Bigg (\frac{c_{3}}{c_{4}}\Bigg )^{2}p\,n(n-1)+O(np^{2}) \ge 2\Bigg (\frac{c_{3}}{c_{4}}\Bigg )^{2}n^{2}p(1+O(p/n)). \end{aligned}$$

(16)

In order to prove (14), we note that the martingale difference sequence $\phi _{n}=2\varvec{\xi }_{n}^\mathsf{T}\varvec{J}_{n-1}+\Vert \varvec{\xi }_{n}\Vert ^{2}-\text{ E }\Vert \varvec{\xi }_{n}\Vert ^{2}$. Hence, we have

$$\begin{aligned} \text{ E }|\phi _{i}|^{3} \le 32\big [\text{ E }(\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1})^{6}\big ]^{1/2}+4Bp^{3} \le c\big [(\text{ E }|\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1}|^{6})^{1/2}+p^{3}\big ]. \end{aligned}$$

From Lemma 7, we know that $\text{ E }(\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1})^{6}\le cp^{6}(i^{3}+i^{2}+i)$. Hence we obtain that

$$\begin{aligned} \sum _{i=1}^{n}\text{ E }|\phi _{i}|^{3}\le c\sum _{i=1}^{n}p^{3}(i^{3/2}+i+\sqrt{i}+1)\le cp^{3}(n^{5/2}+n^{2}+n^{3/2}+n). \end{aligned}$$

Due to (16), if $\frac{p^{3}}{n}\rightarrow 0$, then we derive

$$\begin{aligned} \frac{\sum _{i=1}^{n}\text{ E }|\phi _{i}|^{3}}{w_{n}^{3}}\le cp^{3/2}(1/\sqrt{n}+1/n+1/n^{3/2}+1/n^{2})\rightarrow 0 \end{aligned}$$

and then (14) follows.

We now focus on the proof of (15). It is easy to see that

$$\begin{aligned} \text{ E }[\phi _{i}^{2}|\varvec{\mathcal {F}}_{i-1}] =4\varvec{J}_{i-1}^\mathsf{T}\text{ E }(\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T})\varvec{J}_{i-1} +4\varvec{J}_{i-1}^\mathsf{T}\text{ E }\big [\varvec{\xi }_{i}(\Vert \varvec{\xi }_{i}\Vert ^{2} -\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2})\big ] +O(p^{2}). \end{aligned}$$

Therefore we have

$$\begin{aligned}&\big |\text{ E }[\phi _{i}^{2}\big |\varvec{\mathcal {F}}_{i-1}]-\sigma _{i}^{2}|^{2}\le 12\big \{\big |\varvec{J}_{i-1}^\mathsf{T}\text{ E }(\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T})\varvec{J}_{i-1} -\text{ E }(\varvec{J}_{i-1}^\mathsf{T}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1})\big |^{2}\\&\quad +\,\,\text{ E }\big [\varvec{\xi }_{i}^\mathsf{T}(\Vert \varvec{\xi }_{i}\Vert ^{2}-\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2})\big ]\varvec{J}_{i-1}\varvec{J}_{i-1}^\mathsf{T}\text{ E }\big [\varvec{\xi }_{i}(\Vert \varvec{\xi }_{i}\Vert ^{2} -\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2})\big ] +O(p^{4})\big \} \end{aligned}$$

and then

$$\begin{aligned}&\big [\text{ E }\big |\text{ E }(\phi _{i}^{2}|\varvec{\mathcal {F}}_{i-1})-\sigma _{i}^{2}\big |^{2}\big ]^{1/2}\nonumber \\&\quad \le \sqrt{12}\big \{\text{ E }\big [\varvec{J}_{i-1}^\mathsf{T}\text{ E }(\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T})\varvec{J}_{i-1} -\text{ E }(\varvec{J}_{i-1}^\mathsf{T}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1})\big ]^{2} +\text{ E }\big [\varvec{\xi }_{i}^\mathsf{T}(\Vert \varvec{\xi }_{i}\Vert ^{2} -\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2})\big ]\nonumber \\&\qquad \times \,\,\text{ E }(\varvec{J}_{i-1}\varvec{J}_{i-1}^\mathsf{T})\text{ E }\big [\varvec{\xi }_{i}(\Vert \varvec{\xi }_{i}\Vert ^{2} -\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2})\big ] +O(p^{4})\big \}^{1/2}. \end{aligned}$$

(17)

For the first term in (17), with tedious calculation we obtain that

$$\begin{aligned}&\text{ E }\big |\varvec{J}_{i-1}^\mathsf{T}\text{ E }(\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T})\varvec{J}_{i-1}-\text{ E }(\varvec{J}_{i-1}^\mathsf{T}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1})\big |^{2}\nonumber \\&\quad =\sum _{k,l,s,t=1}^{i-1}\text{ E }[(\varvec{\xi }_{k}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{l})(\varvec{\xi }_{s}^{t}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{t})]\nonumber \\&\qquad +\left[ \,\,\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1}\varvec{V}^{(k)}\varvec{V}_{n}^{-1}\varvec{V}^{(i)})\right] ^{2} -2\left[ \,\,\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1}\varvec{V}^{(k)}\varvec{V}_{n}^{-1}\varvec{V}^{(i)})\right] \nonumber \\&\qquad \times \,\,\text{ E }\left[ \sum _{k,l=1}^{i-1}\varvec{\xi }_{k}^{t}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{l}\right] \end{aligned}$$

(18)

Recalling that $\varvec{\xi }_{i},1\le i\le n$ are independent with mean zero and noting that $\varvec{\xi }_{i}^\mathsf{T}\varvec{A}\varvec{\xi }_{j}=\varvec{\xi }_{j}^\mathsf{T}\varvec{A}\varvec{\xi }_{i}$ if $\varvec{A}$ is a symmetric matrix, we can further simplify the terms of (18) as follows. That is

$$\begin{aligned}&\sum _{k,l,s,t=1}^{i-1}\text{ E }\big [(\varvec{\xi }_{k}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{l})(\varvec{\xi }_{s}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{t})\big ] \nonumber \\&\quad =\sum _{k\ne l=1}^{i-1}\text{ E }\big [\varvec{\xi }_{k}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{k}\big ]\text{ E }\big [\varvec{\xi }_{l}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{l}\big ]\nonumber \\&\qquad +\,\,2\sum _{k\ne l=1}^{i-1}\text{ E }\big [\varvec{\xi }_{k}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{k}\varvec{\xi }_{k}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{l}\big ] \nonumber \\&\qquad +\,\,\sum _{j=1}^{i-1}\text{ E }\big [\varvec{\xi }_{j}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{j}\big ]^{2} \nonumber \\&\quad \le \left[ \,\,\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1/2}\varvec{V}^{(k)}\varvec{V}_{n}^{-1/2}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2})\right] ^{2}\nonumber \\&\qquad +\,\,2\sum _{k\ne l=1}^{i-1}\text{ E }\big [\varvec{\xi }_{l}^\mathsf{T}(\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2})\text{ E }(\varvec{\xi }_{k}\varvec{\xi }_{k}^\mathsf{T})(\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2})\varvec{\xi }_{l}\big ]\nonumber \\&\qquad +\,\,\sum _{j=1}^{i-1}\sum _{k,l,s,t=1}^{p}\big (\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\big )_{kl}\big (\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\big )_{st} \text{ E }(\xi _{jk}\xi _{jl}\xi _{js}\xi _{jt})\nonumber \\&\quad \le \left[ \,\,\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1}\varvec{V}^{(k)}\varvec{V}_{n}^{-1}\varvec{V}^{(i)})\right] ^{2} +2(i-1)(i-2)p\gamma _{1}^{4}(\varvec{V}_{n}^{-1})\nonumber \\&\qquad \times \,\, \gamma _{1}(\varvec{V}^{(l)})\gamma _{1}(\varvec{V}^{(k)})\gamma _{1}(\varvec{V}^{(i)}) + (i-1)p^{4}\gamma _{1}^{4}(\varvec{V}_{n}^{-1/2})\gamma _{1}^{2}(\varvec{V}^{(i)})B \nonumber \\&\quad \le \left[ \,\,\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1}\varvec{V}^{(k)}\varvec{V}_{n}^{-1}\varvec{V}^{(i)})\right] ^{2} +2i^{2}p\left( \frac{c_{4}}{c_{3}}\right) ^{4} +ip^{4}B\left( \frac{c_{4}}{c_{3}}\right) ^{2} \end{aligned}$$

(19)

and

$$\begin{aligned}&\left[ \,\,\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1}\varvec{V}^{(k)}\varvec{V}_{n}^{-1}\varvec{V}^{(i)})\right] \left[ \,\,\sum _{k,l=1}^{i-1} \text{ E }(\varvec{\xi }_{k}^\mathsf{T}\varvec{V}_{n}^{-1/2}\varvec{V}^{(i)}\varvec{V}_{n}^{-1/2}\varvec{\xi }_{l})\right] \nonumber \\&\quad =\sum _{k=1}^{i-1}\text{ tr }(\varvec{V}_{n}^{-1}\varvec{V}^{(k)}\varvec{V}_{n}^{-1}\varvec{V}^{(i)}). \end{aligned}$$

(20)

Finally, by (18), (19) and (20), we derive

$$\begin{aligned} \text{ E }[\varvec{J}_{i-1}^\mathsf{T}\text{ E }(\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T})\varvec{J}_{i-1}-\text{ E }(\varvec{J}_{i-1}^\mathsf{T}\varvec{\xi }_{i}\varvec{\xi }_{i}^\mathsf{T}\varvec{J}_{i-1})]^{2} \le 2i^{2}p\left( \frac{c_{4}}{c_{3}}\right) ^{4}+ip^{4}B\left( \frac{c_{4}}{c_{3}}\right) ^{2}\qquad \end{aligned}$$

(21)

For the second term in (17), it is clear that

$$\begin{aligned}&\text{ E }\big [\varvec{\xi }_{i}^\mathsf{T}(\Vert \varvec{\xi }_{i}\Vert ^{2}-\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2})\big ]\text{ E }(\varvec{J}_{i-1}\varvec{J}_{i-1}^\mathsf{T}) \text{ E }\big [\varvec{\xi }_{i}(\Vert \varvec{\xi }_{i}\Vert ^{2}-\text{ E }\Vert \varvec{\xi }_{i}\Vert ^{2})\big ]\nonumber \\&\quad =\text{ E }(\varvec{\xi }_{i}^\mathsf{T}\Vert \varvec{\xi }_{i}\Vert ^{2})\left[ \,\,\sum _{j,k=1}^{i-1}\text{ E }(\varvec{\xi }_{k}\varvec{\xi }_{k}^\mathsf{T})\,\right] \text{ E }(\varvec{\xi }_{i}\Vert \varvec{\xi }_{i}\Vert ^{2})\nonumber \\&\quad =\sum _{k,l=1}^{p}\sum _{s,t=1}^{p}\left( \sum _{j=1}^{i-1}\varvec{V}_{n}^{-1/2}\varvec{V}^{(j)}\varvec{V}_{n}^{-1/2}\right) _{kl} \text{ E }(\xi _{ik}\xi _{is}^{2})\text{ E }(\xi _{il}\xi _{it}^{2})\nonumber \\&\quad \le \sum _{k,l=1}^{p}\sum _{s,t=1}^{p}\mathcal {M}\left( \sum _{j=1}^{i-1}\varvec{V}_{n}^{-1/2}\varvec{V}^{(j)}\varvec{V}_{n}^{-1/2}\right) \text{ E }(\xi _{ik}\xi _{is}^{2})\text{ E }(\xi _{il}\xi _{it}^{2})\nonumber \\&\quad \le p^{4}\gamma _{1}\left( \sum _{j=1}^{i-1}\varvec{V}_{n}^{-1/2}\varvec{V}^{(j)}\varvec{V}_{n}^{-1/2}\right) B^{2}\nonumber \\&\quad \le p^{4}\gamma _{1}^{2}(\varvec{V}_{n}^{-1/2})\left( \sum _{j=1}^{i-1}\gamma _{1}(\varvec{V}^{(j)})\right) B^{2} <ip^{4}B\frac{c_{3}}{c_{4}} \end{aligned}$$

(22)

Summarizing (17), (21), and (22), we get

$$\begin{aligned} \big [\text{ E }\big |\text{ E }(\phi _{i}^{2}|\varvec{\mathcal {F}}_{i-1})-\sigma _{i}^{2}\big |^{2}\big ]^{1/2}\le c\big \{i^{2}p+ip^{4}+O(p^{4})\big \}^{1/2}, \end{aligned}$$

and further we obtain that

$$\begin{aligned}&\sum _{i=1}^{n}\text{ E }\big |\text{ E }(\phi _{i}^{2}|\varvec{\mathcal {F}}_{i-1})-\sigma _{i}^{2}\big |/w_{n}^{2} \le \sum _{i=1}^{n}\big [\text{ E }\big |\text{ E }(\phi _{i}^{2}|\varvec{\mathcal {F}}_{i-1})-\sigma _{i}^{2}\big |^{2}\big ]^{1/2}/w_{n}^{2}\\&\quad \le c\big (n^{2}p^{1/2}+n^{3/2}p^{2}+O(np^{2})\big )/(n^{2}p) =o(1) \end{aligned}$$

The last equality holds if $p^{3}/n\rightarrow 0$. Therefore, (15) follows.

Applying the martingale central limit theorem of Chow and Teicher (1997), we get

$$\begin{aligned} \frac{W_{n}}{w_{n}}\mathop {\rightarrow }\limits ^{\mathcal {L}}N(0,1),\quad \text{ as }\quad n\rightarrow \infty . \end{aligned}$$

In addition, note that $(n\bar{\varvec{Z}}_{n}^\mathsf{T}\varvec{V}_{n}^{-1}\bar{\varvec{Z}}_{n}-p)/(w_{n}/n)=(W_{n}+\sum _{i=1}^{n}\text{ E }H_{i})/(w_{n}/n)$ and $\sum _{i=1}^{n}\text{ E }H_{i}=0$ whenever $n$ is the sample size. Therefore, the consequence of Proposition 3 follows. $\square$