Theoretical properties of bandwidth selectors for kernel density estimation on the circle

Abstract

We derive the asymptotic properties of the least squares cross-validation (LSCV) selector and the direct plug-in rule (DPI) selector in the kernel density estimation for circular data. The DPI selector has a convergence rate of $O(n^{-5/14})$, although the rate of the LSCV selector is $O(n^{-1/10})$. Our simulation shows that the DPI selector has more stability than the LSCV selector for small and large sample sizes. In other words, the DPI selector outperforms the LSCV selector in theoretical and practical performance.

Appendix A

Proof of Theorem 2.

We set $\gamma (y_{ij})=\gamma _{ij}$ to ease the notation. First, we calculate the expectation of $\overline{{{\mathrm {CV}}}}(\kappa )$, given by

$$\begin{aligned} {{E}}_{f}[\overline{{{\mathrm {CV}}}}(\kappa )]&=\frac{R(K_{\kappa })}{n}+\frac{2}{n^{2}}\sum _{i<j}{{E}}_{f}[ \gamma _{ij}]+\frac{2}{n}\sum _{i}{{E}}_{f}[f(\varTheta _{i})]-R(f). \end{aligned}$$

(16)

We set $\gamma _{i}={{E}}_{f}[\gamma _{ij}|\varTheta _{i}]$. Then, the conditional expectation $\gamma _{i}$ is given by

$$\begin{aligned} \gamma _{i}&=-f(\varTheta _{i})+f^{(4)}(\varTheta _{i})\mu ^{-2}_{0} (L)\mu ^{2}_{2}(L)\kappa ^{-2}+O(\kappa ^{-3}). \end{aligned}$$

(17)

“Appendix B” in ESM presents the details. It follows from (17) that

$$\begin{aligned} {{E}}_{f}[\gamma _{ij}]&={{E}}_{f}[\gamma _{i}] =-R(f)+R(f^{(2)})\mu ^{-2}_{0}(L)\mu ^{2}_{2}(L)\kappa ^{-2}+O(\kappa ^{-3}). \end{aligned}$$

(18)

Lemma 1

(Tsuruta and Sagae 2017) The term $R(K(\theta )\theta ^{t})$ is equal to

$$\begin{aligned} R(K(\theta )\theta ^{t}):=\kappa ^{-(2t-1)/2}[d_{2t}(L)+o(1)], \end{aligned}$$

where $d_{2t}(L):=2^{-1}\mu ^{-2}_{0}(L)\delta _{2t}(L)$ and $d(L):=d_{0}(L)$.

By considering Lemma 1, (18), and ${{E}}_{f}[f(\varTheta _{i})]=R(f)$, we find that ${{E}}_{f}[\overline{{{\mathrm {CV}}}}(\kappa )]$ is equivalent to (6).

We calculate the variance of $\overline{{{\mathrm {CV}}}}(\kappa )$. That is,

$$\begin{aligned} {{\mathrm {Var}}}_{f}[\overline{{{\mathrm {CV}}}}(\kappa )]&\simeq 2n^{-2}{{\mathrm {Var}}}_{f}\left[ \gamma _{ij}\right] + 4n^{-1}{{\mathrm {Var}}}_{f}\left[ f(\varTheta _{i})\right] + 4n^{-1}{\mathrm {Cov}}_{f}\left[ \gamma _{ij},\gamma _{ik}\right] \nonumber \\&\quad +\, 8n^{-1}{\mathrm {Cov}}_{f}\left[ \gamma _{ij},f(\varTheta _{i})\right] , \end{aligned}$$

(19)

where $j\ne k$. Let $I_{1}:=R((f^{(4)})^{1/2}f)$, $I_{2}:=R(f^{(2)})R(f)$, and $I_{3:}=R(f^{3/2})-R(f)^{2}$. Each term on the right-hand side of (19) is given by

$$\begin{aligned}&{{\mathrm {Var}}}_{f}[\gamma _{ij}]=\kappa ^{1/2}[Q(L)R(f)+o(1)],\end{aligned}$$

(20)

$$\begin{aligned}&{{\mathrm {Var}}}_{f}[f(\varTheta _{i})] =I_{3},\end{aligned}$$

(21)

$$\begin{aligned}&{\mathrm {Cov}}_{f}[\gamma _{ij}, \gamma _{ik}]=I_{3}-2\{I_{1}-I_{2} \}\mu ^{-2}_{0}(L)\mu ^{2}_{2}(L)\kappa ^{-2}+o(\kappa ^{-2}), \end{aligned}$$

(22)

and

$$\begin{aligned} {\mathrm {Cov}}_{f}[\gamma _{ij},f(\varTheta _{i})]&=-I_{3}+\{I_{1}-I_{2}\} \mu ^{-2}_{0}(L)\mu ^{2}_{2}(L)\kappa ^{-2}+o(\kappa ^{-2}). \end{aligned}$$

(23)

“Appendix C” in ESM provides the details of (20)–(23). By considering (19)–(23), we find that ${{\mathrm {Var}}}_{f}[\overline{{{\mathrm {CV}}}}(\kappa )]$ is equivalent to (7). $\square$

proof of Corollary 1

We set $c:={\hat{\kappa }}_{{{\mathrm {CV}}}}/\kappa _{*}$. Then, we combine Theorems 1 and 2 and find that

$$\begin{aligned} {\mathrm {AMISE}}(c\kappa _{*})/{\mathrm {MISE}}(c\kappa _{*})&{\mathop {\longrightarrow }\limits ^{p}}1, \end{aligned}$$

(24)

$$\begin{aligned} \overline{{{\mathrm {CV}}}}(c\kappa _{*})/{\mathrm {MISE}}(c\kappa _{*})&{\mathop {\longrightarrow }\limits ^{p}}1, \end{aligned}$$

(25)

and

$$\begin{aligned} {\mathrm {AMISE}}(c\kappa _{*})/{\mathrm {AMISE}} (\kappa _{*})&=\frac{1}{5c^{2}}+\frac{4c^{1/2}}{5}. \end{aligned}$$

(26)

(26) is a convex function with a minimum at $c=1$. Thus, if $c\ne 1$ and n is large, then it follows from combining (24) and (26) that

$$\begin{aligned} {\mathrm {MISE}}(c\kappa _{*})>{\mathrm {MISE}}(\kappa _{*}). \end{aligned}$$

(27)

Suppose that c does not converge to 1. Recall that it is necessary that $\overline{{{\mathrm {CV}}}}(c\kappa _{*})\le \overline{{{\mathrm {CV}}}}(\kappa )$ for any $\kappa$, because ${\hat{\kappa }}_{{{\mathrm {CV}}}}$ is the minimizer of $\overline{{{\mathrm {CV}}}}(\kappa )$. Additionally, if n is large, then $\overline{{{\mathrm {CV}}}}(\kappa )$ is a convex function with a minimum at $\kappa =c\kappa _{*}$, because we find that $\overline{{{\mathrm {CV}}}}(\kappa )$ approximates ${\mathrm {AMISE}}(\kappa )$ from Theorem 2. Therefore, it follows that

$$\begin{aligned} P(\overline{{{\mathrm {CV}}}}(c\kappa _{*})<\overline{{{\mathrm {CV}}}}(\kappa _{*}))\rightarrow 1, \end{aligned}$$

(28)

as $n\rightarrow \infty$. From (25) and (28), then it holds that

$$\begin{aligned} {\mathrm {MISE}}(c\kappa _{*})&<{\mathrm {MISE}}(\kappa _{*}), \end{aligned}$$

(29)

as $n\rightarrow \infty$. The contradiction between (27) and (29) completes the proof. $\square$

Proof of Theorem 3.

Let $U_{ij}=T^{(4)}_{g}(\varTheta _{i}-\varTheta _{j})$, and $U_{i}={{E}}_{f}[U_{ij}|\varTheta _{i}]$. The expectation of ${\hat{\psi }}_{4}(g)$ is given by

$$\begin{aligned} {{E}}_{f}[{\hat{\psi }}_{4}(g)]&= n^{-1}T^{(4)}_{g}(0)+2n^{-2}\sum _{i<j}{{E}}_{f}[U_{ij}]. \end{aligned}$$

(30)

It follows from (9) that

$$\begin{aligned} S^{(4)}_{g}(0)=3g^{2}\left[ S_{g}^{(2)}(0)+O(g^{-1})\right] . \end{aligned}$$

(31)

Lemma 2

(Tsuruta and Sagae 2017) The term $C_{\kappa }(L)$ is given by

$$\begin{aligned} C_{\kappa }(L)&= \kappa ^{-1/2}2^{1/2}\mu _{0}(L)+O\left( \kappa ^{-3/2}\right) . \end{aligned}$$

By combining (31) and Lemma 2, we find that the first term on the right side of (30) is equal to

$$\begin{aligned} n^{-1}T^{(4)}_{g}(0)&=\frac{3g^{5/2}\left[ S_{g}^{(2)}(0)+O(g^{-1})\right] }{2^{1/2}\mu _{0}(S)n}. \end{aligned}$$

(32)

Lemma 3

(Tsuruta and Sagae 2017) We set $\alpha _{j}(K_{\kappa }):=\int ^{\pi }_{-\pi }K_{\kappa }(\theta )\theta ^{j}\mathrm{d}\theta$. The terms $\alpha _{2t}(K_{\kappa })$ for $t=1,2$ are given by

$$\begin{aligned} \alpha _{2}(K_{\kappa })&= 2\mu ^{-1}_{0}(L)\mu _{2}(L)\kappa ^{-1}+O\left( \kappa ^{-2}\right) , \end{aligned}$$

and $\alpha _{4}(K_{\kappa }) = O(\kappa ^{-2})$. Lemma 2 in Tsuruta and Sagae (2017) presents the general form of $\alpha _{2t}(K_{\kappa })$.

It follows from Lemma 3 that

$$\begin{aligned} U_{i}&=\int ^{\pi }_{-\pi }T_{g}(\theta _{j}-\varTheta _{i})f^{(4)}(\theta _{j})\mathrm{d} \theta _{j}\nonumber \\&=f^{(4)}(\varTheta _{i}) + f^{(6)}(\varTheta _{i})\alpha _{2}(T_{g})/2 +O(\alpha _{4}(T_{g}))\nonumber \\&=f^{(4)}(\varTheta _{i})+ f^{(6)}(\varTheta _{i})\mu ^{-1}_{0}(S)\mu _{2}(S)g^{-1}+O(g^{-2}). \end{aligned}$$

(33)

${{E}}_{f}[U_{ij}]$ in (30) is given by the expectation of (33) over $\varTheta _{i}$.

$$\begin{aligned} {{E}}_{f}[U_{ij}]&={{E}}_{f}[U_{i}]=\psi _{4}+\mu ^{-1}_{0}(S) \mu _{2}(S)\psi _{6}g^{-1}+O(g^{-2}). \end{aligned}$$

(34)

We obtain the bias (13) from combining (30), (32), and (34).

We now derive the variance of ${\hat{\psi }}_{4}(g)$. We set $W_{ij}:=U_{ij}-U_{i}-U_{j}+{{E}}_{f}[U_{i}]$ and $Z_{i}:=U_{i}-{{E}}_{f}[U_{i}]$. Then, we obtain ${{E}}_{f}[W_{ij}]=0$, ${{E}}_{f}[Z_{i}]=0$, and ${\mathrm {Cov}}_{f}[Z_{i}W_{ij}]=0$. By using $W_{ij}$ and $Z_{i}$, we present ${\hat{\psi }}_{4}(g)-{{E}}_{f}[{\hat{\psi }}_{4}(g)]$ as

$$\begin{aligned} {\hat{\psi }}_{4}(g)-{{E}}_{f}[{\hat{\psi }}_{4}(g)]&=\frac{2(n-1)}{n^{2}} \sum _{i}Z_{i}+\frac{2}{n^{2}}\sum _{i<j}W_{ij}. \end{aligned}$$

(35)

(35) shows that the variance of ${\hat{\psi }}_{4}$ is equal to

$$\begin{aligned} {{\mathrm {Var}}}_{f}[{\hat{\psi }}_{4}(g)]&=\frac{4(n-1)^{2}}{n^{4}}\sum _{i} {{\mathrm {Var}}}_{f}[Z_{i}]+\frac{4}{n^{4}}\sum _{i<j}{{\mathrm {Var}}}_{f}[W_{ij}]. \end{aligned}$$

(36)

By combining (33) and (34), ${{\mathrm {Var}}}_{f}[Z_{i}]$ reduces to

$$\begin{aligned} {{\mathrm {Var}}}_{f}[Z_{i}]&={{E}}_{f}[U_{i}^{2}]-{{E}}_{f}[U_{i}]^{2}\nonumber \\&={{\mathrm {Var}}}_{f}[f^{(4)}(\varTheta _{i})]+o(1). \end{aligned}$$

(37)

By considering (34), ${{E}}_{f}[U^{2}_{ij}]=g^{9/2}[G_{1,0}(S_{4})\psi _{0}+o(1)]$, and ${{E}}_{f}[U_{i}^{2}]={{E}}_{f}[U_{i}]^{2}=O(1)$ (“Appendix D” in ESM provides the details of ${{E}}_{f}[U^{2}_{ij}]$ and ${{E}}_{f}[U_{i}^{2}]$), we obtain ${{\mathrm {Var}}}_{f}[W_{ij}]$. That is,

$$\begin{aligned} {{\mathrm {Var}}}_{f}[W_{ij}]&={{E}}_{f}[U_{ij}^{2}]-2{{E}}_{f}[U_{i}^{2}]+{{E}}_{f}[U_{i}]^{2}\nonumber \\&=g^{9/2}[G_{1,0}(S_{4})\psi _{0}+o(1)]. \end{aligned}$$

(38)

We obtain (11) from combining (36) (37), and (38). $\square$

Proof of Theorem 4.

If n is large, it follows from Lemma 3 that

$$\begin{aligned} {{\mathrm {CV}}}(\kappa )&\simeq \frac{d(L)\kappa ^{1/2}}{n}+\frac{2}{n^{2}}\sum _{i<j}\gamma (y_{ij}). \end{aligned}$$

(39)

The derivative of (39) is given by

$$\begin{aligned} \frac{\mathrm{d}{{\mathrm {CV}}}(\kappa )}{\mathrm{d}\kappa }&\simeq \frac{d(L)}{2n\kappa ^{1/2}}+\frac{2}{n^{2}\kappa ^{1/2}}\sum _{i<j}V_{ij}, \end{aligned}$$

(40)

where $V_{ij}:=\kappa ^{-1/2}[\gamma (y_{ij})+\rho (y_{ij})+3/4\mu ^{-1}_{0}(L)\mu _{2}(L)\kappa ^{-1}\tau (y_{ij})]$, $\phi _{\kappa }(y_{ij}):=\kappa C^{-1}_{\kappa }(L)\frac{\mathrm{d}}{\mathrm{d}\kappa }L_{\kappa }(y_{ij})$, $\rho (y_{ij}):=K_{\kappa }(y_{ij})+\int ^{\pi }_{-\pi }\{\phi _{\kappa }(w)K_{\kappa }(w+y_{ij})+K_{\kappa }(w)\phi _{\kappa }(w+y_{ij})\}\mathrm{d}w-2\phi _{\kappa }(y_{ij})$, and $\tau (y_{ij}):=\int ^{\pi }_{-\pi } K_{\kappa }(w)K_{\kappa }(w+y_{ij})\mathrm{d}w-K_{\kappa }(y_{ij})$.

“Appendix E” in ESM provides the details. The selector ${\hat{\kappa }}_{{{\mathrm {CV}}}}$ satisfies $\mathrm{d}{{\mathrm {CV}}}(\kappa )/\mathrm{d}\kappa \mid _{\kappa ={\hat{\kappa }}_{{{\mathrm {CV}}}}}=0$. This is equivalent to

$$\begin{aligned} 2n^{-2}\sum _{i<j}V_{ij}\Bigr |_{\kappa ={\hat{\kappa }}_{{{\mathrm {CV}}}}}=-d(L)/(2n). \end{aligned}$$

(41)

Note that $V_{i}:={{E}}_{f}[V_{ij}|\varTheta _{i}]$. Then, we set $H_{ij}:=V_{ij}-V_{i}-V_{j}+{{E}}_{f}[V_{i}]$ and $X_{i}:=V_{i}-{{E}}_{f}[V_{i}]$. Then, we rewrite $2n^{-2}\sum _{i<j}\{V_{ij}-{{E}}_{f}[V_{ij}]\}$ as

$$\begin{aligned} 2n^{-2}\sum _{i<j}V_{ij}-2n^{-2}\sum _{i<j}{{E}}_{f}[V_{ij}]\simeq 2n^{-1}\sum _{i}X_{i}+2n^{-2}\sum _{i<j}H_{ij}, \end{aligned}$$

where $2n^{-2}\sum _{i<j}H_{ij}$ is the degenerate U-statistic. We obtain the asymptotic normality for $2n^{-1}\sum _{i}X_{i}$ from the standard Central Limit Theorem (CLT). That is,

$$\begin{aligned} \frac{2}{n}\sum _{i}X_{i}{\mathop {\longrightarrow }\limits ^{d}}N\left( 0,Bn^{-1}\kappa ^{-5}\right) , \end{aligned}$$

(42)

where, $B:=16\mu ^{4}_{2}(L)\{R(f^{(4)}f^{1/2})-R(f^{\prime \prime })^{2}\}/\{\mu ^{4}_{0}(L)\}$. “Appendix F” in ESM presents the details.

We give the definition of a degenerate U-statistic. A U-statistic is defined as $U_{n}:=\sum _{i<j}H_{ij}$, where $H_{ij}:=H(\varTheta _{i},\varTheta _{j})$ and $H_{ij}$ is symmetric and ${{E}}_{f}[H_{ij}]=0$. Let the degenerate U-statistic be the U-statistic satisfying ${{E}}_{f}[H_{ij}|\varTheta _{i}]=0$. The following lemma describes the asymptotic normality of a degenerate U-statistic.

Lemma 4

(Hall 1984) Assume that $H_{ij}$ is symmetric, and ${{E}}_{f}[H_{ij}|\varTheta _{i}]=0$, almost surely and ${{E}}_{f}[H^{2}_{ij}]<\infty$ for each n. We set $G_{ij}:={{E}}_{f}[H_{ii}H_{ij}]$. if

$$\begin{aligned} \left\{ {{E}}_{f}[G^{2}_{ij}]+n^{-1} {{E}}_{f}[H^{4}_{ij}]\right\} /{{E}}_{f}[H^{2}_{ij}]^{2}\rightarrow 0, \end{aligned}$$

(43)

as $n\rightarrow \infty$, then,

$$\begin{aligned} \sum _{1\le i<j\le n}H_{ij}{\mathop {\longrightarrow }\limits ^{d}} N(0,n^{2}{{E}}_{f}[H_{ij}^{2}]/2). \end{aligned}$$

We obtain the asymptotic normality for $2n^{-2}\sum _{i<j}H_{ij}$ from Lemma 4. that is,

$$\begin{aligned} \frac{2}{n^{2}}\sum _{i<j}H_{ij}{\mathop {\longrightarrow }\limits ^{d}} N(0,2n^{-2}\kappa ^{-1/2}M_{1,0}(L)R(f)). \end{aligned}$$

(44)

See “Appendix G” in ESM for details. We combine (42) and (44) to derive the asymptotically normal for $2n^{-2}\sum _{i<j}V_{ij}$ as

$$\begin{aligned} \frac{2}{n^{2}}\sum _{i<j}V_{ij}&{\mathop {\longrightarrow }\limits ^{d}} N\left( -2R(f^{\prime \prime })\mu ^{-2}_{0}(L)\mu ^{2}_{2}(L)\kappa ^{-5/2}, \sigma ^{2}_{1}\right) , \end{aligned}$$

(45)

where $\sigma ^{2}_{1}:=B n^{-1}\kappa ^{-5}+2n^{-2}\kappa ^{-1/2}M_{1,0}(L)R(f)$. We take $\kappa ={\hat{\kappa }}_{{{\mathrm {CV}}}}$ in (45). Then, we replace ${\hat{\kappa }}_{{{\mathrm {CV}}}}$ in the variance to $\kappa _{*}$ by Corollary 1. Thus, it follows from combining (41) and (45) that

$$\begin{aligned} -2R(f^{\prime \prime })\mu ^{-2}_{0}(L)\mu ^{2}_{2}(L){\hat{\kappa }}_{{{\mathrm {CV}}}}^{-5/2}&{\mathop {\longrightarrow }\limits ^{d}} N\left( -d(L)/(2n), \sigma ^{2}_{2}\right) , \end{aligned}$$

(46)

where $\sigma ^{2}_{2}:=B n^{-1}\kappa _{*}^{-5}+2n^{-2}\kappa _{*}^{-1/2}M_{1,0}(L)R(f)$. We ignore the first term for the variance of (46), because the convergence rate of the first term is $O(n^{-3})$, and that of the second term is $O(n^{-11/5})$ using $\kappa _{*}=O(n^{2/5})$. From (3), we obtain $R(f^{\prime \prime })\mu ^{2}_{2}(L)n/(d(L)\mu _{0}(L))=\kappa ^{5/2}_{*}$. Thus, (46) reduces to

$$\begin{aligned} ({\hat{\kappa }}_{{{\mathrm {CV}}}}/\kappa _{*})^{-5/2}&{\mathop {\longrightarrow }\limits ^{d}} N\left( 1, 8d(L)^{-2}M_{1,0}(L)R(f)\kappa _{*}^{1/2}\right) . \end{aligned}$$

(47)

Let $g(x)=x^{-5/2}$. Then, it follows that $g(1)=1$ and $\{g^{\prime }(1)\}^{2}=25/4$. We obtain the asymptotic normality for ${\hat{\kappa }}_{{{\mathrm {CV}}}}/\kappa _{*}$ by applying the delta method to (47). That is,

$$\begin{aligned} {\hat{\kappa }}_{{{\mathrm {CV}}}}/\kappa _{*}{\mathop {\longrightarrow }\limits ^{d}} N\left( 1, 50d(L)^{-2}M_{1,0}(L)R(f)\beta (L)^{-1/2}R(f^{\prime \prime })^{-1/5} n^{-1/5} \right) . \end{aligned}$$

(48)

Theorem 4 completes the proof from (48). $\square$

Proof of Theorem 5.

The Taylor expansion ${\hat{\kappa }}_{{\mathrm {PI}}}= {\hat{\kappa }}_{{\mathrm {PI}}}({\hat{\psi }}_{4}(g_{*}))$ is given by

$$\begin{aligned} {\hat{\kappa }}_{{\mathrm {PI}}}\left( {\hat{\psi }}_{4}(g_{*})\right)&\simeq \beta (L) n^{2/5}\psi _{4}^{2/5} +\frac{2}{5}\beta (L) n^{2/5}\psi _{4}^{-3/5}({\hat{\psi }}_{4}(g_{*})-\psi _{4})\nonumber \\&=\kappa _{*}\left[ 1+2({\hat{\psi }}_{4}(g_{*})-\psi _{4})/(5\psi _{4})\right] . \end{aligned}$$

(49)

(49) reduces to

$$\begin{aligned} {\hat{\kappa }}_{{\mathrm {PI}}}/\kappa _{*}-1= \frac{2}{5\psi _{4}}\left( {\hat{\psi }}_{4}(g_{*})-\psi _{4}\right) . \end{aligned}$$

(50)

Noting $W_{ij}:=U_{ij}-U_{i}-U_{j}+{{E}}_{f}[U_{i}]$, and $Z_{i}:=U_{i}-{{E}}_{f}[U_{i}]$, it follows that (35) becomes

$$\begin{aligned} {\hat{\psi }}_{4}(g)-{{E}}_{f}[{\hat{\psi }}_{4}(g)]&\simeq 2n^{-1}\sum _{i}Z_{i}+2n^{-2}\sum _{i<j}W_{ij}, \end{aligned}$$

(51)

where $2n^{-2}\sum _{i<j}W_{ij}$ is the degenerate U-statistic. From (37), we obtain the asymptotic normality distribution from the standard CLT. That is,

$$\begin{aligned} n^{-1/2}\sum _{i}Z_{i}{\mathop {\longrightarrow }\limits ^{d}}N(0,{{\mathrm {Var}}}_{f} [f(\varTheta _{i})]). \end{aligned}$$

(52)

If we choose $g_{*}=W(S)n^{2/7}$, then applying Lemma A.4 to $2n^{-2}\sum _{i<j}W_{ij}$ gives

$$\begin{aligned} \frac{2}{n^{2}}\sum _{i<j}W_{ij}{\mathop {\longrightarrow }\limits ^{d}} N\left( 0,2n^{-2}g_{*}^{9/2}G_{1,0}(S_{4})\psi _{0}\right) , \end{aligned}$$

(53)

as $n\rightarrow \infty$. “Appendix H” in ESM presents the details. By combining (52) and (53), we obtain the asymptotic distribution of (51). That is,

$$\begin{aligned} {\hat{\psi }}_{4}(g_{*})-{{E}}_{f}\left[ {\hat{\psi }}_{4}(g_{*})\right] {\mathop {\longrightarrow }\limits ^{d}} N\left( 0,4n^{-1}{{\mathrm {Var}}}_{f}[f(\varTheta _{i})]+2n^{-2}g_{*}^{9/2}G_{1,0} (S_{4})\psi _{0}\right) . \end{aligned}$$

(54)

Theorem 3 shows that the rate of ${{\mathrm {Var}}}_{f}[{\hat{\psi }}_{4}(g^{*})]$ is the order $n^{-5/7}$. Thus, (54) reduces to

$$\begin{aligned} n^{5/14}\left\{ {\hat{\psi }}_{4}(g_{*})-{{E}}_{f} \left[ {\hat{\psi }}_{4}(g_{*})\right] \right\} {\mathop {\longrightarrow }\limits ^{d}} N\left( 0,2W^{9/2}(S)G_{1,0}(S_{4})\psi _{0}\right) . \end{aligned}$$

(55)

The main term ${\hat{\psi }}_{4}(g_{*})-\psi _{4}$ on the right side for (50) is equivalent to

$$\begin{aligned} n^{5/14}\left\{ {\hat{\psi }}_{4}(g_{*})-\psi _{4}\right\} =n^{5/14} \left\{ {\hat{\psi }}_{4}(g_{*})-{{E}}_{f}\left[ {\hat{\psi }}_{4}(g_{*})\right] \right\} -n^{5/14}{{\mathrm {Bias}}}_{f} \left[ {\hat{\psi }}_{4}(g_{*})\right] . \end{aligned}$$

(56)

We show that ${{\mathrm {Bias}}}_{f}[{\hat{\psi }}_{4}(g^{*})]=O(n^{-4/7})$ from Corollary 2. Then, we obtain that $n^{5/14}{{\mathrm {Bias}}}_{f}[{\hat{\psi }}_{4}(g_{*})]$ is $O(n^{-3/14})$. Thus, if n is large, then this term is ignored. Therefore, the asymptotic normal distribution for $n^{5/14}\{{\hat{\psi }}_{4}(g_{*})-\psi _{4}\}$ is given by

$$\begin{aligned} n^{5/14}\{{\hat{\psi }}_{4}(g)-\psi _{4}\}{\mathop {\longrightarrow }\limits ^{d}} N(0,2W^{9/2}(S)G_{1,0}(S_{4})\psi _{0}). \end{aligned}$$

(57)

Therefore, as $n\rightarrow \infty$, Theorem 5 completes the proof from (50) and (57). $\square$