Improving kernel-based nonparametric regression for circular–linear data

Abstract

We discuss kernel-based nonparametric regression where a predictor has support on a circle and a responder has support on a real line. Nonparametric regression is used in analyzing circular–linear data because of its flexibility. However, nonparametric regression is generally less accurate than an appropriate parametric regression for a population model. Considering that statisticians need more accurate nonparametric regression models, we investigate the performance of sine series local polynomial regression while selecting the most suitable kernel class. The asymptotic result shows that higher-order estimators reduce conditional bias; however, they do not improve conditional variance. We show that higher-order estimators improve the convergence rate of the weighted conditional mean integrated square error. We also prove the asymptotic normality of the estimator. We conduct a numerical experiment to examine a small sample of characteristics of the estimator in scenarios wherein the error term is homoscedastic or heterogeneous. The result shows that choosing a higher degree improves performance under the finite sample in homoscedastic or heterogeneous scenarios. In particular, in some scenarios where the regression function is wiggly, higher-order estimators perform significantly better than local constant and linear estimators.

Appendices

Appendix A

Proof

(Proof of Proposition 1) We replace series $(\varTheta _{i}-\theta )^{t}$ in the Taylor extension of $m(\varTheta _{i})$ with $\sin ^{-1}(\sin (\varTheta _{i}-\theta ))^{t}$ for $|\varTheta _{i}-\theta |<\pi /2$. Then, $m(\varTheta _{i})$ is

$$\begin{aligned} m(\varTheta _{i})&= m(\theta ) + m^{\prime }(\theta )\sin ^{-1}(\sin (\varTheta _{i}-\theta )) + m^{\prime \prime }(\theta )\sin ^{-1}(\sin (\varTheta _{i}-\theta ))^{2}/2 + \cdots \end{aligned}$$

(12)

for $|\varTheta _{i}-\theta |<\pi /2$. The Taylor extension of $\sin ^{-1}(u)$ is

$$\begin{aligned} \sin ^{-1}(u) = \sum _{s=0}^{\infty }b_{s}u^{2s+1},\quad |u|<1, \end{aligned}$$

(13)

where $b_{s} =\{(2s-1)!!\}/\{(2s)!!(2s+1)\}$. By combining (12) and (13), we obtain the sine series

$$\begin{aligned} m(\varTheta _{i})&=m(\theta )+\sum ^{p+2}_{t=1}\frac{m^{(t)}(\theta )}{t!}\left( \sum ^{[(p+1)/2]}_{s=0}b_{s}\sin (\varTheta _{i}-\theta )^{2s+1}\right) ^{t}+o_{p}(\sin (\varTheta _{i}-\theta )^{p+2}) \end{aligned}$$

(14)

for $|\varTheta - \theta | <\pi /2$, where $D_{t}:= (\sum ^{[(p+1)/2]}_{s=0}b_{s}\sin (\varTheta _{i}-\theta )^{2s+1})^{t}$ and $D_{t}= O_{p}(\sin (\varTheta _{i}-\theta )^{p+2})$.

When polynomial theorem is applied, its term $(\sum ^{[(p+1)/2]}_{s=0}b_{s}\sin (\varTheta _{i}-\theta )^{2s+1})^{t}$ becomes

$$\begin{aligned} D_{t}&=\sum _{\sum _{s=0}^{[(p+1)/2]}t_{s}=t}\frac{t!}{\prod ^{[(p+1)/2]}_{m=0}t_{m}!}\prod _{l=0}^{[(p+1)/2]}b_{l}^{t_{l}}\sin (\varTheta _{i}-\theta )^{\sum ^{[(p+1)/2]}_{r=0}(2r+1)t_{r}}\nonumber \\&=t!\sum _{q=t}^{p+2}\sum _{\begin{array}{c} \sum _{s=0}^{[(p+1)/2]}t_{s}=t,\\ \sum _{s=0}^{[(p+1)/2]}(2s+1)t_{s}=q \end{array}}\frac{\prod _{l=0}^{[(p+1)/2]}b_{l}^{t_{l}}}{\prod ^{[(p+1)/2]}_{m=0}t_{m}!}\sin (\varTheta _{i}-\theta )^{q}+o_{p}(\sin (\varTheta _{i}-\theta )^{p+2})\nonumber \\&=t!\sum _{q=t}^{p+2}B_{q}(p,t)\sin (\varTheta _{i}-\theta )^{q}+o_{p}(\sin (\varTheta _{i}-\theta )^{p+2}). \end{aligned}$$

(15)

By combining (14) and (15), we obtain

$$\begin{aligned} m(\varTheta _{i})&= m(\theta ) + \sum ^{p+2}_{t=1}m^{(t)}(\theta ) \sum ^{p+2}_{q=t}B_{q}(p,t)\sin (\varTheta _{i}-\theta )^{q} + o_{p}(\sin (\varTheta _{i}-\theta )^{p+2})\nonumber \\&= m(\theta ) + \sum ^{p+2}_{q=1} \left[ \sum ^{q}_{t=1}m^{(t)}(\theta )B_{q}(p,t)\right] \sin (\varTheta _{i}-\theta )^{q} + o_{p}(\sin (\varTheta _{i}-\theta )^{p+2})\nonumber \\&=m(\theta )+\sum _{q=1}^{p+2}M_{q}(\theta )\sin (\varTheta _{i}-\theta )^{q}+o_{p}(\sin (\varTheta _{i}-\theta )^{p+2}),\quad |\theta |<\pi /2. \end{aligned}$$

(16)

The proof of Proposition 1 is complete with (16). $\square$

Appendix B

Proof

(Proof of Lemma 1) Recalling the first-order approximation $\cos (hz)= 1-h^{2}z^{2}/2 + O(h^{4})$, we have

$$\begin{aligned} L_{h}(hz)&=L(h^{-2}[1-\{1-h^{2}z^{2}/2+O(h^{4})\}])\nonumber \\&=c(L)\bar{L}(z)+O(h^{2}). \end{aligned}$$

(17)

The proof of property (i) is complete with (17). We next derive property (ii). Condition (b) indicates that $\bar{L}(z) < Mz^{-(2p + 4 +\alpha )}$ with the upper bound M of $\bar{L}$ if z is large enough. When n is large, we obtain

$$\begin{aligned} \int ^{\infty }_{\pi /h}\bar{L}(z)z^{t}\mathrm{d}z&\le \int ^{\infty }_{\pi /h}\bar{L}(z)z^{2p+2}\mathrm{d}z\nonumber \\&< M\int ^{\infty }_{\pi /h}z^{-(2 + \alpha )}\mathrm{d}z\nonumber \\&=o(h). \end{aligned}$$

(18)

Hence, we can ignore the tail part of $\mu _{t}(\bar{L})$. The proof of property (ii) is complete with (18).

We next derive property (iii). Properties (i) and (ii) provide

$$\begin{aligned} C_{h}(L)&= h\int ^{\pi /h}_{-\pi /h}L(h^{-2}\{ 1 - \cos (hz) \}\mathrm{d}z\nonumber \\&=h\int ^{\pi /h}_{-\pi /h}\{c(L)\bar{L}(z)+O(h^{2})\}\mathrm{d}z\nonumber \\&=h\{c(L)\mu _{0}(\bar{L})+o(h)\}\nonumber \\&=h\{c(L)+o(h)\}. \end{aligned}$$

(19)

Therefore, the normalizing constant $C_{h}(L)$ closes to bandwidth c(L)h. The proof of property (iii) is complete from (19).

We then consider property (iv). Let Q be the upper bound of L. Note that $1 \le 1-\cos (\theta ) \le 2$ for $|\theta | \ge \pi /2$. Recalling that L(r) is a monotonically non-increasing function, we find that $L(h^{-2}\{ 1- \cos (\theta ) \}) \le L(h^{-2} )$ for $|\theta | \ge \pi /2$. When n is large enough, we have $L(h^{-2}\{ 1- \cos (\theta ) \}) \le L(h^{-2} ) \le Q h^{2(p + 2 + \alpha /2)}$ for $\theta \le |\pi /2|$ from condition (b). By combining condition (b) and property (iii), we obtain the order of a kernel’s tail

$$\begin{aligned} K_{h}(\theta )&\le C_{h}^{-1}(L)Q h^{2(p + 2 + \alpha /2)}\nonumber \\&\le Qh^{-1}\{1 + o(h)\}^{-1}h^{2(p + 2 + \alpha /2)} \nonumber \\&=o(h^{p + 2}) \end{aligned}$$

(20)

for $|\theta | \ge \pi /2$. The proof of property (iv) is complete with (20). $\square$

Appendix C

Proof

(Proof of Theorem 1) If n is large enough, we can ignore observing $|\varTheta _{i}-\theta |\ge \pi /2$ because the kernel is $K_{h}(\varTheta _{i}-\theta ) = o_{p}(h^{p+2})$ from (iv) in Lemma 1. Therefore, we assume that all observations satisfy $|\varTheta _{i}-\theta |<\pi /2$ in the sample ${\varvec{\varTheta }}_{n}$ to simplify the proof. Assume that ${\varvec{M}}:=(m(\varTheta _{1}),\ldots ,m(\varTheta _{n}))^{T}$. By applying ${\varvec{M}}$ to the Taylor series of Proposition 1, we obtain

$$\begin{aligned} {\varvec{M}}&={\varvec{S}}_{\theta }\begin{bmatrix}m(\theta )\\ M_{1}(\theta )\\ \vdots \\ M_{p}(\theta )\end{bmatrix}+{\varvec{T}}_{m,\theta }+{\varvec{R}}_{m,\theta } , \end{aligned}$$

(21)

where

$$\begin{aligned} {\varvec{T}}_{m,\theta }&=M_{p+1}(\theta )\begin{bmatrix}\sin (\varTheta _{1}-\theta )^{p+1}\\ \vdots \\ \sin (\varTheta _{n}-\theta )^{p+1}\end{bmatrix}+M_{p+2}(\theta )\begin{bmatrix}\sin (\varTheta _{1}-\theta )^{p+2}\\ \vdots \\ \sin (\varTheta _{n}-\theta )^{p+2}\end{bmatrix}, \end{aligned}$$

(22)

and the remaining ${\varvec{R}}_{m,\theta }$ is ${\varvec{R}}_{m,\theta }=o_{p}( {\varvec{T}}_{m,\theta })$. From (21), we obtain the bias as

$$\begin{aligned} \mathrm {Bias}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]&={\varvec{e}}_{1}^{T}(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }\{{\varvec{T}}_{m,\theta }+{\varvec{R}}_{m,\theta }\}. \end{aligned}$$

(23)

Assume that ${\varvec{A}}:=\mathop {\mathrm{diag}}\nolimits \{1,h,\ldots ,h^{p}\}$ and ${\varvec{\mu }}_{\kappa }:=(\mu _{k},\mu _{k+1},\ldots ,\mu _{k+p})^{T}$ with $\mu _{j}:=\mu _{j}(\bar{L})$. Let ${\varvec{Q}}_{p}$ be the $(p+1)\times (p+1)$ matrix with the (i, j) entry equal to $\mu _{i+j-1}$. Recall that ${\varvec{N}}_{p}$ is the $(p+1)\times (p+1)$ matrix having the (i, j) entry equal to $\mu _{i+j-2}$.

We show the asymptotic forms of ${\varvec{e}}_{1}^{T}(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}$ and $n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{T}}_{m,\theta }$ in (23) as the two following lemmas.

Lemma 2

The term ${\varvec{e}}_{1}^{T}(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}$ is given by

$$\begin{aligned}&{\varvec{e}}_{1}^{T}(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}\\&=f(\theta )^{-1}\left[ {\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}-hf^{\prime }(\theta )f(\theta )^{-1}{\varvec{e}}^{T}_{1}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1}+o_{p}(h)\right] {\varvec{A}}^{-1}. \end{aligned}$$

Lemma 3

The term $n^{-1}{\varvec{A}}^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{T}}_{m,\theta }$ is given by

$$\begin{aligned} n^{-1}&{\varvec{A}}^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{T}}_{m,\theta }\nonumber \\&=h^{p+1}{\varvec{\mu }}_{p+1}f(\theta )M_{p+1}(\theta ) \\&\qquad +h^{p+2}{\varvec{\mu }}_{p+2}\{f^{\prime }(\theta )M_{p+1}(\theta )+f(\theta )M_{p+2}(\theta )\} +o_{p}(h^{p+2}). \end{aligned}$$

Proof

(Proof of Lemma 2) From (i), (ii), and (iii) of Lemma 1, we find that each entry in ${\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }$ is equal to

$$\begin{aligned} \hat{s}_{l}(\theta ;h)&=\int ^{\pi }_{-\pi }K_{h}(\theta _{i}-\theta )\sin (\theta _{i}-\theta )^{l}f(\theta _{i})\mathrm{d}\theta _{i}+O_{p}(n^{-1})\nonumber \\&=\int ^{\pi /h}_{-\pi /h}K_{h}(hz)\sin (hz)^{l}f(\theta +hz)h\mathrm{d}z+O_{p}(n^{-1})\nonumber \\&= \int ^{\pi /h}_{-\pi /h}h^{-1}\{1+o_{p}(h)\}^{-1}\{ \bar{L}(z) + O_{p}(h^{2})\}\sin (hz)^{l}f(\theta +hz)h\mathrm{d}z+O_{p}(n^{-1})\nonumber \\&=\int ^{\pi /h}_{-\pi /h}h^{-1}\{\bar{L}(z)+o_{p}(h)\}\{hz+O_{p}(h^{3})\}^{l}f(\theta +hz)h\mathrm{d}z+O_{p}(n^{-1})\nonumber \\&=\int ^{\pi /h}_{-\pi /h}\{h^{l}\bar{L}(z)z^{l}+o_{p}(h^{l+1})\}[f(\theta )+hf^{\prime }(\theta )z+o_{p}(h)]\mathrm{d}z+O_{p}(n^{-1})\nonumber \\&=h^{l}\left\{ f(\theta )\int ^{\pi /h}_{-\pi /h}\bar{L}(z)z^{l}\mathrm{d}z+hf^{\prime }(\theta )\int ^{\pi /h}_{-\pi /h}\bar{L}(z)z^{l+1}\mathrm{d}z+o_{p}(h)\right\} +O_{p}(n^{-1}) \nonumber \\&=h^{l}\left\{ f(\theta )\mu _{l}(\bar{L})+hf^{\prime }(\theta )\mu _{l+1}(\bar{L})+o_{p}(h)\right\} . \end{aligned}$$

(24)

From (24), we have

$$\begin{aligned} n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }&=A\left[ f(\theta ){\varvec{N}}_{p}+hf^{\prime }(\theta ){\varvec{Q}}_{p}\right] A+o_{p}(hA{\varvec{I}}A). \end{aligned}$$

(25)

Assume that $g(hf^{\prime }(\theta ){\varvec{Q}}_{p}):=[f(\theta ){\varvec{N}}_{p}+hf^{\prime }(\theta ){\varvec{Q}}_{p}]^{-1}$. The Taylor expansion of g is then given by

$$\begin{aligned} g(hf^{\prime }(\theta ){\varvec{Q}}_{p})=f(\theta )^{-1}{\varvec{N}}^{-1}_{p}-hf^{\prime }(\theta )f(\theta )^{-2}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1}+o_{p}(h). \end{aligned}$$

(26)

From combining (25), (26), and ${\varvec{e}}_{1}^{T}{\varvec{A}}^{-1}={\varvec{e}}_{1}^{T}{\varvec{A}}={\varvec{e}}_{1}^{T}$, we find that matrix ${\varvec{e}}_{1}^{T}(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}$ is equal to

$$\begin{aligned} {\varvec{e}}_{1}^{T}(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}&={\varvec{e}}^{T}_{1}\left[ {\varvec{A}}^{-1}\left\{ f(\theta )^{-1}{\varvec{N}}^{-1}_{p}-hf^{\prime }(\theta )f(\theta )^{-2}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1} + o_{p}(h) \right\} {\varvec{A}}^{-1}\right] \nonumber \\&={\varvec{e}}^{T}_{1}\left[ f(\theta )^{-1}{\varvec{N}}^{-1}_{p}-hf^{\prime }(\theta )f(\theta )^{-2}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1}+o_{p}(h)\right\} {\varvec{A}}^{-1}\nonumber \\&=f(\theta )^{-1}\left[ {\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}-hf^{\prime }(\theta )f(\theta )^{-1}{\varvec{e}}^{T}_{1}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1}+o_{p}(h)\right] {\varvec{A}}^{-1}. \end{aligned}$$

(27)

The proof of Lemma 2 is complete with (27). $\square$

Proof

(Proof of Lemma 3) From (24), we derive

$$\begin{aligned} n^{-1}&{\varvec{A}}^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }(\sin (\theta _{1}-\theta )^{k},\ldots ,\sin (\theta _{n}-\theta )^{k})^{T}\nonumber \\&={\varvec{A}}^{-1}(\hat{s}_{k}(\theta ;h),\ldots ,\hat{s}_{k+p}(\theta ;h) )^{T}\nonumber \\&={\varvec{A}}^{-1}{\varvec{A}}[h^{k}f(\theta ){\varvec{\mu }}_{k}+h^{k+1}f^{\prime }(\theta ){\varvec{\mu }}_{k+1}+o_{p}(h^{k+1})]\nonumber \\&=h^{k}f(\theta ){\varvec{\mu }}_{k}+h^{k+1}f^{\prime }(\theta ){\varvec{\mu }}_{k+1}+o_{p}(h^{k+1}). \end{aligned}$$

(28)

By combining (22) and (28), we find that

$$\begin{aligned} n^{-1}&{\varvec{A}}^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{T}}_{m,\theta }\nonumber \\&=M_{p+1}(\theta )\left\{ h^{p+1}f(\theta ){\varvec{\mu }}_{p+1} +h^{p+2}{\varvec{\mu }}_{p+2}f^{\prime }(\theta ) +o_{p}(h^{p+2})\right\} \nonumber \\&\quad +M_{p+2}(\theta )\left\{ h^{p+2}f(\theta ){\varvec{\mu }}_{p+2}+o_{p}(h^{p+2})\right\} \nonumber \\&=h^{p+1}{\varvec{\mu }}_{p+1}f(\theta )M_{p+1}(\theta ) +h^{p+2}{\varvec{\mu }}_{p+2}\{f^{\prime }(\theta )M_{p+1}(\theta )\nonumber \\&\quad +f(\theta )M_{p+2}(\theta )\} +o_{p}(h^{p+2}). \end{aligned}$$

(29)

The proof of Lemma 3 is complete with (29). $\square$

By combining (23) and Lemmas 2 and 3, we derive the bias as equal to

$$\begin{aligned}&\mathrm {Bias}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]\nonumber \\&\quad =f(\theta )^{-1}\left[ {\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}-hf^{\prime }(\theta )f(\theta )^{-1}{\varvec{e}}^{T}_{1}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1}+o_{p}(h)\right] \nonumber \\&\qquad \times [h^{p+1}{\varvec{\mu }}_{p+1}f(\theta )M_{p+1}(\theta ) +h^{p+2}{\varvec{\mu }}_{p+2}\{f^{\prime }(\theta )M_{p+1}(\theta )+f(\theta )M_{p+2}(\theta )\} +o_{p}(h^{p+2})]\nonumber \\&\quad = h^{p+1}M_{p+1}(\theta ){\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p} {\varvec{\mu }}_{p+1}+h^{p+2}\{ M_{p+1}(\theta )f^{\prime }(\theta )f(\theta )^{-1} + M_{p+2}(\theta )\}{\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p} {\varvec{\mu }}_{p+2}\nonumber \\&\qquad -h^{p+2}M_{p+1}(\theta )f^{\prime }(\theta )f(\theta )^{-1}{\varvec{e}}^{T}_{1}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1}{\varvec{\mu }}_{p+1}+o_{p}(h^{p+2})\nonumber \\&\quad =h^{p+1}M_{p+1}(\theta )\sum ^{p+1}_{j=1}({\varvec{N}}_{p}^{-1})_{1j}\mu _{p+j}\nonumber \\&\qquad +h^{p+2}\left\{ M_{p+1}(\theta )f^{\prime }(\theta )f(\theta )^{-1}+M_{p+2}(\theta ) \right\} \sum ^{p+1}_{j=1}({\varvec{N}}_{p}^{-1})_{1j}\mu _{p+j+1}\nonumber \\&\qquad -h^{p+2}M_{p+1}(\theta )f^{\prime }(\theta )f(\theta )^{-1}{\varvec{e}}^{T}_{1}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}{\varvec{N}}_{p}^{-1}{\varvec{\mu }}_{p+1}+o_{p}(h^{p+2}). \end{aligned}$$

(30)

We employ the following lemma given by Ruppert and Wand (1994) to simplify (30). $\square$

Lemma 4

It holds that

• (i) if j is odd, then $\mu _{j}=0$, otherwise $\mu _{j}\ne 0$;

• (ii) if $i+j$ is odd, then $({\varvec{N}}_{p})_{ij}=({\varvec{N}}_{p}^{-1})_{ij}=0$;

• (iii) if $i+j$ is even, then $({\varvec{Q}}_{p})_{ij}=0$.

When p is odd, we find that the first term on the right-hand side (RHS) of (30) does not become zero if (i) and (ii) of Lemma 4 are combined. This leads to

$$\begin{aligned} \mathrm {Bias}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]&=h^{p+1}M_{p+1}(\theta )\sum ^{p+1}_{j=1}({\varvec{N}}_{p}^{-1})_{1j}\mu _{p+j}+o_{p}(h^{p+1}) \end{aligned}$$

(31)

for odd p.

Next, we consider the case where p is even. When we combine (i) and (ii) in Lemma 4, we find that the first term on the RHS of (30) is zero. Additionally, the first p columns of ${\varvec{Q}}_{p}$ are identical to the last p columns of ${\varvec{N}}_{p}$. Ruppert and Wand (1994) found that ${\varvec{e}}^{T}_{1}{\varvec{N}}_{p}^{-1}{\varvec{Q}}_{p}$ becomes zero when they combine (ii) and (iii) in Lemma 4. This demonstrates that the last term on the RHS of (30) vanishes. Therefore, we obtain

$$\begin{aligned} \mathrm {Bias}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]&=h^{p+2}\left\{ M_{p+1}(\theta )f^{\prime }(\theta )f(\theta )^{-1}+M_{p+2}(\theta ) \right\} \nonumber \\&\quad \times \sum ^{p+1}_{j=1}({\varvec{N}}_{p}^{-1})_{1j}\mu _{p+j+1}+o_{p}(h^{p+2}) \end{aligned}$$

(32)

for even p.

Assume that a cofactor of the determinant $|{\varvec{N}}_{p}|$ is $c_{ij}$ and that $({\varvec{N}}_{p}^{-1})_{ij}=c_{ij}/|{\varvec{N}}_{p}|$ and $|{\varvec{M}}_{p}(z)|=\sum _{j}c_{1j}z^{j-1}$. We provide the relation between the k-th moment of $\bar{L}_{(p)}$ and $\sum ^{p+1}_{j=1}({\varvec{N}}_{p}^{-1})_{1j}\mu _{k-1+j}$ as the following equation:

$$\begin{aligned} \mu _{k}(\bar{L}_{(p)})&=\int _{\mathbb {R}}\bar{L}_{(p)}(z)z^{k}\mathrm{d}z\nonumber \\&= \int _{\mathbb {R}} \frac{ |{\varvec{M}}_{p}(z)| }{ |{\varvec{N}}_{p}| }\bar{L}(z)z^{k}\mathrm{d}z\nonumber \\&=\int _{\mathbb {R}}\sum _{j=1}^{p+1}\frac{c_{1j}z^{j-1}}{|{\varvec{N}}_{p}|}\bar{L}(z)z^{k}\mathrm{d}z\nonumber \\&=\sum _{j=1}^{p+1}({\varvec{N}}_{p}^{-1})_{1j}\mu _{k+j-1}. \end{aligned}$$

(33)

When we apply (33) to (31) and (32), we obtain biases (6) and (7).

We next consider the variance. Assume that ${\varvec{V}}:=\mathop {\mathrm{diag}}\nolimits \{v(\varTheta _{1}),\ldots ,v(\varTheta _{n})\}$. The variance is given by

$$\begin{aligned}&\mathrm {Var}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]\nonumber \\&\quad =n^{-1}{\varvec{e}}_{1}^{T}(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{V}}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }(n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta })^{-1}{\varvec{e}}_{1}. \end{aligned}$$

(34)

Moreover, let ${\varvec{T}}_{p}$ be the $(p+1)\times (p+1)$ matrix with (i, j) entry equal to $\mu _{i+j-2}(\bar{L}^{2})$. We approximate matrix $n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{V}}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }$ with the following lemma.

Lemma 5

Matrix $n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{V}}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }$ is given by

$$\begin{aligned} n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{V}}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }&={\varvec{A}}\{h^{-1}v(\theta )f(\theta ){\varvec{T }}_{p}+o_{p}(h^{-1})\}{\varvec{A}}. \end{aligned}$$

Proof

(Proof of Lemma 5) when n is large enough, using the same procedure (ii) in Lemma 1, we have

$$\begin{aligned} \int ^{\pi /h}_{-\pi /h}\bar{L}(z)^{2}z^{l}\mathrm{d}z=\mu _{l}(\bar{L}^{2})+o(h) \end{aligned}$$

(35)

for $l \le 2p+2$. Assume that $\hat{r}_{l}(\theta ;h):=n^{-1}\sum _{i}K_{h}(\varTheta _{i}-\theta )^{2}\sin (\varTheta _{i}-\theta )^{l}v(\varTheta _{i})$. Then, matrix $n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{V}}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }$ becomes the matrix $(p+1)\times (p+1)$ with (i, j) entry to $\hat{r}_{i+j-2}(\theta ;h)$. When we combine (1) and (iii) in Lemma 1 and (35), we obtain

$$\begin{aligned} \hat{r}_{l}(\theta ;h)&=\int ^{\pi }_{-\pi }K_{h}(\theta _{i}-\theta )^{2}\sin (\theta _{i}-\theta )^{l}v(\theta _{i})f(\theta _{i})\mathrm{d}\theta _{i}+O_{p}(n^{-1})\nonumber \\&=\int ^{\pi /h}_{-\pi /h}K_{h}(hz)^{2}\sin (hz)^{l}v(\theta +hz)f(\theta +hz)h\mathrm{d}z+O_{p}(n^{-1})\nonumber \\&=h^{-1}\int ^{\pi /h}_{-\pi /h}\{\bar{L}(z)+o_{p}(h)\}^{2}\{hz+O_{p}(h^{3})\}^{l}\{v(\theta )f(\theta ) + o_{p}(1)\} \mathrm{d}z\nonumber \\&=h^{l-1}v(\theta ) f(\theta ) \int ^{\pi /h}_{-\pi /h}\{\bar{L}(z)^{2}z^{l} + o_{p}(1)\}\mathrm{d}z \nonumber \\&=h^{l-1}\{v(\theta ) f(\theta )\mu _{l}(\bar{L}^{2})+o_{p}(1)\}. \end{aligned}$$

(36)

From (36), we have

$$\begin{aligned} n^{-1}{\varvec{S}}_{\theta }^{T}{\varvec{W}}_{\theta }{\varvec{V}}{\varvec{W}}_{\theta }{\varvec{S}}_{\theta }&={\varvec{A}}[h^{-1}\{v(\theta )f(\theta ){\varvec{T }}_{p}+o_{p}(1)\}]{\varvec{A}}. \end{aligned}$$

(37)

The proof of Lemma 5 is complete with (37). $\square$

By combining (34), Lemmas 2, and 5, we show that the variance is equal to

$$\begin{aligned} \mathrm {Var}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]\nonumber \&=n^{-1}f(\theta )^{-1}\left[ {\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}+o_{p}(1)\right] \nonumber \\&\quad {\varvec{A}}^{-1}{\varvec{A}}[h^{-1}\{v(\theta )f(\theta ){\varvec{T }}_{p}+o_{p}(1)\}]{\varvec{A}}\nonumber \\&\quad \times {\varvec{A}}^{-1}\left[ {\varvec{N}}^{-1}_{p}{\varvec{e}}_{1}+o_{p}(1)\right] f(\theta )^{-1}\nonumber \\ \&=n^{-1}h^{-1}\{v(\theta )f(\theta )^{-1}{\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}{\varvec{T }}_{p}{\varvec{N}}^{-1}_{p}{\varvec{e}}_{1}+o_{p}(1)\}. \end{aligned}$$

(38)

The term ${\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}{\varvec{T }}_{p}{\varvec{N}}^{-1}_{p}{\varvec{e}}_{1}$ on the RHS of (38) is given by

$$\begin{aligned} {\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}{\varvec{T }}_{p}{\varvec{N}}^{-1}_{p}{\varvec{e}}_{1}&=\frac{1}{|{\varvec{N}}_{p}|^{2}}\sum ^{p+1}_{i=1}\sum ^{p+1}_{j=1}c_{1i}c_{1j}\mu _{i+j-2}(\bar{L}^{2}). \end{aligned}$$

(39)

The following equation can be used to simplify ${\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}{\varvec{T }}_{p}{\varvec{N}}^{-1}_{p}{\varvec{e}}_{1}$.

$$\begin{aligned} \mu _{0}(\bar{L}_{(p)}^{2})&=\int _{\mathbb {R}}\bar{L}_{(p)}(z)^{2}\mathrm{d}z\nonumber \\&=\int _{\mathbb {R}}\left\{ \sum ^{p+1}_{j=1}c_{1j}z^{j-1}|{\varvec{N}}_{p}|^{-1}\bar{L}(z)\right\} ^{2}\mathrm{d}z\nonumber \\&=\frac{1}{|{\varvec{N}}_{p}|^{2}}\sum ^{p+1}_{i=1}\sum ^{p+1}_{j=1}c_{1i}c_{1j}\int _{\mathbb {R}}z^{i+j-2}\bar{L}(z)^{2}\mathrm{d}z\nonumber \\&=\frac{1}{|{\varvec{N}}_{p}|^{2}}\sum ^{p+1}_{i=1}\sum ^{p+1}_{j=1}c_{1i}c_{1j}\mu _{i+j-2}(\bar{L}^{2}). \end{aligned}$$

(40)

By combining (39) and (40), we obtain

$$\begin{aligned} {\varvec{e}}^{T}_{1}{\varvec{N}}^{-1}_{p}{\varvec{T }}_{p}{\varvec{N}}^{-1}_{p}{\varvec{e}}_{1}=\mu _{0}(\bar{L}_{(p)}^{2}). \end{aligned}$$

(41)

Furthermore, by combining (38) and (41), we obtain variance (8). $\square$

Appendix D

Proof

(Proof of Theorem 2) Assume vector ${\varvec{e}}^{T}_{1}(n^{-1}{\varvec{S_{\theta }}}^{T}{\varvec{W_{\theta }}}{\varvec{S_{\theta }}})^{-1}{\varvec{S_{\theta }}}^{T}{\varvec{W_{\theta }}}=(c_{1},\ldots ,c_{n})$. We then have $\hat{m}(\theta ;p,h)=n^{-1}\sum _{i=1}^{n}c_{i}Y_{i}$. Further, we assume that the sum of the conditional variance of $h^{1/2}c_{i}Y_{i}$ is $S^{2}_{n}:=\sum _{i=1}^{n}\mathrm {Var}[h^{1/2}c_{i}Y_{i}|{\varvec{\varTheta }}_{n}]$. From (8), we find that $S^{2}_{n}$ is equal to

$$\begin{aligned} S^{2}_{n}&=n^{2}h\mathrm {Var}\left[ \sum _{i=1}^{n}\hat{m}(\theta ;p,h) \left| {\varvec{\varTheta }}_{n}\right] \right. \nonumber \\&=nv(\theta )\mu _{0}(\bar{L}_{(p)}^{2})f(\theta )^{-1}\{1+o_{p}(1)\}. \end{aligned}$$

(42)

From (42), we have

$$\begin{aligned} \lim _{n\rightarrow \infty }&\mathrm {E}[(Y_{i}-\mathrm {E}[Y_{i}|{\varvec{\varTheta }}_{n}])^{2}\text {I}_{\{|Y_{i}-\mathrm {E}[Y_{i}|{\varvec{\varTheta }}_{n}]|>\varepsilon S_{n}\}}|{\varvec{\varTheta }}_{n}] =0 \end{aligned}$$

(43)

for any $\varepsilon >0$. By combining (42) and (43), we have

$$\begin{aligned} \lim _{n\rightarrow \infty }&\frac{1}{S^{2}_{n}}\sum _{i=1}^{n}\mathrm {E}[(h^{1/2}c_{i}Y_{i}-\mathrm {E}[h^{1/2}c_{i}Y_{i}|{\varvec{\varTheta }}_{n}])^{2}\text {I}_{\{|h^{1/2}c_{i}Y_{i}-\mathrm {E}[h^{1/2}c_{i}Y_{i}|{\varvec{\varTheta }}_{n}]|>\varepsilon S_{n}\}}|{\varvec{\varTheta }}_{n}] =0 \end{aligned}$$

(44)

for any $\varepsilon >0$. Equation (44) indicates that the Lindeberg condition for $h^{1/2}c_{i}Y_{i}$ holds. By combining (42) and the central limit theorem, we obtain

$$\begin{aligned} \frac{n^{1/2}h^{1/2}}{\sqrt{v(\theta )\mu _{0}(\bar{L}_{(p)}^{2})/f(\theta )}}&[\hat{m}(\theta ;p,h)-\mathrm {E}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]]\nonumber \\&=\frac{1}{S_{n}}\sum _{i=1}^{n}\{h^{1/2}c_{i}Y_{i}-\mathrm {E}[h^{1/2}c_{i}Y_{i}|{\varvec{\varTheta }}_{n}]\}\nonumber \\&{\mathop {\longrightarrow }\limits ^{d}} \text {N}(0,1), \end{aligned}$$

(45)

as $n\rightarrow \infty$.

Thus, we have

$$\begin{aligned} n^{1/2}h^{1/2}[\hat{m}(\theta ;p,h)-m(\theta )]&=n^{1/2}h^{1/2}[\hat{m}(\theta ;p,h)-\mathrm {E}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]]\nonumber \\&\quad +n^{1/2}h^{1/2}\mathrm {Bias}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]. \end{aligned}$$

(46)

When p is odd, we obtain $n^{1/2}h^{1/2}\mathrm {Bias}[\hat{m}(\theta ;p,h)|{\varvec{\varTheta }}_{n}]=O_{p}(n^{\{1+\gamma (2p+3)\}/2})$ from (6). Thus, the second term on the RHS of (46) vanishes when $\gamma < -1/(2p+3)$. Therefore, if $\gamma <-1/(2p+3)$ and $n\rightarrow \infty$, it then follows that

$$\begin{aligned} n^{1/2}h^{1/2}[\hat{m}(\theta ;p,h)-m(\theta )]{\mathop {\longrightarrow }\limits ^{d}} \text {N}(0,v(\theta )\mu _{0}(\bar{L}_{(p)}^{2})/f(\theta )). \end{aligned}$$

(47)

When p is even, we find that (47) holds if $\gamma <-1/(2p+5)$ and $n\rightarrow \infty$ use the same procedure applied when p is odd. $\square$