A smooth simultaneous confidence band for conditional variance function

Abstract

A smooth simultaneous confidence band (SCB) is obtained for heteroscedastic variance function in nonparametric regression by applying spline regression to the conditional mean function followed by Nadaraya–Waston estimation using the squared residuals. The variance estimator is uniformly oracally efficient, that is, it is as efficient as, up to order less than $n^{-1/2}$, the infeasible kernel estimator when the conditional mean function is known, uniformly over the data range. Simulation experiments provide strong evidence that confirms the asymptotic theory while the computing is extremely fast. The proposed SCB has been applied to test for heteroscedasticity in the well-known motorcycle data and Old Faithful geyser data with different conclusions.

Appendix A

Throughout this Appendix, we denote by $\left\| \xi \right\|$ the Euclidean norm and $\left| \xi \right|$ means the largest absolute value of the elements of any vector $\xi$. We use $c$, $C$ to denote any positive constants in the generic sense. We denote for any given constant $C>0$, a class of Lipschitz continuous functions by $\text {Lip}\left( \left[ 0,1\right] ,C\right) =\left\{ \varphi \left| \left| \varphi \left( x\right) -\varphi \left( x^{\prime }\right) \right| \le C\left| x-x^{\prime }\right| \text {, }\forall x,x^{\prime }\in \left[ 0,1\right] \right. \right\}$.

1.1 A.1 Preliminaries

The Lemmas of this Subsection are needed for the proof of Propositions 1, 2 and 3. These Propositions clearly establish Theorems 1 and 2.

Lemma 1

Under Assumptions (A1)–(A5), there exists a constant $C_{p}>0$, $p>1$, such that for any $m\in C^{p}\left[ 0,1\right]$ there is a spline function $g_{p}\in G_{N}^{\left( p-2\right) }$ satisfying $\left\| m-g_{p}\right\| _{\infty }$ $\le CH^{p}$ and $m-g_{p}\in \text {Lip}\left( \left[ 0,1\right] ,CH^{p-1}\right)$. The function $\tilde{m}_{p}\left( x\right)$ given in Equation (12)

$$\begin{aligned} \left\| \tilde{m}_{p}\left( x\right) -m\left( x\right) \right\| _{\infty }\le C_{p}\underset{g\in G_{N}^{\left( p-2\right) }}{\inf }\ \left\| g-m\right\| _{\infty }=\mathcal {O}_{p}\left( H^{p}\right) . \end{aligned}$$

Moreover, for the function $\tilde{\varepsilon }_{p}\left( x\right)$ given in Equation (12)

$$\begin{aligned} \left\| \tilde{\varepsilon }_{p}\left( x\right) \right\| _{2,n}= \mathcal {O}_{p}\left( n^{-1/2}N^{1/2}\right) ,\left\| \tilde{ \varepsilon }_{p}\left( x\right) \right\| _{\infty }=\mathcal {O}_{p}\left( n^{-1/2}N^{1/2}{(\log n)}^{1/2}\right) . \end{aligned}$$

See Lemma A.1 of Song and Yang (2009), and also Wang and Yang (2009) for detailed proof.

Lemma 2

Under Assumption (A6), as $n\rightarrow \infty$,

$$\begin{aligned} \underset{x\in \left[ 0,1\right] }{\sup }\left\{ \left| \left\{ B_{j,p}\left( x\right) \right\} _{j=1-p}^{N}\right| +\left\| \left\{ B_{j,p}\left( x\right) \right\} _{j=1-p}^{N}\right\| \right\} =\mathcal {O} \left( H^{-1/2}\right) \end{aligned}$$

(21)

See Lemma A.4 of Song and Yang (2009) for detailed proof.

The strong approximation result of Tusnády (1977) is also needed.

Lemma 3

Let $U_{1},\ldots ,U_{n}$ be i.i.d. r.v.’s on the $2$ -dimensional unit square with $P\left( U_{i}<\mathbf {t}\right) =\lambda \left( \mathbf {t}\right) ,\mathbf {0\le t\le 1,}$ where $\mathbf {t=(} t_{1,}t_{2}\mathbf {)}$ and $\mathbf {1=(}1,1\mathbf {)}$ are $2$-dimensional vectors, $\lambda \left( \mathbf {t}\right) =t_{1}t_{2}.$ The empirical distribution function $F_{n}^{u}\left( \mathbf {t}\right)$ based on sample $\left( U_{1},\ldots ,U_{n}\right)$ is defined as $F_{n}^{u}\left( \mathbf {t} \right) =n^{-1}\sum _{i=1}^{n}\mathrm{I}_{\left\{ U_{i}<\mathbf {t}\right\} }$ for $\mathbf {0}\le \mathbf {t\le 1.}$ The $2$-dimensional Brownian bridge $B\left( \mathbf {t}\right)$ is defined by $B\left( \mathbf {t}\right) =W\left( \mathbf {t}\right) -\lambda \left( \mathbf {t}\right) W\left( \mathbf { 1}\right)$ for $\mathbf {0}\le \mathbf {t\le 1}$, where $W\left( \mathbf {t} \right)$ is a $2$-dimensional Wiener process. Then there is a version $B_{n}\left( \mathbf {t}\right)$ of $B\left( \mathbf {t}\right)$ such that

$$\begin{aligned} P\left[ \sup _{\mathbf {0\le t\le 1}}\left| n^{1/2}\left\{ F_{n}^{u}\left( \mathbf {t}\right) -\lambda \left( \mathbf {t}\right) \right\} -B_{n}\left( \mathbf {t}\right) \right| >\left( C\log n+x\right) \frac{ \log n}{n^{1/2}}\right] <Ke^{-\lambda x} \end{aligned}$$

holds for all $x$, where $C,K,$ $\lambda$ are positive constants.

Denote the well-known Rosenblatt transformation for bivariate continuous $\left( X,\varepsilon \right)$ as

$$\begin{aligned} \left( X^{\prime },\varepsilon ^{\prime }\right) =M\left( X,\varepsilon \right) =\left\{ F_{X}\left( x\right) ,F_{\varepsilon |X}\left( \varepsilon |x\right) \right\} , \end{aligned}$$

(22)

so that $\left( X^{\prime },\varepsilon ^{\prime }\right)$ has uniform distribution on $\left[ 0,1\right] ^{2}$, therefore

$$\begin{aligned} Z_{n}\left\{ M^{-1}\left( x^{\prime },\varepsilon ^{\prime }\right) \right\} =Z_{n}\left( x,\varepsilon \right) =\sqrt{n}\left\{ F_{n}\left( x,\varepsilon \right) -F\left( x,\varepsilon \right) \right\} , \end{aligned}$$

(23)

with $F_{n}\left( x,\varepsilon \right)$ denoting the empirical distribution of $\left( X,\varepsilon \right)$. Lemma 3 implies that there exists a version $B_{n}$ of $2$-dimensional Brownian bridge such that

$$\begin{aligned} \sup _{x,\varepsilon }\left| Z_{n}\left( x,\varepsilon \right) -B_{n}\left\{ M\left( x,\varepsilon \right) \right\} \right| =\mathcal {O} _{a.s.}\left( n^{-1/2}\log ^{2}n\right) . \end{aligned}$$

(24)

Lemma 4

Under Assumptions (A2)-(A5), as $n\rightarrow \infty$, for any sequence of functions $r_{n}$ $\in \text {Lip}\left( \left[ 0,1\right] ,l_{n}\right) ,l_{n}>0$ with ${\left\| r_{n}\right\| _{\infty }=\rho }_{n}\ge 0$

$$\begin{aligned} {n}^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \varepsilon _{i}=\mathcal {\ U}_{p}\left( n^{-1/2}h^{-1/2}{\rho }_{n}\log ^{1/2}n+n^{-1/2}h^{1/2}l_{n}\right) \nonumber \\ \end{aligned}$$

(25)

Proof

Step 1. We first discretize the problem by letting $0=x_{0}<x_{1}\,<\cdots <x_{M_{n}}=1,M_{n}=n^{4}$ by equally spaced points, the smoothness of kernel $K$ in Assumption (A4) imples that

$$\begin{aligned}&\!\!\!\underset{x\in \left[ 0,1\right] }{\sup }\left| n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \varepsilon _{i}\right| \le \underset{0\le j\le M_{n}}{\max } \left| n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x_{j}\right) r_{n}\left( X_{i}\right) \varepsilon _{i}\right| \\&\!\!\!\qquad +\underset{0\le j<M_{n}}{\max }\sup _{x\in \left[ x_{j},x_{j+1}\right] }\left| n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x_{j}\right) r_{n}\left( X_{i}\right) \varepsilon _{i}-n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \varepsilon _{i}\right| \\&\!\!\!\quad \le \underset{0\le j<M_{n}}{\max }\left| n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x_{j}\right) r_{n}\left( X_{i}\right) \varepsilon _{i}\right| +CM_{n}^{-1}h^{-2}n^{-1}\sum _{i=1}^{n}\left| r_{n}\left( X_{i}\right) \varepsilon _{i}\right| \end{aligned}$$

and the moment conditions on error $\varepsilon$ in Assumption (A2), the rate of $h$ in Assumption (A5) imply next that

$$\begin{aligned}&\underset{x\in \left[ 0,1\right] }{\sup }\left| n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \varepsilon _{i}\right| \nonumber \\&\quad \le \underset{0\le j<M_{n}}{\max }\left| n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x_{j}\right) r_{n}\left( X_{i}\right) \varepsilon _{i}\right| +\mathcal {O}_{p}\left\{ {\rho } _{n}n^{-2}\right\} . \end{aligned}$$

(26)

Step 2. To truncate the error, we denote $D_{n}=n^{\alpha }$ with $\alpha$ as in Assumption (A5). Assumption (A5) implies that $D_{n}n^{-1/2}h^{-1/2} \log ^{2}n\rightarrow 0,$ $n^{1/2}h^{1/2}D_{n}^{-\left( 1+\eta \right) }\rightarrow 0$, $\sum _{n=1}^{\infty }D_{n}^{-\left( 2+\eta \right) }<\infty$. Write $\varepsilon _{i}=\varepsilon _{i,1}^{D_{n}}+\varepsilon _{i,2}^{D_{n}}$, where $\varepsilon _{i,1}^{D_{n}}=\varepsilon _{i}\mathrm{I}\left\{ \left| \varepsilon _{i}\right| >D_{n}\right\} ,\varepsilon _{i,2}^{D_{n}}=\varepsilon _{i}\mathrm{I}\left\{ \left| \varepsilon _{i}\right| \le D_{n}\right\}$, and denote $\mu ^{D_{n}}\left( x\right) =\mathsf {E}\left\{ \varepsilon _{i}\mathrm{I}\left\{ \left| \varepsilon _{i}\right| \le D_{n}\right\} \mid X_{i}=x\right\}$. One immediately obtains that

$$\begin{aligned} \underset{x\in \left[ 0,1\right] }{\sup }\left| \mu ^{D_{n}}\left( x\right) \right| \le \mathsf {E}\left\{ \left| \varepsilon \right| ^{2+\eta }\mid X=x\right\} D_{n}^{-\left( 1+\eta \right) }=o\left( n^{-1/2}h^{-1/2}\right) , \nonumber \\ \underset{x\in \left[ 0,1\right] }{\sup }\left| \mathsf {E} n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \mu ^{D_{n}}\left( X_{i}\right) \right| =o\left( n^{-1/2}h^{-1/2}{\rho } _{n}\right) . \end{aligned}$$

(27)

Next, since $P\left( \left| \varepsilon _{i}\right| >D_{n}\right) \le \mathsf {E}\left| \varepsilon \right| ^{2+\eta }D_{n}^{-(2+\eta )}$, $\sum \nolimits _{n=1}^{\infty }P\left( \left| \varepsilon _{n}\right| >D_{n}\right) \le$ $\mathsf {E}\left| \varepsilon \right| ^{2+\eta }\sum \nolimits _{n=1}^{\infty }D_{n}^{-\left( 2+\eta \right) }<+\infty$, Borel–Cantelli Lemma then implies that

$$\begin{aligned} P\left\{ \omega \mid \exists N\left( \omega \right) ,\varepsilon _{i,1}^{D_{n}}\left( \omega \right) =0,i=1,2,\ldots n,\text {for }n>N\left( \omega \right) \right\} =1. \end{aligned}$$

So one has for any $k>0$

$$\begin{aligned} \underset{x\in \left[ 0,1\right] }{\sup }{n}^{-1}\left| \sum \nolimits _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \varepsilon _{i,1}^{D_{n}}\right| =\mathcal {O}_{a.s.}\left( n^{-k}{\rho } _{n}\right) . \end{aligned}$$

(28)

Step 3. The truncated sum $n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}\!-\!x\right) r_{n}\left( X_{i}\right) \varepsilon _{i,2}^{D_{n}}$ equals $\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u\!-\!x\right) r_{n}\left( u\right) \varepsilon dF_{n}\left( u,\varepsilon \right)$, while

$$\begin{aligned}&\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dF\left( u,\varepsilon \right) =\mathsf {E} n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \varepsilon _{i,2}^{D_{n}} \\&\quad =\mathsf {E}n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \mu ^{D_{n}}\left( X_{i}\right) =u\left( n^{-1/2}h^{-1/2}{\rho } _{n}\right) , \end{aligned}$$

according to (27). The above two Equations imply that

$$\begin{aligned} n^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) r_{n}\left( X_{i}\right) \varepsilon _{i,2}^{D_{n}}&= n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dZ_{n}\left( u,\varepsilon \right) \nonumber \\&+u\left( n^{-1/2}h^{-1/2}{\rho }_{n}\right) . \end{aligned}$$

(29)

Step 4. The term $n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dZ_{n}\left( u,\varepsilon \right)$ equals

$$\begin{aligned}&-n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}d\left\{ K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon \right\} Z_{n}\left( u,\varepsilon \right) \\&\quad =n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}d\left\{ K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon \right\} \left[ B_{n}\left\{ M\left( u,\varepsilon \right) \right\} -Z_{n}\left( u,\varepsilon \right) \right] \\&\qquad -n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}d\left\{ K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon \right\} B_{n}\left\{ M\left( u,\varepsilon \right) \right\} . \end{aligned}$$

Note that

$$\begin{aligned}&n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}d\left\{ K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon \right\} \left[ B_{n}\left\{ M\left( u,\varepsilon \right) \right\} -Z_{n}\left( u,\varepsilon \right) \right] \nonumber \\&\quad =n^{-1/2}\!\int _{\left| \varepsilon \right| \le D_{n}}\!\left\{ dK_{h}\left( u\!-\!x\right) \!r_{n}\left( u\right) \!+\!K_{h}\left( u\!-\!x\right) dr_{n}\!\left( u\right) \right\} d\varepsilon \left[ B_{n}\!\left\{ M\left( u,\varepsilon \right) \right\} \!-\!Z_{n}\left( u,\varepsilon \right) \right] \nonumber \\&\quad =U_{a.s.}\left\{ n^{-1/2}\log ^{2}n\times D_{n}\left( h^{-1}{\rho } _{n}+l_{n}\right) n^{-1/2}\right\} \nonumber \\&\quad =U_{a.s.}\left\{ D_{n}n^{-1}h^{-1}\log ^{2}n\left( {\rho } _{n}+hl_{n}\right) \right\} \nonumber \\&\quad =u_{a.s.}\left\{ n^{-1/2}h^{-1/2}\left( {\rho }_{n}+hl_{n}\right) \right\} \end{aligned}$$

(30)

by the growth constraint on $D_{n}=n^{\alpha }$. Note also that

$$\begin{aligned}&-n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}d\left\{ K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon \right\} B_{n}\left\{ M\left( u,\varepsilon \right) \right\} \\&\quad =n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dB_{n}\left\{ M\left( u,\varepsilon \right) \right\} \\&\quad =n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dW_{n}\left\{ M\left( u,\varepsilon \right) \right\} \\&\qquad -n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dF\left( \varepsilon \left| u\right. \right) f\left( u\right) duW_{n}\left( 1,1\right) \end{aligned}$$

in which

$$\begin{aligned}&\left| n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dF\left( \varepsilon \left| u\right. \right) f\left( u\right) duW_{n}\left( 1,1\right) \right| \nonumber \\&\quad \le n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}\left| \varepsilon \right| dF\left( \varepsilon \left| u\right. \right) \int K_{h}\left( u-x\right) \left| r_{n}\left( u\right) \right| f\left( u\right) du\left| W_{n}\left( 1,1\right) \right| \nonumber \\&\quad \le n^{-1/2}\mathsf {E}\left| \varepsilon \right| ^{2+\eta }D_{n}^{-\left( 1+\eta \right) }{\rho }_{n}C_{f}\left| W_{n}\left( 1,1\right) \right| =U_{p}\left( n^{-1/2}D_{n}^{-\left( 1+\eta \right) }{ \rho }_{n}\right) . \qquad \end{aligned}$$

(31)

Meanwhile

$$\begin{aligned}&\mathsf {E}\left[ n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) r_{n}\left( u\right) \varepsilon dW_{n}\left\{ M\left( u,\varepsilon \right) \right\} \right] ^{2} \\&\quad =n^{-1}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x\right) ^{2}r_{n}^{2}\left( u\right) \varepsilon ^{2}dF\left( \varepsilon \left| u\right. \right) f\left( u\right) du \\&\quad =n^{-1}\int \left\{ \int _{\left| \varepsilon \right| \le D_{n}}\varepsilon ^{2}dF\left( \varepsilon \left| u\right. \right) \right\} K_{h}\left( u-x\right) ^{2}r_{n}^{2}\left( u\right) f\left( u\right) du \\&\quad \le n^{-1}\int K_{h}\left( u-x\right) ^{2}r_{n}^{2}\left( u\right) \sigma ^{2}\left( u\right) f\left( u\right) du\le n^{-1}h^{-1}{\rho } _{n}^{2}C_{\sigma }^{2}C_{f}, \end{aligned}$$

so the $M_{n}$ Gaussian variables $n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x_{j}\right) r_{n}\left( u\right) \varepsilon dW_{n}\left\{ M\left( u,\varepsilon \right) \right\} ,0\le j<M_{n}$ each has variance less than $n^{-1}h^{-1}{\rho }_{n}^{2}C_{\sigma }^{2}C_{f}$, hence

$$\begin{aligned}&\underset{0\le j<M_{n}}{\max }\left| n^{-1/2}\int _{\left| \varepsilon \right| \le D_{n}}K_{h}\left( u-x_{j}\right) r_{n}\left( u\right) \varepsilon dW_{n}\left\{ M\left( u,\varepsilon \right) \right\} \right| \nonumber \\&\quad =\mathcal {O}_{p}\left( n^{-1/2}h^{-1/2}{\rho }_{n}\log ^{1/2}n\right) . \end{aligned}$$

(32)

Finally, putting together Eqs. (26), (28 ), (29), (30), (31 ) and (32) proves the Lemma.

1.2 A.2 Proof of Propositions

Proof of Proposition

1 It is obvious that $\left| \mathrm{I}_{i,p}\right| \le 2\left\{ \tilde{m}_{p}\left( X_{i}\right) -m\left( X_{i}\right) \right\} ^{2}+2\tilde{\varepsilon }_{p}^{2}\left( X_{i}\right) .$ Meanwhile applied Lemma 1, $\left\| m-\tilde{m}_{p}\right\| _{2,n}^{2}\le \left\| m-\tilde{m} _{p}\right\| _{\infty }^{2}=\mathcal {O}_{p}\left( H^{2p}\right)$, $\left| \mathrm{I}\right|$ is bounded by

$$\begin{aligned}&2{n}^{-1}{h}^{-1}c^{-1}\sum \nolimits _{i=1}^{n}\left| K\left\{ (X_{i}-x)/h\right\} \right| \left\{ \left( m\left( X_{i}\right) -\tilde{m }_{p}\left( X_{i}\right) \right) ^{2}+\tilde{\varepsilon }_{p}^{2}\left( X_{i}\right) \right\} \\&\quad \le 2{h}^{-1}c^{-1}\left\| K\right\| _{\infty }\left\{ \left\| m- \tilde{m}_{p}\right\| _{2,n}^{2}+\left\| \tilde{\varepsilon } _{p}^{2}\right\| _{2,n}^{2}\right\} \\&\quad \le 2c^{-1}\left\| K\right\| _{\infty }\left\{ h^{-1}\left( H^{2p}+n^{-1}H^{-1}\right) \right\} . \end{aligned}$$

Proof of Proposition

2 By (12), $\tilde{\varepsilon }_{p}( X_{i}) =\sum _{J=1-p}^{N}\tilde{a}_{J,p}B_{J,p}( X_{i})$, Lemma 1 and Wang and Yang (2009) entail that $(\sum _{J=1-p}^{N}\tilde{a}_{J,p}^{2}) ^{1/2}=\mathcal {O}_{p}(\Vert \tilde{\varepsilon }_{p}(x) \Vert _{2,n}) =\mathcal {O}_{p}(n^{-1/2}N^{1/2})$. Set $r_{n}\left( x\right) =B_{J,p}\left( x\right)$, then Lemma 2 entails that $\rho _{n}=\mathcal {O}\left( H^{-1/2}\right)$ and it is easy to verify that $l_{n}=\mathcal {O}\left( H^{-3/2}\right)$. Applying Lemma 4, one obtains that

$$\begin{aligned}&{n}^{-1}\sum \limits _{i=1}^{n}K_{h}\left( X_{i}-x\right) \varepsilon _{i}B_{J,p}\left( X_{i}\right) \nonumber \\&\quad =\mathcal {U}_{p}\left( n^{-1/2}h^{-1/2}H^{-1/2}\log ^{1/2}n+n^{-1/2}h^{1/2}H^{-3/2}\right) \end{aligned}$$

(33)

and hence

$$\begin{aligned} \left| \mathrm{II}\left( x\right) \right|&= \left| {n} ^{-1}\sum \limits _{i=1}^{n}K_{h}\left( X_{i}-x\right) 2\varepsilon _{i} \tilde{\varepsilon }_{p}\left( X_{i}\right) \right| \\&= \left| 2{n}^{-1}\sum \limits _{i=1}^{n}K_{h}\left( X_{i}-x\right) \varepsilon _{i}\sum \limits _{J=1-p}^{N}\tilde{a}_{J,p}B_{J,p}\left( X_{i}\right) \right| \\&\le c^{-1}\left\{ \sum \limits _{J=1-p}^{N}\tilde{a}_{J,p}^{2}\sum \limits _{J=1-p}^{N}\left\{ 2{n}^{-1}\sum \limits _{i=1}^{n}K_{h}\left( X_{i}-x\right) \varepsilon _{i}B_{J,p}\left( X_{i}\right) \right\} ^{2}\right\} ^{1/2} \\&= \mathcal {O}_{p}\left( n^{-1/2}N^{1/2}\right) \times \left( N+p\right) ^{1/2}\\&\,\times \mathcal {U}_{p}\left( n^{-1/2}h^{-1/2}H^{-1/2}\log ^{1/2}n+n^{-1/2}h^{1/2}H^{-3/2}\right) \\&= \mathcal {U}_{p}\left( n^{-1}h^{-1/2}H^{-3/2}\log ^{1/2}n+n^{-1}h^{1/2}H^{-5/2}\right) , \end{aligned}$$

the lemma is proved.

Proof of Proposition

3

$$\begin{aligned} \mathrm{III}\left( x\right)&= 2{n}^{-1}\sum \limits _{i=1}^{n}K_{h}\left( X_{i}-x\right) \left\{ \left( m\left( X_{i}\right) -g_{p}\left( X_{i}\right) \right) \varepsilon _{i}\right\} \\&+2{n}^{-1}\sum \limits _{i=1}^{n}K_{h}\left( X_{i}-x\right) \left\{ \left( g_{p}\left( X_{i}\right) -\tilde{m}_{p}\left( X_{i}\right) \right) \varepsilon _{i}\right\} \end{aligned}$$

in which the spline function $g_{p}\in G_{N}^{\left( p-2\right) }$ satisfies $\left\| m-g_{p}\right\| _{\infty }$ $\le CH^{p},m-g_{p}\in \text {Lip }\left( \left[ 0,1\right] ,CH^{p-1}\right)$ as in Lemma 1. Set $r_{n}\left( x\right) =m\left( x\right) -g_{p}\left( x\right)$, then $\rho _{n}=\mathcal {O}\left( H^{p}\right) ,l_{n}=\mathcal {O} \left( H^{p-1}\right)$, so applying Lemma 4 yields

$$\begin{aligned}&{n}^{-1}\sum _{i=1}^{n}K_{h}\left( X_{i}-x\right) \left\{ \left( m\left( X_{i}\right) -g_{p}\left( X_{i}\right) \right) \varepsilon _{i}\right\} \nonumber \\&\quad = \mathcal {U}_{p}\left\{ {(nh/\log n)}^{-1/2}H^{p}+n^{-1/2}h^{1/2}H^{p-1} \right\} . \end{aligned}$$

(34)

Denoting $g_{p}\left( x\right) -\tilde{m}_{p}\left( x\right) =\sum _{J=1-p}^{N}\gamma _{J,p}B_{J,p}\left( x\right)$ and applying Lemma 1, one has

$$\begin{aligned} \left( \sum \nolimits _{J=1-p}^{N}\gamma _{J,p}^{2}\right) ^{1/2}\le C\left\| g_{p}\left( x\right) -\tilde{m}_{p}\left( x\right) \right\| _{2}=\mathcal {O}\left( H^{p}\right) , \end{aligned}$$

which, together with (33) imply that

$$\begin{aligned}&\left| 2{n}^{-1}\sum \limits _{i=1}^{n}K_{h}\left( X_{i}-x\right) \left\{ g_{p}\left( X_{i}\right) -\tilde{m}_{p}\left( X_{i}\right) \right\} \varepsilon _{i}\right| \\&\quad =\left| 2{n}^{-1}\sum \limits _{J=1-p}^{N}\gamma _{J,p}\sum \nolimits _{i=1}^{n}K_{h}\left( X_{i}-x\right) B_{J,p}\left( X_{i}\right) \varepsilon _{i}\right| \\&\quad \le \left( \sum \limits _{J=1-p}^{N}\gamma _{J,p}^{2}\right) ^{1/2}\left[ \sum \limits _{J=1-p}^{N}\left\{ 2{n}^{-1}\sum \nolimits _{i=1}^{n}K_{h}\left( X_{i}-x\right) B_{J,p}\left( X_{i}\right) \varepsilon _{i}\right\} ^{2} \right] ^{1/2} \\&\quad =\mathcal {O}\left( H^{p}\right) \times \mathcal {O}\left( N^{1/2}\right) \times \mathcal {U}_{p}\left( n^{-1/2}h^{-1/2}H^{-1/2}\log ^{1/2}n+n^{-1/2}h^{1/2}H^{-3/2}\right) \\&\quad =\mathcal {U}_{p}\left( n^{-1/2}h^{-1/2}H^{p-1}\log ^{1/2}n+n^{-1/2}h^{1/2}H^{p-2}\right) , \end{aligned}$$

which, together with (34), prove the lemma.