Unified estimation of densities on bounded and unbounded domains

Abstract

Kernel density estimation in domains with boundaries is known to suffer from undesirable boundary effects. We show that in the case of smooth densities, a general and elegant approach is to estimate an extension of the density. The resulting estimators in domains with boundaries have biases and variances expressed in terms of density extensions and extension parameters. The result is that they have the same rates at boundary and interior points of the domain. Contrary to the extant literature, our estimators require no kernel modification near the boundary and kernels commonly used for estimation on the real line can be applied. Densities defined on the half-axis and in a unit interval are considered. The results are applied to estimation of densities that are discontinuous or have discontinuous derivatives, where they yield the same rates of convergence as for smooth densities on ${\mathbb {R}}$.

Appendix: Proofs

Proof of Theorem 1

(1) By the IID assumption

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)= & {} \frac{1}{h^{m+1}}E\left[ K^{(m)}\left( \frac{x-X_{1}}{h} \right) +\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}K^{(m)}\left( \frac{x+X_{1}/w_{j} }{h}\right) \right] \nonumber \\= & {} \frac{1}{h^{m+1}}\left[ \int _{0}^{\infty }K^{(m)}\left( \frac{x-t}{h} \right) f(t)\mathrm{d}t\right. \nonumber \\&\left. +\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\int _{0}^{\infty }K^{(m)}\left( \frac{x+t/w_{j}}{h}\right) f(t)\mathrm{d}t\right] . \end{aligned}$$

(30)

In the first integral let $u=\frac{x-t}{h}$, in the others $u=\frac{x+t/w_{j}}{h}$. Then

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)= & {} \frac{1}{h^{m}}\left[ -\int _{x/h}^{-\infty }K^{(m)}\left( u\right) f(x-hu)\mathrm{d}u\right. \\&\left. +\sum _{j=1}^{s+1}k_{j}\int _{x/h}^{\infty }K^{(m)}(u)f\left( -w_{j}(x-hu)\right) \mathrm{d}u\right] \\= & {} \frac{1}{h^{m}}\left[ \int _{-\infty }^{x/h}K^{(m)}\left( u\right) f(x-hu)\mathrm{d}u\right. \\&\left. +\int _{x/h}^{\infty }K^{(m)}(u)\sum _{j=1}^{s+1}k_{j}f\left( -w_{j}(x-hu)\right) \mathrm{d}u\right] . \end{aligned}$$

In the first integral we have $x-hu>0$ and $f(x-hu)=g_s(x-hu)$; in the second one $x-hu<0,$ so $\sum _{j=1}^{s+1}k_{j}f\left( -w_{j}(x-hu)\right) =g_s(x-hu)$. Hence,

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)=\frac{1}{h^{m}}\int _{{\mathbb {R}}}K^{(m)}\left( u\right) g_s(x-hu)\mathrm{d}u. \end{aligned}$$

(31)

By Assumption 1$\left| K^{(j)}(u)g_s^{(m-1-j)}(x-hu)\right| =o(1),$ as $|u|\rightarrow \infty$ for $j=0,\ldots ,m-1$, $h>0$. Therefore, integration by parts gives (31)

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)= & {} \sum _{j=0}^{m-1} \frac{1}{h^{m-j}} K^{(m-1-j)}(u)g_s^{(j)}(x-hu) \mathop \vert \limits _{-\infty }^{\infty } + \int _{{\mathbb {R}}}K( u) g_s^{(m)}(x-hu)\mathrm{d}u\nonumber \\= & {} \int _{{\mathbb {R}}}K\left( u\right) g_s^{(m)}(x-hu)\mathrm{d}u. \end{aligned}$$

(32)

Since $\int _{\mathbb {R}} K(t)\mathrm{d}t=1$, this implies (6).

(2) Plug the definition of $M_{k}$ in (30) to get

$$\begin{aligned} E{\hat{f}}_{s,k}^{(m)}(x) =&-\frac{1}{C_{2k}^{k}}\sum _{|l|=1}^{k}\frac{(-1)^{l}C_{2k}^{l+k}}{|l|l^{m}h^{m+1}}\left[ \int _{0}^{\infty }K^{(m)}\left( \frac{x-t}{lh}\right) f(t)\mathrm{d}t\right. \nonumber \\&\left. + \sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\int _{0}^{\infty }K^{(m)}\left( \frac{x+t/w_{j}}{lh}\right) f(t)\mathrm{d}t\right] . \end{aligned}$$

(33)

For $l<0$, and after putting $\frac{x-t}{lh} =u$ and $\frac{x+t/w_{j}}{lh}=u$ on the first and second integrals

$$\begin{aligned}&\int _{0}^{\infty }K^{(m)}\left( \frac{x-t}{lh}\right) f(t)\mathrm{d}t+\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\int _{0}^{\infty }K^{(m)}\left( \frac{x+t/w_{j}}{lh}\right) f(t)\mathrm{d}t \\&\quad =-lh\left[ \int _{x/(lh)}^{\infty }K^{(m)}\left( u\right) f(x-lhu)\mathrm{d}u\right. \\&\qquad \left. +\sum _{j=1}^{s+1}k_{j}\int _{-\infty }^{x/(lh)}K^{(m)}\left( u\right) f(-w_{j}(x-lhu))\mathrm{d}u\right] \\&\quad =-lh\int _{{\mathbb {R}}}K^{(m)}\left( u\right) g_s(x-lhu)\mathrm{d}u. \end{aligned}$$

Similarly, we have for $l>0$

$$\begin{aligned}&\int _{0}^{\infty }K^{(m)}\left( \frac{x-t}{lh}\right) f(t)\mathrm{d}t+\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\int _{0}^{\infty }K^{(m)}\left( \frac{x+t/w_{j}}{lh} \right) f(t)\mathrm{d}t\\&\quad =lh\int _{{\mathbb {R}}}K^{(m)}\left( u\right) g_s(x-lhu)\mathrm{d}u. \end{aligned}$$

Therefore, (33) gives

$$\begin{aligned} E{\hat{f}}_{s,k}^{(m)}(x)= & {} -\frac{1}{C_{2k}^{k}}\sum _{|l|=1}^{k}\frac{(-1)^{l}C_{2k}^{l+k}}{(lh)^{m}}\int _{{\mathbb {R}}}K^{(m)}\left( u\right) g_s(x-lhu)\mathrm{d}u \\&\text {(integrating by parts as above)} \\= & {} -\frac{1}{C_{2k}^{k}}\sum _{|l|=1}^{k}(-1)^{l}C_{2k}^{l+k}\int _{{\mathbb {R}}}K\left( u\right) g_s^{(m)}(x-lhu)\mathrm{d}u. \end{aligned}$$

Finally,

$$\begin{aligned}&E{\hat{f}}_{k}^{(m)}(x)-f^{(m)}(x) =-\frac{1}{(-1)^kC_{2k}^{k}} \sum _{|l|=1}^{k}(-1)^{l+k}C_{2k}^{l+k}\int _{{\mathbb {R}}}K\left( u\right) g_s^{(m)}(x-lhu)\mathrm{d}u \\&-\frac{(-1)^{k}C_{2k}^{k}}{(-1)^{k}C_{2k}^{k}}\int _{{\mathbb {R}}}K\left( u\right) g_s^{(m)}(x)\mathrm{d}u =-\frac{1}{(-1)^{k}C_{2k}^{k}}\int _{{\mathbb {R}}}K(u)\Delta _{hu}^{2k}g_s^{(m)}(x)\mathrm{d}u \end{aligned}$$

which is (7). $\square$

Proof of Theorem 2

For part (1), we note that since $\int K(t)\mathrm{d}t =1$ we have from Theorem 1

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)-f^{(m)}(x)=&\int K(t)(g_s^{(m)}(x-ht)-g_s^{(m)}(x))\mathrm{d}t\\ =&\int K(t)\left( g_s^{(m+1)}(x)(-ht)+ \frac{1}{2!}g_s^{(m+2)}(x)(-ht)^2\right. \\&\left. + \cdots + \frac{1}{(s-m)!}g_s^{(s)}(x-th\tau )(-ht)^{s-m} \right) \mathrm{d}t \end{aligned}$$

for some $\tau \in (0,1)$. If $\int _{{\mathbb {R}}}K(t)t^{j}\mathrm{d}t=0$, for $j=1,\ldots ,s-m-1$, then

$$\begin{aligned} |E{\hat{f}}_s^{(m)}(x)-f^{(m)}(x)|&\le \frac{h^{s-m}}{(s-m)!} \int |t|^{s-m}|K(t)||g^{(s)}(x-th\tau )|\mathrm{d}t \le C h^{s-m}, \end{aligned}$$

where the last inequality follows from the assumptions that $\int _{{\mathbb {R}}}\left| K(t)t^{s-m}\right| \mathrm{d}t<\infty$, $\left| f^{(s)}(x)\right| <C$ for all $x\ge 0$ and the structure of $g^{(s)}$. For part 2), using (7) and Hölder’s inequality we have

$$\begin{aligned} \left| E{\hat{f}}_{s,k}^{(m)}(x)-f^{(m)}(x)\right|= & {} c\left| \int _{{\mathbb {R}}}K(t)\left| ht\right| ^{r+1/q}\frac{\Delta _{ht}^{2k}g_s^{(m)}(x)}{\left| ht\right| ^{r+1/q}}\mathrm{d}t\right| \nonumber \\\le & {} c\left( \int _{{\mathbb {R}}}|K(t)|^{q^{\prime }}\left| ht\right| ^{(r+1/q)q^{\prime }}\mathrm{d}t\right) ^{1/q^{\prime }}\\&\times \left( \int _{{\mathbb {R}}}\left( \frac{ \mathop {\text {sup}}\limits _{x}|\Delta _{ht}^{2k}g_s^{(m)}(x)|}{\left| ht\right| ^{r}}\right) ^{q}\frac{\mathrm{d}t}{\left| ht\right| }\right) ^{1/q} \nonumber \\&\text {(changing variables on the second integral)} \nonumber \\\le & {} ch^{r}\left( \int |K(t)|^{q^{\prime }}\left| t\right| ^{(r+1/q)q^{\prime }}\mathrm{d}t\right) ^{1/q^{\prime }}\left\| g_s^{(m)}\right\| _{b_{\infty ,q}^{r}({\mathbb {R}})}\\= & {} O(h^{r}). \end{aligned}$$

In the last line, we used the bound $\left\| g_s^{(m)}\right\| _{B_{p,q}^{r}({\mathbb {R}})}\le c\left\| f^{(m)}\right\| _{B_{p,q}^{r}(0,\infty )}$. $\square$

Proof of Theorem 4

(1) Let $I_{A}$ denote the indicator of a set A. Then, for an arbitrary function g, $\sum _{X_{i}<aw_{j}}g(X_{i})=\sum _{i=1}^{n}I_{\{X_{i}<aw_{j}\}}g(X_{i})$. Using indicators in (15) and the fact that $\{X_i\}_{i=1}^n$ is IID, we have

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)= & {} \frac{1}{h^{m+1}}\left\{ \int _{0}^{1}K^{(m)}\left( \frac{x-t}{h}\right) f(t)\mathrm{d}t \right. \nonumber \\&+\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\left[ \int _{0}^{aw_{j}}K^{(m)}\left( \frac{x+t/w_{j}}{h}\right) f(t)\mathrm{d}t \right. \nonumber \\&+\left. \left. \int _{1-aw_{j}}^{1}K^{(m)}\left( \frac{x-1+(t-1)/w_{j}}{h} \right) f(t)\mathrm{d}t\right] \right\} . \end{aligned}$$

(34)

Changing variables using $\frac{x-t}{h}=u$, $\frac{x+t/w_{j}}{h}=u$, $\frac{x-1+(t-1)/w_{j}}{h}=u$, we have

$$\begin{aligned} E{\hat{f}}^{(m)}_s(x)= & {} \frac{1}{h^{m}}\left\{ -\int _{x/h}^{(x-1)/h}K^{(m)}\left( u\right) f(x-hu)\mathrm{d}u\right. \\&+\sum _{j=1}^{s+1}k_{j}\left[ \int _{x/h}^{(x+a)/h}K^{(m)}\left( u\right) f(-w_{j}(x-hu))\mathrm{d}u\right. \\&+\left. \left. \int _{(x-1-a)/h}^{(x-1)/h}K^{(m)}\left( u\right) f(1-w_{j}(x-hu-1))\mathrm{d}u\right] \right\} . \end{aligned}$$

Applying (14), we have

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)= & {} \frac{1}{h^{m}}\left\{ \int _{(x-1)/h}^{x/h}K^{(m)}\left( u\right) f(x-hu)\mathrm{d}u\right. \nonumber \\&+\int _{x/h}^{(x+a)/h}K^{(m)}\left( u\right) \sum _{j=1}^{s+1}k_{j}f(-w_{j}(x-hu))\mathrm{d}u \nonumber \\&+\left. \int _{(x-1-a)/h}^{(x-1)/h}K^{(m)}\left( u\right) \sum _{j=1}^{s+1}k_{j}f(1-w_{j}(x-hu-1))\mathrm{d}u\right\} \nonumber \\= & {} \frac{1}{h^{m}}\int _{(x-1-a)/h}^{(x+a)/h}K^{(m)}\left( u\right) g_s(x-hu)\mathrm{d}u. \end{aligned}$$

(35)

Regardless of $x\in [0,1],$ the interval $((x-1-a)/h,(x+a)/h)$ contains $(-a/h,a/h)$ which contains ${\mathrm {supp}} K$ for all small h. Therefore,

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)=\frac{1}{h^{m}}\int _{{\mathbb {R}}}K^{(m)}\left( u\right) g_s(x-hu)\mathrm{d}u. \end{aligned}$$

For this to hold formally, $g_s$ should be extended outside $(-a,1+a)$ smoothly; the manner of extension does not affect the above integral. Finally, integration by parts and the condition $\int _{\mathbb {R}} K(t)\mathrm{d}t=1$ prove the statement.

(2) Since K is assumed to have finite support, we do not need Assumption 1. Calculations done in the proof of Theorem 2 after Eq. (33) include change of variables and integration by parts and can be easily repeated here. $\square$

Proof of Theorem 5

Define

$$\begin{aligned} u_{i} =&\frac{1}{h^{m+1}}\left\{ K^{(m)}\left( \frac{x-X_{i}}{h}\right) +\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\left[ I_{\{X_{i}<aw_{j}\}}K^{(m)}\left( \frac{x+X_{i}/w_{j}}{h}\right) \right. \right. \\&+\left. \left. I_{\{X_{i}>1-aw_{j}\}}K^{(m)}\left( \frac{x-1+(X_{i}-1)/w_{j} }{h}\right) \right] \right\} . \end{aligned}$$

Then $V\left( {\hat{f}}_{s}^{(m)}(x)\right) =\frac{1}{n}[Eu_1^2-(Eu_1)^2]$. It will be shown that $Eu_1^2$ is of order $h^{-(2m+1)}$ in all cases. Since $Eu_1=O(1)$ by Theorem 4, it is enough to find the exact order of $Eu_1^2$. Letting $F=K^{(m)}$, denote

$$\begin{aligned} g= & {} F\left( \frac{x-X_{1}}{h}\right) , \quad g_{i}^{l}=I_{\{X_{1}<aw_{i}\}}F\left( \frac{x+X_{1}/w_{i}}{h}\right) , \\ g_{i}^{r}= & {} I_{\{X_{1}>1-aw_{i}\}}F\left( \frac{x-1+(X_{1}-1)/w_{i}}{h} \right) . \end{aligned}$$

g is used at internal points of the domain, $g_{i}^{l}$ and $g_{i}^{r}$ are used for correction at the left and right boundaries, respectively. Their contributions to variances reflect this. From

$$\begin{aligned} u_{1}=\frac{1}{h^{m+1}}\left( g+\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\left( g_{j}^{l}+g_{j}^{r}\right) \right) \end{aligned}$$

we see that $Eu_{1}^{2}$ contains (a) $Eg^{2},$ (b) $Egg_{i}^{l},$ (c) $Eg_{i}^{l}g_{j}^{l},$ (d) $Egg_{j}^{r},$ (e) $Eg_i^{l}g_{j}^{r}$, (f) $Eg_{i}^{r}g_{j}^{r}.$

• (I) Let $x\in (0,1).$

  • (a) Replacing $\frac{x-t}{h}=u,$ we have

    $$\begin{aligned} \frac{1}{h}Eg^{2}=\frac{1}{h}\int _{0}^{1}F^{2}\left( \frac{x-t}{h}\right) f(t)\mathrm{d}t=\int _{(x-1)/h}^{x/h}F^{2}(u)f(x-hu)\mathrm{d}u. \end{aligned}$$

    (36)

    Since $x/h\rightarrow \infty ,$$(x-1)/h\rightarrow -\infty$ and f is bounded and continuous, in the equation

    $$\begin{aligned} \frac{1}{h}Eg^{2}= & {} -\int _{x/h}^{\infty }F^{2}(u)f(x-hu)\mathrm{d}u-\int _{-\infty }^{(x-1)/h}F^{2}(u)f(x-hu)\mathrm{d}u\\&+\int _{\mathbb {R}}F^{2}(u)f(x-hu)\mathrm{d}u \end{aligned}$$

    the first two integrals on the right tend to zero and the last integral tends to $f(x)\Gamma$ by the dominated convergence theorem. Thus,

    $$\begin{aligned} \frac{1}{h}Eg^{2}=f(x)\Gamma +o(1). \end{aligned}$$

    (37)

    Similar arguments below will be omitted.

  • (b) Here we use boundedness of f, F and integrability of F:

    $$\begin{aligned} \left| \frac{1}{h}Egg_{i}^{l}\right|= & {} \left| \frac{1}{h} \int _{0}^{aw_{i}}F\left( \frac{x-t}{h}\right) F\left( \frac{x+t/w_{i}}{h} \right) f(t)\mathrm{d}t\right| \nonumber \\&\left( \text {replacing }\frac{x+t/w_{i}}{h} =u\text { and using dots in place of inconsequential arguments}\right) \nonumber \\= & {} w_{i}\left| \int _{x/h}^{(x+a)/h}F\left( \cdots \right) F\left( u\right) f(\cdots )\mathrm{d}u\right| \le w_{i}\sup \left| fF\right| \int _{x/h}^{(x+a)/h}|F\left( u\right) |\mathrm{d}u\rightarrow 0. \nonumber \\ \end{aligned}$$

    (38)

  • (c) Denoting $\lambda =\min \{w_{i},w_{j}\}$, we have

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{l}g_{j}^{l}= & {} \frac{1}{h}\int _{0}^{a\lambda }F\left( \frac{x+t/w_{i}}{h}\right) F\left( \frac{x+t/w_{j}}{h}\right) f(t)\mathrm{d}t \nonumber \\&\left( \text {replacing }\frac{x+t/w_{i}}{h} =u\right) \nonumber \\= & {} w_{i}\int _{x/h}^{(x+a\lambda /w_{i})/h}F\left( \cdots \right) F\left( u\right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (39)

  • (d) Replacing $\frac{x-1+(t-1)/w_{i}}{h}=u$, we get

    $$\begin{aligned} \frac{1}{h}Egg_{i}^{r}= & {} \frac{1}{h}\int _{1-aw_{i}}^{1}F\left( \frac{x-t}{h} \right) F\left( \frac{x-1+(t-1)/w_{i}}{h}\right) f(t)\mathrm{d}t \nonumber \\= & {} w_{i}\int _{(x-1-a)/h}^{(x-1)/h}F\left( \cdots \right) F\left( u\right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (40)

  • (e) Replacing $\frac{x+t/w_{i}}{h}=u$

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{l}g_{j}^{r}= & {} \frac{1}{h}\int _{1-aw_{j}}^{aw_{i}}F\left( \frac{x+t/w_{i}}{h}\right) F\left( \frac{x-1+(t-1)/w_{j}}{h}\right) f(t)\mathrm{d}t \nonumber \\= & {} w_{i}\int _{[x+(1-aw_{j})/w_{i}]/h}^{(x+a)/h}F\left( u\right) F\left( \cdots \right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (41)

    Here we take into account that $aw_{j}\le 1$ for all j.

  • (f) Letting $\lambda =\min \{w_{i},w_{j}\}$ we have

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{r}g_{j}^{r}= & {} \frac{1}{h}\int _{1-a\lambda }^{1}F\left( \frac{x-1+(t-1)/w_{i}}{h}\right) \nonumber \\&\times F\left( \frac{x-1+(t-1)/w_{j}}{h}\right) f(t)\mathrm{d}t \nonumber \\&\left( \text {replacing }\frac{x-1+(t-1)/w_{i}}{h}=u\right) \nonumber \\= & {} w_{i}\int _{(x-1-a\lambda /w_{i})/h}^{(x-1)/h}F\left( u\right) F\left( \cdots \right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (42)

    The conclusion from (37) to (42) is that $Eu_{1}^{2}=\frac{1}{h^{2m+1}} \left\{ f(x)\Gamma +o(1)\right\}$ which proves statement I).

• (II) Let $x=0.$

  • (a) From (36)

    $$\begin{aligned} \frac{1}{h}Eg^{2}= & {} \int _{-1/h}^{0}F^{2}(u)f(-hu)\mathrm{d}u\rightarrow f(0)\int _{-\infty }^{0}F^{2}(u)\mathrm{d}u\nonumber \\= & {} f(0)\int _{0}^{\infty }F^{2}\left( \frac{u}{ w_{0}}\right) \mathrm{d}u. \end{aligned}$$

    (43)

  • (b) By (38)

    $$\begin{aligned} \frac{1}{h}Egg_{i}^{l}= & {} \frac{1}{h}\int _{0}^{aw_{i}}F\left( -\frac{t}{h} \right) F\left( \frac{t}{hw_{i}}\right) f(t)\mathrm{d}t\ \ \ \ \ \ \ \ \ \left( \text { replacing }\frac{t}{h}=u\right) \nonumber \\= & {} \int _{0}^{aw_{i}/h}F\left( -u\right) F\left( \frac{u}{w_{i}}\right) f(hu)\mathrm{d}u\rightarrow f(0)\nonumber \\&\times \int _{0}^{\infty }F\left( -u\right) F\left( \frac{u}{ w_{i}}\right) \mathrm{d}u \nonumber \\= & {} f(0)\int _{0}^{\infty }F\left( \frac{u}{w_{0}}\right) F\left( \frac{u}{ w_{i}}\right) \mathrm{d}u. \end{aligned}$$

    (44)

  • (c) By (39)

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{l}g_{j}^{l}= & {} \frac{1}{h}\int _{0}^{a\lambda }F\left( \frac{t/w_{i}}{h}\right) F\left( \frac{t/w_{j}}{h}\right) f(t)\mathrm{d}t\ \ \ \ \ \ \left( \text {replacing }\frac{t}{h}=u\right) \nonumber \\= & {} \int _{0}^{a\lambda /h}F\left( \frac{u}{w_{i}}\right) F\left( \frac{u}{ w_{j}}\right) f(hu)\mathrm{d}u\rightarrow f(0)\nonumber \\&\times \int _{0}^{\infty }F\left( \frac{u}{w_{i} }\right) F\left( \frac{u}{w_{i}}\right) \mathrm{d}u. \end{aligned}$$

    (45)

  • (d) By (40)

    $$\begin{aligned} \frac{1}{h}Egg_{j}^{r}=w_{j}\int _{(-1-a)/h}^{-1/h}F\left( \cdots \right) F\left( u\right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (46)

  • (e) From (41)

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{l}g_{j}^{r}= & {} \frac{1}{h}\int _{1-aw_{j}}^{aw_{i}}F\left( \frac{t/w_{i}}{h}\right) F\left( \frac{-1+(t-1)/w_{j}}{h}\right) f(t)\mathrm{d}t \nonumber \\&\left( \text {here we replace }\frac{-1+(t-1)/w_{j}}{h} =u\right) \nonumber \\= & {} w_{j}\int _{(-1-a)/h}^{[-1+(aw_{i}-1)/w_{j}]/h}F\left( \cdots \right) F\left( u\right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (47)

    Here we remember that $aw_{i}\le 1.$

  • (f) From (42)

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{r}g_{j}^{r}=w_{i}\int _{(-1-a\lambda /w_{i})/h}^{-1/h}F\left( u\right) F\left( \cdots \right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (48)

    From (43) to (48), we conclude that

    $$\begin{aligned} Eu_{1}^{2}= & {} \frac{1}{h^{2m+1}}\left\{ f(0)\int _{0}^{\infty }F^{2}\left( \frac{u}{w_{0}}\right) \mathrm{d}u\right. \\&+\,2f(0)\sum _{i=1}^{s+1}\frac{k_{i}}{w_{i}} \int _{0}^{\infty }F\left( \frac{u}{w_{0}}\right) F\left( \frac{u}{w_{i}} \right) \mathrm{d}u\\&\left. +\,f(0)\sum _{i,j=1}^{s+1}\frac{k_{i}}{w_{i}}\frac{k_{j}}{w_{j}} \int _{0}^{\infty }F\left( \frac{u}{w_{i}}\right) F\left( \frac{u}{w_{j}} \right) \mathrm{d}u+o(1)\right\} \\= & {} \frac{1}{h^{2m+1}}\left\{ f(0)\int _{0}^{\infty }\left[ \sum _{i=0}^{s+1} \frac{k_{i}}{w_{i}}F\left( \frac{u}{w_{i}}\right) \right] ^{2}\mathrm{d}u+o(1)\right\} . \end{aligned}$$

• (III) Let $x=1.$

  • (a) From (36)

    $$\begin{aligned} \frac{1}{h}Eg^{2}= & {} \int _{0}^{1/h}F^{2}(u)f(1-hu)\mathrm{d}u\rightarrow f(1)\int _{0}^{\infty }F^{2}\left( u\right) \mathrm{d}u\nonumber \\= & {} f(1)\int _{-\infty }^{0}F^{2}\left( \frac{u}{w_{0}}\right) \mathrm{d}u. \end{aligned}$$

    (49)

  • (b) From the second line of (38)

    $$\begin{aligned} \frac{1}{h}Egg_{i}^{l}=w_{i}\int _{1/h}^{(1+a)/h}F\left( \cdots \right) F\left( u\right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (50)

  • (c) From the last line of (39)

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{l}g_{j}^{l}=w_{i}\int _{1/h}^{(1+a\lambda /w_{i})/h}F\left( \cdots \right) F\left( u\right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (51)

  • (d) From (40)

    $$\begin{aligned} \frac{1}{h}Egg_{i}^{r}= & {} \frac{1}{h}\int _{1-aw_{i}}^{1}F\left( \frac{1-t}{h} \right) \nonumber \\&\times F\left( \frac{(t-1)/w_{i}}{h}\right) f(t)\mathrm{d}t\ \ \ \ \ \ \left( \text { replacing }\frac{t-1}{h}=u\right) \nonumber \\= & {} \int _{-aw_{i}/h}^{0}F\left( -u\right) F\left( \frac{u}{w_{i}}\right) f(1+hu)\mathrm{d}u \nonumber \\\rightarrow & {} f(1)\int _{-\infty }^{0}F\left( -u\right) F\left( \frac{u}{w_{i} }\right) \mathrm{d}u\nonumber \\= & {} f(1)\int _{-\infty }^{0}F\left( \frac{u}{w_{0}}\right) F\left( \frac{u}{w_{i}}\right) \mathrm{d}u. \end{aligned}$$

    (52)

  • (e) From (41)

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{l}g_{j}^{r}= & {} \frac{1}{h}\int _{1-aw_{j}}^{aw_{i}}F\left( \frac{1+t/w_{i}}{h}\right) F\left( \frac{(t-1)/w_{j}}{h}\right) f(t)\mathrm{d}t \nonumber \\&\left( \text {replace }\frac{1+t/w_{i}}{h} =u\right) \nonumber \\= & {} w_{i}\int _{[1+(1-aw_{j})/w_{i}]/h}^{(1+a)/h}F\left( u\right) F\left( \cdots \right) f(\cdots )\mathrm{d}u\rightarrow 0. \end{aligned}$$

    (53)

  • (f) From (42)

    $$\begin{aligned} \frac{1}{h}Eg_{i}^{r}g_{j}^{r}= & {} \frac{1}{h}\int _{1-a\lambda }^{1}F\left( \frac{t-1}{hw_{i}}\right) \nonumber \\&\times F\left( \frac{t-1}{hw_{j}}\right) f(t)\mathrm{d}t \ \ \ \ \left( \text {replacing }\frac{t-1}{h} =u\right) \nonumber \\= & {} \int _{-a\lambda /h}^{0}F\left( \frac{u}{w_{i}}\right) F\left( \frac{u}{ w_{j}}\right) f(1+hu)\mathrm{d}u\nonumber \\\rightarrow & {} f(1)\int _{-\infty }^{0}F\left( \frac{u}{ w_{i}}\right) F\left( \frac{u}{w_{j}}\right) \mathrm{d}u. \end{aligned}$$

    (54)

    Collecting nonzero limits from (49), (52), (54)

    $$\begin{aligned} Eu_{1}^{2}= & {} \frac{1}{h^{2m+1}}\left\{ f(1)\int _{-\infty }^{0}F^{2}\left( \frac{u}{w_{0}}\right) \mathrm{d}u\right. \\&+\,2f(1)\sum _{i=1}^{s+1}\frac{k_{i}}{w_{i}} \int _{-\infty }^{0}F\left( \frac{u}{w_{0}}\right) F\left( \frac{u}{w_{i}} \right) \mathrm{d}u\\&\left. +\,f(1)\sum _{i,j=1}^{s+1}\frac{k_{i}}{w_{i}}\frac{k_{j}}{w_{j}} \int _{-\infty }^{0}F\left( \frac{u}{w_{i}}\right) F\left( \frac{u}{w_{i}} \right) \mathrm{d}u+o(1)\right\} \\= & {} \frac{1}{h^{2m+1}}\left\{ f(1)\int _{-\infty }^{0}\left[ \sum _{i=0}^{s+1} \frac{k_{i}}{w_{i}}F\left( \frac{u}{w_{i}}\right) \right] ^{2}\mathrm{d}u+o(1)\right\} . \end{aligned}$$

$\square$

Proof of Theorem 6

• (1) Instead of (30), we have

  $$\begin{aligned} E{\hat{f}}_{+,s}^{(m)}(x)= & {} \frac{1}{h^{m+1}}E\left\{ I_{\{X_{1} \ge 0\}}\left[ K^{(m)}\left( \frac{x-X_{1}}{h}\right) \right. \right. \\&\left. \left. +\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}} K^{(m)}\left( \frac{x+X_{1}/w_{j}}{h}\right) \right] \right\} \\= & {} \frac{1}{h^{m+1}}\left[ \int _{0}^{\infty }K^{(m)}\left( \frac{x-t}{h} \right) f_{+}(t)\mathrm{d}t\right. \\&\left. +\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\int _{0}^{\infty }K^{(m)}\left( \frac{x+t/w_{j}}{h}\right) f_{+}(t)\mathrm{d}t\right] . \end{aligned}$$

  Repeating calculations that led from (30) to (32), we get

  $$\begin{aligned} E{\hat{f}}_{+,s}^{(m)}(x)=\int _{{\mathbb {R}}}K\left( s\right) g_{+,s}^{(m)}(x-hs)ds \end{aligned}$$

  (those calculations did not use the fact that f was a density).

• (2) The proof is similar to that of Theorem 5.

$\square$

Proof of Theorem 7

By the i.i.d. assumption

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)= & {} \frac{1}{h^{m+1}}\left\{ \int _{c}^{d}K^{(m)}\left( \frac{x-t}{h}\right) f_{r}(t)\mathrm{d}t\right. \\&+\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}\left[ \int _{c}^{c+a_1w_{j}}K^{(m)} \left( \frac{w_{j}(x-c)+(t-c)}{w_{j}h}\right) f_{r}(t)\mathrm{d}t\right. \\&\left. \left. +\int _{d-a_1w_{j}}^{d}K^{(m)}\left( \frac{w_{j}(x-d)+(t-d)}{ w_{j}h}\right) f_{r}(t)\mathrm{d}t\right] \right\} . \end{aligned}$$

The obvious changes of variables are:

$$\begin{aligned} \frac{x-t}{h}=u,\ \frac{w_{j}(x-c)+(t-c)}{w_{j}h}=u,\ \frac{w_{j}(x-d)+(t-d) }{w_{j}h}=u. \end{aligned}$$

The mean value becomes

$$\begin{aligned} E{\hat{f}}_s^{(m)}(x)= & {} \frac{1}{h^{m}}\left\{ -\int _{(x-c)/h}^{(x-d)/h}K^{(m)}\left( u\right) f_{r}(x-hu)\mathrm{d}u\right. \\&+\sum _{j=1}^{s+1}k_{j}\left[ \int _{(x-c)/h}^{(x-c+a_1)/h}K^{(m)}\left( u\right) f_{r}(c-w_{j}(x-c)+w_{j}hu)\mathrm{d}u\right. \\&\left. +\left. \int _{(x-d-a_1)/h}^{(x-d)/h}K^{(m)}\left( u\right) f_{r}(d-w_{j}(x-d)+w_{j}hu)\mathrm{d}u\right] \right\} . \end{aligned}$$

Applying (19) we see that this is the same as

$$\begin{aligned} E{\hat{f}}^{(m)}_s(x)= & {} \frac{1}{h^{m}}\left\{ \int _{(x-d)/h}^{(x-c)/h}K^{(m)}\left( u\right) f_{r}(x-hu)\mathrm{d}u\right. \\&+\int _{(x-c)/h}^{(x-c+a_1)/h}K^{(m)}\left( u\right) \sum _{j=1}^{s+1}k_{j}f_{r}(c-w_{j}(x-hu-c))\mathrm{d}u \\&+\left. \int _{(x-d-a_1)/h}^{(x-d)/h}K^{(m)}\left( u\right) \sum _{j=1}^{s+1}k_{j}f_{r}(d-w_{j}(x-hu-d))\mathrm{d}u\right\} \\= & {} \frac{1}{h^{m}}\int _{(x-d-a_1)/h}^{(x-c+a_1)/h}K^{(m)}\left( u\right) g_{r}(x-hu)\mathrm{d}u. \end{aligned}$$

Regardless of $x\in [c,d],$ the interval $((x-d-a)/h,(x-c+a)/h)$ contains $(-a/h,a/h)$ which contains ${\mathrm {supp}}K$ for all small h. Therefore, also integrating by parts,

$$\begin{aligned} E{\hat{f}}^{(m)}_s(x)=\frac{1}{h^{m}}\int _{{\mathbb {R}}}K^{(m)}\left( u\right) g_{r}(x-hu)\mathrm{d}u=\int _{{\mathbb {R}}}K\left( u\right) g_{r}^{(m)}(x-hu)\mathrm{d}u. \end{aligned}$$

The derivation of the expression for variance largely repeats that from Theorem 5. $\square$

Proof of Theorem 8

(21) implies

$$\begin{aligned} \left( \frac{d}{dx}\right) ^{j}{\tilde{f}}^{(m)}(x)\left| _{x=0+}\right.= & {} \sum _{i=0}^{j}C_{j}^{i}\left[ \left( \frac{d}{dx}\right) ^{i}\psi (xh^{-\alpha })\right] \\&\times \left[ \left( \frac{d}{dx}\right) ^{j-i}{\hat{f}} ^{(m)}(x)\right] \left| _{x=0+}\right. =0, \end{aligned}$$

for $j=0,\ldots ,l-m$, so (23) is satisfied.

To prove (24), consider two cases.

Case$xh^{-\alpha }\ge 1.$ (24) follows trivially from (8) because ${\tilde{f}}^{(m)}(x)={\hat{f}}^{(m)}(x).$

Case$xh^{-\alpha }\le 1.$ Obviously, in the equation

$$\begin{aligned} E{\tilde{f}}^{(m)}(x)-f^{(m)}(x)= & {} \psi (xh^{-\alpha })\left[ E{\hat{f}} ^{(m)}(x)-f^{(m)}(x)\right] \\&+\left[ \psi (xh^{-\alpha })-1\right] f^{(m)}(x) \end{aligned}$$

the first term on the right is $O(h^{s-m}),$ and it remains to prove that $\left[ \psi (xh^{-\alpha })-1\right] f^{(m)}(x)=O(h^{s-m}).$ Suppose (22) is true, so that $\alpha =1.$ Then

$$\begin{aligned} f^{(m)}(x)= & {} f^{(m)}(0+)+\cdots +f^{(s)}(0+)\frac{x^{s-m}}{(s-m)!}+o(x^{s-m}) \nonumber \\= & {} o\left( (h^{\alpha })^{s-m}\right) =o(h^{s-m}). \end{aligned}$$

(55)

Suppose (22) is wrong. Then $\alpha =\frac{s-m}{k-m}$ and

$$\begin{aligned} f^{(m)}(x)= & {} f^{(m)}(0+)+\cdots +f^{(k)}(0+)\frac{x^{k-m}}{(k-m)!}+o(x^{k-m}) \nonumber \\= & {} O\left( x^{k-m}\right) =O(h^{s-m}). \end{aligned}$$

(56)

(55) and (56) prove what we need.

To prove (25), consider two cases.

Case$xh^{-\alpha }\ge 1.$ The first part of (25) is obvious because ${\tilde{f}}^{(m)}(x)={\hat{f}}^{(m)}(x).$

Case$xh^{-\alpha }\le 1.$ From (35) it follows that

$$\begin{aligned} \psi (xh^{-\alpha })= & {} \psi (0+)+\cdots +\psi ^{(l-m+1)}(0+)\frac{\left( xh^{-\alpha }\right) ^{(l-m+1)}}{(l-m+1)!}\\= & {} \psi ^{(l-m+1)}(0+)\left( xh^{-\alpha }\right) ^{(l-m+1)} \end{aligned}$$

which proves the second part of (25). $\square$

Proof of Theorem 9

For almost all samples $\min _{i}X_{i}>0$ and for $0<x\le \frac{1 }{3}\min _{i}X_{i}$, one has $\psi (X_{i}/x)=0,$$i=1,\ldots ,n.$ Hence ${\hat{f}} ^{(m)}(x)$ vanishes, together with all its derivatives, in the neighborhood of zero for almost all samples. Following (30), we see that the mean is

$$\begin{aligned} E{\hat{f}}^{(m)}(x)= & {} \frac{1}{h^{m+1}}\int _{0}^{\infty }\left[ K^{(m)}\left( \frac{x-t}{h}\right) \right. \\&\left. +\sum _{j=1}^{s+1}\frac{k_{j}}{w_{j}}K^{(m)}\left( \frac{ x+t/w_{j}}{h}\right) \right] \psi \left( \frac{t}{x}\right) f(t)\mathrm{d}t. \end{aligned}$$

Here the function $f_{x}(t)=f(t)\psi \left( t/x\right)$ has support $\mathrm{supp}f_{x}\subseteq [0,3x].$ Implementing changes applied after (31), including integration by parts, we obtain an analog of (32) with $g_{x}$ instead of g. $g_{x}$ is obtained by replacing f in (2)–(3) by $f_{x}.$ The rest is familiar.

The statement about variance is obtained by repeating the corresponding part of the proof of Theorem 5. $\square$