Distribution function estimation via Bernstein polynomial of random degree

Abstract

The problem of distribution function (df) estimation arises naturally in many contexts. The empirical and the kernel df estimators are well known. There is another df estimator based on a Bernstein polynomial of degree m. For a Bernstein df estimator, plays the same role as the bandwidth in a kernel estimator. The asymptotic properties of the Bernstein estimator has been studied so far assuming m is non random, chosen subjectively. We propose algorithms for data driven choice of m. Such an m is a function of the data, i.e. random. We obtain the convergence rates of a Bernstein df estimator, using a random m, for i.i.d., strongly mixing and a broad class of linear processes. The estimator is shown to be consistent for any stationary, ergodic process satisfying some conditions. Using simulations and analysis of real data the finite sample performance of the different df estimators are compared.

Appendix

Proof of Lemma 1

In this case $||F_n-F||=\sup _{[0,1]}|F_n(x)-F(x)|$. Let us partition the interval [0, 1] into $k_n=[n^{2/3}]+1$ subintervals $J_{ni}=[\frac{(i-1)}{n^{2/3}}, \frac{i}{n^{2/3}}),\ i=1,2,\ldots ,[n^{2/3}],$ and $J_{nk_n}=([n^{2/3}]/n^{2/3},\ 1]$ each of length $n^{-2/3}$ or less (the last interval). Then

$y_{i-1}$ is the lower endpoint of the interval $J_{ni},\ i=,1,\ldots ,k_n$. Under the assumption that F has a bounded density

$$\begin{aligned} \sup _{x\in J_{ni}}|F(x)-F(y_{i-1})|\le \frac{||f||}{n^{2/3}},\ i=1,\ldots ,k_n. \end{aligned}$$

The right-hand side of the above inequality is free of i and goes to zero as n is increased. Therefore there exists N (depending on $\epsilon >0$) such that for $n>N$

$$\begin{aligned} P\left( n^{1/3}\sup _{x\in J_{ni}}|F(x)-F(y_{i-1})|>\epsilon \right) =0,\quad \ x\in J_{ni}, i=1,\ldots ,k_n. \end{aligned}$$

Moreover

$$\begin{aligned}&F_n(y_{i-1})-F(y_{i-1})=\!\frac{1}{n}\sum ^n_{j=1} Z^i_{nj}\\&\text {and}\sup _{x\in J_{ni}}|F_n(x)-F_n(y_{i-1})|\le \frac{1}{n}\sum ^n_{j=1} I\left( X_j\in J_{n,i}\right) \le \frac{1}{n}\sum ^n_{j=1} Y^i_{nj} + \frac{||f||}{n^{2/3}}, \end{aligned}$$

where $Z^i_{nj}\!=\!I\left( X_j\le y_{i-1}\right) - P\left( X_j\!\le \! y_{i-1}\right) ,\ Y^i_{nj}=I\left( X_j\in J_{n,i}\right) - P\left( X_j\in J_{n,i}\right) , j=1,\ldots , n,$ and $i=1,\ldots , k_n$. Therefore for $n>N$,

$$\begin{aligned} P\left( n^{1/3}\sup _{[0,1]}|F_n(x)-F(x)|>\epsilon \right)\le & {} \sum ^{k_n}_{i=1}\left[ P\left( \left| \frac{1}{n}\sum ^n_{j=1} Z^i_{nj}\right| >\frac{\epsilon }{n^{1/3}}\right) \right. \\&\left. +\, P\left( \left| \frac{1}{n}\sum ^n_{j=1} Y^i_{nj}\right| >\frac{\epsilon }{n^{1/3}}\right) \right] \end{aligned}$$

If $\{X_n\}$ is a stationary strongly mixing process with mixing coefficient $\alpha (n)$, then for each i, $\{Z^i_{nj}, j = 1, . . ., n\},\ \{Y^i_{nj}, j = 1, . . ., n\}$ represent strongly mixing stationary sequences of mean zero bounded random variables with a sequence of mixing coefficients bounded by $\alpha (n)$.

Under the stated conditions $\alpha (n)\le \exp (-2cn)$ for some $c > 0$. Under this condition, using Theorem 1 of Merlevède et al. (2009) we get that for $\epsilon > 0$ and $n\ge 4$,

$$\begin{aligned}&P\left( \left| \frac{1}{n}\sum ^n_{j=1} Z^i_{nj}\right| >\frac{\epsilon }{n^{1/3}}\right) ,\ P\left( \left| \frac{1}{n}\sum ^n_{j=1} Y^i_{nj}\right| >\epsilon n^{-1/3}\right) \\&\quad \le \exp \left( -\frac{Cn^{2/3}\epsilon ^2}{n^{1/3}+\epsilon \log (n) \log \log (n)}\right) . \end{aligned}$$

Therefore as $n\rightarrow \infty$

$$\begin{aligned} P\left( n^{1/3}\sup _{[0,1]}|F_n(x)-F(x)|>\epsilon \right)= & {} O\left( k_n\exp \left( -n^{1/3}C/(\log (n)\log \log (n))\right) \right) \\= & {} O\left( n^{2/3}\exp \left( -n^{1/3}C/(\log (n)\log \log (n))\right) \right) . \end{aligned}$$

$\square$

Lemma 2

Let $\{X_n\}_{n=1,2,\ldots }$ be a strongly mixing process with differentiable marginal density f supported on [0, 1] and $\alpha (n)<D\rho ^n$, where $0<\rho <1$ and $D>0$. Let $f^{(1)}$ be continuous on [0, 1] and the kernel K satisfies Assumption A. Then for any $x_0$ as $n\rightarrow \infty$,

$$\begin{aligned} f^{(1)}_n(x_0)\rightarrow f^{(1)}(x_0)\ \text {a.s.}, \end{aligned}$$

where $f^{(1)}_n(x)=\frac{1}{nh^2}\sum ^n_{i=1} K^{(1)}\left( \frac{x-X_i}{h}\right) \ \text {and}\ h$ is a multiple of $n^{-1/9}$.

Proof of Lemma

Under the stated conditions on f we see that as $n\rightarrow \infty$

$$\begin{aligned} E(f^{(1)}_n(x_0))=f^{(1)}(x_0)+o(1). \end{aligned}$$

Therefore for every $\epsilon >0$ there exists N such that for $n>N$

$$\begin{aligned} \left| E(f^{(1)}_n(x_0))-f^{(1)}(x_0)\right| <\epsilon /2. \end{aligned}$$

Therefore for $n>N$,

$$\begin{aligned} P\left( |f^{(1)}_n(x_0)-f^{(1)}(x_0)|>\epsilon \right)\le & {} P\left( |f^{(1)}_n(x_0)-E\left( f^{(1)}_n(x_0)\right) |>\epsilon /2\right) \\= & {} P\left( \frac{1}{n}\left| \sum ^n_{i=1}Y_{ni}\right| >\epsilon h^2/2\right) , \end{aligned}$$

where $Y_{ni}=\left[ K^{(1)}\left( \frac{x_0-X_i}{h}\right) -E\left\{ K^{(1)}\left( \frac{x_0-X_i}{h}\right) \right\} \right] ,\ i=1,\ldots ,n,$ which represents a sequence of stationary strongly mixing mean zero bounded random variables, with mixing coefficient bounded above by $\alpha (n)$. Therefore using the Bernstein type inequality for strongly mixing processes in Merlevède et al. (2009) we get that for $n\ge 4$

$$\begin{aligned} P\left( |f^{(1)}_n(x_0)-f^{(1)}(x_0)|>\epsilon \right) \le \exp \left( -\frac{Cnh^4\epsilon ^2}{||K^{(1)}||+\epsilon h^2\log (n)\log \log (n)}\right) \end{aligned}$$

For h equal to a multiple of $n^{-1/9}$,

$$\begin{aligned} P\left( |f^{(1)}_n(x_0)-f^{(1)}(x_0)|>\epsilon \right) \le \exp \left( -\frac{C_1n^{5/9}\epsilon ^2}{\log (n)\log \log (n)}\right) ,\ C_1>0. \end{aligned}$$

Now using Borel–Cantelli Lemma we see that $f^{(1)}_n(x_0)\rightarrow f^{(1)}(x_0)$, almost surely, as $n\rightarrow \infty$. $\square$