Optimal robust estimators for families of distributions on the integers

Abstract

Let $F_{\theta }$ be a family of distributions with support on the set of nonnegative integers $Z_{0}$. In this paper we derive the M-estimators with smallest gross error sensitivity (GES). We start by defining the uniform median of a distribution F with support on $Z_{0}$ (umed(F)) as the median of $x+u,$ where x and u are independent variables with distributions F and uniform in [-0.5,0.5] respectively. Under some general conditions we prove that the estimator with smallest GES satisfies umed$(F_{n})=$umed$(F_{\theta }),$ where $F_{n}$ is the empirical distribution. The asymptotic distribution of these estimators is found. This distribution is normal except when there is a positive integer k so that $F_{\theta }(k)=0.5.$ In this last case, the asymptotic distribution behaves as normal at each side of 0, but with different variances. A simulation Monte Carlo study compares, for the Poisson distribution, the efficiency and robustness for finite sample sizes of this estimator with those of other robust estimators.

Appendix: Proofs of results

1.1 Proof of Theorem 1

For the purposes of this proof it will be more convenient to state Hampel’s problem in its equivalent dual form, namely, to minimize the GES under a bound K on the asymptotic variance. It is known that the solution is again given by (9), where now m is a decreasing function of the bound K. For given m call $\widehat{\theta }_{m}$ the Hampel-optimal estimator given by (9). In this case (10) takes on the form

$$\begin{aligned} {\displaystyle \sum _{k=0}^{\infty }} \psi _{m}^{H}(\psi _{0}(k,\widehat{\theta }_{m})-c(m,\widehat{\theta } _{m}))p(k,\widehat{\theta }_{m})=0. \end{aligned}$$

(19)

When the bound K tends to infinity, $m\rightarrow 0$ and the GES of $\widehat{\theta }_{m}$ tends to its lower bound. Then to prove the Theorem it is enough to show that there exists $m_{0}$ such that for $m\le m_{0},$ this estimator coincides with the estimator given by (11).

We will suppose that $\psi _{0}(k,\theta )$ is strictly increasing in k. The proof when it is strictly decreasing is similar. Put $k_{0}^{*}(\theta )=k_{0}(F_{\theta })$ and let

$$\begin{aligned} m_{0}=\frac{1}{2}\min (\psi _{0}(k_{0}^{*}(\theta ),\theta )-\psi _{0} (k_{0}^{*}(\theta )-1,\theta ),\psi _{0}(k_{0}^{*}(\theta )+1,\theta )-\psi _{0}(k_{0}^{*},\theta )). \end{aligned}$$

(20)

It will be shown that if $m\le m_{0}$ then

$$\begin{aligned} \psi _{0}(k_{0}^{*}(\theta ),\theta )-m_{0}\le c(m,\theta )\le \psi _{0} (k_{0}^{*}(\theta ),\theta )+m_{0}. \end{aligned}$$

(21)

Suppose that $c(m,\theta )<\psi _{0}(k_{0}^{*}(\theta ),\theta )-m_{0}.$ Then we have $\psi _{0}(k,\theta )-c(m,\theta )>m_{0}$ for all $k\ge k_{0}^{*}(\theta ),$ and hence

$$\begin{aligned} {\displaystyle \sum _{k=k_{0}^{*}(\theta )}^{\infty }} \psi _{m}^{H}(\psi _{0}(k,\theta )-c(m,\theta ))p(k,\theta )=m(1-F_{0}(k_{0}^{*}(\theta )-1))\ge \frac{m}{2}. \end{aligned}$$

We also have

$$\begin{aligned} {\displaystyle \sum _{k=0}^{k_{0}^{*}-1}} \psi _{m}^{H}(\psi _{0}(k,\theta )-c(m,\theta ))\ p(k,\theta )\ge -m_{0}F_{0} (k_{0}^{*}(\theta )-1)>-\ \frac{m}{2}, \end{aligned}$$

which implies

$$\begin{aligned} {\displaystyle \sum _{k=0}^{\infty }} \psi _{m}^{H}(\psi _{0}(k,\theta )-c(m,\theta ))p(k,\theta )>0, \end{aligned}$$

contradicting ( 19 ), Similarly it can be proved that we can not have $c(m,\theta )>\psi _{0}(k_{0}^{*}(\theta ),\theta )-m_{0}.$

Then from (21) and (20) we get that $\psi _{m}^{H}(\psi _{0}(k,\theta )-c(m,\theta ))\le -m$ for $k<k_{0}^{*}(\theta )$ and $\psi _{m}^{H}(\psi _{0}(k,\theta )-c(m,\theta ))\ge m$ for $k>k_{0}^{*}(\theta ).$ Then it follows from (19) that

$$\begin{aligned} -mF(k_{0}^{*}(\theta )-1,\theta )+m(1-F(k_{0}^{*}(\theta ),\theta ))+(\psi _{0}(k_{0}^{*}(\theta ),\theta )-c(m,\theta ))p(k_{0}^{*} (\theta ),\theta )=0, \end{aligned}$$

(22)

or similarly

$$\begin{aligned} m\frac{(1-2F(k_{0}^{*}(\theta ),\theta ))+p(k_{0}^{*}(\theta ),\theta )}{p(k_{0}^{*}(\theta ),\theta )}=\psi _{m}^{H}(\psi _{0}(k_{0}^{*} (\theta ),\theta )-c(m,\theta )). \end{aligned}$$

(23)

From (23) we derive

$$\begin{aligned} k_{0}^{*}(\theta )-0.5+\frac{(0.5-F(k_{0}^{*}(\theta ),\theta )}{p(k_{0}^{*}(\theta ),\theta )}=k_{0}^{*}(\theta )-1+\frac{1}{2m}\psi _{m}^{H}(\psi _{0}(k_{0}^{*}(\theta ),\theta )-c(m,\theta )). \end{aligned}$$

(24)

Define

$$\begin{aligned} G_{m}(\kappa ,\theta )=k-1\ +\frac{2}{2m}\psi _{m}^{H}(\psi _{0}(k,\theta )-c(m,\theta )), \end{aligned}$$

and note that according to (24) for $m\le m_{0},$ $G_{m}$ does not depend on m. Then (24) and (4) imply that for $m\le m_{0}$

$$\begin{aligned} {\text {umed}}(F_{\theta })=G_{m}(k_{0}^{*}(\theta ),\theta ). \end{aligned}$$

(25)

If $m\le m_{0}$ , (9) is equivalent to

$$\begin{aligned} -mF_{n}(k_{0}^{*}(\widehat{\theta }_{m})-1)+m(1-F_{n}(k_{0}^{*} (\widehat{\theta }_{m})))+&(\psi _{0}(k_{0}^{*}(\widehat{\theta }_{m} ),\widehat{\theta }_{m}) \nonumber \\&-c(m,\widehat{\theta }_{m}))p_{n}(k_{0}^{*} (\widehat{\theta }_{m}))=0, \end{aligned}$$

(26)

and using the same arguments that lead to (25), we can prove that (26) is equivalent to

$$\begin{aligned} {\text {umed}}(F_{n})=G_{m}(k_{0}^{*}(\widehat{\theta }_{m}),\widehat{\theta }_{m}). \end{aligned}$$

(27)

Consider the estimator $\widehat{\mathbf {\theta }}$ defined by umed $(\widehat{\theta })=$umed$(F_{n}).$ By (7) we have $k_{0}(F_{n})=$ $k_{0}^{*}(\widehat{\theta })$ and by (25) we get

$$\begin{aligned} {\text {umed}}(F_{n})={\text {umed}(}F_{\widehat{\theta }})=G_{m}(k_{0}^{*}(\widehat{\theta }),\widehat{\theta }). \end{aligned}$$

Then (27) holds and this implies that (9) holds too. This proves the Theorem.

1.2 Proof of Lemma1

Let $X_{n}\sim F^{\left( n\right) }$ and $X\sim F,$ and call $G^{\left( n\right) }$ and G the distributions of $X_{n}+u$ and $X+u,$ respectively, where u has a uniform distribution on $[-0.5,0.5]$ independent of $X_{n}$ or of X. Then $G^{(n)}$ and G have a positive density, and $G^{(n)}\rightarrow _{w}G$. Since umed$(F^{\left( n\right) })=\mathrm {med}\left( G^{\left( n\right) }\right)$ and umed$(F)=\mathrm {med}\left( G\right) ,$ and the median is a weakly continuous functional, the result is shown.

1.3 Proof of Theorem 3

Recall the notation defined in (12) and (13).

(a) If $F(K)>0.5,$ then then for large n we have $k_{0}(F_{n})=K,$ and therefore

$$\begin{aligned} Z_{n}=n^{1/2}\left( \frac{0.5-F_{n}(K-1)}{p_{n}(K)}-\frac{0.5-F(K-1)}{p_{0}}\right) . \end{aligned}$$

To derive the asymptotic distribution of $Z_{n}$ we need first to calculate that of the vector

$$\begin{aligned} \mathbf {d}_{n}=n^{1/2}\left[ \left( F_{n}(K-1)-F^{1}\right) ,\left( \ p_{n}(K)-p_{0}\right) \right] ^{\prime }. \end{aligned}$$

The vector $d_{n}$ converges in distribution to a bivariate normal distribution with mean (0, 0) and covariance matrix

$$\begin{aligned} \left[ \begin{array}[c]{cc} a &{}\quad c\\ c &{}\quad b \end{array} \right] \end{aligned}$$

where

$$\begin{aligned} a=F^{1}(1-F^{1}),~b=p_{0}(1-p_{0}),\ c=-F^{1}p_{0}. \end{aligned}$$

Then since for large n

$$\begin{aligned} Z_{n}\backsimeq -n^{1/2}\frac{F_{n}(K-1)-F^{1}}{p_{0}}-n^{1/2}\frac{(0.5-F^{1})(p_{n}(K)-p_{0})}{p_0^{2}}, \end{aligned}$$

the delta method yields that $Z_{n}\rightarrow ^{D}N(0,\sigma ^{2})$ where

$$\begin{aligned} \sigma ^{2}&=\frac{a}{p_{0}^{2}}+\frac{b(0.5-F^{1})^{2}}{p_{0}^{4}}+\frac{2c(0.5-F^{1})\ }{p_{0}^{3}}\\&=\frac{F^{1}(1-F^{1})}{p_{0}^{2}}+\frac{(1-p_{0})(0.5-F^{1})^{2}}{p_{0}^{3}}-\frac{2F^{1}(0.5-F^{1})}{p_{0}^{2}} \end{aligned}$$

and a straightforward calculation yields (15).

(b) If $F\left( K\right) =0.5$ it follows from (6) that $\mathrm {umed}(F)=K+0.5.$ On the other hand, for large n we have

$$\begin{aligned} k_{0}(F_{n})=\left\{ \begin{array} [c]{ccc} K &{} \text {if} &{} F_{n}(K)\ge 0.5\\ K+1 &{} \text {if} &{} F_{n}(K)<0.5 \end{array} \right. \end{aligned}$$

We are going to calculate $Z_{n}$ in both cases. If $F_{n}(K)\ge 0.5$ we have

$$\begin{aligned} \mathrm {umed}(F_{n}) =K+0.5+\frac{0.5-F_{n}(K)}{p_{n}(K)}, \end{aligned}$$

(28)

and therefore

$$\begin{aligned} Z_{n}=\ \frac{n^{1/2}(0.5-F_{n}(K)}{p_{n}(K)}\le 0. \end{aligned}$$

(29)

If $F_{n}(K)<0.5$ it follows in the same way that

$$\begin{aligned} Z_{n}=\frac{n^{1/2}(0.5-F_{n}(K)}{p_{n}(K+1)}>0. \end{aligned}$$

Note that, conversely, $Z_{n}\le 0$ implies $F_{n}(K)\ge 0.5$ and $Z_{n}>0$ implies $F_{n}(K)<0.5.$ Since $n^{1/2}(0.5-F_{n}(K))\rightarrow _{d}N(0,0.25),$ the Central Limit Theorem and Slutsky’s Lemma yield (16).

1.4 Proof of Theorem 4

Recall that $\widehat{\theta }_{n}=g^{-1}\left( {\text {umed}}(F_{n})\right)$ and $\theta =g^{-1}\left( {\text {umed}}(F_{\theta })\right) ,$ Put for brevity $K=k_{0}\left( F_{\theta }\right) .$

(a) If $F_{\theta }\left( K\right) >0.5$ there exists an interval I containing $\theta$ such that $t\in I$ implies that $F_{t}\left( K\right) >0.5$ and $F_{t}\left( K-1\right) <0.5,$ and therefore $k_{0}\left( F_{t}\right) =K.$ Therefore g is differentiable at $\theta ,$ and the result follows from Theorem 2, part (a), Theorem 3, and Slutsky’s Lemma.

(b) Assume now $F_{\theta }\left( K\right) =0.5.$ Then $t<\theta$ implies that $F_{t}\left( K\right) >0.5,$ and therefore for sufficiently small $\delta$ we have $k_{0}\left( F_{\theta -\delta }\right) =K$ and $k_{0}\left( F_{\theta +\delta }\right) =K+1.$ Then the left- and right side derivatives of g at $\theta$ are $g_{-}^{\prime }$ and $g_{+}^{\prime }$ given by (18), and therefore the left- and right side derivatives of $g^{-1}$ are $1/g_{-}^{\prime }$ and $g_{+}^{\prime },$ respectively.

We have

$$\begin{aligned} n^{1/2}(\widehat{\theta }_{n}-\theta )=n^{1/2}\left( g^{-1}({\text {umed}}(F_{n}))-g^{-1}({\text {umed}}(F_{\theta }))\right) . \end{aligned}$$

(30)

Note that for $\tau <0$ we have $g\left( t\right) -g\left( \theta \right) =\left( t-\theta \right) g_{-}^{\prime }+o\left( t-\theta \right) ,$ and that

$$\begin{aligned} \mathrm {P}\left( n^{1/2}\left( {\text {umed}}(F_{n})-{\text {umed}}(F_{\theta })\right) <t\right) \rightarrow H\left( t\right) , \end{aligned}$$

with H defined in (16). The result follows by applying the delta method.