New designs to consistently estimate the isotonic regression

Abstract

The usual estimators of the regression under isotonicity assumptions are too sensitive at the tails. In order to avoid this problem, some new strategies for fixed designs are analyzed. The uniform consistency of certain estimators on a closed and bounded working interval are obtained. It is shown that the usual isotonic regression can be employed when the number of observations at the edges of the interval is suitably controlled. Moreover, two modifications are proposed which substantially improve the results. One modification is based on the reallocation of part of the edge observations, and the other one forces the isotonic regression to take values within some horizontal bands. The theoretical results are complemented with some examples and simulation studies that illustrate the performance of the proposed estimators in practice.

Appendix: Proofs

The following lemma is a slightly generalization of Lemma 3 in Hanson et al. (1973) that takes into account weights. We will focus on the part of this lemma that will be used for the different proofs.

Lemma 1

Let F be a real-valued function on $[0,\infty )$ satisfying condition (C2) and let $\{Z_{i,n}\}_{i=1}^n$ $(n\in \mathbb {N})$ be a triangular array of row-wise independent random variables so that $E(Z_{i,n})=0$ and $P(|Z_{i,n}|\ge z)\le F(z)$ for all $i=1,\ldots ,n$, $n \in \mathbb {N}$ and $z\ge 0$. Let $\{w_{i,n}\}_{i=1}^n$ $(n\in \mathbb {N})$ be a family of positive and bounded real numbers, then

$$\begin{aligned} \max _{1\le k \le n} \left| \sum _{i=1}^k Z_{i,n}w_{i,n}\right| /n {\mathop {\longrightarrow }\limits ^{n \rightarrow \infty }} 0 \quad a.s.-[P]. \end{aligned}$$

Proof

Fix $0<M<\infty$ so that $w_{i,n}\le M$ for all $i=1,\ldots ,n$ and $n\in \mathbb {N}$. Obviously,

$$\begin{aligned} P(|Z_{i,n}w_{i,n}|\ge z)\le F_M(z)=F(z/M) \end{aligned}$$

for all $z\ge 0.$ In addition, it is easy to check that both $\lim _{z\rightarrow \infty } F_M(z)=0$ and $\int _{0}^{\infty } z|dF_M(z)|<\infty$. Consequently, we can apply Lemma 3 in Hanson et al. (1973) to the triangular array of row-wise independent random variables $\{Z_{i,n}w_{i,n}\}_{i=1}^n$ ($n\in \mathbb {N}$) to prove the result. $\square$

Proof Theorem 1

As in Hanson et al. (1973), the result can be proven by combining the reasoning of Theorem 4.1 of Brunk (1970) with Lemma 1. $\square$

Proof Theorem  2

As Conditions (C1)–(C5) are satisfied, Theorem 1 guarantees the pointwise $a.s.-[P]$ convergence of $\widehat{m}_I^*$ in (a, b). Thus, in order to prove the result, it is enough to check the pointwise $a.s.-[P]$ convergence in a and b.

First of all, we focus on the pointwise $a.s.-[P]$ convergence in a. In this sense, taking into account the well-known max-min formula and the isotonicity of m, it is easy to check that

$$\begin{aligned} \left| \widehat{m}_I^*(a) - m(a)\right| \le \left| m(x_{1,n}) - m(a)\right| + \displaystyle \max _{k=1,\ldots ,n} \left| \frac{\sum _{i=1}^k \sum _{j=1}^{r_n(i)}w(x_{i,n}) \varepsilon ^j(x_{i,n})}{\sum _{i=1}^k w(x_{i,n}) r_n(i)}\right| . \end{aligned}$$

The continuity of m ensures that the first term in the preceding sum converges to 0 as $n \rightarrow \infty$. On the other hand, the second term is bounded by

$$\begin{aligned} \displaystyle \left( \frac{1}{\inf _{x \in A}w(x)} \right) \left( \frac{N_n(A)}{r_n(1)}\right) \max _{k=1,\ldots ,n} \left| \frac{\sum _{i=1}^k \sum _{j=1}^{r_n(i)} w(x_{i,n}) \varepsilon ^j(x_{i,n})}{N_n(A)}\right| \end{aligned}$$

which converges $a.s.-[P]$ to 0 as $n \rightarrow \infty$ as a consequence of Lemma 1 and Conditions (C5) and (C6). Consequently, the pointwise $a.s.-[P]$ convergence in a follows. The pointwise $a.s.-[P]$ convergence in b can be deduced in the same way. $\square$

Proof Theorem 3

By considering $A=[a-1,b-1]$, $m(x)=m(a)$, $w(x)=w(a)$ and $\varepsilon (x)$ distributed as $\varepsilon (a)$ for all $x \in [a-1,a)$ (analogous at the point b) and the set of design points $B_n$, we have that the conditions (C1)–(C5) are satisfied. Thus, the result follows directly from Theorem 1.

In order to prove the next two theorems, it should be noted that the increasing of $\widehat{m}_I^*$ together with Conditions (C1) and (C3) guarantee that it is enough to prove the pointwise $a.s.-[P]$ convergence in [a, b]. $\square$

Proof Theorem 4

Let $I_n=\min (\overline{Y(x_{1,n})},\overline{Y})$ and $S_n=\max (\overline{Y},\overline{Y(x_{n,n})})$. Based on Domínguez-Menchero and González-Rodríguez (2007) it can be deduced that

$$\begin{aligned} \widehat{m}_3^*(x) =\max \{\min \{\widehat{m}_I^*(x),S_n\},I_n\} \end{aligned}$$

for all $x \in [a,b]$ for certain isotonic extension $\widehat{m}_I^*$ of the non-restricted isotonic regression $\widehat{m}_I$. In addition, $\widehat{m}_3^*(a)=I_n$ and $\widehat{m}_3^*(b)=S_n$ which, taking into account Condition (C8) and the SLLN, converges $a.s.-[P]$ to m(a) and m(b), respectively. Finally, note that as Conditions (C1)–(C5) are satisfied, Theorem 1 guarantees the pointwise $a.s.-[P]$ convergence of $\widehat{m}_I^*$ to m in (a, b), and consequently the pointwise $a.s.-[P]$ convergence of $\widehat{m}_3^*$ to m in [a, b] is obtained, which finishes the proof. $\square$