Localization processes for functional data analysis

Abstract

We propose an alternative to k-nearest neighbors for functional data whereby the approximating neighboring curves are piecewise functions built from a functional sample. Using a locally defined distance function that satisfies stabilization criteria, we establish pointwise and global approximation results in function spaces when the number of data curves is large. We exploit this feature to develop the asymptotic theory when a finite number of curves is observed at time-points given by an i.i.d. sample whose cardinality increases up to infinity. We use these results to investigate the problem of estimating unobserved segments of a partially observed functional data sample as well as to study the problem of functional classification and outlier detection. For such problems our methods are competitive with and sometimes superior to benchmark predictions in the field. The R package localFDA provides routines for computing the localization processes and the estimators proposed in this article.

Appendix A Proofs of main results and additional figures

Here we provide the proofs of the main results of Sect. 2, the proof of Proposition 1, and three additional figures.

1.1 A.1 Auxiliary results

We prepare for the proofs with two lemmas.

Lemma A.1

Fix $k \in \mathbb {N}$. For almost all $t \in [0,1]$, there are random variables $\{X'_j(t) \}_{j = 1}^n$, coupled to $\{X_j(t) \}_{j = 1}^n,$ and a Cox process $\mathcal{P}_{{\kappa }_t(X'_1(t))}$, also coupled to $\{X_j(t) \}_{j = 1}^n$, such that as $n \rightarrow \infty$

$$\begin{aligned} {W'_n}^{(k)}(t)&= W^{(k)}(X'_1(t), \{X'_j(t) \}_{j = 1}^n ) \nonumber \\&= \frac{2}{k} L^{(k)}(\mathbf{0}, n( \{X'_j(t) \}_{j = 1}^n - X'_1(t)) )\\&{\mathop {\longrightarrow }\limits ^{\mathcal{P}}}\frac{2}{k} L^{(k)}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(X'_1(t))}) =: {W'_{\infty }}^{(k)}(t). \end{aligned}$$

Proof

The convergence may be deduced from Section 3 of Penrose and Yukich (2003) and we provide details as follows. For $x \in \mathbb {R}^d$ and $r > 0$ let B(x, r) denote the Euclidean ball centered at x with radius r. Note that $L^{(k)}$ is a stabilizing score function on Poisson input $\mathcal{P}$ having constant intensity density, that is to say that its value at the origin is determined by the local data consisting of the points in the intersection of the realization of $\mathcal{P}$ and the ball $B(\mathbf{0}, R^{L^{(k)}}(\mathbf{0}, \mathcal{P}))$, where $R^{L^{(k)}}(\mathbf{0}, \mathcal{P})$ is a radius of stabilization. For precise definitions we refer to Penrose and Yukich (2003), Section 3 and Penrose (2007).

The coupling of Section 3 of Penrose and Yukich (2003) shows that we may find $\{X'_j(t) \}_{j = 2}^n$, where $X'_j(t) {\mathop {=}\limits ^{\mathcal{D}}}X_j(t), j = 2,\ldots ,n$ and a Cox process $\mathcal{P}_{{\kappa }_t(X'_1(t))}$ such that if we put $\mathcal{X}'_{n - 1}(t) = \{X'_j(t) \}_{j = 2}^n$, then for all $K > 0$

$$\begin{aligned} \lim _{n \rightarrow \infty } \mathbb {P}\left( n(\mathcal{X}'_{n - 1}(t) - X'_1(t)) \cap B(\mathbf{0},K) = \mathcal{P}_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0},K)\right) = 1, \end{aligned}$$

(A1)

which is a consequence of the convergence of the point process $n(\mathcal{X}'_{n - 1}(t) - X'_1(t)) \cap B(\mathbf{0},K)$ to the point process $\mathcal{P}_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0},K)$ as $n \rightarrow \infty$. See Lemma 3.1 of Penrose and Yukich (2003) for details. Fix $\epsilon > 0$. Now write for all $\delta > 0$

$$\begin{aligned}&\mathbb {P}\left( |\frac{2}{k} L^{(k)}(\mathbf{0}, n(\mathcal{X}'_{n - 1}(t) - X'_1(t)) ) - \frac{2}{k} L^{(k)}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(X'_1(t))})|> \epsilon \right) \nonumber \\&\le \mathbb {P}( n(\mathcal{X}'_{n - 1}(t) - X'_1(t)) \cap B(\mathbf{0},K) \ne \mathcal{P}_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0},K)) \nonumber \\&+ \mathbb {P}( R^{L^{(k)}}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(X'_1(t))})> K) \nonumber \\&\le \mathbb {P}( n(\mathcal{X}'_{n - 1}(t) - X'_1(t)) \cap B(\mathbf{0},K) \ne \mathcal{P}_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0},K)) \nonumber \\&+ \mathbb {P}( R^{L^{(k)}}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(X'_1(t))}) > K, {\kappa }_t(X'_1(t)) \ge \delta ) + \mathbb {P}( {\kappa }_t(X'_1(t)) \le \delta ). \end{aligned}$$

(A2)

Given $\epsilon > 0$, the last term in (A2) may be made less than $\epsilon /3$ if $\delta$ is small. The penultimate term is bounded by $\mathbb {P}( R^{L^{(k)}}(\mathbf{0}, \mathcal{P}_{\delta }) > K)$, which is less than $\epsilon /3$ if K is large, since $R^{L^{(k)}}(\mathbf{0}, \mathcal{P}_{\delta })$ is finite a.s. By (A1) the first term is less than $\epsilon /3$ if n is large. Thus, for $\delta$ small and K and n large, the right-hand side of (A2) is less than $\epsilon$, which concludes the proof. $\square$

Lemma A.2

Assume that the data are bounded and regular from below for all $t \in T_0 \subseteq [0,1]$ as at (6), and where $T_0$ has Lebesgue measure 1. Then $\sup _{n \le \infty }\sup _{t \in T_0} \mathbb {E}W_n^{(k)}(t)^2 \le C,$ where C is a finite constant.

Proof

We treat the case $1 \le n < \infty$, as the case $n = \infty$ follows by similar methods. Without loss of generality we may assume the data are bounded above by 1 and that $S({\kappa }_t) = [0,1]$ for all $t \in T_0$. We first prove the lemma for $k = 1$ and then for general k. Let $S_\delta$ be the subinterval of [0, 1] such that ${\kappa }_t(x) \ge {\kappa }_{\text {min}}$ for all $x \in S_\delta$. Note that $|S_\delta |=\delta \in (0,1]$ by assumption. We have for all $r> 0$ and all $t \in T_0$

$$\begin{aligned} \mathbb {P}( W_n^{(1)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge r)&= \mathbb {P}\left( L^{(1)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge \frac{r}{2n}\right) \\&= \Pi _{j = 2}^n \mathbb {P}\left( |X_1(t) - X_j(t) |\ge \frac{r }{2n}\right) \\&= \left( 1 - \mathbb {P}\left( |X_1(t) - X_2(t) |\le \frac{r }{2n}\right) \right) ^{n - 1}\\&= \left( 1 - \int _{[0,1]} \int _{ |x_1 - x_2 |\le \frac{r}{2n} } {\kappa }_t(x_2) dx_2 {\kappa }_t(x_1) dx_1 \right) ^{n - 1}. \end{aligned}$$

By the boundedness assumption on the data, we have $\mathbb {P}( L^{(1)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge \frac{r}{2n}) = 0$ for all $r \in (2n, \infty )$. Thus we may assume without loss of generality that $r \in [0,2n]$. We bound the double integral from below by

$$\begin{aligned} \int _{[0,1]} \int _{ |x_1 - x_2 |\le \frac{r}{2n} } {\kappa }_t(x_2) dx_2 {\kappa }_t(x_1) dx_1&\ge \int _{S_\delta } \int _{|x_1 - x_2 |\le \frac{r}{2n} } {\kappa }_t(x_2) dx_2 {\kappa }_t(x_1) dx_1 \nonumber \\&\ge \int _{S_\delta } \int _{|x_1 - x_2 |\le \frac{r \delta }{2n} } {\kappa }_t(x_2) dx_2 {\kappa }_t(x_1) dx_1 \nonumber \\&\ge c \delta {\kappa }_{\text {min}}^2 \cdot \frac{r \delta }{2n}, \end{aligned}$$

where here and elsewhere $c > 0$ is a generic constant, possibly changing from line to line. This gives

$$\begin{aligned} \mathbb {P}( W_n^{(1)}(X_1(t), \{X_j(t) \}_{j = 1}^n ) \ge r)&\le \left( 1 - \delta {\kappa }_{\text {min}}^2 \cdot \frac{r \delta }{2n} \right) ^{n - 1} \\&\le c \exp {\left( - \frac{ \delta ^2 {\kappa }_{\text {min}}^2 r}{c} \right) }, \ r > 0. \end{aligned}$$

Random variables having exponentially decaying tails have finite moments of all orders and this proves the lemma for $k = 1$.

The proof for general k is similar. We show this holds for $k = 2$ as follows. We have

$$\begin{aligned} \mathbb {P}( W_n^{(2)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge r) = \mathbb {P}\left( L^{(2)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge \frac{r}{n}\right) . \end{aligned}$$

Given the event $\{ L^{(2)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge \frac{r}{n} \}$, either the first nearest neighbor to $X_1(t)$ is at a distance greater than $\frac{r }{n}$ to $X_1(t)$ or the first nearest neighbor to $X_1(t)$ is at a distance less than $\frac{r}{n}$ to $X_1(t)$ and the first nearest neighbor among the remaining $n - 2$ sample points exceeds $\frac{r}{n}$.

This gives

$$\begin{aligned} \mathbb {P}\left( L^{(2)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge \frac{r }{n}\right) \le \mathbb {P}\left( L^{(1)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge \frac{r}{n}\right) \end{aligned}$$

$$\begin{aligned} + \sum _{i = 2}^n \mathbb {P}\left( L^{(1)}(X_1(t), \{X_j(t) \}_{j = 2, j \ne i}^n ) \ge \frac{r }{n}\right) \mathbb {P}( |X_1 - X_i |\le \frac{r }{n}). \end{aligned}$$

Since $\mathbb {P}( |X_1 - X_i |\le \frac{r }{n}) = O(n^{-1})$ for all $i = 2,\ldots ,n$ we may use the bounds for the case $k = 1$ to show that $\mathbb {P}\left( L^{(2)}(X_1(t), \{X_j(t) \}_{j = 2}^n ) \ge \frac{r}{n}\right)$ decays exponentially fast in r. The proof for general k follows in a similar fashion and we leave the details to the reader. This proves the lemma. $\square$

1.2 A.2 Proof of Theorem 2.1

We first prove (3). Let $t \in [0,1]$ be such that the marginal density $\kappa _t$ exists. By translation invariance of $L^{(k)}$ we have as $n \rightarrow \infty$

$$\begin{aligned} \mathbb {E}W_n^{(k)}(t)&= \mathbb {E}L^{(k)}\Big ( \frac{2n}{k } X_1(t), \frac{2n}{k } \{X_j(t) \}_{j = 1}^n \Big ) \\&= \frac{2}{k } \mathbb {E}L^{(k)}(\mathbf{0}, n( \{X_j(t) \}_{j = 1}^n - X_1(t) ) ) \\&\rightarrow \int _{ S(\kappa _t)} \frac{2}{k} \mathbb {E}L^{(k)} (\mathbf{0}, \mathcal{P}_{\kappa _t(y)} ) \kappa _t(y) dy, \end{aligned}$$

where the limit follows since convergence in probability (Lemma A.1) combined with uniform integrability (Lemma A.2) gives convergence in mean. For any constant $\tau$ we have $\mathbb {E}L^{(k)} (\mathbf{0}, \mathcal{P}_{\tau } ) = \tau ^{-1} \mathbb {E}L^{(k)} (\mathbf{0}, \mathcal{P}_{1} )$. Notice that $L^{(k)} (\mathbf{0}, \mathcal{P}_{1} )$ is a Gamma $\Gamma (k,2)$ random variable with shape parameter k and scale parameter 2 and thus $\mathbb {E}L^{(k)} (\mathbf{0}, \mathcal{P}_{1}) = k/2$. The proof of (3) is complete.

To prove (4), we replace $W_n^{(k)}(t)$ by its square in the above computation. This yields

$$\begin{aligned} \lim _{n \rightarrow \infty } \mathbb {E}W_n^{(k)}(t)^2 = \int _{ S(\kappa _t)} \frac{4}{k^2} \mathbb {E}(L^{(k)} (\mathbf{0}, \mathcal{P}_{\kappa _t(y)} ))^2 \kappa _t(y) dy. \end{aligned}$$

For any constant $\tau \in (0, \infty )$ we have

$$\begin{aligned} \mathbb {E}(L^{(k)} (\mathbf{0}, \mathcal{P}_{\tau } ))^2 = \tau ^{-2} \mathbb {E}( L^{(k)} (\mathbf{0}, \mathcal{P}_{1} ))^2 = \tau ^{-2} \frac{(k + 1)k}{4}, \end{aligned}$$

(A3)

where we recall that the second moment of a Gamma $\Gamma (k,2)$ random variable equals $(k + 1)k/4$. These facts yield (4). The limit (5) is a consequence of Lemma A.1. $\square$

1.3 A.3 Proof of Theorem 2.2

Recall that on $T_0 \subseteq [0,1]$ we have that $\kappa _t$ exists, the data are bounded, and the data are regular from below, as at (6). By Lemmas A.1 and A.2, the random variables ${W'_n}^{(k)}(t) = W^{(k)}(X'_1(t), \{X'_j(t) \}_{j = 1}^n ), t \in T_0,$ converge in probability and also in mean. It follows that for all $t \in T_0$ as $n \rightarrow \infty$

$$\begin{aligned} F_n(t) = \mathbb {E}|{W'_n}^{(k)}(t) - {W'_{\infty }}^{(k)}(t) |= \mathbb {E}|{W}_n^{(k)}(t) - {W}_{\infty }^{(k)}(t) |\rightarrow 0. \end{aligned}$$

Now $\sup _n \sup _{t \in T_0} F_n(t) \le C$ and the bounded convergence theorem gives $\lim _{n \rightarrow \infty } \int _0^1 F_n(t) dt = 0.$ This gives the first statement of Theorem 2.2. The identity $\lim _{n \rightarrow \infty } \int _0^1 \mathbb {E}{W}_{\infty }^{(k)}(t) dt = 1$ follows from

$$\begin{aligned} \int _0^1 F_n(t) dt \ge |\int _0^1 \mathbb {E}W_n^{(k)}(t)dt - \int _0^1 \mathbb {E}{W}_{\infty }^{(k)}(t) dt |\end{aligned}$$

and the identity $\mathbb {E}{W}_{\infty }^{(k)}(t) = |S(\kappa _t) |$, $t \in T_0$. $\square$

1.4 A.4 Proof of Theorem 2.3

Lemma A.1 assumes that k is fixed. The lemma will not always hold if k is growing arbitrarily fast with n. Thus our proof techniques and coupling arguments need to be modified. We break the proof of Theorem 2.3 into five parts.

Part (i) Coupling. We start with a general coupling fact. Given Poisson point processes $\Sigma _1$ and $\Sigma _2$ with densities $f_1$ and $f_2$, we may find coupled Poisson point processes $\Sigma '_1$ and $\Sigma '_2$ with $\Sigma '_1 {\mathop {=}\limits ^{\mathcal{D}}}\Sigma _1$ and $\Sigma '_2 {\mathop {=}\limits ^{\mathcal{D}}}\Sigma _2$ such that the probability that the two point processes are not equal on $[-A,A]$ is bounded by

$$\begin{aligned} \int _{-A}^{A} |f_1(x) - f_2(x) |dx. \end{aligned}$$

Let $t \in [0,1]$ be such that the marginal density $\kappa _t$ exists. As in Theorem 2.3, we assume that $\kappa _t$ is $\alpha$-Hölder continuous for $\alpha \in (0,1]$. Note that the point process $n(\mathcal{P}_{n \kappa _t} - y)$ has intensity density ${\kappa }_t( \frac{x}{n} + y), \ x \in n (S(\kappa _t) - y)$. For each $y \in S(\kappa _t)$, we may find coupled Poisson point processes $\mathcal{P}'_{n \kappa _t}$ and $\mathcal{P}'_{\kappa _t(y)}$ with $n(\mathcal{P}'_{n \kappa _t} - y) {\mathop {=}\limits ^{\mathcal{D}}}n(\mathcal{P}_{n \kappa _t} - y)$ and $\mathcal{P}'_{\kappa _t(y)} {\mathop {=}\limits ^{\mathcal{D}}}\mathcal{P}_{\kappa _t(y)}$ such that the probability that the point processes $n(\mathcal{P}'_{n \kappa _t} - y)$ and $\mathcal{P}'_{\kappa _t(y)}$ are not equal on $[-A, A]$ is bounded uniformly in $y \in S({\kappa }_t)$ by

$$\begin{aligned} \int _{-A}^{A} |{\kappa }_t\left( \frac{x}{n} + y\right) - {\kappa }_t(y) |dx \le 2A \left( \frac{ A}{n}\right) ^{\alpha }. \end{aligned}$$

(A4)

We will need this coupling in what follows.

Part (ii) Poissonization. We will first show a Poissonized version of (9). Write k instead of k(n). We assume that we are given a Poisson number of data curves $\{X_j(t) \}_{j = 1}^{N(n)}$, where N(n) is an independent Poisson random variable with parameter n. We aim to show for almost all $t \in [0,1]$

$$\begin{aligned} \lim _{n \rightarrow \infty } \mathbb {E}W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) = |S(\kappa _t) |. \end{aligned}$$

(A5)

By translation invariance of $W_n^{(k)}$ we have

$$\begin{aligned} \mathbb {E}W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} )&= \frac{2n}{k} \mathbb {E}L^{(k)}\left( X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} \right) \nonumber \\&= \frac{2n}{k} \mathbb {E}L^{(k)}(\mathbf{0}, ( \{X_j(t) \}_{j = 1}^{N(n)} - X_1(t) ) ). \end{aligned}$$

(A6)

We assert that as $n \rightarrow \infty$

$$\begin{aligned} |\mathbb {E}L^{(k)}(\mathbf{0}, \frac{2n}{k} ( \{X_j(t) \}_{j = 1}^{N(n)} - X_1(t) ) ) - \frac{2}{k} \mathbb {E}L^{(k)}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(X_1(t))}) |\rightarrow 0. \end{aligned}$$

(A7)

Combining (A6)-(A7) and recalling that $\frac{2}{k} \mathbb {E}L^{(k)}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(X_1(y))}) = |S(\kappa _t) |,$ we obtain

$$\begin{aligned} \lim _{n \rightarrow \infty } \mathbb {E}W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) = |S(\kappa _t) |, \end{aligned}$$

which establishes (A5).

It remains to establish (A7). A Poisson point process on a set S with intensity density nh(x), where h is itself a density, may be expressed as the realization of random variables $X_1,\ldots .,X_{N(n)}$, where N(n) is an independent Poisson random variable with parameter n and where each $X_i$ has density h on S. Thus the point process $\{X_j(t) \}_{j = 1}^{N(n)}$ is the Poisson point process $\mathcal{P}_{n{\kappa }_t}$. To show the assertion (A7) we thus need to show

$$\begin{aligned} |\frac{2}{k} \mathbb {E}L^{(k)}(\mathbf{0}, n(\mathcal{P}_{n{\kappa }_t} - X_1(t))) - \frac{2}{k} \mathbb {E}L^{(k)}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(X_1(t))}) |\rightarrow 0 \end{aligned}$$

or equivalently,

$$\begin{aligned} |\frac{1}{k} \mathbb {E}L^{(k)}(\mathbf{0}, n(\mathcal{P}'_{n {\kappa }_t} - X'_1(t))) - \frac{1}{k} \mathbb {E}L^{(k)}(\mathbf{0}, \mathcal{P}'_{{\kappa }_t(X'_1(t))}) |\rightarrow 0, \end{aligned}$$

where $\mathcal{P}'_{n \kappa _t}$ and $\mathcal{P}'_{\kappa _t(y)}$ are as in part (i).

   Fix $\epsilon > 0.$ As in the bound (A2), we have for all $\delta> 0, K > 0$

$$\begin{aligned}&\quad \mathbb {P}\left( |\frac{2}{k} L^{(k)}(\mathbf{0}, n(\mathcal{P}'_{n{\kappa }_t} - X'_1(t))) - \frac{2}{k} L^{(k)}(\mathbf{0}, \mathcal{P}'_{{\kappa }_t(X'_1(t))})|> \epsilon \right) \nonumber \\&\le \mathbb {P}\left( \frac{2n}{k} (\mathcal{P}'_{n{\kappa }_t} - X'_1(t)) \cap B(\mathbf{0},K) \ne \frac{2}{k} \mathcal{P}'_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0},K) \right) \nonumber \\&+ \mathbb {P}\left( R^{L^{(k)}}(\mathbf{0}, \frac{2}{k} \mathcal{P}'_{{\kappa }_t(X'_1(t))})> K\right) \nonumber \\&\le \mathbb {P}\left( \frac{2n}{k} (\mathcal{P}'_{n{\kappa }_t} - X'_1(t)) \cap B(\mathbf{0},K) \ne \frac{2}{k} \mathcal{P}'_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0},K)\right) \nonumber \\&+ \mathbb {P}\left( R^{L^{(k)}}(\mathbf{0}, \frac{2}{k} \mathcal{P}'_{{\kappa }_t(X'_1(t))}) > K, {\kappa }_t(X'_1(t)) \ge \delta \right) + \mathbb {P}( {\kappa }_t(X'_1(t)) \le \delta ). \end{aligned}$$

(A8)

The last term in (A8) may be made less than $\epsilon /3$ if $\delta$ is small. The penultimate term is bounded $\mathbb {P}( R^{L^{(k)}}(\mathbf{0}, \mathcal{P}_{\delta k})> K) = \mathbb {P}( R^{L^{(k)}}(\mathbf{0}, \frac{1}{\delta k} \mathcal{P}_{1})> K) = \mathbb {P}( \frac{1}{\delta k} \Gamma (k,2) >K)$, which by Chebyshev’s inequality is less than $\epsilon /3$ if K is large. The first term satisfies

$$\begin{aligned}&\quad \mathbb {P}\left( \frac{2n}{k} (\mathcal{P}'_{n {\kappa }_t} - X'_1(t)) \cap B(\mathbf{0},K) \ne \frac{2}{k} \mathcal{P}'_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0},K) \right) \nonumber \\&= \mathbb {P}\left( n (\mathcal{P}'_{n {\kappa }_t} - X'_1(t)) \cap B(\mathbf{0}, \frac{Kk}{2} ) \ne \mathcal{P}'_{{\kappa }_t(X'_1(t))} \cap B(\mathbf{0}, \frac{Kk}{2}) \right) \nonumber \\&\le Kk ( \frac{Kk}{2n})^{\alpha } \end{aligned}$$

(A9)

where the inequality follows from (A4). By assumption, we have $\lim _{n \rightarrow \infty } \frac{ k^{1 + \alpha }}{n^{\alpha }} = 0$ and it follows that the first term is less than $\epsilon /3$ if n is large. Thus, for $\delta$ small and K and n large, the right-hand side of (A2) is less than $\epsilon$. Thus

$$\begin{aligned} |\frac{2}{k} L^{(k)}(\mathbf{0}, n(\mathcal{P}'_{{\kappa }_t} - X'_1(t))) - \frac{2}{k} L^{(k)}(\mathbf{0}, \mathcal{P}'_{{\kappa }_t(X'_1(t))})|{\mathop {\longrightarrow }\limits ^{\mathcal{P}}}0. \end{aligned}$$

The assertion (A7) follows since convergence in probability combined with uniform integrability gives convergence in mean.

Part (iii) de-Poissonization. We de-Poissonize the above equality to obtain (9). In other words we need to show that the limit does not change when N(n) is replaced by n. Put

$$\begin{aligned} \mathcal{Y}_n = {\left\{ \begin{array}{ll} X_1(t),\ldots ,X_{N(n)- (N(n) - n)^+}(t) , &{} \mathrm{if} \ N(n) \ge n \\ X_1(t),\ldots ,X_{N(n) + (n - N(n)^+}(t) , &{} \mathrm{if} \ N(n) < n. \end{array}\right. } \end{aligned}$$

Then $\mathcal{Y}_n {\mathop {=}\limits ^{\mathcal{D}}}\{ X_1(t), X_2(t),\ldots ,X_{n}(t)\}.$ We use this coupling of Poisson and binomial input in all that follows.

We wish to show that $\hat{X}_{n,1}^{(k)}(t)$ coincides with $\hat{X}_{N(n),1}^{(k)}(t)$ on a high probability event; in other words we wish to show that the sample points with indices between $\min (n, N(n))$ and $\max (n, N(n))$ do not, in general, modify the value of $\hat{X}_{n,1}^{(k)}(t)$. Consider the event that the Poisson random variable does not differ too much from its mean, i.e.,

$$\begin{aligned} E_n= \{ |N(n) - n |\le c \sqrt{n} \log n \} \end{aligned}$$

and note that tail bounds for Poisson random variables show that there is $c > 0$ such that $\mathbb {P}(E_n^c) = O( n^{-2})$. Write

$$\begin{aligned}&\mathbb {E}|W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) - W_n(t) |\\&\le \mathbb {E}|[W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) - W_n^{(k)}(t)] \mathbf{1}(E_n) |\\&+ \mathbb {E}|[W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) - W_n^{(k)}(t)] \mathbf{1}(E_n^c) |. \end{aligned}$$

The last summand is o(1), which may be seen using the Cauchy-Schwarz inequality, Lemma A.2, and $\mathbb {P}(E_n^c) = O(n^{-2})$.

For any $1 \le j \le c \sqrt{n} \log n$ we define

$$\begin{aligned} A_{n,j} = A_{n,j}(t) = \{ |X_{\min (n, N(n)) + j}(t) - X_1(t) |\ge |\hat{X}_{n,1}^{(k)}(t) - X_1(t) |\}. \end{aligned}$$

This is the event that the data curves having index larger than $\min (n, N(n))$ are farther away from $X_1(t)$ than is the data curve $\hat{X}_{n,1}^{(k)}(t)$.

Given i.i.d. random variables $Z_i, 1 \le i \le n$, we let $Z_i^{(k)}$ denote the kth nearest neighbor to $Z_i$. Given an independent random variable $Z_0$ having the same distribution as $Z_i$, the probability that $Z_0$ belongs to $[Z_i, Z_i^{(k)}]$ coincides with the probability that a uniform random variable on [0, 1] belongs to $[U_i, U_i^{(k)}]$ where $U_i, 1 \le i \le n,$ are i.i.d. uniform random variables on [0, 1]. By exchangeability this last probability equals $k/(n-1)$.

It follows that for any $j = 1,2,\dots$

$$\begin{aligned} \mathbb {P}(A_{n,j} \mid n \le N(n) ) = \frac{(n - 1) - k}{ n -1 }, \end{aligned}$$

whereas

$$\begin{aligned} \mathbb {P}(A_{n,j} \mid n \ge N(n) ) = \frac{(N(n) - 1) - k}{N(n) -1 }. \end{aligned}$$

We have

$$\begin{aligned} \mathbb {P}(A_{n,j} \mid N(n)) = 1 - \frac{k}{ \min ((n -1), (N(n) - 1)) } \end{aligned}$$

and thus $\mathbb {P}(A_{n,j} \cap E_n) \ge 1 - \frac{k}{ (n - 1) - c \sqrt{n} \log n}.$ Thus

$$\begin{aligned}&\mathbb {P}((W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) - W_n^{(k)}(t)) \mathbf{{1}}_{\{E_n\}} \ne 0 ) \\&= 1 - \left( \mathbb {P}(A_{n,1} \cap E_n)\right) ^{c \sqrt{n} \log n} \\&\le 1 - \left( 1 - \frac{k}{ (n - 1) - c \sqrt{n} \log n }\right) ^{c \sqrt{n} \log n}\\&\le \frac{k c' \log n}{ \sqrt{n} }. \end{aligned}$$

When $\frac{k c' \log n}{ \sqrt{n} } = o(1)$ we find that $\left( W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) - W_n^{(k)}(t)\right) \mathbf{{1}}(E_n)$ converges to zero in probability as $n \rightarrow \infty$, and thus so does $\left( W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} )\right. \left. - W_n^{(k)}(t)\right) .$ By uniform integrability we obtain that $(W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} ) - W_n^{(k)}(t))$ converges to zero in mean. This completes the proof of (9).

Part (iv) Variance convergence. Replacing $W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)})$ by its square in the above computation gives

$$\begin{aligned}&\lim _{n \rightarrow \infty }[ \mathbb {E}W_n^{(k)}(X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} )^2 - (\mathbb {E}W_n^{(k)}X_1(t), \{X_j(t) \}_{j = 1}^{N(n)} )^2] \\&= \lim _{n \rightarrow \infty } \frac{4}{k^2 } \int _{ S(\kappa _t) } \mathbb {E}L^{(k)}(\mathbf{0}, \mathcal{P}_{{\kappa }_t(y)})^2 {\kappa }_t(y) dy - |S(\kappa _t) |^2\\&= \int _{S(\kappa _t)} \frac{ 1 }{{\kappa }_t(y)} dy - |S(\kappa _t) |^2, \end{aligned}$$

where the last equality makes use of (A3). This gives (10), as desired.

Part (v) $L^1$ convergence. The limit (11) follows exactly as in the proof of Theorem 2.2. $\square$

1.5 A.5 Proof of Theorem 2.4

This result is a straightforward consequence of the central limit theorem for M-dependent random variables. It is enough to prove the central limit theorem for the re-scaled random variables $\{L(mT_r)\}_{r = 1}^m$. Indeed these random variables $\{L(mT_r)\}_{r = 1}^m$ have moments of all orders and they are M-dependent since $\{L(mT_r)\}_{r \in A}$ and $\{L(mT_r)\}_{r \in B}$ are independent whenever the distance between the index sets A and B exceeds 2M. The asserted asymptotic normality follows by the classical central limit theorem for M-dependent random variables.

$\square$

1.6 A.6 Proof of Proposition 1

Proof. It is enough to show for any fixed j and all $\varepsilon >0$ that

$$\begin{aligned} \lim _{n\rightarrow \infty } \mathbb {P}\big ( \int _0^1 |X^{(n,j)}(t) -X_1(t) |dt <\varepsilon \big ) = 1. \end{aligned}$$

We have

$$\begin{aligned}&\quad \mathbb {P}\big ( \int _0^1 |X^{(n,j)}(t) -X_1(t) |dt>\varepsilon \big ) \nonumber \\&\le \mathbb {P}\big ( \int _0^1 |X^{(n,j)}(t) -X_1(t) |\mathbf{1}_{I^{(k)}(X^{(n,j)})}(t) dt> \frac{\varepsilon }{2} \big ) \nonumber \\&+ \mathbb {P}\big ( \int _0^1 |X^{(n,j)}(t) -X_1(t) |\mathbf{1}_{[0,1] \setminus I^{(k)}(X^{(n,j)})} (t) dt >\frac{\varepsilon }{2} \big ) \nonumber \\&\le \frac{2}{\varepsilon } \mathbb {E}\int _0^1 |\hat{X}_1^{(k)}(t) -X_1(t) |dt + \mathbb {E}\int _0^1 \mathbf{1}_{[0,1] \setminus I^{(k)}(X^{(n,j)})} (t) dt \nonumber \\&= \frac{k}{n\varepsilon } \mathbb {E}\int _0^1 W_n^{(k)}(t) dt + \mathbb {E}\int _0^1 \mathbf{1}_{[0,1] \setminus I^{(k)}(X^{(n,j)})} (t) dt . \end{aligned}$$

(A10)

Since $\kappa _t$ is $\alpha$-Hölder continuous with $\alpha = 1$, we may apply Theorem 2.3 for $\alpha = 1$ and $k= k(n) = o(\sqrt{n})$. Thus, as $n \rightarrow \infty$, the right-hand side goes to 0 by Theorem 2.3, the finiteness of $\mathbb {E}\int _0^1 W_n^{(k)}(t) dt$, and (18). $\square$

1.7 A.7 See Figs. 5, 6 and 7

Fig. 5

Estimated values of $\mathbb {P}\big ( I^{(k)}\big (X^{(n,j)}\big ) \ \text{ contains } \ t \big )$, $0\le t \le 1$, $1\le j\le 4$, $1 \le k\le 250$, $n=2500$. The estimation is based on 1000 independent replicates of $(X_1,O_1)$ when $O_1$ is obtaining by removing randomly one interval of the partition of [0, 1] induced by two independent Uniform(0,1) variables. $X_1$ is a linear combination of sines and cosines with independent Gaussian coefficients

Fig. 6

Boxplots of Relative MSE from 1000 reconstruction exercises based on 50, 200 and 1000 curves randomly selected from the Spanish daily temperatures

Fig. 7

Log age-specific mortality rates and localization distances boxplot for $k=9$. Each outlier value and its corresponding curve are highlighted in yellow (color figure online)