On visual distances for spectrum-type functional data

Abstract

A functional distance ${\mathbb H}$, based on the Hausdorff metric between the function hypographs, is proposed for the space ${\mathcal E}$ of non-negative real upper semicontinuous functions on a compact interval. The main goal of the paper is to show that the space $({\mathcal E},{\mathbb H})$ is particularly suitable in some statistical problems with functional data which involve functions with very wiggly graphs and narrow, sharp peaks. A typical example is given by spectrograms, either obtained by magnetic resonance or by mass spectrometry. On the theoretical side, we show that $({\mathcal E},{\mathbb H})$ is a complete, separable locally compact space and that the ${\mathbb H}$-convergence of a sequence of functions implies the convergence of the respective maximum values of these functions. The probabilistic and statistical implications of these results are discussed, in particular regarding the consistency of k-NN classifiers for supervised classification problems with functional data in ${\mathbb H}$. On the practical side, we provide the results of a small simulation study and check also the performance of our method in two real data problems of supervised classification involving mass spectra.

Appendix

Proof of Proposition 2 To prove this Proposition we first must prove an auxiliary result:

Lemma 1

If $f,g\in \mathcal {E}$, then there exist $u\in \partial H_f$ and $v\in \partial H_g$ such that

$$\begin{aligned} \mathbb {H}(f,g)=d(u,H_g)=d(v,H_f)=\Vert u-v \Vert . \end{aligned}$$

(4)

Proof

We have, by definition of $\mathbb {H}$:

$$\begin{aligned} \mathbb {H}(f,g)=d_H(H_f,H_g)=\max \left\{ \sup _{a\in H_f} d(a,H_g), \sup _{b\in H_g}d(b,H_f)\right\} . \end{aligned}$$

Assume ${\mathbb H}(f,g)>0$. Otherwise the result is trivial. Let us suppose by contradiction that there is no pair $(u,v)\in \partial H_f\times \partial H_g$ such that (4) is fulfilled. In any case, the compactness of $H_f$ and $H_g$ guarantees the existence of $x=(x_1,x_2)$ and $y=(y_1,y_2)$ fulfilling (4) but, according to our contradiction argument, either x or y must be an interior point. For example, if $x\in int(H_f)$, then $0<x_1<1$. We will see that $d((x_1,f(x_1)), H_g)\ge \mathbb {H}(f,g)$. For every $t\in [0,1]$ such that $|t-x_1|<\mathbb {H}(f,g)$ let us denote $u^t=(t,u_2^t)$ and $v^t=(t,v^t_2)$ the intersection points of $\partial B(x,\mathbb {H}(f,g))$ and the line $x_1=t$; with $u^t_2<v^t_2$. From the assumption on x, $d(x,H_g)={\mathbb H}(f,g)$. This entails that $\mathring{B}(x,\mathbb {H}(f,g))\cap H_g=\emptyset$ and, since $H_g$ is a hypograph (which implies that if $(a,b)\in H_g$ then the segment joining (a, b) and (a, 0) is included in $H_g$) it is clear that $g(t)\le u^t_2$, for all $t\in [0,1]$ with $|t-x_1|< \mathbb {H}(f,g)$. Therefore, if we move upwards the point $x=(x_1,x_2)$ to $(x_1,f(x_1))$ (recall that from the USC assumption, $x_2\le f(x_1)$), we have $\mathring{B}((x_1,f(x_1)),\mathbb {H}(f,g))\cap H_g=\emptyset$ and then $d((x_1,f(x_1)), H_g)\ge \mathbb {H}(f,g)$. We cannot have $d((x_1,f(x_1)), H_g)>\mathbb {H}(f,g)$ since $(x_1,f(x_1))\in H_f$ and $\mathbb {H}(f,g)=d_H(H_f,H_g)$. So, we must have $d((x_1,f(x_1)), H_g)= \mathbb {H}(f,g)$ with $u:=(x_1,f(x_1))\in \partial H_f$. As a consequence, we must also have a point $v\in \partial H_g$ such that $\Vert u-v\Vert = \mathbb {H}(f,g)$. This contradicts the assumption we made about the non-existence of such a pair (u, v). $\square$

We now prove Proposition 2.

Proof

Let us denote

$$\begin{aligned} d=\max \left\{ \sup _{\left\{ \begin{array}{c} x=(x_1,x_2)\in \partial H_g \\ g(x_1)\ge f(x_1) \end{array}\right\} }d\left( x,\partial H_f\right) ,\ \sup _{\left\{ \begin{array}{c} y=(y_1,y_2)\in \partial H_f \\ f(y_1)> g(y_1) \end{array}\right\} } d \left( y,\partial H_g\right) \right\} . \end{aligned}$$

The case $\mathbb {H}(f,g)=0$ is trivial, so let us assume $\mathbb {H}(f,g)>0$. We will first see that $\mathbb {H}(f,g)\le d$. Since $H_f$ and $H_g$ are compact, there are two possibilities:

• (1) there exists $x\in H_g$ such that $\mathbb {H}(f,g)=d(x,\partial H_f)$, or

• (2) there exists $y\in H_f$ such that $\mathbb {H}(f,g)= d(y,\partial H_g)$.

Let us suppose that we are in the first case. By Lemma 1 we can assume that $x=(x_1,x_2)\in \partial H_g$. Since $\mathbb {H}(f,g)>0$ and $H_f$ is a hypograph it must be $g(x_1)> f(x_1)$, then $x\in \{x=(x_1,x_2)\in \partial H_g: g(x_1)\ge f(x_1)\}$ from where it follows that $\mathbb {H}(f,g)\le d$. If we are in case 2, again by Lemma 1, we can assume $y=(y_1,y_2)\in \partial H_f$, as $\mathbb {H}(f,g)>0$ and $H_g$ is a hypograph it must be $f(y_1)>g(y_1)$, then $y \in \{y=(y_1,y_2)\in \partial H_f: f(y_1)> g(y_1) \}$. from where it follows that $\mathbb {H}(f,g)\le d$. The inequality ${\mathbb H}(f,g)\ge d$ follows directly from the definition of $\mathbb {H}$. $\square$

We also use the following proposition in the algorithm to calculate ${\mathbb H}(f,g)$ for f and g continuous.

Proposition 4

Let $f,g\in \mathcal {E}$ be continuous functions, let u and v be the points of Lemma 1. Then, there exist $t \in [0,1]$ and $s\in [0,1]$ such that $u=(t,f(t))$ and $v=(s,g(s))$.

Proof

Again, assume ${\mathbb H}(f,g)>0$. By Lemma 1 $u\in \partial H_f$, $v\in \partial H_g$, and $\mathbb {H}(f,g)=\Vert u-v\Vert$. So it is enough to prove that $u=(t,f(t))$ and $v= (s,g(s))$. Since f is continuous and $u\in \partial H_f$, there are four possibilities: (and the same holds for $v\in \partial H_g$):

• 1. u is in the left border: $u=(0,u_2)$ with $u_2<f(0)$.

• 2. u is in the right border: $u=(1,u_2)$ with $u_2<f(1)$.

• 3. u is in the lower border: $u=(u_1,0)$ with $0\le u_1\le 1$.

• 4. u is in the upper border: $0\le u_1\le 1$ y $u_2=f(u_1)$.

We now prove that u can only be in Case 4. It is clear that Case 3 is not possible because both functions are non-negative. Cases 1 and 2 are also excluded following the ideas used in Lemma 1. For example, let us suppose that we are in Case 1 (see Fig. 4). First observe that $\mathring{B}((0,f(0)),\mathbb {H}(f,g))\cap H_g=\emptyset$; otherwise there would exist $(t_1,t_2)\in \mathring{B} ((0,f(0)),\mathbb {H}(f,g))\cap H_g$, then, the segment joining the points $(t_1,0)$ and $(t_1,t_2)$ (which is included in $H_g$) intersects $\mathring{B} (u,\mathbb {H}(f,g))$. But this contradicts $\mathbb {H}(f,g)=d_H(H_f,H_g)$. So we conclude $d ((0,f(0)),H_g)\ge \mathbb {H}(f,g)$. However $d ((0,f(0)),H_g)> \mathbb {H}(f,g)$ leads to a contradiction with the definition of $\mathbb {H}(f,g)$. Also, $d(u,H_g)= d((0,f(0)),H_g)$ leads to another contradiction. Indeed, if this were the case, we would have two points ($(0,u_2)$ and (0, f(0))) on the vertical axis $x_1=0$ which are equidistant to the hypograph $H_g$. Then we have three possibilities:

• (a) $u_2<g(0)<f(0)$. This contradicts $d(u,H_g)= d ((0,f(0)),H_g)$, since all the points $(0,u_3)$ with $u_2<u_3<g(0)$ belong to $H_g$.

• (b) $g(0)\le u_2$: this contradicts the continuity of g since $H_g$ must have a point in the boundary of $B((0,f(0)), d((0,f(0)),H_g))$ and no point in the open ball $\mathring{B} ((0,u_2), d ((0,u_2),H_g))$.

• (c) $g(0)\ge f(0)$: this is not compatible with $d(u,H_g)= d ((0,f(0)),H_g)$.

Fig. 4

We cannot have $d(u,H_g)= d ((0,f(0)),H_g)$ with $u=(0,u_2)$ and $u_2<f(0)$

Proof of Theorem 1

(a) To state the local compactness we will in fact prove a slightly stronger property: we will show that any closed and bounded set in $({\mathcal E},{\mathbb H})$ is compact. Indeed, this would imply that the closed balls are compact. Since the family of balls with center at a given point is a local base, the local compactness will follow.

Since we are in a metric space compactness is equivalent to sequential compactness. Let us take $\{f_n\}\subset \mathcal {E}$ a bounded sequence; we will prove that this sequence has necessarily a convergent subsequence. To see this, note that the corresponding sequence of compact sets $H_{f_n}$ is bounded. So it has convergent subsequence, which we may denote again by $H_{f_n}$, in the Hausdorff metric (since the closed and bounded sets are compact in the space of compact sets with the Hausdorff metric). Denote by C the limit of that subsequence. Therefore it is enough to prove that

$$\begin{aligned}&\text{ if } \left\{ H_{f_n}\right\} _n \text{ is } \text{ fulfils } H_{f_n}\rightarrow C \text{ for } \text{ some } \text{ compact } \text{ set } C, \text{ then } \text{ there } \nonumber \\&\text{ exists } \text{ a } \text{ USC } \text{ function } f:[0,1]\rightarrow [0,\infty )\text{, } \text{ such } \text{ that } C=H_{f}. \end{aligned}$$

(5)

Let us take $(x,y)\in C$ and $(x_n,y_n)\in H_{f_n}$ converging to (x, y); note that there exists at least one such sequence because $d_H(H_{f_n},C)\rightarrow 0$. Now, since the $H_{f_n}$ are hypographs the vertical segment $[(x_n,0),(x_n,y_n)]$ joining the points $(x_n,0)$ and $(x_n,y_n)$ is included in $H_{f_n}$. So $H_{f_n}\rightarrow C$, implies

$$\begin{aligned} \left[ \left( x,0\right) ,\left( x,\limsup f_n(x_n)\right) \right] \subset C. \end{aligned}$$

(6)

Indeed, since $H_{f_n}$ is a hypograph, $f_n(x_n)\ge y_n$. Then if we take $\limsup$ we obtain $y\le \limsup f_n(x_n)$ and $\{(x,z): 0\le z \le \limsup f(x_n)\}\subset C$.

Let us now define $f:[0,1]\rightarrow [0,\infty )$ by

$$\begin{aligned} f(x)= \sup _{\{x_n\}:x_n\rightarrow x} \limsup f_n(x_n). \end{aligned}$$

Since $\{f_n\}$ is bounded, f is well defined as a real-valued function. Let us prove that $C=H_f$. Since C is closed, we have, by (6), $H_{f}\subset C$. Moreover, if $(x,y)\in C$, taking $(x_n,y_n)\rightarrow (x,y)$ with $(x_n,y_n)\in H_{f_n}$, $f(x)\ge \limsup f_n(x_n)\ge \limsup y_n=\, y$, we obtain $(x,y)\in H_f$.

It remains to prove that f is USC. Suppose by contradiction that there exists $a \in [0,1]$ such that $\limsup _{x\rightarrow a}f(x)>f(a)$. Then, we can take a constant $\delta >0$ and a sequence $x_n\rightarrow a$, $x_n\ne a$ for all n such that $f(x_n)>f(a)+\delta$ for n large enough, say $n>n_0$. By the definition of f, for every $x_n$ we can take a sequence $z^k_n\rightarrow _k x_n$ (dependent on n), such that $f(x_n)=\lim _{z^k_n\rightarrow _k x_n} \limsup f_k(z^k_n)$.

Given $\varepsilon >0$, for every $n>n_0$ let us take an increasing sequence $k(n)>n_0$ with

$$\begin{aligned} \left| z_n^{k(n)}-x_n\right| <\frac{1}{n}\quad \text{ and }\quad \left| f_{k(n)}(z_n^{k(n)})-\limsup f_k(z^k_n)\right| <\varepsilon , \end{aligned}$$

that is, $|f_{k(n)}(z_n^{k(n)})-f(x_n)|<\varepsilon$. But as $z_n^{k(n)}\rightarrow _n a$ this contradicts $f(x_n)>f(a)+\delta$ for $n>n_0$.

Completeness follows directly from the fact that the space of compact sets endowed with the Hausdorff metric is complete, together with (5).

To prove separability, let ${\mathcal P}_n$ be the set of all partitions of [0, 1] defined by $0=x_0<x_1<\dots <x_{n-1}<x_n=1$ where the $x_i$ are rational numbers. Denote $\mathcal {P}=\cup _n {\mathcal P}_n$. Note that $\mathcal {P}$ in numerable.

Given a partition $\mathcal {P}\in \mathcal {P}_n$ and a set $q_0,\dots ,q_{n-1}$ of rational numbers, let us define

$$\begin{aligned} f_{\mathcal {P}}(x)={\left\{ \begin{array}{ll} q_0 &{} \text { if } x \in [0,x_1)\\ q_i &{}\text { if } x\in (x_i,x_{i+1})\quad 1 \le i \le n-3\\ q_{n-1} &{}\text { if } x\in (x_{n-1},1]\\ \max \{q_i,q_{i+1}\} &{}\text { if } x=x_i \quad 1 \le i \le n-2 \end{array}\right. } \end{aligned}$$

(7)

It is immediately seen that this function is USC and bounded. Let us see that the (numerable) set of all functions defined by (7), for all possible partitions $\mathcal {P}$ and rational values $q_i$ is dense in $\mathcal {E}$ with respect to $\mathbb {H}$. Let f be a non-negative USC function and take $\varepsilon >0$. Consider $\mathcal {P}\in \mathcal {P}_n$ a partition of the form $0=x_0<x_1<\dots <x_{n-1}<x_n=1$ where $x_i$ are rational numbers and such that $\max _{i=0,\dots ,n-1} |x_{i+1}-x_i|<\varepsilon /2$. By Proposition 1 there exists $f_i=\max _{x\in [x_i,x_{i+1}]} f(x)$. Let us take $q_0,\dots ,q_{n-1}$ rational numbers such that $q_i>f_i$ and $q_i-f_i<\varepsilon /2$ for all i. For this partition and this set of rational numbers let us define $f_{\mathcal {P}}$ as in (7). Now we claim that $\mathbb {H}(f_\mathcal {P},f)\le \varepsilon$. Indeed, it is clear that $f_\mathcal {P}(x)>f(x)$ for all $x\in [0,1]$ so that $H_f\subset H_{f_\mathcal {P}}$, and $\mathbb {H}(f_\mathcal {P},f)= \sup _{z\in H_{f_\mathcal {P}}} d(z,H_f).$ Given $z=(z_1,z_2)\in H_{f_\mathcal {P}}$, there exists $0\le i_0 \le n-1$ such that $x_{i_0}\le z_1\le x_{i_0+1}$. Now, choose t such that $x_{i_0}\le t\le x_{i_0+1}$ and $f_{i_0}= f(t)$. We have $z_2< f(t)+\varepsilon /2$ and then $d(z,H_f)< \varepsilon$. Since z was an arbitrary point in $H_{f_\mathcal {P}}$ we finally get $\sup _{z\in H_{f_\mathcal {P}}} d(z,H_f)\le \epsilon$.

(b) By Proposition 1 (i) we know that there exists $z\in [0,1]$ such that $f(z)=\max _{x\in [0,1]} f(x)$. As $\mathbb {H}(f_n,f)\rightarrow 0$ there exist $x^n=(x_1^n,x_2^n)\in H_{f_n}$ such that $x_n\rightarrow (z,f(z))$. Then, $x^n_2\le f_n(x_1^n)\le \max _{x\in [0,1]} f_n(x)$ and, since $x_2^n\rightarrow f(z)$, we obtain

$$\begin{aligned} \max _{x\in [0,1]} f(x) = f(z)\le \liminf _{n\rightarrow +\infty } \max _{x\in [0,1]} f_n(x) \le \limsup _{n\rightarrow +\infty } \max _{x\in [0,1]} f_n(x). \end{aligned}$$

Finally, let us prove that $\limsup _n \max _{x\in [0,1]} f_n(x) \le \max _{x\in [0,1]} f(x)$. Denote $z_0=\limsup _{n} \max _{x\in [0,1]} f_n(x)$. There exists $x_n\in [0,1]$ such that $f_n(x_n)\rightarrow z_0$ with $f_n(x_n)= \max _{x\in [0,1]} f_n(x)$. Taking if necessary a subsequence, we can assume that $x_n\rightarrow x_0\in [0,1]$. Since $(x_0,z_0)\in H_f$ we have $f(x_0)\ge z_0$ then $\max _{x\in [0,1]} f(x)\ge z_0$.

Proof of Proposition 3

Proof

This result is just a direct corollary from Th. 2 in Cérou and Guyader (2006) (recall that the continuity of $\eta (x)$ is a sufficient condition for (2)), combined with the fact that the regression function $\eta _Q(x)={\mathbb P}(Y=1|X=x)$ (i.e., the regression function under Q) can be approximated by a continuous compact supported function; we use here the local compactness of ${\mathcal E}$ (see Folland 1999, Proposition 7.9). Indeed, note that the joint distribution of (X, Y) is completely determined by $\eta (x)={\mathbb P}(Y=1|X=x)$ and by the marginal distribution $\mu$ of X. Then, given Q, one can construct P by just approximating $\eta _Q(x)={\mathbb P}(Y=1|X=x)$ by a continuous compact-supported function $\eta _P(x)$ which, without loss of generality, can be taken $0\le \eta _P\le 1$. Then, the distribution P determined by $\eta _P$ and the marginal distribution $\mu$ of X is arbitrarily close to Q (just taking $\eta _P$ close enough to $\eta$). Indeed, given any Borel set $C \subset \mathcal E \times \{0,1\}$, consider the sets $C_0 = \{x \in \mathcal E: (x,0) \in C\}$ and $C_1 = \{x \in \mathcal E: (x,1) \in C\}$. Then,

$$\begin{aligned} Q(C) = \int _{C_0} (1-\eta _Q(x)) d\mu (x) + \int _{C_1} \eta _Q(x) d\mu (x), \end{aligned}$$

and

$$\begin{aligned} P(C) = \int _{C_0} (1-\eta _P(x)) d\mu (x) + \int _{C_1} \eta _P(x) d\mu (x), \end{aligned}$$

which can be made arbitrarily close. $\square$