Functional summaries of persistence diagrams

Abstract

One of the primary areas of interest in applied algebraic topology is persistent homology, and, more specifically, the persistence diagram. Persistence diagrams have also become objects of interest in topological data analysis. However, persistence diagrams do not naturally lend themselves to statistical goals, such as inferring certain population characteristics, because their complicated structure makes common algebraic operations—such as addition, division, and multiplication—challenging (e.g., the mean might not be unique). To bypass these issues, several functional summaries of persistence diagrams have been proposed in the literature (e.g. landscape and silhouette functions). The problem of analyzing a set of persistence diagrams then becomes the problem of analyzing a set of functions, which is a topic that has been studied for decades in statistics. First, we review the various functional summaries in the literature and propose a unified framework for the functional summaries. Then, we generalize the definition of persistence landscape functions, establish several theoretical properties of the persistence functional summaries, and demonstrate and discuss their performance in the context of classification using simulated prostate cancer histology data, and two-sample hypothesis tests comparing human and monkey fibrin images, after developing a simulation study using a new data generator we call the Pickup Sticks Simulator (STIX).

Appendices

A Proofs

The proofs from the propositions of Sect. 3 are presented below.

Proof of Proposition 1

This proof uses ideas from Rubin (1956) and Yuan (1997).

Let $\epsilon >0$ be given and ${\widehat{F}}_n = {\widehat{F}} = \frac{1}{n}\sum _{i=1}^n F_i$. Because ${\mathcal {B}}_F$ is equicontinuous, the collection of differences

$$\begin{aligned} {\mathcal {B}}_{\varDelta } = \{{\widehat{F}}_n - {\bar{F}}:n = 1,2,\ldots \} \end{aligned}$$

is also equicontinuous. Since ${\mathbb {T}}$ is compact, there exists a number $M>0$ and points $t_1,\ldots ,t_M\in {\mathbb {T}}$ such that

$$\begin{aligned} \sup _{\varDelta \in {\mathcal {B}}_{\varDelta }} \min _{j =1,\ldots ,M}|\varDelta (t)-\varDelta (t_j)| < \epsilon /2. \end{aligned}$$

Namely, $t_1,\ldots ,t_M$ forms a $\epsilon /2$-covering of ${\mathcal {B}}_{\varDelta }$.

Let $\varDelta _n(t) = {\widehat{F}}(t)-{\bar{F}}(t)$, and note that $\varDelta _n\in {\mathcal {B}}_{\varDelta }$. Then

$$\begin{aligned} \sup _{t\in {\mathbb {T}}}|{\widehat{F}}(t)-{\bar{F}}(t)|&= \sup _{t\in {\mathbb {T}}}|\varDelta _n(t)|\\&\le \sup _{t\in {\mathbb {T}}} \min _{j\in 1,\ldots ,M} |\varDelta _n(t)-\varDelta _n(t_j)| + \sup _{j\in 1,\ldots ,M} |\varDelta _n(t_j)|\\&\le \epsilon /2 + \sup _{j\in 1,\ldots ,M} |\varDelta _n(t_j)|. \end{aligned}$$

By (5), every $t\in {\mathbb {T}}$ satisfies ${{\mathbb {E}}}|F_i(t)|<\infty$ so by the strong law of large numbers,

$$\begin{aligned} |\varDelta _n(t_j)| \overset{a.s.}{\rightarrow } 0 \end{aligned}$$

for every $j=1,\ldots ,M$ and this implies that for any $\delta >0$, there exists $N>0$ such that

$$\begin{aligned} P\left( \sup _{n\ge N}|\varDelta _n(t_j)|>\frac{\epsilon }{2M}\right) <\frac{\delta }{M} \end{aligned}$$

for every $j=1,\ldots ,M$.

As a result,

$$\begin{aligned} P\left( \sup _{t\in {\mathbb {T}}}|{\widehat{F}}(t)-{\bar{F}}(t)|>\epsilon \right)&\le P\left( \epsilon /2 + \sup _{j\in 1,\ldots ,M} |\varDelta _n(t_j)|>\epsilon \right) \\&\le P\left( \sup _{n\ge N} \sup _{j\in 1,\ldots ,M}|\varDelta _n(t_j)|>\frac{\epsilon }{2}\right) \\&\le \sum _{j=1}^M P\left( \sup _{n\ge N}|\varDelta _n(t_j)|>\frac{\epsilon }{2M}\right) \le \delta \end{aligned}$$

and the result follows. $\square$

Proof of Corollary 1

The case of persistence landscapes is just a special case of the generalized landscape so we will only prove the case of persistence intensity and generalized landscape. Note that Chazal et al. (2014) already show that persistence silhouettes with a power-weighted silhouette is 1-Lipschitz so the equicontinuity follows so we ignore this case.

Persistence intensity Given a persistence diagram ${\textsf {D}}$, the persistence intensity is the function

$$\begin{aligned} F(t,s;{\textsf {D}})={\mathbb {F}}({\textsf {D}};t,s) = \frac{1}{\left| {\textsf {D}}\right| }\sum _{j=1}^{\left| {\textsf {D}}\right| }\omega (d_j-b_j) \cdot K\left( \frac{\sqrt{|b_j-t|^2+|d_j-s|^2}}{h}\right) , \end{aligned}$$

Thus, for any $(t_1,s_1)$ and $(t_2,s_2)$,

$$\begin{aligned} \left| F(t_1,s_1;{\textsf {D}}) - F(t_2,s_2;{\textsf {D}}) \right|&= \left| \frac{1}{\left| {\textsf {D}}\right| }\sum _{j=1}^{\left| {\textsf {D}}\right| }\omega (d_j-b_j) \cdot \left\{ K\left( \frac{\sqrt{|b_j-t_1|^2+|d_j-s_1|^2}}{h}\right) \right. \right. \\&\quad \left. \left. - K\left( \frac{\sqrt{|b_j-t_2|^2+|d_j-s_2|^2}}{h}\right) \right\} \right| \\&\le \sup _{a,b\in {\mathbb {T}}}\Vert \omega (a,b)\Vert \times \left| K\left( \frac{\sqrt{|b_j-t_1|^2+|d_j-s_1|^2}}{h}\right) \right. \\&\quad \left. - K\left( \frac{\sqrt{|b_j-t_2|^2+|d_j-s_2|^2}}{h}\right) \right| \\&\le \sup _{a,b\in {\mathbb {T}}}\Vert \omega (a,b)\Vert L_0 \cdot \frac{\sqrt{(t_1-t_2)^2+(s_1-s_2)^2}}{h}, \end{aligned}$$

where $L_0$ is the Lipschitz constant of the kernel function K, i.e., $|K(x)-K(y)|\le L_0|x-y|$. The bound $\sup _{a,b\in {\mathbb {T}}}\Vert \omega (a,b)\Vert L_0/h$ is uniform for all $(t_1,s_1)$ and $(t_2,s_2)$ so the corresponding persistence intensity functions form an equicontinuous family.

Generalized landscape Recall that generalized landscapes use the kth maximum value of $\{{\widetilde{\varLambda }}_j(t; h):j =1,\ldots , |{\textsf {D}}|\}$, where

$$\begin{aligned} {\widetilde{\varLambda }}_j(t; h) = {\left\{ \begin{array}{ll} \frac{y_j}{K_h(0)} K_h\left( {t - x_j}\right) , &{}\quad \text {for } | \frac{t - x_j}{h} | \le 1 \\ 0, &{}\quad \text {otherwise, } \end{array}\right. } \end{aligned}$$

and $y_j =|b_j-d_j|$ is the persistence, $x_j = \frac{b_j+d_j}{2}$, and $K_h(\cdot /h) = \frac{1}{h}K(\cdot /h)$. Therefore, for any $t_1,t_2$,

$$\begin{aligned} |{\widetilde{\varLambda }}_j(t_1; h)-{\widetilde{\varLambda }}_j(t_2; h)|&\le \frac{y_j}{K_h(0)} |K_h\left( {t_1 - x_j}\right) -K_h\left( {t_2 - x_j}\right) |\\&\le \frac{y_j}{K(0)/h} \frac{1}{h}\left| K\left( \frac{t_1 - x_j}{h}\right) -K\left( \frac{t_2 - x_j}{h}\right) \right| \\&\le \frac{\mathsf{diam}({\mathbb {T}})}{K(0)} L_0 \frac{|t_1-t_2|}{h}, \end{aligned}$$

where $L_0$ is the Lipschitz constant of the kernel function K and $\mathsf{diam}({\mathbb {T}})$ is the diameter of ${\mathbb {T}}$. Thus, for a given $h>0$, ${\widetilde{\varLambda }}_j(t; h)$ is Lipschitz with a constant $\frac{\mathsf{diam}({\mathbb {T}}) L_0}{K(0)h}$. Moreover, this analysis applies to every $j=1,\ldots , |{\textsf {D}}|$ so the kth maximum value of $\{{\widetilde{\varLambda }}_j(t; h):j =1,\ldots , |{\textsf {D}}|\}$ is still Lipschitz with the same constant. This implies that the generalized landscape forms an equicontinuous family. $\square$

Proof of Proposition 2

Note that by Eq. (5), the functional summary satisfies

$$\begin{aligned} \sup _{t\in {\mathbb {T}}}\mathsf{Var}(F_i(t)) \le \sup _{t\in {\mathbb {T}}}{{\mathbb {E}}}(F^2_i(t))\le {\bar{U}}^2<\infty . \end{aligned}$$

Pointwise normality The first assertion $\sqrt{n}({\widehat{F}}(t)-{\bar{F}}(t))\rightarrow N(0,\sigma ^2(t))$ follows from the usual central limit theorem because the variance is finite.

Normality of integrated difference To show the normality for the integrated difference, note that

$$\begin{aligned} \sqrt{n}\int ({\widehat{F}}(t)-{\bar{F}}(t))dt&= \sqrt{n}\int \left( \frac{1}{n}\sum _{i=1}^n F_i(t) - {\bar{F}}(t)\right) dt\\&= \sqrt{n} \frac{1}{n}\sum _{i=1}^n \underbrace{\int \left( F_i(t)-{\bar{F}}(t)\right) dt}_{=Y_i}\\&= \sqrt{n} {\bar{Y}}_n, \end{aligned}$$

where ${\bar{Y}}_n = \frac{1}{n}\sum _{i=1}^n Y_i$ and $Y_1,\ldots ,Y_n$ are i.i.d. with mean ${{\mathbb {E}}}(Y_i) = 0$ and variance

$$\begin{aligned} \mathsf{Var}(Y_i )= \int \mathsf{Var}(F_i(t))dt = \int \sigma ^2(t)dt<\infty . \end{aligned}$$

Thus, by the usual central limit theorem again, we obtain the normality.

Convergence to a Gaussian process The proof of convergence toward a Gaussian process involves the concept of a Donsker class; see Section 2.2 of Kosorok (2007) for an introduction. Simply put, given i.i.d. random elements $S_1,\ldots , S_n\in {\mathcal {S}}$ and a collection of mappings $f_t: {\mathcal {S}}\mapsto {\mathbb {R}}$ with an index set $t\in {\mathbb {T}}$, the collection $\{f_t: t\in {\mathbb {T}}\}$ is called Donsker if the average function ${\bar{f}}(t) = \frac{1}{n}\sum _{i=1}^nf_t(S_i)$ converges to a tight Gaussian process in ${\mathcal {L}}_\infty$ metric. The assumption in Eq. (6) along with Theorem 2.5 in Kosorok (2007) implies that the class ${\mathcal {W}}$ is Donsker, which implies that the sample mean functional ${\widehat{F}}(t)$ converges to a Gaussian process. $\square$

Proof of of Proposition 3

Because Eq. (6) implies that the class ${\mathcal {W}}$ is Donsker, this proposition is a well-known result in the empirical process theory. For instance, see the discussion on page 21 of Kosorok (2007). Here we briefly highlight the basic idea.

By Proposition 3 and the continuous mapping theorem [see, e.g., page 16 of Kosorok (2007)],

$$\begin{aligned} \sqrt{n}\sup _{t\in {\mathbb {T}}}|{\widehat{F}}(t)-{\bar{F}}(t)| \overset{D}{\rightarrow } \sup _{t\in {\mathbb {T}}}|{\mathbb {B}}(t)|. \end{aligned}$$

In the case of the bootstrap, using the same proposition but now apply it to the bootstrap version, we obtain

$$\begin{aligned} \sqrt{n}\sup _{t\in {\mathbb {T}}}|{\widehat{F}}^*(t)-{\widehat{F}}(t)| \overset{D|F^{\otimes n}}{\rightarrow } \sup _{t\in {\mathbb {T}}}|{\mathbb {B}}(t)|, \end{aligned}$$

where $\overset{D|F^{\otimes n}}{\rightarrow }$ denotes convergences in distribution given $F_1,\ldots ,F_n$.

Therefore, the bootstrap quantile converges to the corresponding quantile of the original difference. $\square$

Proof of Proposition 4

This proof consists of two parts. In the first part, we prove that ${\widehat{q}}_\gamma \overset{P}{\rightarrow } q_\gamma$. In the second part, we prove the desired result.

Part 1 Given $F_1,\ldots ,F_n$, we define

$$\begin{aligned} {\widehat{Q}}(t) = \frac{1}{n}\sum _{i=1}^n I(d(F_i,{\bar{F}})\le t) \end{aligned}$$

to be the empirical version of Q(t) and define

$$\begin{aligned} {\widetilde{q}}_\gamma : {\widehat{Q}}({\widetilde{q}}_\gamma ) = \gamma , \end{aligned}$$

to be the corresponding $\gamma$ quantile.

Because ${\widehat{Q}}$ is just the empirical distribution function (EDF) of $Y_1,\ldots ,Y_n$ where $Y_i = d(F_i,{\bar{F}})$ and Q(t) is the cumulative distribution function (CDF) of $Y_i$,

$$\begin{aligned} \sup _{t} |{\widehat{Q}}(t)-Q(t)| = O_P\left( \frac{1}{\sqrt{n}}\right) . \end{aligned}$$

By the mean value theorem and the fact that ${\widehat{Q}}({\widetilde{q}}_\gamma ) -Q({\widetilde{q}}_\gamma ) = o_P(1)$,

$$\begin{aligned} {\widehat{Q}}({\widetilde{q}}_\gamma ) -Q({\widetilde{q}}_\gamma )&=Q(q_\gamma ) -Q({\widetilde{q}}_\gamma ) \\&= Q'(q_\gamma ^*) (q_\gamma -{\widetilde{q}}_\gamma ) \end{aligned}$$

for some $q_\gamma ^* \in [q_\gamma , {\widetilde{q}}_\gamma ]$. By assumption, $Q'(q_\gamma ^*)\ge q_0>0$ so

$$\begin{aligned} |q_\gamma -{\widetilde{q}}_\gamma | \le \frac{1}{q_0} \left| {\widehat{Q}}({\widetilde{q}}_\gamma ) -Q({\widetilde{q}}_\gamma )\right| =O_P\left( \frac{1}{\sqrt{n}}\right) . \end{aligned}$$

Now, because $\max {i=1,\ldots ,n}|d(F_i,{\bar{F}})-d(F_i,{\widehat{F}})| \le d({\bar{F}},{\widehat{F}})$, ${\widetilde{q}}_\gamma$, the quantile of $d(F_1,{\bar{F}}),\ldots , d(F_n,{\bar{F}})$, and ${\widehat{q}}_\gamma$, the quantile of $d(F_1,{\widehat{F}}),\ldots , d(F_n,{\widehat{F}})$, are bounded by

$$\begin{aligned} |{\widetilde{q}}_\gamma -{\widehat{q}}_\gamma | \le d({\bar{F}},{\widehat{F}}). \end{aligned}$$

which implies

$$\begin{aligned} |{\widehat{q}}_\gamma - q_\gamma |=O_P\left( \frac{1}{\sqrt{n}}\right) +d({\bar{F}},{\widehat{F}}). \end{aligned}$$

Part 2 Let

$$\begin{aligned} {\widehat{A}}_\gamma = P(F_\mathsf{new}\subset \widehat{{\mathcal {F}}}_\gamma |F_1,\ldots ,F_n) =P(d(F_\mathsf{new}, {\widehat{F}})\le {\widehat{q}}_\gamma |F_1,\ldots ,F_n). \end{aligned}$$

be the probability of interest. By the triangular inequality,

$$\begin{aligned} |d(F_\mathsf{new}, {\bar{F}})-d(F_\mathsf{new}, {\widehat{F}}) | \le d({\bar{F}}, {\widehat{F}}). \end{aligned}$$

Thus,

$$\begin{aligned} A_\gamma&= P(d(F_\mathsf{new}, {\widehat{F}})\le {\widehat{q}}_\gamma |F_1,\ldots ,F_n) \\&\ge P(d(F_\mathsf{new}, {\bar{F}})\le {\widehat{q}}_\gamma -d({\widehat{F}},{\bar{F}})|F_1,\ldots ,F_n)\\&= Q({\widehat{q}}_\gamma -d({\widehat{F}},{\bar{F}}))\\&\ge Q\left( q_\gamma -2d({\widehat{F}},{\bar{F}})+O_P\left( \frac{1}{\sqrt{n}}\right) \right) \\&= Q(q_\gamma ) + O_P\left( \left( \frac{1}{\sqrt{n}}\right) +d({\widehat{F}},{\bar{F}})\right) .\\ A_\gamma&= P(d(F_\mathsf{new}, {\widehat{F}})\le {\widehat{q}}_\gamma |F_1,\ldots ,F_n) \\&\le P(d(F_\mathsf{new}, {\bar{F}})\le {\widehat{q}}_\gamma +d({\widehat{F}},{\bar{F}})|F_1,\ldots ,F_n)\\&=Q({\widehat{q}}_\gamma +d({\widehat{F}},{\bar{F}}))\\&\le Q\left( q_\gamma +2d({\widehat{F}},{\bar{F}})+O_P\left( \frac{1}{\sqrt{n}}\right) \right) \\&= Q(q_\gamma ) + O_P\left( \left( \frac{1}{\sqrt{n}}\right) +d({\widehat{F}},{\bar{F}})\right) .\\ \end{aligned}$$

Thus,

$$\begin{aligned} A_\gamma = P(F_\mathsf{new}\subset \widehat{{\mathcal {F}}}_\gamma |F_1,\ldots ,F_n)= \underbrace{Q(q_\gamma )}_{=\gamma }+ O_P\left( \left( \frac{1}{\sqrt{n}}\right) +d({\widehat{F}},{\bar{F}})\right) , \end{aligned}$$

which completes the proof. $\square$

Fig. 16

STIX simulation results for homology dimension 0 by landscape and generalized landscape function order. The median permutation p values (a–c) and $\log _{10}$ (p values) (d–f) are plotted along with their interquartile range (the vertical lines) for two-sample hypothesis tests comparing samples drawn from the null population, ${\mathcal {P}}_{F,1}$, with an average thickness of $t_1 = 5$. The alternative hypotheses include average thicknesses of $t_2 = 5, 5.25, 5.5, 5.75, 6, 7, 8$, corresponding to images (a–f), respectively (except average thickness of 8 is not displayed). The permutation p values are based on 100 repetitions of 12 STIX images drawn from the null and alternative hypothesis, with 10,000 random permutations. The different plot colors and symbols represent the different functions and bandwidths considered; the function order is the ordering of the landscape and generalized landscape functions (color figure online)

Fig. 17

STIX simulation results for homology dimension 1 by landscape and generalized landscape function order. The median permutation p values (a–c) and $\log _{10}$ (p values) (d–f) are plotted along with their interquartile range (the vertical lines) for two-sample hypothesis tests comparing samples drawn from the null population, ${\mathcal {P}}_{F,1}$, with an average thickness of $t_1 = 5$. The alternative hypotheses include average thicknesses of $t_2 = 5, 5.25, 5.5, 5.75, 6, 7, 8$, corresponding to images (a–f), respectively (except average thickness of 8 is not displayed). The permutation p values are based on 100 repetitions of 12 STIX images drawn from the null and alternative hypothesis, with 10,000 random permutations. The different plot colors and symbols represent the different functions and bandwidths considered; the function order is the ordering of the landscape and generalized landscape functions (color figure online)

B STIX simulation results

For completeness, results for the STIX simulation study from Sect. 4.2.1 are displayed for $t_2 = 5, 5.25, 5.5, 5.75, 6, 7$. Figures 16 and 17 include the 0th and 1st homology dimensions, respectively, for the landscape and generalized landscape functions for their first 10 function orders; a subset of the results for the 1st homology dimension are displayed in Fig. 12 and discussed in the main text. Note that the results for the setting with $t_2 = 8$ are not displayed because the permutation p values for all function orders for the 1st homology dimension achieve the minimum possible p value, and most of the function orders for the 0th homology dimension achieve this minimum as well.

Similarly, the results for the other functional summaries and the concatenated orders of the landscape and generalized landscape functions for $t_2 = 5, 5.25, 5.5, 5.75, 6, 7$ are displayed in Figs. 18 and 19 for the 0th and 1st homology dimensions, respectively. A subset of the results for the 1st homology dimension are displayed in Fig. 13 and discussed in the main text. Note that the results for the setting with $t_2 = 8$ are not displayed because the permutation p values for all functional summaries for the 1st homology dimension achieve the minimum possible p value, and all functional summaries (except the PDT for the 0th homology dimension) achieve this minimum as well.

Fig. 18

STIX simulation results for homology dimension 0. The median permutation p values (a–c) and $\log _{10}$ (p values) (d–f) are plotted along with their interquartile range (the vertical lines) for two-sample hypothesis tests comparing samples drawn from the null population, ${\mathcal {P}}_{F,1}$, with an average thickness of $t_1 = 5$. The alternative hypotheses include average thicknesses of $t_2 = 5, 5.25, 5.5, 5.75, 6, 7, 8$, corresponding to images (a–f), respectively (except average thickness of 8 is not displayed). The permutation p values are based on 100 repetitions of 12 STIX images drawn from the null and alternative hypothesis, with 10,000 random permutations. The different plot colors and symbols represent the different functional summaries and bandwidths considered; the landscape and generalized landscape functions used orders 1–10 (color figure online)

Fig. 19

STIX simulation results for homology dimension 1. The median permutation p values (a–c) and $\log _{10}$ (p values) (d–f) are plotted along with their interquartile range (the vertical lines) for two-sample hypothesis tests comparing samples drawn from the null population, ${\mathcal {P}}_{F,1}$, with an average thickness of $t_1 = 5$. The alternative hypotheses include average thicknesses of $t_2 = 5, 5.25, 5.5, 5.75, 6, 7, 8$, corresponding to images (a–f), respectively (except average thickness of 8 is not displayed). The permutation p values are based on 100 repetitions of 12 STIX images drawn from the null and alternative hypothesis, with 10,000 random permutations. The different plot colors and symbols represent the different functional summaries and bandwidths considered; the landscape and generalized landscape functions used orders 1–10 (color figure online)

C Alternative generalized landscape function kernel results

1.1 C.1 Simulated Gleason data

Figure 20 displays classification test error results for generalized landscape functions using an alternative kernel function (the tricubic kernel) to the one considered in Sect. 4.1 (the triangle kernel). The ratio of the classification test errors are plot with the tricubic kernel in the numerator and the corresponding triangle kernel plotted in the denominator. The ratios vary between about 0.98 and 1.02 suggesting that the kernel choice is not strongly affecting the results in this setting.

Fig. 20

Classification test error ratios of gland classification tests results for generalized landscape functions using tricubic or triangle kernels. The different plot colors and symbols represent the different functions and bandwidths considered; the function order is the ordering of the generalized landscape functions. The ratios are the gland classification test results for the tricubic kernel function divided by the gland classification test results for the corresponding triangle kernels (as displayed in Fig. 8). The dashed line at a ratio of 1 is where the two kernels have identical classification test error (color figure online)

Fig. 21

STIX simulation results using a tricubic kernel function for the generalized landscape functions. The median permutation $\log _{10}$ (p values) using the tricubic kernel function are plotted for a$H_0$ and b$H_1$ along with their interquartile range (the vertical lines) for two-sample hypothesis tests comparing samples drawn from the null population, ${\mathcal {P}}_{F,1}$, with an average thickness of $t_1 = 5$. The alternative hypothesis displayed have an average thicknesses of $t_2 = 5.75$. The permutation p values are based on 100 repetitions of 12 STIX images drawn from the null and alternative hypothesis, with 10,000 random permutations. The ratio of the median p values using the tricubic kernel function over the median p values using the triangle kernel function are displayed in c, d for the $H_0$ and $H_1$, respectively. The different plot colors and symbols represent the different functions and bandwidths considered; the function order is the ordering of the landscape and generalized landscape functions. The dashed line at a ratio of 1 is where the two kernels have identical classification test error (color figure online)

1.2 C.2 Pick-up sticks simulation data

Figure 21 displays the two-sample hypothesis test results for generalized landscape functions using an alternative kernel function (the tricubic kernel) for the setting with samples drawn from the null population, ${\mathcal {P}}_{F,1}$, with an average thickness of $t_1 = 5$ and an alternative hypothesis with an average thicknesses of $t_2 = 5.75$. Additionally, a comparison between the median p values using the tricubic kernel compared to the triangle kernal considered in Sect. 4.2.1. There appears to be more variability in these results compared to what we saw in “Appendix C.1”; however, the p values used in this setting tend to be smaller values than the classification test error values so a larger variation may imply only a small magnitude difference. For $H_0$, the largest magnitude difference in p values (across all function orders and thicknesses considered) is 0.089 (in the setting with $t_2 = 5$, function order 3, bandwidth of 0.60), with an interquartile range of 0 to 0.014. For $H_1$, the largest magnitude difference in p values (across all function orders and thicknesses considered) is 0.065 (in the setting with $t_2 = 5$, function order 2, bandwidth of 0.40), with an interquartile range of 0 to 0.006. There does not appear to be a discernible pattern in the variations between the tricubic and triangle kernel function results.