Plant-insect interactions in the upper Oligocene of Enspel (Westerwald, Germany), including an extended mathematical framework for rarefaction

Abstract

Insect herbivory as indicated by damage to leaves is an important tool in understanding insect diversity in deep time. Here we use leaves from the Oligocene Enspel Fossil-Lagerstätte (Westerwald, Germany) to examine changes in insect damage with a high temporal resolution. The layer-by-layer changes are interpreted as the effects of migration among insects. Extending the statistical framework employed in studying biodiversity, explicit solutions are given for rarefaction problems, and these methods are applied to the data. In particular, the analytical solutions are developed further to allow individuals to belong to multiple classes (here, leaves to have multiple types of damage) and to reconstruct exact probability distributions for the richness estimators. This allows non-parametric comparisons between datasets and resolves issues like the asymmetry of confidence intervals. Additionally, rarefaction was used to detect collection effects that can produce misleading results, and the dataset was restricted to a range where no bias was detected.

Appendix—analytical method

Firstly, a data matrix H is defined, where H(i,j) = 1 if the ith individual belongs to class j and H(i,j) = 0 if the ith individual does not belong to class j.

We then define

$$ {a}_{l_1\dots {l}_k}={\displaystyle \sum_{i=1}^n1-{\displaystyle \prod_{j=1}^k\left(1-H\left(i,{l}_j\right)\right.}} $$

(1)

with N being the number of individuals in the original data matrix.

Therefore, a gives the number of individuals that do belong to at least one of the indexed classes.

We further define two functions g(N,a,n) for hypergeometric and for binomial resampling:

$$ {g}_{\mathrm{hypergeometric}}\left(N,a,n\right)=\frac{\left(N-n\right)!\left(N-a\right)!}{\left(N-a-n\right)!N!}={\displaystyle \prod_{i=1}^a\frac{N-i-n+1}{N-i+1}}={\displaystyle \prod_{j=1}^n\frac{N-j-a+1}{N-j+1}} $$

(2)

$$ {g}_{\mathrm{binomial}}\left(N,a,n\right)={\left(\frac{N-a}{N}\right)}^n $$

(3)

introducing the parameter n, the size of the drawn sample.

g(N,a,n) gives the probability of drawing none of a subset of size a, when drawing n individuals form a total sample of size N. The hypergeometric model gives the probability without replacement; the binomial model gives the probability with replacement. It is worth noting that g _{hypergeometric} ≤ g _binomial, with equality iff n = 0, n = 1 or a = 0.

We are interested in reconstructing the probability distribution of a random variable X that gives us the number of classes not contained in a sample of size n. It should be noted that the probability distribution of X is conditioned on the matrix H. Using the number of classes not contained is done for ease of calculation.

Due to the linearity of expected values, we find that the expectation of X is given by

$$ E(X)={\displaystyle \sum_{i=1}^mg\left(N,{a}_i,n\right)} $$

(4)

where m is the total number of classes in the original dataset, while the expectation of X ² (the second-order moment) is given by

$$ E\left({X}^2\right)={\displaystyle \sum_{j=1}^m{\displaystyle \sum_{i=1}^mg\left(N,{a}_{i,j},n\right)}} $$

(5)

Equations (4) and (5) allow the recovery of the earlier exact formula for the mean and variance (Heck et al. 1975) but expand it to include cases where individuals can belong to more than one class.

In general, the expectation of X ^k or the kth order moment is given by

$$ E\left({X}^k\right)={\displaystyle \sum_{l_1=1}^m\dots {\displaystyle \sum_{l_k=1}^mg\left(N,{a}_{l_1\dots {l}_k},n\right)}} $$

(6)

As an arbitrary number of moments can be calculated in this way, the problem is a moments problem. Since X is a variable that can only take a finite number of states, (m + 1) in the most inclusive case, the distribution can be constructed using the first m moments. Since

$$ E\left({X}^k\right)={\displaystyle \sum_{i=0}^m{i}^k{p}_i} $$

(7)

by the definition of the expected value, using Eqs. (6) and (7) yields

$$ {\displaystyle \sum_{i=0}^m{i}^k{p}_i}={\displaystyle \sum_{l_1=1}^m\dots {\displaystyle \sum_{l_k=1}^mg\left(N,{a}_{l_1\dots {l}_k},n\right)}} $$

(8)

and thus a system of linear equations. Using the convention 0⁰ = 1, this holds for the 0th moment as well, and using the equations up to the mth moment yields m + 1 equations with m + 1 unknowns. It is worth noting that in cases where the possible states can be further reduced so can the number of moments necessary to construct the probability distribution. In this inclusive case, it is assumed that all categories as well as none can be represented in a subsample, and all intermediate states are possible.

Equation (6) is rather costly in terms of calculations when m and k are large, requiring g to be evaluated m ^k times. Because a only depends on the set of indices and not their multiplicity or order, it can be restated as

$$ E\left({X}^k\right)={\displaystyle \sum_{j=1}^kc\left(j,k\right)T(j)} $$

(9)

where

$$ T(j)={\displaystyle \sum_{\begin{array}{l}{l}_1=1\dots m\\ {}{l}_2<{l}_1\\ {}\vdots \\ {}{l}_j<{l}_{j-1}\end{array}}g\left(N,{a}_{l_1\dots {l}_j},n\right)} $$

(10)

and the coefficient c(j,k) is recursively defined:

$$ \begin{array}{l}c\left(1,k\right)=1\hfill \\ {}c\left(j,k\right)={j}^k-{\displaystyle \sum_{i=1}^{j-1}\left(\begin{array}{c}\hfill j\hfill \\ {}\hfill i\hfill \end{array}\right)c\left(i,k\right)}\hfill \end{array} $$

(11)

Using T(k) as defined in Eq. (10) also allows a general solution to the linear equations given in Eq. (8).

$$ p\left(X=x\right)={\displaystyle \sum_{i=\mathrm{x}}^mT(i){\left(-1\right)}^{x-i}\left(\begin{array}{c}\hfill i\hfill \\ {}\hfill x\hfill \end{array}\right)} $$

(12)

Calculating T(k) takes $\left(\begin{array}{c}\hfill m\hfill \\ {}\hfill k\hfill \end{array}\right)$ evaluations of g, substantially reducing the computational cost. In the most inclusive case, g has to be evaluated 2^m times, which can usually be further reduced by calculating the values for g that actually occur, depending on the values for a. This leads to a maximum of N–1 such calculations, reducing the computational cost even further. In this case, the bulk attends the calculation of a. In the special case examined by Heck et al. (1975), it is worth noting that

$$ {a}_{l_1\dots {l}_k}={\displaystyle \sum_{i=1}^k{a}_{l_k}} $$

(13)

for l ₁ ≠ l ₂ ≠ … ≠ l _k which is satisfied when using Eq. (9).A data matrix can be split into sub-matrices to speed up calculations. In this case the matrix has to take the form

$$ H=\left(\begin{array}{llllll}{H}_{11}\hfill & \cdots \hfill & {H}_{1c}\hfill & 0\hfill & \cdots \hfill & 0\hfill \\ {}\vdots \hfill & \ddots \hfill & \vdots \hfill & \vdots \hfill & \ddots \hfill & \vdots \hfill \\ {}{H}_{b1}\hfill & \cdots \hfill & {H}_{bc}\hfill & 0\hfill & \cdots \hfill & 0\hfill \\ {}0\hfill & \cdots \hfill & 0\hfill & {H}_{b+1\;c+1}\hfill & \cdots \hfill & {H}_{b+1m}\hfill \\ {}\vdots \hfill & \ddots \hfill & \vdots \hfill & \vdots \hfill & \ddots \hfill & \vdots \hfill \\ {}0\hfill & \cdots \hfill & 0\hfill & {H}_{Nc+1}\hfill & \cdots \hfill & {H}_{Nm}\hfill \end{array}\right) $$

(13)

yielding the sub-matrices

$$ \begin{array}{cc}\hfill \begin{array}{cc}\hfill {H}^{\ast }=\hfill & \hfill \left(\begin{array}{ccc}\hfill {H}_{11}\hfill & \hfill \cdots \hfill & \hfill {H}_{1c}\hfill \\ {}\hfill \vdots \hfill & \hfill \ddots \hfill & \hfill \vdots \hfill \\ {}\hfill {H}_{b1}\hfill & \hfill \cdots \hfill & \hfill {H}_{bc}\hfill \end{array}\right)\hfill \end{array};\hfill & \hfill \begin{array}{cc}\hfill {H}^{\sim }=\hfill & \hfill \left(\begin{array}{ccc}\hfill {H}_{b+1\;c+1}\hfill & \hfill \cdots \hfill & \hfill {H}_{b+1m}\hfill \\ {}\hfill \vdots \hfill & \hfill \ddots \hfill & \hfill \vdots \hfill \\ {}\hfill {H}_{Nc+1}\hfill & \hfill \cdots \hfill & \hfill {H}_{Nm}\hfill \end{array}\right)\hfill \end{array}\hfill \end{array} $$

(14)

Given the results for these two matrices, the probability can be reconstructed using the hypergeometric (15) or binomial (16) distribution:

$$ P\left(X=x\right)={\sum}_{j=0}^x{\sum}_{i=0}^nP\left({X}^{\ast }=j\Big|{n}^{\ast }=i\right)\kern0.7em P\left({X}^{\sim }=x-j\Big|{n}^{\sim }=n-i\right)\kern0.28em \frac{\left(\overset{b}{i}\right)\kern0.28em \left(\overset{N-b}{n-i}\right)}{\left(\underset{n}{N}\right)} $$

(15)

$$ P\left(X=x\right)={\sum}_{j=0}^x{\sum}_{i=0}^nP\left({X}^{\ast }=j\Big|{n}^{\ast }=i\right)\kern0.7em P\left({X}^{\sim }=x-j\Big|{n}^{\sim }=n-i\right)\kern0.24em \left(\overset{n}{i}\right)\kern0.28em {\left(\frac{b}{N}\right)}^i{\left(\frac{N-b}{N}\right)}^{n-i} $$

(16)