Effective Sample Size for Line Transect Sampling Models with an Application to Marine Macroalgae

Abstract

This paper provides a framework for estimating the effective sample size in a spatial regression model context when the data have been sampled using a line transect scheme and there is an evident serial correlation due to the chronological order in which the observations were collected. We propose a linear regression model with a partially linear covariance structure to address the computation of the effective sample size when spatial and serial correlations are present. A recursive algorithm is described to separately estimate the linear and nonlinear parameters involved in the covariance structure. The kriging equations are also presented to explore the kriging variance between our proposal and a typical spatial regression model. An application in the context of marine macroalgae, which motivated the present work, is also presented.

Appendices

Appendix 1: The Estimation Algorithm

Appendix 2: The Box–Cox Transformation

The following function was proposed by Box and Cox (1964):

$$\begin{aligned} Z_{\varvec{\delta }}(\varvec{s}) = \displaystyle \left\{ \begin{array}{ccc} \frac{(Y(\varvec{s})+\delta _2)^{\delta _1}-1}{\delta _1} &{};&{} \delta _1\ne 0,\\ \ln (Y(\varvec{s})+\delta _2)&{};&{}\delta _1=0, \end{array}\right. \end{aligned}$$

(18)

where $Y(\varvec{s})$ is the original variable and $\varvec{\delta }=(\delta _1,\delta _2)$ is an unknown parameter vector to be estimated to achieve normality of the transformed variable $Z(\cdot ).$ Given the vector of spatial observations $(Z(\varvec{s}_1),\ldots ,Z(\varvec{s}_n))^{\top }$, $\varvec{\delta }$ can be estimated (Box and Cox 1964) by maximizing the likelihood function

$$\begin{aligned} L(\varvec{\delta })= -\frac{n}{2} \ln \left( \frac{1}{n}\sum _{i=1}^{n}(Z_{\varvec{\delta }}(\varvec{s}_i)-\bar{Z}(\varvec{s}_i)^2\right) +(\delta _1-1)\sum _{i=1}^{n}\ln (Y(\varvec{s}_i)+\delta _2). \end{aligned}$$

(19)

An alternative way to estimate $\varvec{\delta }$ is to find the optimal value that maximizes the correlation between $\Phi ^{-1}\left( (i-0.5)/n\right)$ and $Z_{(i)},$ where $\Phi ^{-1}$ is the inverse of the cumulative distribution function of $Z(\varvec{s}_i)$ and $Z_{(i)}$ is the order statistic associated with $Z(\varvec{s}_i)$, for $i=1,\ldots ,n$ (see Kutner et al. 2004).

Using the logarithm transformation ($\delta _1=0$), it is possible to obtain an approximated expression that relates the ESS for the original and transformed variables.

Proposition 1

Suppose that $\varvec{Z}=(Z(\varvec{s}_1),\ldots ,Z(\varvec{s}_n))^{\top }\sim \mathcal {N}(\varvec{\mu }_{Z},\varvec{\Sigma }_Z )$ and consider the transformation

$$\begin{aligned} Z(\varvec{s}_i)=\ln ( Y(\varvec{s}_i)+\delta _2), \quad i=1,\dots ,n, \end{aligned}$$

(20)

where $\varvec{Y}=(Y(\varvec{s}_1),\dots ,Y(\varvec{s}_n))^{\top }$ is the original spatial sample. Let $\varvec{R}_{\varvec{Z}}$ and $\varvec{R}_{\varvec{Y}}$ be the correlation matrices of $\varvec{Z}$ and $\varvec{Y}$, respectively. If $(\varvec{\mu }_{\varvec{Z}})_i= \mu ,$ and $(\varvec{\Sigma }_{\varvec{Z}})_{ii}=\sigma _{ii}=\widetilde{\sigma }^2, i=1,\ldots ,n,$ then

$$\begin{aligned} (\varvec{R}_{\varvec{Y}})_{ij} = c\cdot (\varvec{R_Z})_{ij}k_{ij}+o\left( |\tilde{\sigma }^2|^{m}\right) , \end{aligned}$$

(21)

where $\displaystyle c=\dfrac{\tilde{\sigma }^2}{\exp (\tilde{\sigma }^2)-1}$ and $\displaystyle k_{ij}=1+\dfrac{\sigma _{ij}}{2}+\cdots +\dfrac{\sigma _{ij}^{m-1}}{m!}$. Furthermore, $(\varvec{R}_{\varvec{Y}})_{ij} = \dfrac{c}{\tilde{\sigma }^2}(e^{\sigma _{ij}}-1)$ as $m\rightarrow \infty$.

Proof

Because $Z(\varvec{s}_i)$ is normally distributed, $Y(\varvec{s}_i)$ has a lognormal distribution with

$$\begin{aligned} {{\mathrm{E}}}[Y(\varvec{s}_i)]&=e^{\mu _i+0.5\tilde{\sigma }^2}-\delta _2,\\ {{\mathrm{cov}}}(Y(\varvec{s}_i),Y(\varvec{s}_j))&=e^{\mu _i+\mu _j+\tilde{\sigma }^2}\left( e^{\sigma _{ij}}-1\right) , \end{aligned}$$

for $i,j=1,\dots ,n$.

Using a Taylor expansion for the function $e^{\sigma _{ij}},$ one obtains

$$\begin{aligned} {{\mathrm{cor}}}(Y(\varvec{s}_i),Y(\varvec{s}_j))= & {} \dfrac{e^{\mu _i+\mu _j+\tilde{\sigma }^2}\left( e^{\sigma _{ij}}-1\right) }{e^{\mu _i+\mu _j+\tilde{\sigma }^2}\left( e^{\tilde{\sigma }^2}-1\right) }\\= & {} \dfrac{1}{e^{\tilde{\sigma }^2}-1}\sigma _{ij}\left( 1+\frac{\sigma _{ij}}{2}+\dots +\frac{\sigma ^{m-1}_{ij}}{m!}\right) +\dfrac{1}{e^{\tilde{\sigma }^2}-1}o\left( |\tilde{\sigma }^2|^{m}\right) \\= & {} \dfrac{\tilde{\sigma }^2}{e^{\tilde{\sigma }^2}-1}\dfrac{\sigma _{ij}}{\tilde{\sigma }^2}k_{ij}+o\left( |\tilde{\sigma }^2|^{m}\right) \\= & {} c\cdot (\varvec{R_Z})_{ij}k_{ij}+o\left( |\tilde{\sigma }^2|^{m}\right) , \end{aligned}$$

where $\displaystyle {{\mathrm{cor}}}(Z(\varvec{s}_i),Z(\varvec{s}_j))=(\varvec{R_Z})_{ij}=\dfrac{\sigma _{ij}}{\tilde{\sigma }^2}$, $\displaystyle k_{ij}=1+\dfrac{\sigma _{ij}}{2}+\cdots +\dfrac{\sigma _{ij}^{m-1}}{m!}$, and $\displaystyle c=\dfrac{\tilde{\sigma }^2}{\exp (\tilde{\sigma }^2)-1}$. Moreover, if $m\rightarrow \infty$ we have that

$$\begin{aligned} {{\mathrm{cor}}}(Y(\varvec{s}_i),Y(\varvec{s}_j)) = \dfrac{e^{\sigma _{ij}}-1}{e^{\tilde{\sigma }^2}-1} = \dfrac{c}{\tilde{\sigma }^2}(e^{\sigma _{ij}}-1). \end{aligned}$$

$\square$