Several Proportions or Probabilities

Abstract

We discuss the Multi-hypergeometric and Multinomial distributions and their properties with the focus on exact and large sample inference for comparing two proportions or probabilities from the same or different populations. Relative risks and odds ratios are also considered. Maximum likelihood estimation, asymptotic normality theory, and simultaneous confidence intervals are given for the Multinomial distribution. The chapter closes with some applications to animal populations, including multiple-recapture methods, and the delta method.

Appendix: Delta Method

In this section we consider a well-known method for finding large sample variances. The theory is then applied to the Multinomial distribution. We also consider functions of Normal random variables.

3.1.1 General Theory

We consider general ideas only without getting too involved with technical details about limits. Let $X$ be a random variable with mean $\mu$ and variance $\sigma _X^2$, and let $Y=g(X)$ be a “well-behaved” function of $X$ that has a Taylor expansion

$$\begin{aligned} g(X)-g(\mu )=(X-\mu )g^{\prime }(\mu ) +\frac{1}{2}(X-\mu )^2g^{\prime }(X_0), \end{aligned}$$

where $X_0$ lies between $X$ and $\mu$ and $g^{\prime }(\mu )$ is the derivative of $g$ evaluated at $X=\mu$. Assuming second order terms can be neglected, we have $\mathrm{E}(Y)\approx g(\mu )$ and

$$\begin{aligned} \mathrm{{var}}(Y)&\approx \mathrm{E}[(g(X)-g(\mu ))^2]\\&\approx \mathrm{E}[(X-\mu )^2][g^{\prime }(\mu )]^2\\&= \sigma _X^2[g^{\prime }(\mu )]^2. \end{aligned}$$

For example, if $g(X)=\log X$ then, for large $\mu$,

$$\begin{aligned} \mathrm{{var}}(\log X)\approx \frac{\sigma _X^2}{\mu ^2}. \end{aligned}$$

(3.31)

If $\mathbf{X }=(X_1,X_2,\ldots ,X_k)^{\prime }$ is a vector with mean ${\varvec{\mu }}$, then for suitable $g$,

$$\begin{aligned} Y=g(\mathbf{X })-g({\varvec{\mu }})\approx \sum _{i=1}^k(X_i-\mu _i)g_i^{\prime }({\varvec{\mu }}) +\ldots , \end{aligned}$$

where $g_i^{\prime }({\varvec{\mu }})$ is $\partial g/\partial X_i$ evaluated at $\mathbf{X }={\varvec{\mu }}$. If second order terms can be neglected, we have

$$\begin{aligned} \mathrm{{var}}(Y)&\approx \mathrm{E}[(g(\mathbf{X })-g({\varvec{\mu }}))^2]\nonumber \\&\approx \mathrm{E}\left[ \sum _{i=1}^k\sum _{j=1}^k(X_i-\mu _i)(X_j-\mu _j)g_{i}^{\prime }({\varvec{\mu }})g_j^{\prime }({\varvec{\mu }})\right] \nonumber \\&= \sum _{i=1}^k\sum _{j=1}^k\mathrm{{cov}}(X_i,X_j)g_{i}^{\prime }({\varvec{\mu }})g_j^{\prime }({\varvec{\mu }}). \end{aligned}$$

(3.32)

3.1.2 Application to the Multinomial Distribution

Suppose $\mathbf{X }$ has the Multinomial distribution given by (3.10) and

$$\begin{aligned} g(\mathbf{X })=\frac{X_1X_2\cdots X_r}{X_{r+1}X_{r+2}\cdots X_s}\quad (s\le k). \end{aligned}$$

Then, using the above approach with $\mu _i=np_i$,

$$\begin{aligned} \frac{g(\mathbf{X })-g({\varvec{\mu }})}{g({\varvec{\mu }})}\approx \sum _{i=1}^r\frac{X_i-\mu _i}{\mu _i }-\sum _{i=r+1}^s\frac{X_i-\mu _i}{\mu _i },\end{aligned}$$

and it can be shown that (Seber 1982, pp. 8–9)

$$\begin{aligned} \mathrm{{var}}[g(\mathbf{X })]\approx \frac{[g({\varvec{\mu }})]^2}{n}\left\{ \sum _{i=1}^s\frac{1}{p_i}-(s-2r)^2\right\} . \end{aligned}$$

(3.33)

Two cases of interest in this monograph are, $s=2r=2$ and $s=2r=4$. In the first case $g(\mathbf{X })=X_1/X_2$ and

$$\begin{aligned} \mathrm{{var}}[g(X)]\approx [g({\varvec{\mu }})]^2\left( \frac{1}{\mu _1}+\frac{1}{\mu _2}\right) . \end{aligned}$$

(3.34)

We are particularly interested in $Y=\log g(\mathbf{X })$, so that from (3.31),

$$\begin{aligned} \mathrm{{var}}(Y)\approx \frac{\mathrm{{var}}[g(\mathbf{X })]}{[g({\varvec{\mu }})]^2}=\frac{1}{\mu _1}+\frac{1}{\mu _2}. \end{aligned}$$

(3.35)

If $g(\mathbf{X })$ is a product of two such independent ratios from independent Binomial distributions, then we just add two more terms to $\mathrm{{var}}(Y)$. We can estimate $\mathrm{{var}} (Y)$ by replacing each $\mu _i$ by $X_i$ in (3.35).

Using similar algebra, we find that

$$\begin{aligned} \mathrm{{var}}\left[ \log \left( \frac{X_1X_2}{X_3X_4}\right) \right] \approx \sum _{i=1}^4\frac{1}{\mu _i}. \end{aligned}$$

(3.36)

3.1.3 Asymptotic Normality

In later chapters we are interested in functions of a maximum likelihood estimator, which we know is asymptotically Normally distributed under fairly general conditions. For example, suppose $\sqrt{n}(\widehat{{\varvec{\mu }}}_n - {\varvec{\mu }})$ is asymptotically $N(\mathbf{{0}}, {\varvec{\Sigma }}({\varvec{\mu }}))$. Then using the delta method above, $\sqrt{n}(g(\widehat{{\varvec{\mu }}})-g({\varvec{\mu }}))$ is asymptotically distributed as $N(0,\sigma _g^2)$ as $n\rightarrow \infty$, where

$$\begin{aligned} \sigma _g^2=\left[ \left( \frac{\partial g}{\partial {\varvec{\mu }}}\right) {\varvec{\Sigma }}({\varvec{\mu }})\left( \frac{\partial g}{\partial {\varvec{\mu }}}\right) ^{\prime }\right] . \end{aligned}$$

This result also holds if we replace $g$ by a vector function $\mathbf{g }$ giving us $N(\mathbf{{0}}, {{\varvec{\Sigma }}}_{\mathbf{g }})$.