Usefulness of Akaike information criterion for making decision in two-sample problems when sample sizes are too small

Abstract

There are studies in medical science that researchers want to show the equivalence, or non-equivalence, of two treatment effects, but an application of the routine statistical test is useless when sample size is small. Concentrating on a simple two-sample problem that assumes normality with unknown common variance, and comparing power functions and positive and negative predictive values, it is shown in this paper that the decision rule based on Akaike information criterion is superior to those rules based on Bayes information criterion and on statistical test when sample sizes are too small.

Appendices

Appendix A

Proof of Proposition 2

The statistic T follows t-distribution with degrees of freedom N under $H_0$ and non-central t-distribution with degree of freedom N with non-centrality parameter $\lambda =\delta _1/\bigl (\sigma (m^{-1}+n^{-1})\bigr )$ under $H_1: \delta =\delta _1$. Let $F_N(y)$ and $F_N(y|\lambda )$ be cumulative distribution functions of a central t-distribution of N degrees of freedom and non-central t-distribution of N degrees of freedom with non-centrality parameter $\lambda$. Then $\gamma _0(c)$ and $\gamma _1(c)$ may be represented as follows.

$$\begin{aligned} \gamma _0(c)= & {} 1-2F_N(-c),\\ \gamma _1(c)= & {} 1+F_N(-c|\lambda )-F_N(c|\lambda ). \end{aligned}$$

Thus by putting

$$\begin{aligned} g(y)=\frac{F_N(-y)}{1-F_N(y|\lambda )+F_N(-y|\lambda )},~~~~ h(y)=\frac{F_N(y|\lambda )-F_N(-y|\lambda )}{1-2F_N(-y|\lambda )}, \end{aligned}$$

${\mathrm{PPV}}(c)$ and ${\mathrm{NPV}}(c)$ at $\delta =\delta _1$ may be represented as follows.

$$\begin{aligned} {\mathrm{PPV}}(c)=\frac{R}{R+2g(c)},~~~~{\mathrm{NPV}}(c)=\frac{1}{1+Rh(c)}. \end{aligned}$$

Proof of Proposition 2 (1). Suppose that $c_1<c_2$ for fixed N, then to prove ${\mathrm{PPV}}(c_1)<{\mathrm{PPV}}(c_2)$ it is sufficient to prove the following lemma.

Lemma 1

(1) g(y) is a monotone decreasing function of $y>0.$ (2) h(y) is a monotone increasing function of $y>0$.

Proof

(1) We have

$$\begin{aligned} \frac{dg(y)}{dy}=\frac{u(y)}{\bigl [1-F_N(y|\lambda )+F_N(-y|\lambda )\big ]^2}, \end{aligned}$$

where

$$\begin{aligned} u(y)=F_N(-y)\bigl [f_N(y|\lambda )+f_N(-y|\lambda )\bigr ]-f_N(y)\bigl [1-F_N(y)+F_N(-y|\lambda )\bigr ] , \end{aligned}$$

and $f_N(y)$ and $f_N(y|\lambda )$ are density functions of $F_N(y)$ and $F_N(y|\lambda )$. Substituting functional forms of $f_N(y)$ and $f_N(y|\lambda )$ we may represent u(y) as follows.

$$\begin{aligned} u(y)=\int _{-\infty }^{-y} c_3(x,y)\sum _{j=1}^{\infty } c_2(2j)\left [\left(\frac{y^2}{N+y^2}\right)^j-\left(\frac{x^2}{N+x^2}\right)^j\right ] dx, \end{aligned}$$

where

$$\begin{aligned} c_1(y)=\frac{\exp (-y^2/2)}{(N\pi )^{1/2}\Gamma (N/2)}\left (\frac{N}{N+y^2} \right )^{(N+1)/2},~~~ c_2(j)=\lambda ^j 2^{j/2}\frac{\Gamma \bigl ((N+j+1)/2\bigr )}{j!}, \end{aligned}$$

and $c_3(x,y)=2c_1(y)f_N(x).$ Since $c_2(2j)>0$, $c_3(x,y)>0$ and furthermore

$$\begin{aligned} \left(\frac{y^2}{N+y^2}\right)^j<\left(\frac{x^2}{N+x^2}\right)^j \end{aligned}$$

for x such that $-\infty<x<-y<0$ it follows that $u(y)<0$ for $y>0$. Thus we have $dg(y)/dy<0$ for $y>0$ and the proof of (1) is completed.

We next prove (2). We have

$$\begin{aligned} \frac{dh(y)}{dy}=\frac{v(y)}{\bigl (1-2F_N(-y)\bigr )^2,} \end{aligned}$$

where

$$\begin{aligned} v(y)=\bigl [f_N(y|\lambda )+f_N(-y|\lambda )\bigr ]\bigl [1-2F_N(-y)\bigr ]-2f_N(y)\bigl [F_N(y|\lambda )-F_N(-y|\lambda )\bigr ]. \end{aligned}$$

Similarly as above v(y) is represented by

$$\begin{aligned} v(y)=2\int _0^y c_3(x,y)\sum _{j=1}^{\infty } c_2(2j)\left[(\frac{y^2}{N+y^2})^j - (\frac{x^2}{N+x^2})^j\right ]dx, \end{aligned}$$

where $c_2(j)$ and $c_3(x,y)$ are the same quantities as defined above. Since

$$\begin{aligned} \left(\frac{y^2}{N+y^2}\right)^j > \left(\frac{x^2}{N+x^2}\right)^j \end{aligned}$$

for x such that $0<x<y$ we have $v(y)>0$. Thus $dh(y)/dy>0$ and h(y) is a monotone increasing function in $y>0$. This completes the proof of the lemma.

Proof of Proposition 2 (2) is straightforward from Proposition 1 (2).

Proof of Proposition 2 (3). Suppose that $c/N^{1/2}$ is a decreasing function of N, then from Proposition 1 (3) $\gamma _1(c)$ and $\gamma _0(c)$ are increasing function of N. Thus $1-\gamma _0(c)/\gamma _1(c)$ is a decreasing function of N and PPV(c) is an increasing function of N.

Proof of Proposition 2 (4). Since ${\mathrm{NPV}}(c)=1/(1+Rh(c))$, Proposition  2 (4) is clear from Lemma 1 (2).

Appendix B

Proof of Proposition 3

(1) Since $P(|T|>t_N(\alpha /2)~|~H_0)=\alpha$, we have $2F_N(-t_N(\alpha /2))=\alpha$. Thus $c_A <t_N(\alpha /2)$ if and only if $2F_N(-c_A)>2F_N(-t_N(\alpha /2))=\alpha$. (2) May be proved similarly. (3) It follows immediately that $c_A < c_B$ if and only if $\exp (2/(N+2)) < (N+2)^{1/(N+2)}$, that is equivalent to $N \ge 6$.

Appendix C

Proof of Proposition 6

When $R \ge 1$ and $\gamma _1(c) \ge \gamma _1^*$

$$\begin{aligned} {\mathrm{PPV}}(c) \ge \frac{\gamma _1^*}{\gamma _1^* + 1 - \gamma _0(c)} . \end{aligned}$$

Thus to prove the proposition we may show

$$\begin{aligned} \frac{\gamma _1^*}{\gamma _1^* + 1 - \gamma _0(c)} \ge \gamma _1^* . \end{aligned}$$

The inequality is identical to $\gamma _0(c) \ge \gamma _1^*$.