On the equivalence between the LRT and F-test for testing variance components in a class of linear mixed models

Abstract

For the special case of balanced one-way random effects ANOVA, it has been established that the generalized likelihood ratio test (LRT) and Wald’s test are largely equivalent in testing the variance component. We extend these results to explore the relationships between Wald’s F test, and the LRT for a much broader class of linear mixed models; the generalized split-plot models. In particular, we explore when the two tests are equivalent and prove that when they are not equivalent, Wald’s F test is more powerful, thus making the LRT test inadmissible. We show that inadmissibility arises in realistic situations with common number of degrees of freedom. Further, we derive the statistical distribution of the LRT under both the null and alternative hypotheses $H_0$ and $H_1$ where $H_0$ is the hypothesis that the between variance component is zero. Providing an exact distribution of the test statistic for the LRT in these models will help in calculating a more accurate p-value than the traditionally used p-value derived from the large sample chi-square mixture approximations.

Appendices

Appendix 1

Supplementary material includes (i) Proof for the PPOs Properties in (15), (ii) Illustration for the proof of Lemma 2, and (iii) Proof of Lemma 3.

1.1 Proof for the PPOs Properties in (15)

Firstly, ${\tilde{M}}=M_{*}+M_2$: This result is an immediate consequence of conditions (b) and (c) of Sect. 1.3. In particular, since $C({\tilde{X}})= C(X_{*}, (I-M_1)X_2)$ then by defintion of PPO

$$\begin{aligned} {\tilde{M}}= & {} \left[ X_{*}, (I-M_1)X_2\right] \left( \left[ \begin{array}{c} X_{*}^{'}\\ \left[ (I-M_1)X_2\right] ^{'} \end{array} \right] \left[ X_{*}, (I-M_1)X_2\right] \right) ^{-1}\nonumber \\&\times \left[ \begin{array}{c} X_{*}^{'}\\ \left[ (I-M_1)X_2\right] ' \end{array} \right] \nonumber \\= & {} X_{*}\left( X_{*}^{'}X_{*}\right) ^{-1}X_{*}^{'}+ (I-M_1)X_2\left( X_{2}^{'}(I-M_1)X_2\right) ^{-1} X_2^{'}(I-M_1)\nonumber \\= & {} \quad M_{*}+M_2. \end{aligned}$$

(83)

The equality in the third line of (83) is due to condition (b) which implies that $(I-M_1)X_{*}=X_{*}^{'}(I-M_1)=0$.

Secondly, $M_{*}M_1=M_{*}$: This result is an immediate consequence of condition (b). In particular, since $C(X_{*})\subset C(X_1)$ then $X_{*}=X_1B$ for some matrix B and therefore, by definition of PPO,

$$\begin{aligned} M_{*}= & {} X_{*}\left( X_{*}^{'}X_{*}\right) ^{-1}X_{*}^{'} =X_1B\left( B^{'}X_1^{'}X_1B\right) ^{-1}B^{'}X_1^{'}. \end{aligned}$$

(84)

Thus, using $M_{*}$ from (84) and $M_1$ from (13) gives

$$\begin{aligned} M_{*}M_1= & {} X_1B\left( B^{'}X_1^{'}X_1B\right) ^{-1}B^{'}X_1^{'}X_1\left( X_1^{'}X_1\right) ^{-1}X_1^{'}\nonumber \\= & {} X_1B\left( B^{'}X_1^{'}X_1B\right) ^{-1}B^{'}X_1^{'}=M_{*}. \end{aligned}$$

(85)

Thirdly, $M_{1}M_2=0$: This results is trivially obtained by simply multiplying $M_1$ from (13) and $M_2$ from (14).

Fourthly, $M=M_1+M_2$: Let $X=[X_1, X_2]$ such that M is the PPO onto C(X). Since $M_1M_2=0$, then $C(M_1)\perp C(M_2)$ and hence $M=M_1+M_2$ is a PPO onto $C(M_1,M_2)$ by Theorem B.45 of Christensen (2011). But $C(M_1,M_2)=C(X_1,(I-M_1)X_2)$ since $C(M_1)=C(X_1)$ and $C(M_2)=C((I-M_1)X_2)$. So, it remains to prove that $C(X_1,X_2)=C(X_1,(I-M_1)X_2)$ to complete the proof. To do so, we use the fact that $C(A_1) = C(A_2)$ iff there exist $B_1$ and $B_2$ such that $A_1 = A_2 B_2$ and $A_2=A_1 B_1$ as follows.

$$\begin{aligned} \left[ X_{1}, (I-M_1)X_2\right]= & {} \left[ X_{1}, X_2-M_1X_2\right] \nonumber \\= & {} \left[ X_{1}, X_2-X_1(X_1^{'}X_1)^{-1}X_1^{'}X_2\right] \nonumber \\= & {} \left[ X_{1}, X_2\right] \begin{bmatrix} I &{} -(X_1^{'}X_1)^{-1}X_1^{'}X_2 \\ 0 &{} I \end{bmatrix} \end{aligned}$$

(86)

and

$$\begin{aligned} \left[ X_{1}, X_2\right]= & {} \left[ X_{1}, (I-M_1)X_2\right] \begin{bmatrix} I &{} (X_1^{'}X_1)^{-1}X_1^{'}X_2 \\ 0 &{} I \end{bmatrix}. \end{aligned}$$

(87)

That is, $C(X_1, (I-M_1)X_2)\subset C(X_1, X_2)$ and $C(X_1, X_2)\subset C(X_1, (I-M_1)X_2)$ so that $C(X_1,X_2)=C(X_1,(I-M_1)X_2)$ as desired. $\square$

Appendix 2

1.1 Illustration for the proof of Lemma 2

Let $\lambda =-\,\frac{a}{b}$. Then

$$\begin{aligned} |aI_n+bP|= & {} |b\left( P-\lambda I_n\right) |=b^n|P-\lambda I_n|. \end{aligned}$$

(88)

However, the determinant $|P-\lambda I_n|$ in (88) is the characteristic polynomial of P which equals to (89) since 1 and 0 are the eigenvalues for P with multiplicity r(P) and $n - r(P)$ respectively.

$$\begin{aligned} |P-\lambda I_n|\equiv & {} p_P(\lambda )=(-\lambda )^{n-r(P)}(1-\lambda )^{r(P)}. \end{aligned}$$

(89)

Hence, substituting (89) in (88) gives the desired result

$$\begin{aligned} |aI_n+bP|= & {} b^n|P-\lambda I_n| =b^n(-\lambda )^{n-r(P)}(1-\lambda )^{r(P)}\nonumber \\= & {} \frac{b^na^{n-r(P)}}{b^{n-r(P)}}\frac{(a+b)^{r(P)}}{b^{r(P)}}=a^{n-r(P)} (a+b)^{r(P)}. \end{aligned}$$

(90)

$\square$

Appendix 3

1.1 Proof of Lemma 3

When $x_2> x_1> 0$, we have a standard maximization problem for a function of two variables. Setting the partial derivatives to zero gives

$$\begin{aligned} \frac{\partial g}{\partial x_1}=0\iff \frac{q_1(Q_1-x_1)}{x_1^2}=0 \iff x_1=Q_1, \end{aligned}$$

(91)

and

$$\begin{aligned} \frac{\partial g}{\partial x_2}=0\iff \frac{q_2(Q_2-x_2)}{x_2^2}=0 \iff x_2=Q_2. \end{aligned}$$

(92)

Let $g_{x_ix_j}=\frac{\partial }{\partial x_j}\left( \frac{\partial }{\partial x_i}g(x_i,x_j)\right)$ for $i,j\in \{1,2\}$. Then, according to the second derivative test, we have

$$\begin{aligned} D(x_1, x_2)= & {} g_{x_1x_1}(x_1, x_2)g_{x_2x_2}(x_1, x_2)-[g_{x_1x_2}(x_1, x_2)]^2\nonumber \\= & {} \left( \frac{q_1}{x_1^2}-\frac{2q_1Q_1}{x_1^3}\right) \left( \frac{q_2}{x_2^2}-\frac{2q_2Q_2}{x_2^3}\right) \end{aligned}$$

(93)

with $D(Q_1, Q_2)=\frac{q_1q_2}{Q_1^2Q_2^2}>0$ and $g_{x_1x_1}(Q_1, Q_2)=\frac{-q_1}{Q_1^2}<0$ so that $(x_1, x_2)=(Q_1, Q_2)$ is a maximum point. Thus, if $Q_2>Q_1>0$ then the point $(Q_1,Q_2)$ is in the interior and maximizes the function within the interior; i.e. a local maximum.

When $x_1=x_2:=x$, using direct substitution, the problem reduces to maximizing the function of one variable

$$\begin{aligned} g(x)=-\,\left[ \text {constant}+(q_1+q_2)\log (x)+\frac{q_1Q_1+q_2Q_2}{x}\right] \end{aligned}$$

(94)

over $R^{+}$. So, setting the partial derivative of g(x) to zero gives

$$\begin{aligned} \frac{\partial g}{\partial x}=0\iff -\frac{q_1+q_2}{x}+\frac{q_1Q_1+q_2Q_2}{x^2}=0\iff x=\frac{q_1Q_1+q_2Q_2}{q_1+q_2}.\nonumber \\ \end{aligned}$$

(95)

Now, using the second derivative test, we have

$$\begin{aligned} \frac{\partial g(x)}{\partial x^2}=\frac{q_1+q_2}{x^2}-\frac{2(q_1Q_1+q_2Q_2)}{x^3} \end{aligned}$$

(96)

with

$$\begin{aligned} \frac{\partial g(x)}{\partial x^2}\mid _{x=\frac{q_1Q_1+q_2Q_2}{q_1+q_2}}=\frac{-(q_1+q_2)^3}{(q_1Q_1+q_2Q_2)^2}<0 \end{aligned}$$

(97)

so that $(x_1, x_2)=\left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}, \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right)$ is a maximum point on the boundary of the domain.

Now, we show that if $Q_2>Q_1>0$ then the maximum in the interior is a global maximum. Note that when the maximum is in the interior at $(Q_1,Q_2)$ it attains the value

$$\begin{aligned} g(Q_1,Q_2)=-\,\left[ constant+(q_1+q_2)+\log \left( Q_1^{q_1}Q_2^{q_2}\right) \right] . \end{aligned}$$

Further, when the maximum is on the boundary at $\left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}, \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right)$ it attains the value

$$\begin{aligned} g\left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right) =-\,\left[ constant+(q_1+q_2)+(q_1+q_2)\log \left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right) \right] . \end{aligned}$$

Showing that $g(Q_1,Q_2)>g\left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right)$ is the same as showing

$$\begin{aligned} \log \left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right) >\frac{q_1\log (Q_1)}{q_1+q_2}+\frac{q_2\log (Q_2)}{q_1+q_2}, \end{aligned}$$

which is true due to Jensen’s Inequality:

Let Q be a r.v. such that $P(Q=Q_1)=\frac{q_1}{q_1+q_2}$ and $P(Q=Q_2)=\frac{q_2}{q_1+q_2}$ then by Jensen’s Inequality we have

$$\begin{aligned}&\log \left[ E(Q)\right]>E\left[ \log (Q)\right] \\&\quad \iff \log \left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right) >\frac{q_1\log (Q_1)}{q_1+q_2}+\frac{q_2\log (Q_2)}{q_1+q_2}. \end{aligned}$$

Now, we show that if $Q_1>Q_2>0$ then the maximum in the boundary is a global maximum. Note that if $Q_1> Q_2>0$, there are no critical points of the function within the interior. Further, we know that $g(x_1,x_2)$ goes to $-\infty$ in both $x_1$ and $x_2$ which forces the maximum on the boundary at $\left( \frac{q_1Q_1+q_2Q_2}{q_1+q_2}, \frac{q_1Q_1+q_2Q_2}{q_1+q_2}\right)$ to be a global maximum. $\square$