A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models

Abstract

Group-level variance estimates of zero often arise when fitting multilevel or hierarchical linear models, especially when the number of groups is small. For situations where zero variances are implausible a priori, we propose a maximum penalized likelihood approach to avoid such boundary estimates. This approach is equivalent to estimating variance parameters by their posterior mode, given a weakly informative prior distribution. By choosing the penalty from the log-gamma family with shape parameter greater than 1, we ensure that the estimated variance will be positive. We suggest a default log-gamma(2,λ) penalty with λ→0, which ensures that the maximum penalized likelihood estimate is approximately one standard error from zero when the maximum likelihood estimate is zero, thus remaining consistent with the data while being nondegenerate. We also show that the maximum penalized likelihood estimator with this default penalty is a good approximation to the posterior median obtained under a noninformative prior.

Our default method provides better estimates of model parameters and standard errors than the maximum likelihood or the restricted maximum likelihood estimators. The log-gamma family can also be used to convey substantive prior information. In either case—pure penalization or prior information—our recommended procedure gives nondegenerate estimates and in the limit coincides with maximum likelihood as the number of groups increases.

Appendices

Appendix A. Derivation of Properties in Section 4

Here, we derive Properties 1 and 2.

Properties 1 and 2

With the quadratic approximation of the profile log-likelihood in Section 3.2 using Equation (5), the MPL estimator is given by

(A.1)

With a simple calculation, we can show that $\partial\widehat{\sigma }_{\theta} / \partial\lambda\leq0$. Therefore, as λ→0 for fixed α and $\widehat{\mathrm {se}}(\hat{\sigma}_{\theta}^{\operatorname{ML}})$, the MPL estimate increases monotonically to the maximum. When $\hat{\sigma}_{\theta}^{\operatorname{ML}}=0$, the maximum is $\widehat {\mathrm{se}}(\hat{\sigma}_{\theta}^{\operatorname{ML}}) \sqrt{\alpha -1}$. When $\hat{\sigma}_{\theta}^{\operatorname{ML}} >0$, (A.1) is reduced into

$$\widehat{\sigma}_\theta=\frac{\hat{\sigma}_\theta^{\operatorname {ML}}}{2} + \frac {\hat{\sigma}_\theta^{\operatorname{ML}}}{2} \sqrt {1 + 4(\alpha -1)\widehat{\mathrm{se}} \bigl(\hat{\sigma}_\theta^{\operatorname{ML}} \bigr)^2\big/\bigl(\hat{\sigma}_\theta^{\operatorname{ML}} \bigr)^2}>\hat{\sigma}_\theta^{\operatorname{ML}}. $$

In addition, ${\partial\widehat{\sigma}_{\theta}}/{\partial\widehat {\mathrm{se}} (\hat{\sigma}_{\theta}^{\operatorname{ML}})}$ becomes

$$\frac{\partial\widehat{\sigma}_\theta}{\partial\widehat{\mathrm {se}}(\hat{\sigma}_\theta^{\operatorname{ML}})} = \frac{\alpha-1}{\sqrt{\alpha-1+(\hat{\sigma}_\theta^{\operatorname {ML}})^2/\{ 4\widehat{\mathrm{se}}(\hat{\sigma}_\theta^{\operatorname{ML}})^2\}}}, $$

which decreases as $\hat{\sigma}_{\theta}^{\operatorname{ML}}$ increases.

Property 3

If we assign the log-gamma(α,λ) penalty on $\sigma_{\theta}^{2}$ instead of σ _θ , the penalty becomes $\log p(\sigma_{\theta}^{2})=2(\alpha-1) \log\sigma_{\theta}- \lambda\sigma_{\theta}^{2}$. In the limit λ→0, the term 2(α−1)logσ _θ is the same as the corresponding term of the log-gamma(2α−1,λ) penalty on σ _θ .

Property 6

Let t=g _γ (σ _θ ). Then the Jacobian is $\partial g_{\gamma}^{-1} (t) = (\gamma t+1)^{1/\gamma-1}$, which is $\sigma_{\theta}^{1-\gamma}$ when written as a function of σ _θ . Therefore, the prior p(g _γ (σ _θ )) of g _γ (σ _θ ) is proportional to $\sigma_{\theta}^{\alpha-\gamma} e^{-\lambda\sigma_{\theta}}$, which is proportional to gamma(α−γ+1,λ).

Appendix B. Proof of Theorem 4

Proof

Let $S_{nJ} = ( \sum_{j} (\bar{y}_{\cdot j} -\mu)^{2} ) /n^{2}J$ and $T_{nJ} = S_{nJ} - ( \sigma_{\epsilon}^{0} )^{2}/n - \sigma_{\theta}^{2}$. Then S _nJ follows $(\sigma_{\epsilon}^{2}/n + ( \sigma_{\theta}^{0} )^{2}) \chi^{2}_{J}/J$, T _nJ =O _p (J ^−1/2), E(T _nJ )=0 and $\mathit{Var}(T_{nJ})= 2 (\sigma_{\epsilon}^{2} + ( \sigma_{\theta}^{0} )^{2} )^{2}/J$. Using these terms, we can expand $\hat{\sigma}_{\theta}^{\operatorname{ML}}$ as

Therefore, we have

$$E\bigl(\hat{\sigma}_{\theta}^{\operatorname{ML}}\bigr) = \sigma_\theta^0- \frac{1}{4 ( \sigma_\theta^0 )^3 J} \biggl(\frac{\sigma_\epsilon^2}{n} + \bigl( \sigma_\theta^0 \bigr)^2 \biggr)^2+o\bigl(J^{-1}\bigr). $$

For the asymptotic bias of $\hat{\sigma}_{\theta}^{\operatorname {MPL}}$, here we describe the outline of the proof. Details are in Dorie (2013). We will work with an estimating equation ψ _nJ (σ _θ ), given by

and $\hat{\sigma}_{\theta}^{\operatorname{MPL}}$ will be a root of ψ _nJ (σ _θ )=0. The expression above Theorem 4 gives $\hat{\sigma}_{\theta}^{\operatorname{MPL}} - \sigma_{\theta}^{0}= O_{p}(J^{-1/2})$. Therefore, the Taylor expansion of ψ _nJ around $\sigma_{\theta}^{0}$ is given by

As the left-hand side of the approximation is 0, we can complete the square to obtain:

Note that each of ψ, ψ′ and ψ″ are of O _p (J), so that when we pass in 1/J under the root we make each term O _p (1),

The difference $\sqrt{J}(\hat{\sigma}_{\theta}^{\operatorname{MPL}} - \sigma_{\theta}^{0})$ will blow up unless we take the positive root so that the leading terms cancel. Using the expansions of ψ, ψ′ and ψ″ and the expansion of the square root, we can reduce the numerator to

$$ a_1 T_{nJ} + a_2 J^{-1} + a_3 T_{nJ}^2 + o_p\bigl(J^{-1}\bigr) $$

(B.1)

with some constants a ₁, a ₂, and a ₃.

Similarly, Taylor expansion of the reciprocal of the denominator is written as

$$ b_1 +b_2 T_{nJ} + o_p\bigl(J^{-1/2}\bigr) $$

(B.2)

with constants b ₁ and b ₂. Multiplication of (B.1) by (B.2) gives the bias up to the order of J ⁻¹ and it follows that

Since $\hat{\sigma}_{\theta}^{\operatorname{MPL}}$ is uniformly integrable, the expectation of the above is

$$E\bigl(\hat{\sigma}_\theta^{\operatorname{MPL}}\bigr) =\sigma_\theta^0 + \biggl(\frac {\alpha +\lambda\sigma_\theta^0 -1}{2} - \frac{1}{4} \biggr) \frac{1}{\sigma_\theta^3 J } \biggl(\frac{\sigma_\epsilon^2}{n} + \bigl( \sigma_\theta^0 \bigr)^2 \biggr)^2 + o\bigl(J^{-1}\bigr). $$

 □

Appendix C. Proof of Equation (9)

The model in (2) can be written as y=X β+ϵ, where X is a covariate matrix, ϵ follows N(0,V), V is a block-diagonal matrix with n×n blocks V _j , and each V _j contains $\sigma_{\theta}^{2}+\sigma_{\epsilon}^{2}$ on the diagonal and $\sigma_{\theta}^{2}$ on the off-diagonals. As noted in Section 4.4, the REML log-likelihood can be written as the log-likelihood with an additive penalty term, −log{det(X ^T V ⁻¹ X)}/2.

The inverse of V is also block-diagonal of the same structure as V but with $\{ \sigma_{\epsilon}^{2} + (n_{j}-\nobreak 1) \sigma_{\theta}^{2} \} /\allowbreak \sigma_{\epsilon}^{2}(\sigma_{\epsilon}^{2} + n_{j} \sigma_{\theta}^{2})$ in the diagonals and $-{\sigma_{\theta}^{2}}/{\sigma_{\epsilon}^{2}(\sigma_{\epsilon}^{2} + n_{j} \sigma_{\theta}^{2})}$ in the off-diagonals.

Let the columns of X consist of a vector of ones, q level-1 covariates (z ₁,…,z _q ) and r level-2 covariates (w ₁,…,w _r ). When we assume that w ₁,…,w _r are dummy variables for the first r groups and $\boldsymbol{z}_{i}^{T} \boldsymbol {z}_{i}=1$ and $\boldsymbol{z}_{i}^{T} \boldsymbol{z}_{j}=0$ for all i≠j and the data are balanced, X ^T V ⁻¹ X can be simplified to a block-diagonal with

and $\frac{J}{\sigma_{\epsilon}^{2}} I_{q \times q}$.

Therefore it follows that

$$\det\bigl(X^TV^{-1} X\bigr) = \biggl( \frac{n }{\sigma_\epsilon^2 + n \sigma_\theta^2} \biggr)^{(r+1)} (J-r) \biggl( \frac{J}{\sigma_\epsilon^2} \biggr)^q $$

and

$$-\frac{1}{2} \log\bigl\{\det\bigl(X^TV^{-1} X\bigr)\bigr\} = \frac{r+1}{2} \log \biggl( \sigma_\theta^2 + \frac{\sigma_\epsilon^2}{n} \biggr) + \mbox {constant}. $$

Appendix D. REML and Log-Gamma Penalty in General Cases (Referred in Section 4.4)

Figure 8 compares the REML penalty function in (9), the log of the gamma density with corresponding α=(r+1)/2+1, and the REML penalty function in the second term of (8) for a dataset with n=30, J=5, q=1, r=0, 1, or 2, which does not have the form assumed when deriving (9). For evaluating the REML penalty term in (8), the columns of the covariate matrix X consist of a vector of ones, a level-1 covariate z ₁ with z _1ij =i and two level-2 covariates w ₁ and w ₂, where w _1j =j for all j=1,…,J and w ₂ is the same as w ₁ except that the values for the last group are 0 instead of J. Comparing Figures 8(a) and (c), the penalties differ by a constant which does not affect the mode, so formula (9) appears to hold more generally.

Figure 8.

REML log-penalty function compared with log-gamma((r+1)/2+1,0) penalty. The shapes of the curves agree quite well, except when $\sigma_{\theta}^{2}$ is close to 0 where the log-gamma penalty tends to 0.

For Figures 8(a) and (b), the constant terms were ignored to make the figures easier to compare. The REML penalty functions with r=0, 1, and 2 look very similar to the gamma penalty on $\sigma_{\theta}^{2}$ with α=2, 3, and 4, respectively, except where $\sigma_{\theta}^{2}$ is close to zero. At $\sigma_{\theta}^{2}=0$, the log-gamma penalty is −∞ for α>1, whereas the REML penalty approaches −∞ only if σ _ϵ →0 or n→∞. This explains why REML can produce boundary estimates. Further, it implies that the log-gamma penalty assigns more penalty on $\sigma_{\theta}^{2}$ close to zero than REML for small n and large σ _ϵ . Otherwise, REML can approximately be viewed as a special case of our method with a log-gamma penalty.

Appendix E. Simulation of Unbalanced Variance Component Model

Swallow and Monahan (1984) compared several variance estimation methods for the one-way model, given by

$$ y_{ij}=\mu+\theta_j + \epsilon_{ij}, \quad i=1, \ldots, n_j,\ j=1, \ldots, J $$

(E.1)

where $\theta_{j} \sim N(0,\sigma_{\theta}^{2})$ and $\epsilon_{ij} \sim N(0,\sigma_{\epsilon}^{2})$. They considered unbalanced data with eight different patterns of group sizes (n ₁,…,n _J ), and compared the bias and RMSE of estimators of σ _θ using simulated datasets.

In this appendix, we picked two of the patterns Swallow and Monahan (1984) considered, (n ₁,…,n _J )=(1,5,9) and (1,1,1,1,13,13) with σ _ϵ =1, and compared ML and REML with the performance of the MPL estimates with log-gamma(2,0) penalty on σ _θ , which approximates the REML penalty for this model.

As for the balanced case in Section 6, both ML and REML tend to underestimate σ _θ for σ _θ >0. (See the left column of Figure 9.) On the other hand, MPL tends to overestimate σ _θ but the magnitude of the bias decreases as σ _θ increases. For σ _θ =1, the MPL estimator has the smallest bias for both patterns of group sizes. The RMSE is smallest for the MPL estimator when σ _θ >0 as shown in the middle column of Figure 9.

Figure 9.

Bias and RMSE of $\hat{\sigma}_{\theta}$, and bias of the standard error of $\hat{\mu}$. + is MPL, △ is REM and ∘ is ML.

The last column in Figure 9 shows the estimated bias of the standard error of $\hat{\mu}$. When σ _θ is zero, there is almost no difference in the bias between the ML and REML estimators. As σ _θ increases, the bias for the MPL estimator becomes increasingly smaller than the bias for the other estimators.