Robust Estimation of General Linear Mixed Effects Models

Abstract

The classical REML estimator for fitting a general linear mixed effects model is modified by bounding the terms appearing in the scoring equations. This leads to a generally applicable robust M-type estimator that we call robust scoring equations estimator. It requires only minor assumptions on the covariance matrices (block diagonal for the random effects and diagonal, known up to scale for the residual errors) additional to those of the classical methods. The structure of the data is arbitrary as long as the model is estimable in the classical sense. The estimator can detect and contain the effect of outliers in moderately contaminated datasets. Contamination is detected and treated at all levels of variability of the model, e.g., at both the subject and the observation level for a one-way ANOVA model. The estimator’s properties are studied by simulation and two examples. One example implies crossed random effects, for which the known robust methods are not applicable.

Appendix

1.1 Linear Approximation of Estimated Quantities

In this section, we develop linear approximations to the residuals and the estimated random effects . We use these linear approximations to compute the expected values in the estimating equations as well as the scaling factors τ used in the DAS approach.

Let , , $\mathbf \psi _e^* = {} \mathbf \psi _e\kern -1pt\left ({\mathbf \varepsilon ^{*}}/{\sigma }\right )\kern +0.5pt/\lambda _e$, $\mathbf \psi _b^* = \boldsymbol {\Lambda }_b^{-1}\mathbf \psi _b\kern -1pt\left (\mathbf b^{*}/\sigma \right )\kern +0.5pt$, $\boldsymbol {D}_e =\mathbf {Diag}\kern -1pt\left (\mathbf \psi ^{\prime }_e\kern -1pt\left ({\mathbf \varepsilon ^{*}}/{\sigma }\right )\kern +0.5pt\right )\kern +0.5pt/\lambda _e$, with , , and .

We linearize around β and b^∗, which will be the “true” β and B^∗ later on,

Plugging these expressions into the estimating equations (7), divided by λ_e, and (11) and combining both equations into one yields

where

Using the formula for the inversion of a partitioned matrix, we have

(14)

where

or, equivalently,

Plugging this into (14), we get an approximation for the residuals and for the estimated random effects,

(15)

(16)

with

1.2 Covariance Matrices

The approximations (15) and (16) are used in the computation of covariance matrices. In simpler setups, covariance matrices are calculated on the basis of influence functions IF by integrating . IF is obtained, in the same way as for any M-estimator, from a linear approximation and results proportional to the ψ-function, the factor being the integral of its derivative, $\lambda = {\mathbf {E}}_{0}\left [\psi ^{\prime }_e\right ]$. Even though we have no rigorous proof for a generalization to our case, we apply this idea here.

The expected values of D_e and D_b are the identity matrices. When these expected values are used as approximations, the matrices M_.. and A_.. depend only on θ. The calculation of covariance matrices is then straightforward. They will contain the following expectations under the standard normal distribution:

$$\displaystyle \begin{aligned} \gamma_{.}^{(1)} = {\mathbf E}_o\left[z\psi_e\kern-1pt\left(z\right)\kern+0.5pt\right]/\lambda_. \qquad \gamma_{.}^{(2)} = {\mathbf E}_o\left[\psi_e\kern-1pt\left(z\right)\kern+0.5pt^2\right]/\lambda_.^2 \end{aligned}$$

where the dot (_.) stands for e or b. The corresponding expressions for the block-diagonal case are and . These are diagonal matrices with entries $\gamma _{b}^{(1)}\kern -1pt\left (k\kern -1pt\left (j\right )\kern +0.5pt,m_{k\kern -1pt\left (j\right )\kern +0.5pt}\right )\kern +0.5pt$ and $\gamma _{b}^{(2)}\kern -1pt\left (k\kern -1pt\left (j\right )\kern +0.5pt,m_{k\kern -1pt\left (j\right )\kern +0.5pt}\right )\kern +0.5pt$, respectively, which depend on the dimensions of the blocks k. They are given by

$$\displaystyle \begin{aligned} \gamma_{b}^{(p)}\kern-1pt\left(k,m\right)\kern+0.5pt = m^{-1}{\mathbf E}_{0,m}\left[w_k\kern-1pt\left(u\right)\kern+0.5pt^pu\right]\big/ \lambda_b\kern-1pt\left(k,m\right)\kern+0.5pt \qquad p=1,2 \;. \end{aligned}$$

For fully diagonal V_b, m = 1 and these formulas reduce to $\gamma _{b}^{(1)}$ and $\gamma _{b}^{(2)}$.

The covariance matrix of the estimated fixed effects is

(17)

For the derivation of the last equality, we have used the following two identities:

$$\displaystyle \begin{aligned} \begin{array}{rcl} & &\displaystyle \begin{aligned} &\displaystyle {}\boldsymbol{M}_{\beta\beta} \boldsymbol{M}_{XX}\boldsymbol{M}_{\beta\beta} + \boldsymbol{M}_{\beta\beta} \boldsymbol{M}_{XZ} \boldsymbol{M}_{b\beta} = \left(\boldsymbol{M}_{\beta\beta} \boldsymbol{M}_{XX} + \boldsymbol{M}_{\beta b} \boldsymbol{M}_{ZX} \right)\boldsymbol{M}_{\beta\beta} = {} \\ & \left(\boldsymbol{I} + \boldsymbol{M}_{XX}^{-1} \boldsymbol{M}_{XZ} \boldsymbol{M}_{bb} \boldsymbol{M}_{ZX} - \boldsymbol{M}_{XX}^{-1} \boldsymbol{M}_{XZ} \boldsymbol{M}_{bb} \boldsymbol{M}_{ZX} \right)\boldsymbol{M}_{\beta\beta} = \boldsymbol{M}_{\beta\beta} \;,\quad \text{and} \end{aligned} \\[5pt] & &\displaystyle \begin{aligned} &\displaystyle \boldsymbol{M}_{\beta b} \boldsymbol{M}_{ZX} \boldsymbol{M}_{\beta\beta} + \boldsymbol{M}_{\beta b} \boldsymbol{M}_{ZZ} \boldsymbol{M}_{b\beta} = \\ & \boldsymbol{M}_{\beta b} \left(\boldsymbol{M}_{ZX} \boldsymbol{M}_{\beta\beta} - \left(\widehat{\boldsymbol{M}}_{ZZ} - \boldsymbol{D}_b\right) \widehat{\boldsymbol{M}}_{ZZ}^{-1}\boldsymbol{M}_{ZX}\boldsymbol{M}_{\beta\beta} \right) = - \boldsymbol{M}_{\beta b} \boldsymbol{D}_b \boldsymbol{M}_{b\beta}. \end{aligned} \end{array} \end{aligned} $$

For the DAS standardization, we need the covariance matrix of the residuals $\widehat {\varepsilon }^{*}_i$ and the $\widehat {b}^{*}_j$,

1.3 Refined Design Adaptive Scale

We first write down the equation determining τ_e,i for the determination of $\widehat \sigma$ through (9). The requirement that the ith term in the sum should be zero in expectation translates to the implicit equation

$$\displaystyle \begin{aligned} \int \psi^{(\sigma)}\kern-1pt\left((e-\psi_e\kern-1pt\left(e\right)\kern+0.5pt - r)/\tau_{e,i}\right)\kern+0.5pt \varphi\kern-1pt\left(r/\sigma^{(R)}_i\right)\kern+0.5pt/\sigma^{(R)}_i \mbox dr \varphi\kern-1pt\left(e\right)\kern+0.5pt \mbox de &\\ = \kappa_\sigma\int w^{(\sigma)}\kern-1pt\left((e-\psi_e\kern-1pt\left(e\right)\kern+0.5pt - r)/\tau_{e,i}\right)\kern+0.5pt \varphi\kern-1pt\left(r/\sigma^{(R)}_i\right)\kern+0.5pt/\sigma^{(R)}_i \mbox dr \varphi\kern-1pt\left(e\right)\kern+0.5pt \mbox de & \; \end{aligned}$$

for τ_i, where $\psi ^{(\sigma )}\kern -1pt\left (e\right )\kern +0.5pt=e^2w^{(\sigma )}\kern -1pt\left (e\right )\kern +0.5pt$, φ is the standard normal density, $\sigma ^{(R)}_i$ is the standard deviation of R_i, and κ_σ is defined below (9). The modification for the variance components θ_ℓ in the case of diagonal V_b is straightforward.

For random effects with block-diagonal covariance structure, we have

and T_k is determined by

where $\psi ^{(\eta )}\kern -1pt\left (\mathbf b\right )\kern +0.5pt=\mathbf b\; w^{(\eta )}\kern -1pt\left (\|\mathbf b\|\right )\kern +0.5pt^{1/2}$ and $\boldsymbol {V}^{(R)}_k$ is the covariance matrix of R_k. (Note that the normalizing constants of the densities cancel.) Integration thus extends over $2m\kern -1pt\left (k\right )\kern +0.5pt$ dimensions. With this choice of T_k, each term in the sum (12) has approximate expectation zero. To see this, note that . Therefore, multiplying the last equation by from the right and forming the trace proves the result.

The last equation resembles the problem of estimating a robust covariance matrix and can be computed along the same lines.