Expectation-robust algorithm and estimating equations for means and dispersion matrix with missing data

Abstract

Means and covariance/dispersion matrix are the building blocks for many statistical analyses. By naturally extending the score functions based on a multivariate $t$-distribution to estimating equations, this article defines a class of M-estimators of means and dispersion matrix for samples with missing data. An expectation-robust (ER) algorithm solving the estimating equations is obtained. The obtained relationship between the ER algorithm and the corresponding estimating equations allows us to obtain consistent standard errors when robust means and dispersion matrix are further analyzed. Estimating equations corresponding to existing ER algorithms for computing M- and S-estimators are also identified. Monte Carlo results show that robust methods outperform the normal-distribution-based maximum likelihood when the population distribution has heavy tails or when data are contaminated. Applications of the results to robust analysis of linear regression and growth curve models are discussed.

Appendix A

This appendix shows that the converged values of the ER algorithm in (2), (3), (12) and (13) satisfy (8) and (9). For simple notation we use ${\varvec{\mu }}$ and $\mathbf{\Sigma }$ to denote the converged values and rewrite (12) and (13) as

$$\begin{aligned} \sum _{i=1}^nw_{i1}(\hat{\mathbf{x}}_{ic}-{\varvec{\mu }})=\mathbf{0}\end{aligned}$$

(28)

and

$$\begin{aligned} \sum _{i=1}^n \left[ w_{i2}(\hat{\mathbf{x}}_{ci}-{\varvec{\mu }}) (\hat{\mathbf{x}}_{ic}-{\varvec{\mu }})'+w_{i3}(\mathbf{C}_i-\mathbf{\Sigma })\right] =\mathbf{0}, \end{aligned}$$

(29)

where

$$\begin{aligned} \mathbf{C}_i=\left( \begin{array}{ll} \mathbf{0}&{}\quad \mathbf{0}\\ \mathbf{0}&{}\quad \mathbf{C}_{imm} \end{array} \right) \end{aligned}$$

with $\mathbf{C}_{imm}=\mathbf{\Sigma }_{imm}-\mathbf{\Sigma }_{imo}\mathbf{\Sigma }_i^{-1}\mathbf{\Sigma }_{iom}$ being the converged $\mathbf{C}_{imm}^{(j)}$ in (3). Notice that

$$\begin{aligned} \hat{\mathbf{x}}_{ic}-{\varvec{\mu }}=\left( \begin{array}{c} \mathbf{x}_i-{\varvec{\mu }}_i\\ \mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1}(\mathbf{x}_i-{\varvec{\mu }}_{i}) \end{array} \right) \end{aligned}$$

and

$$\begin{aligned} \mathbf{\Sigma }^{-1}=\left( \begin{array}{ll} \mathbf{I}_{p_i}&{}\quad -\mathbf{\Sigma }_{i}^{-1}\mathbf{\Sigma }_{iom}\\ \mathbf{0}&{}\quad \mathbf{I}_{q_i} \end{array} \right) \left( \begin{array}{ll} \mathbf{\Sigma }_{i}^{-1}&{}\quad \mathbf{0}\\ \mathbf{0}&{}\quad \mathbf{\Sigma }_{i(m|o)}^{-1} \end{array} \right) \left( \begin{array}{ll} \mathbf{I}_{p_i}&{}\quad \mathbf{0}\\ -\mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1}&{}\quad \mathbf{I}_{q_i} \end{array} \right) , \end{aligned}$$

where $q_i=p-p_i$ and $\mathbf{\Sigma }_{i(m|o)}=\mathbf{C}_{imm}$. Direct matrix multiplication yields

$$\begin{aligned} \mathbf{\Sigma }^{-1}(\hat{\mathbf{x}}_{ic}-{\varvec{\mu }})= \left( \begin{array}{l} \mathbf{\Sigma }_{i}^{-1}(\mathbf{x}_i-{\varvec{\mu }}_{i})\\ \mathbf{0}\end{array} \right) . \end{aligned}$$

(30)

The equivalence of (8) and (28) follows from (30) and $\partial {\varvec{\mu }}_i/\partial {\varvec{\mu }}'=(\mathbf{I}_{p_i}, \mathbf{0})$.

For showing equivalence of (9) and (29), let $\mathbf{H}_i=(\hat{\mathbf{x}}_{ic}-{\varvec{\mu }})(\hat{\mathbf{x}}_{ic}-{\varvec{\mu }})'$. When $p_i<p$,

$$\begin{aligned} \mathbf{H}_i=\left( \begin{array}{ll} \mathbf{H}_{ioo}&{}\quad \mathbf{H}_{ioo}\mathbf{\Sigma }_{i}^{-1}\mathbf{\Sigma }_{iom}\\ \mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1}\mathbf{H}_{ioo} &{}\quad \mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1}\mathbf{H}_{ioo}\mathbf{\Sigma }_{i}^{-1}\mathbf{\Sigma }_{iom} \end{array} \right) , \end{aligned}$$

where $\mathbf{H}_{ioo}=(\mathbf{x}_i-{\varvec{\mu }}_i)(\mathbf{x}_i-{\varvec{\mu }}_i)'$. Matrix multiplications yield

$$\begin{aligned} \mathbf{\Sigma }^{-1}\mathbf{H}_i\mathbf{\Sigma }^{-1} = \left( \begin{array}{l@{\quad }l} \mathbf{\Sigma }_{i}^{-1}\mathbf{H}_{ioo}\mathbf{\Sigma }_{i}^{-1}&{}\mathbf{0}\\ \mathbf{0}&{}\mathbf{0}\end{array} \right) \end{aligned}$$

(31)

and

$$\begin{aligned} \mathbf{\Sigma }^{-1}\mathbf{C}_i\mathbf{\Sigma }^{-1}= \left( \begin{array}{l@{\quad }l} \mathbf{\Sigma }_{i}^{-1}\mathbf{\Sigma }_{iom}\mathbf{\Sigma }_{i(m|o)}^{-1} \mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1} &{}-\mathbf{\Sigma }_{i}^{-1}\mathbf{\Sigma }_{iom}\mathbf{\Sigma }_{i(m|o)}^{-1}\\ -\mathbf{\Sigma }_{i(m|o)}^{-1}\mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1} &{}\mathbf{\Sigma }_{i(m|o)}^{-1} \end{array} \right) . \end{aligned}$$

Notice that

$$\begin{aligned} \mathbf{\Sigma }^{-1}= \left( \begin{array}{ll} \mathbf{\Sigma }_{i}^{-1}+\mathbf{\Sigma }_{i}^{-1}\mathbf{\Sigma }_{iom}\mathbf{\Sigma }_{i(m|o)}^{-1} \mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1} &{}-\mathbf{\Sigma }_{i}^{-1}\mathbf{\Sigma }_{iom}\mathbf{\Sigma }_{i(m|o)}^{-1}\\ -\mathbf{\Sigma }_{i(m|o)}^{-1}\mathbf{\Sigma }_{imo}\mathbf{\Sigma }_{i}^{-1} &{}\mathbf{\Sigma }_{i(m|o)}^{-1} \end{array} \right) . \end{aligned}$$

There exists

$$\begin{aligned} \mathbf{\Sigma }^{-1}=\mathbf{\Sigma }^{-1}\mathbf{C}_i\mathbf{\Sigma }^{-1}+ \left( \begin{array}{ll} \mathbf{\Sigma }_{i}^{-1}&{}\mathbf{0}\\ \mathbf{0}&{}\mathbf{0}\end{array} \right) . \end{aligned}$$

(32)

The equivalence of (9) and (29) follows from (31), (32), and by noticing that (9) can be rewritten as

$$\begin{aligned} \sum _{i=1}^n\mathrm{tr}\left\{ \left[ w_{i2}\mathbf{\Sigma }_i^{-1}(\mathbf{x}_i-{\varvec{\mu }}_i) (\mathbf{x}_i-{\varvec{\mu }}_i)'\mathbf{\Sigma }_i^{-1}-w_{i3}\mathbf{\Sigma }_i^{-1}\right] (d\mathbf{\Sigma }_i)\right\} =0, \end{aligned}$$

where $d\mathbf{\Sigma }_i$ is the differential of $\mathbf{\Sigma }_i$.