Computing Robust Statistics via an EM Algorithm

Abstract

Maximum likelihood is perhaps the most common method to estimate model parameters in applied statistics. However, it is well known that maximum likelihood estimators often have poor properties when outliers are present. Robust estimation methods are often used for estimating the model parameters in the presence of outliers, but these methods lack a unified approach. We propose a unified method using EM algorithm to make statistical modelling more robust. In this paper, we describe the proposed method of robust estimation and demonstrate it using the example of estimating the location parameter. Well known real data sets with outliers were used to demonstrate the application of proposed estimator. Finally, the proposed estimator is compared with standard M-estimator. In this talk, the location case was considered for simplicity, but directly extends to the robust estimation of parameters in a broad range of statistical models. Hence this proposed method aligns with the classical statistical modelling, in terms of a unified approach.

Appendix A: Proof of Equation (2.9)

$$\displaystyle \begin{aligned} \begin{array}{rcl} l_c(\theta) &\displaystyle =&\displaystyle \log L_c(\theta)\\ &\displaystyle =&\displaystyle \sum_{i = 1}^n \left[\hat z_i \log (\lambda) + \hat z_i \log f(y_i - \theta) + (1-\hat z_i) \log (1-\lambda) + (1-\hat z_i) \log (g) \right]\\ &\displaystyle =&\displaystyle \sum_{i = 1}^n \hat z_i \log f(y_i - \theta) + \mbox{constant}\\ \end{array} \end{aligned} $$

For MLE, $l_c(\theta ) = \sum _{i = 1}^n \hat z_i (y_i - \theta )^2 + \mbox{constant}$ is to be maximized with respect to θ,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{d l_c(\theta)}{d \theta} &\displaystyle =&\displaystyle 0\\ \sum_{i = 1}^n \hat z_i (y_i - \theta) &\displaystyle =&\displaystyle 0. \end{array} \end{aligned} $$