M-Estimators

Abstract

The most important method of constructing statistical estimators is to choose the estimator to maximize a certain criterion function. We shall call such estimators M-estimators (from “maximum” or “minimum”). In the case of i.i.d. observations X ₁,..., X _n , a common type of criterion function is of the form

$$\theta \mapsto {{\text{P}}_n}{m_\theta } = \frac{1}{n}\sum\limits_{i = 1}^n {{m_\theta }} ({X_i}),$$

for known given functions m _θ on the sample space. In particular, the method of maximum likelihood estimation corresponds to the choice m _θ = log p _θ , where p _θ is the density of the observations.† The theory of empirical processes comes in naturally when studying the asymptotic properties of these estimators. In this chapter we present several results that give the asymptotic distribution of M-estimators. Some results are of a general nature, while others presume the set-up of i.i.d. observations.