Theory of Estimation

No, no; I never guess. It is a shocking habit—destructive to the logical faculty. Sherlock Holmes in “The Sign of Four” The basic objective of statistics is to understand and model the underlying processes that generate the data. This involves statistical inference, where we extract information contained in a sample by applying a model. In general, we assume an i.i.d. random sample {xi}n i=1 from which we extract unknown characteristics of its distribution. In parametric statistics these are condensed in a p-variate vector θ characterizing the unknown properties of the population pdf f(x, θ): this could be the mean, the covariance matrix, kurtosis, or something else.