Estimation

Abstract

In the previous chapter, we have seen how to make optimal decisions with respect to a given utility function and belief. One important question is how to compute an updated belief from observations and a prior belief. More generally, we wish to examine how much information we can obtain about an unknown parameter from observations, and how to bound the respective estimation error. While most of this chapter will focus on the Bayesian framework for estimating parameters, we shall also look at tools for making conclusions about the value of parameters without making specific assumptions about the data distribution, i.e., without providing specific prior information. In the Bayesian setting, we calculate posterior distributions of parameters given data. The basic problem can be stated as follows. Let $\mathscr {P}\mathrel {\triangleq }\left\{ P_{\omega } ~|~ \omega \in \Omega \right\}$ be a family of probability measures on $({\mathcal {S}}, {\mathcal {F}}_{\mathcal {S}})$ and $\xi$ be our prior probability measure on $(\varOmega , {\mathcal {F}}_{\varOmega })$.