Information tradeoff

Summary

A prior may be noninformative for one parameter at the cost of being informative for another parameter. This leads to the idea of tradeoff priors: priors that give up noninformativity for some parameters to achieve noninformativity for others. We propose a general framework where priors are selected by optimizing a functional with two components. The first component formalizes the requirement that the optimal prior be noninformative for the parameter of interest. The second component is a penalty term that forces the optimizing prior to be close to some target prior. Optimizing such a functional results in a parameterized family of priors from which a specific prior may be selected as the tradeoff prior. An important particular example of such functionals is provided by choosing the first term to be the marginal missing information for the parameter of interest (generalizing Bernardo’s notion of missing information) and the second term to be the relative entropy between the unknown prior and the Jeffreys prior. In this case we find a closed form expression for the tradeoff prior and we make explicit connections with the Berger-Bernardo prior. In particular, we show that under certain conditions, the Berger-Bernardo prior and the Jeffreys prior are special cases of the tradeoff prior. We consider several examples.