A general guide in Bayesian and robust Bayesian estimation using Dirichlet processes

Abstract

In this paper, we investigate Bayesian and robust Bayesian estimation of a wide range of parameters of interest in the context of Bayesian nonparametrics under a broad class of loss functions. Dealing with uncertainty regarding the prior, we consider the Dirichlet and the Dirichlet invariant priors, and provide explicit form of the resulting Bayes and robust Bayes estimators. Tractability of the results is supported by numerous examples of different well-known loss functions. The practical utility of the proposed Bayes and robust Bayes estimators are examined for a real data set.

Appendix

Lemma A

Let ${\varvec{X}}=({\varvec{X}}_{1},\ldots ,{\varvec{X}}_{m})^{T}$ be a sample of size m of random vectors from unknown distribution $P\in {\mathcal {F}}$, where ${\mathcal {F}}$ is a general class of beliefs on $(\mathbb {R},{\mathcal {B}})$. Suppose $\theta _P=P(X_1\in S)$, where S is an arbitrary subset of the real line. Assuming that $P\sim DP(a,H)\ [$or $P\sim DIP(a,H)]$, suppose that $P|{\varvec{X}}\sim DP(\tilde{a},\tilde{H})\ [$or $P|{\varvec{X}}\sim DIP(\tilde{a},\tilde{H})]$. Then, for some known $\alpha$, $\beta$ and $\gamma$,

• (i)
  $$\begin{aligned} E[\theta _P^{\alpha }(1-\theta _P)^{\beta }|{\varvec{X}}] =\frac{B\left( \tilde{a}\tilde{H}{(S)}+\alpha ,\tilde{a}(1-\tilde{H}{(S)}) +\beta \right) }{B\left( \tilde{a}\tilde{H}{(S)}, \tilde{a}(1-\tilde{H}{(S)})\right) }, \end{aligned}$$
• (ii)
  $$\begin{aligned} E[e^{\gamma \theta _P}|{\varvec{X}}]= 1+\sum _{l=1}^{\infty } \frac{c^l}{l!} \prod _{j=0}^{l-1} \frac{\tilde{a}\tilde{H}{(S)}+j}{\tilde{a}+j}, \end{aligned}$$
• (iii)
  $$\begin{aligned} E[\ln \theta _P|{\varvec{X}}]=-\sum _{l=1}^{\infty }\frac{1}{l} \frac{B\left( \tilde{a}\tilde{H}{(S)},\tilde{a}(1-\tilde{H}{(S)})+l\right) }{B\left( \tilde{a}\tilde{H}{(S)},\tilde{a}(1-\tilde{H}{(S)})\right) }, \end{aligned}$$
• (iv)
  $$\begin{aligned} E[\ln (1-\theta _P)|{\varvec{X}}]=-\sum _{l=1}^{\infty }\frac{1}{l} \frac{B\left( \tilde{a}\tilde{H}{(S)}+l,\tilde{a}(1-\tilde{H}{(S)})\right) }{B\left( \tilde{a}\tilde{H}{(S)},\tilde{a}(1-\tilde{a})\right) }, \end{aligned}$$

where $B(\alpha ,\beta )=\frac{\varGamma (\alpha )\varGamma (\beta )}{\varGamma (\alpha +\beta )}$, and $\varGamma (u)=\int _0^{\infty } v^{u-1} e^{-v}\,dv$ is the gamma function.

Proof

In Definition 1 take $k=2$ and consider ${\mathcal {X}}$ as a union of two disjoint sets. This yields that $P\sim Beta(aH{(S)},a(1-H{(S)}))$. Part (i) is simply verified using simple algebraic manipulation. Part (ii) is in fact obtained by using the moment generating function of the Beta distribution, see Gupta and Nadarajah (2004). Part (iii)¹ is verified using the Taylor’s expansion of $\ln \theta _P$, i.e., $\ln \theta _P=-\sum _{l=1}^{\infty }\frac{1}{l}(1-\theta _P)^l$, along with applying part (i) of the lemma with the choices $\alpha =0$ and $\beta =l$. Part (iv) is verified using the Taylor’s expansion of $\ln (1-\theta _P)$, i.e., $\ln (1-\theta _P)=-\sum _{l=1}^{\infty } \frac{1}{l}{\theta _P^l}$, along with applying part (i) of the lemma with the choices $\alpha =l$ and $\beta =0$.

Proof of Theorem 3

• (i) The function $\eta _j^{\mathcal {F}}(a,\theta _0)$ is increasing in $\theta _0$ and thus, $\eta _j^{\mathcal {F}} (a,\theta _0)$ attains its infimum at $\theta _0=\underline{\theta }_0$. Similarly, $\eta _j^{\mathcal {F}}(a,\theta _0)$ attains its supremum at $\theta _0=\overline{\theta }_0$.

• (ii) The function $\eta _j^{\mathcal {F}}(a,\theta _0)$ is increasing in a, provided that $\theta _0>\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$, and is decreasing in a, provided that $\theta _0<\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$. Otherwise, $\eta _j^{\mathcal {F}}(a,\theta _0)$ is constant. Thus, if $\theta _0>\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$, $\underline{\eta }_j^{\mathcal {F}}= \eta _j^{\mathcal {F}}(a_1,\theta _0)$ and if $\theta _0<\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$, $\overline{\eta }_j^{\mathcal {F}}= \eta _j^{\mathcal {F}}(a_2,\theta _0)$. The same discussion leads to the determination of $\overline{\eta }_j^{\mathcal {F}}$.

• (iii) First, suppose that $\underline{\theta }_0 \ge \frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$. The fact that $\theta _0\in [\underline{\theta }_0,\overline{\theta }_0]$ yields that $\theta _0 \ge \underline{\theta }_0 \ge \frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$ which results in $\eta _j^{\mathcal {F}}(a,\theta _0)\ge \eta _j^{\mathcal {F}}(a_1,\theta _0)$, as well as $\theta _0-\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j} \ge \underline{\theta }_0 -\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}\ge 0$. The latter inequality implies that $\eta _j^{\mathcal {F}}(a_1,\theta _0)\ge \eta _j^{\mathcal {F}}(a_1,\underline{\theta }_0)$. Now, using the fact that $\eta _j^{\mathcal {F}}(a,\theta _0)\ge \eta _j^{\mathcal {F}}(a_1,\theta _0)$, it is concluded that for every a and H, $\eta _j^{\mathcal {F}}(a,\theta _0)\ge \eta _j^{\mathcal {F}}(a_1,\underline{\theta }_0)$ or equivalently $\underline{\eta }_j^{\mathcal {F}}(a,\theta _0) =\eta _j^{\mathcal {F}}(a_1,\underline{\theta }_0)$. The same discussion leads us to $\overline{\eta }_j^{\mathcal {F}} =\eta _j^{\mathcal {F}}(a_2,\overline{\theta }_0)$.       If $\overline{\theta }_0 \le \frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$, the fact that $\theta _0\in [\underline{\theta }_0,\overline{\theta }_0]$ yields that $\underline{\theta }_0 \le \theta _0 \le \overline{\theta }_0 \le \frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$ which results in $\eta _j^{\mathcal {F}}(a,\theta _0)\le \eta _j^{\mathcal {F}}(a_2,\theta _0)$, as well as $\underline{\theta }_0-\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j} \le \theta _0-\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j} \le \overline{\theta }_0 -\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j} \le 0$. The latter inequality implies that $\eta _j^{\mathcal {F}}(a_2,\theta _0)\le \eta _j^{\mathcal {F}}(a_2,\underline{\theta }_0)$. The fact that $\eta _j^{\mathcal {F}}(a,\theta _0)\le \eta _j^{\mathcal {F}}(a_2,\theta _0)$ yields that for every a and $\theta _0$, $\eta _j^{\mathcal {F}}(a,\theta _0)\le \eta _j^{\mathcal {F}}(a_2,\underline{\theta }_0)$ or equivalently $\underline{\eta }_j^{\mathcal {F}}=\eta _j^{\mathcal {F}}(a_2, \underline{\theta }_0)$. The same discussion leads us to $\overline{\eta }_j^{\mathcal {F}}= \eta _j^{\mathcal {F}}(a_1,\overline{\theta }_0)$.       Now, if $\underline{\theta }_0< \frac{j+{v^{\mathcal {F}} (Z,{\varvec{X}})}}{m+j} < \overline{\theta }_0$, the fact that $\theta _0\in [\underline{\theta }_0,\overline{\theta }_0]$ yields to the two possibilities $\underline{\theta }_0 \le \theta _0<\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j} < \overline{\theta }_0$ and $\underline{\theta }_0<\frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j} < \theta _0 \le \overline{\theta }_0$. In both cases, $\underline{\theta }_0- \frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j} \le \theta _0- \frac{j+{v^{\mathcal {F}}(Z,{\varvec{X}})}}{m+j}$ which yields that $\eta _j^{\mathcal {F}}(a,\theta _0)\ge \eta _j^{\mathcal {F}}(a,\underline{\theta }_0)$. On the other hand $\eta _j^{\mathcal {F}}(a,\underline{\theta }_0)\ge \eta _j^{\mathcal {F}}(a_2,\underline{\theta }_0)$. Combining these facts results that for every a and H, $\eta _j^{\mathcal {F}} (a,\theta _0)\ge \eta _j^{\mathcal {F}}(a_2,\underline{\theta }_0)$ or equivalently $\underline{\eta }_j^{\mathcal {F}}=\eta _j^{\mathcal {F}} (a_2,\underline{\theta }_0)$. The same discussion proves that $\overline{\eta }_j^{\mathcal {F}}= \eta _j^{\mathcal {F}}(a_2, \overline{\theta }_0)$.