The variety-of-evidence thesis: a Bayesian exploration of its surprising failures

Abstract

Diversity of evidence is widely claimed to be crucial for evidence amalgamation to have distinctive epistemic merits. Bayesian epistemologists capture this idea in the variety-of-evidence thesis: ceteris paribus, the strength of confirmation of a hypothesis by an evidential set increases with the diversity of the evidential elements in that set. Yet, formal exploration of this thesis has shown that it fails to be generally true. This article demonstrates that the thesis fails in even more circumstances than recent results would lead us to expect. Most importantly, it can fail whatever the chance that the evidential sources are unreliable. Our results hold for two types of degrees of variety: reliability independence and testable aspect independence. We conclude that the variety-of-evidence thesis can, at best, be interpreted as an exception-prone rule of thumb.

Appendix A. Proofs

1.1 A.1. Likelihood ratios with Cs as consequences : general and specific evidential structures

$P(h|e_1,e_2)$ can be represented by the likelihood-ratio form:

$$\begin{aligned} P^{*}(h) = \frac{h_{0}}{h_{0} + {\bar{h}}_{0}L}, \quad \text {where} \quad L = \frac{P(e_{1}, e_{2}|\lnot h)}{P(e_1, e_2|h)} \end{aligned}$$

Given that the ceteris paribus clause imposes a common $h_0$ across structures, we can make pairwise comparisons of structures in terms of likelihood ratios directly, noting that:

$$\begin{aligned} P^{*}_X(h)> P^{*}_Y(h) \Leftrightarrow L_Y > L_X. \end{aligned}$$

Likelihood ratio: proof for general structure G

The likelihood ratio for the general structure G can be calculated with the first panel of Table 4 and the two last columns of Table 5:

$$\begin{aligned} L_{\text {G}}&= \frac{P_{\text {G}}(e_{1}, e_{2}|\lnot h)}{P_{\text {G}}(e_{1}, e_{2}|h)} \\&= \frac{\sum _{C_{1},C_{2},R_{1},R_{2}}(\prod _{i=1,2}P_{\text {G}}(e_{i}|C_{i}, R_{i})) P_{\text {G}}(C_{1}, C_{2}|\lnot h) P_{\text {G}}(R_{1}, R_{2}) }{\sum _{C_{1},C_{2},R_{1},R_{2}}(\prod _{i=1,2}P_{\text {G}}(e_{i}|C_{i}, R_{i})) P_{\text {G}}(C_{1}, C_{2}|h) P_{\text {G}}(R_{1}, R_{2})} \\&= \frac{\omega _{rr}(q^{1+\gamma }) + \omega _{rh}\left( 2q^{1+\gamma } + 2\theta _{\lnot h}\right) + \omega _{hh}\left( q^{1+\gamma } + 2\theta _{\lnot h} + {\bar{q}}^{1+\gamma }\right) }{\omega _{rr}(p^{1+\gamma }) + \omega _{rh}\left( 2p^{1+\gamma } + 2\theta _{h}\right) + \omega _{hh}\left( p^{1+\gamma } + 2\theta _{h} + {\bar{p}}^{1+\gamma }\right) } \\&= \frac{\omega _{rr}(q^{1+\gamma }) + \omega _{rh}(q^{1+\gamma } - {\bar{q}}^{(1+\gamma )} + 1) + \omega _{hh} }{\omega _{rr}(p^{1+\gamma }) + \omega _{rh}(p^{1+\gamma } - {\bar{p}}^{(1+\gamma )} + 1) + \omega _{hh} },&\text {using (13)} \end{aligned}$$

which is the expression in Table 1. $\square$

We can expand this likelihood by substituting the original parameters for the $\omega$. Using Eqs. (11) and (12), we have:

$$\begin{aligned} L_{\text {G}}&= \frac{r^{1+\delta }(q^{1+\gamma }) + (r + \alpha {\bar{r}} - .5(1 + \omega _{rr} + \omega _{hh} - \omega _{\lnot h \lnot h}))(q^{1+\gamma } - {\bar{q}}^{1+\gamma } + 1) + (\alpha {\bar{r}})^{1+\delta } }{r^{1+\delta }(p^{1+\gamma }) + (r + \alpha {\bar{r}} - .5(1 + \omega _{rr} + \omega _{hh} - \omega _{\lnot h \lnot h}))(p^{1+\gamma } - {\bar{p}}^{1+\gamma } + 1) + (\alpha {\bar{r}})^{1+\delta }} \\&= \frac{r^{1+\delta }(q^{1+\gamma }) + (r + \alpha {\bar{r}} - .5(1 + r^{1+\delta } + (\alpha {\bar{r}})^{1+\delta } - ({\bar{\alpha }}{\bar{r}})^{1+\delta }))(q^{1+\gamma } - {\bar{q}}^{1+\gamma } + 1) + (\alpha {\bar{r}})^{1+\delta } }{r^{1+\delta }(p^{1+\gamma }) + (r + \alpha {\bar{r}} - .5(1 + r^{1+\delta } + (\alpha {\bar{r}})^{1+\delta } - ({\bar{\alpha }}{\bar{r}})^{1+\delta }))(p^{1+\gamma } - {\bar{p}}^{1+\gamma } + 1) + (\alpha {\bar{r}})^{1+\delta }} \end{aligned}$$

Likelihood ratios for specific evidential structures can then be calculated by attributing a value of 0 to $\delta$ and $\gamma$ for shared reliability and shared consequence, and of 1 for independent reliability and independent consequences:

Likelihood ratio: proof for structure A (shared reliability, shared consequence)

$$\begin{aligned} L_{\text {A}}&= \frac{r^{1}q^{1} + (r + \alpha {\bar{r}} - .5(1 + r^{1} + (\alpha {\bar{r}})^{1} - ({\bar{\alpha }}{\bar{r}})^{1}))(q^{1} - {\bar{q}}^{1} + 1) + (\alpha {\bar{r}})^{1} }{r^{1}(p^{1}) + (r + \alpha {\bar{r}} - .5(1 + r^{1} + (\alpha {\bar{r}})^{1} - ({\bar{\alpha }}{\bar{r}})^{1}))(p^{1} - {\bar{p}}^{1} + 1) + (\alpha {\bar{r}})^{1}} \\&= \frac{qr + 0 \times (q - {\bar{q}} + 1) + \alpha {\bar{r}}}{pr + 0 \times (p - {\bar{p}} + 1) + \alpha {\bar{r}}} \\ L_{\text {A}}&= \frac{qr + \alpha {\bar{r}}}{pr + \alpha {\bar{r}}}\\ \end{aligned}$$

$\square$

Likelihood ratio: proof for structure B (independent reliability, shared consequence)

$$\begin{aligned} L_{\text {B}}&= \frac{r^{2}q^{1} + (r + \alpha {\bar{r}} - .5(1 + r^{2} + (\alpha {\bar{r}})^{2} - ({\bar{\alpha }}{\bar{r}})^{2}))(q^{1} - {\bar{q}}^{1} + 1) + (\alpha {\bar{r}})^{2} }{r^{2}(p^{1}) + (r + \alpha {\bar{r}} - .5(1 + r^{2} + (\alpha {\bar{r}})^{2} - ({\bar{\alpha }}{\bar{r}})^{2}))(p^{1} - {\bar{p}}^{1} + 1) + (\alpha {\bar{r}})^{2}} \\&= \frac{qr^{2} + (r\alpha {\bar{r}})(2q) + (\alpha {\bar{r}})^{2}}{pr^{2} + (r\alpha {\bar{r}})(2p) + (\alpha {\bar{r}})^{2}} \\ L_{\text {B}}&= \frac{qr^{2} + 2qr\alpha {\bar{r}} + (\alpha {\bar{r}})^{2} }{pr^{2} + 2pr\alpha {\bar{r}} + (\alpha {\bar{r}})^{2}}\\ \end{aligned}$$

$\square$

Likelihood ratio: proof for structure C (shared reliability, independent consequences)

$$\begin{aligned} L_{\text {C}}&= \frac{r^{1}q^{2} + (r + \alpha {\bar{r}} - .5(1 + r^{1} + (\alpha {\bar{r}})^{1} - ({\bar{\alpha }}{\bar{r}})^{1}))(q^{2} - {\bar{q}}^{2} + 1) + (\alpha {\bar{r}})^{1} }{r^{1}(p^{2}) + (r + \alpha {\bar{r}} - .5(1 + r^{1} + (\alpha {\bar{r}})^{1} - ({\bar{\alpha }}{\bar{r}})^{1}))(p^{2} - {\bar{p}}^{2} + 1) + (\alpha {\bar{r}})^{1}} \\&= \frac{q^{2}r + 0 \times (q^{2} - {\bar{q}}^{2} + 1) + \alpha {\bar{r}} }{p^{2}r + 0 \times (p^{2} - {\bar{p}}^{2} + 1) + \alpha {\bar{r}}} \\ L_{\text {C}}&= \frac{q^{2}r + \alpha {\bar{r}}}{p^{2}r + \alpha {\bar{r}}}\\ \end{aligned}$$

$\square$

Likelihood ratio: proof for structure D (independent reliability and consequences)

$$\begin{aligned} L_{D}&= \frac{r^{2}q^{2} + (r + \alpha {\bar{r}} - .5(1 + r^{2} + (\alpha {\bar{r}})^{2} - ({\bar{\alpha }}{\bar{r}})^{2}))(q^{2} - {\bar{q}}^{2} + 1) + (\alpha {\bar{r}})^{2} }{r^{2}(p^{2}) + (r + \alpha {\bar{r}} - .5(1 + r^{2} + (\alpha {\bar{r}})^{2} - ({\bar{\alpha }}{\bar{r}})^{2}))(p^{2} - {\bar{p}}^{2} + 1) + (\alpha {\bar{r}})^{2}} \\&= \frac{(qr)^{2} + (r\alpha {\bar{r}})(2q) + (\alpha {\bar{r}})^{2}}{(pr)^{2} + (r\alpha {\bar{r}})(2p) + (\alpha {\bar{r}})^{2}} \\ L_{D}&= \frac{(qr + \alpha {\bar{r}})^{2}}{(pr + \alpha {\bar{r}})^{2}} \\ \end{aligned}$$

$\square$

1.2 A.2. Comparison of confirmatory strengths with Cs as consequences

Proof of Proposition 1

In terms of likelihood-ratio the proposition is: $L_{\text {A}} > L_{\text {D}}$, for all admissible values of $p,q,r,\alpha$.

$$\begin{aligned} L_{\text {A}}&> L_{\text {D}} \\ \frac{qr + \alpha {\bar{r}}}{pr + \alpha {\bar{r}}}&> \frac{(qr + \alpha {\bar{r}})^{2}}{(pr + \alpha {\bar{r}})^{2}} \\ pr + \alpha {\bar{r}}&> qr + \alpha {\bar{r}} \\ p&> q, \end{aligned}$$

which is true by assumption 7. $\square$

Proof of Proposition 2

$L_{\text {A}} > L_{\text {B}}$ and $L_{\text {C}} > L_{\text {D}}$, for all admissible values of $p,q,r,\alpha$.

We start by the first inequality:

$$\begin{aligned} L_{\text {A}}&> L_{\text {B}} \\ \frac{qr + \alpha {\bar{r}}}{pr + \alpha {\bar{r}}}&> \frac{qr^{2} + 2qr\alpha {\bar{r}} + (\alpha {\bar{r}})^{2}}{pr^{2} + 2pr\alpha {\bar{r}} + (\alpha {\bar{r}})^{2}} \\ qr(\alpha {\bar{r}})^{2} + pr^{2}\alpha {\bar{r}} + 2pr(\alpha {\bar{r}})^{2}&> pr(\alpha {\bar{r}})^{2} + qr^{2}\alpha {\bar{r}} + 2qr(\alpha {\bar{r}})^{2} \\ r^{2}\alpha {\bar{r}}(p - q) + 2r(\alpha {\bar{r}})^{2}(p - q)&> r(\alpha {\bar{r}})^{2}(p - q) \\ r + \alpha {\bar{r}}&> 0 \end{aligned}$$

The division by $p-q$ to get to the last line leaves the direction of inequality unchanged because of assumption 7 $p > q$. Since r and $\alpha$ are assumed to be always strictly positive, the inequality holds.

Now the second inequality:

$$\begin{aligned} L_{\text {C}}&> L_{\text {D}} \\ \frac{q^{2}r + \alpha {\bar{r}}}{p^{2}r + \alpha {\bar{r}}}&> \frac{(qr + \alpha {\bar{r}})^{2}}{(pr + \alpha {\bar{r}})^{2}} \\ p^{2}r + 2pq^{2}r + 2p\alpha {\bar{r}} + q^{2}\alpha {\bar{r}}&> p^{2}r + 2pq^{2}r + 2p\alpha {\bar{r}} + q^{2}\alpha {\bar{r}} \\ r(p + q)(p - q) + 2\alpha {\bar{r}}(p - q)&> 2pqr(p - q) + \alpha {\bar{r}}(p + q)(p - q) \\ r(p + q - 2pq) + \alpha {\bar{r}}(2 - (p + q))&> 0 \\ \end{aligned}$$

Since p $< 1$ and q $< 1, 2 - \mathrm{(p + q)} > 0$ is always true. Thus, we need to prove that p + q − 2pq $> 0$ is always true :

$$\begin{aligned} p + q - 2pq&> 0 \\ \frac{p + q}{pq}&> 2 \\ \frac{1}{q} + \frac{1}{p}&> 2 \\ \end{aligned}$$

Since 0 $\le$ q $\le$ 1 and 0 $\le$ p $\le$ 1, then $\frac{1}{q}$$\ge$ 1 and $\frac{1}{p}$$\ge$ 1, which entail that the sum of both fractions is superior to 2.

$\square$

Proof of Proposition 3

$$\begin{aligned} L_{\text {A}}&> L_{\text {C}} \text { iff } \, rpq + \alpha {\bar{r}} (p + q - 1)> 0.\\ L_{\text {B}}&> L_{\text {D}} \text { iff } \, ( r^2 + 2 r \alpha {\bar{r}} ) p q + (\alpha {\bar{r}})^2 (p + q - 1) >0 \end{aligned}$$

Starting with the first inequality:

$$\begin{aligned} L_{\text {A}}&> L_{\text {C}} \nonumber \\ \frac{qr + \alpha {\bar{r}}}{pr + \alpha {\bar{r}}}&> \frac{q^{2}r + \alpha {\bar{r}}}{p^{2}r + \alpha {\bar{r}}} \nonumber \\ r^{2}p^{2}q + qr\alpha {\bar{r}} + p^{2}r\alpha {\bar{r}}&> r^{2}q^{2}p + pr\alpha {\bar{r}} + q^{2}r\alpha {\bar{r}} \nonumber \\ r^{2}pq(p - q) + r\alpha {\bar{r}}(p + q)(p - q)&> r\alpha {\bar{r}}(p - q) \nonumber \\ rpq + \alpha {\bar{r}}(p + q - 1)&> 0 \end{aligned}$$

(14)

Expression 14 being as claimed in the proposition.

To ease the proof for the second inequality, define the following expressions:

$$\begin{aligned} A = r^2&B = 2 r \alpha {\bar{r}}&C = (\alpha {\bar{r}})^2 \end{aligned}$$

Starting from the inequality implied by the variety-of-evidence thesis, we have:

$$\begin{aligned} L_{\text {B}}> & {} L_{\text {D}} \\ \frac{ qr^{2} + q2r\alpha {\bar{r}} + (\alpha {\bar{r}})^{2} }{ pr^{2} + p2r\alpha {\bar{r}} + (\alpha {\bar{r}})^{2} }> & {} \frac{ q^2r^{2} + q2r\alpha {\bar{r}} + (\alpha {\bar{r}})^{2} }{ p^2r^{2} + p2r\alpha {\bar{r}} + (\alpha {\bar{r}})^{2} } \\ \frac{ qA + qB + C }{ pA + pB + C }> & {} \frac{ q^2 A + qB + C }{ p^2 A + pB + C } \\ pq(p A + B)&(A + B) + C(p^2 A + pB + qA + qB) \\> & {} pq(q A + B)(A + B) + C(q^2 A + qB + pA + pB) \end{aligned}$$

$$\begin{aligned}&A ( A + B ) p q ( p-q ) + A C ( p^2 + q - q^2 -p )>0 \\&( A + B ) p q ( p-q ) + C (p + q - 1) (p - q)>0 \\&( r^2 + 2 r \alpha {\bar{r}} ) p q + (\alpha {\bar{r}})^2 (p + q - 1) >0 \end{aligned}$$

$\square$

Proof of Proposition 4

To prove that the inequality holds if the first disjunct holds, we start with the likelihood ratio for the general structure and impose $p=1$ and $q=0$:

$$\begin{aligned} L_{\text {G}}&= \frac{\omega _{rr}(q^{1+\gamma }) + \omega _{rh}(q^{1+\gamma } - {\bar{q}}^{(1+\gamma )} + 1) + \omega _{hh} }{\omega _{rr}(p^{1+\gamma }) + \omega _{rh}(p^{1+\gamma } - {\bar{p}}^{(1+\gamma )} + 1) + \omega _{hh} } \\&= \frac{ \omega _{hh} }{\omega _{rr} + 2\omega _{rh} + \omega _{hh} } \\ \end{aligned}$$

This likelihood ratio is identical to the expression in Claveau (2013, eq. 11) which implies that the results in expression (13) and in Fig. 4 of this article also hold here.

Proving the second part of the proposition requires more work. Starting from the likelihood ratio of the general structure and imposing $\gamma =0$:

$$\begin{aligned} L_{\text {G}}&= \frac{\omega _{rr}(q^{1+\gamma }) + \omega _{rh}(q^{1+\gamma } - {\bar{q}}^{(1+\gamma )} + 1) + \omega _{hh} }{\omega _{rr}(p^{1+\gamma }) + \omega _{rh}(p^{1+\gamma } - {\bar{p}}^{(1+\gamma )} + 1) + \omega _{hh} } \\&= \frac{\omega _{rr}q + 2\omega _{rh}q + \omega _{hh} }{\omega _{rr}p + 2\omega _{rh}p + \omega _{hh} } \end{aligned}$$

Taking the derivative of this expression with respect to $\delta$:

$$\begin{aligned} \frac{\partial L_{\text {G}}}{\partial \delta }&= \frac{\omega '_{rr}q + 2\omega '_{rh}q + \omega '_{hh}}{\omega _{rr}p + 2\omega _{rh}p + \omega _{hh}} - \frac{(\omega _{rr}q + 2\omega _{rh}q + \omega _{hh})(\omega '_{rr}p + 2\omega '_{rh}p + \omega '_{hh})}{(\omega _{rr}p + 2\omega _{rh}p + \omega _{hh})^{2}} \\ \end{aligned}$$

Using the inequality claimed by the degree-of-reliability-independence thesis:

$$\begin{aligned}&\frac{\partial L_{\text {G}}}{\partial \delta }< 0 \\&\quad (\omega '_{rr}q + 2\omega '_{rh}q + \omega '_{hh})(\omega _{rr}p + 2\omega _{rh}p + \omega _{hh}) \\&\qquad< (\omega _{rr}q + 2\omega _{rh}q + \omega _{hh})(\omega '_{rr}p + 2\omega '_{rh}p + \omega '_{hh}) \\&\quad \omega '_{rr}\omega _{hh}q + 2\omega '_{rh}\omega _{hh}q + \omega '_{hh}\omega _{rr}p + 2\omega '_{hh}\omega _{rh}p\\&\qquad< \omega '_{rr}\omega _{hh}p + 2\omega '_{rh}\omega _{hh}p + \omega '_{hh}\omega _{rr}q + 2\omega '_{hh}\omega _{rh}q \\&\quad \omega '_{hh}\omega _{rr} - \omega '_{rr}\omega _{hh} + 2\omega '_{hh}\omega _{rh} - 2\omega '_{rh}\omega _{hh} <0 \end{aligned}$$

The derivatives of the $\omega$ with respect to $\delta$ are:

$$\begin{aligned} \omega '_{rr} = \frac{\partial \omega _{rr}}{\partial \delta }&= r^{1+\delta }\ln (r) = \omega _{rr}\ln (r) \\ \omega '_{hh} =\frac{\partial \omega _{hh}}{\partial \delta }&= (\alpha {\bar{r}})^{1+\delta }\ln (\alpha {\bar{r}}) = \omega _{hh}\ln (\alpha {\bar{r}}) \\ \omega '_{rh} =\frac{\partial \omega _{rh}}{\partial \delta }&= -\,0.5(\omega _{rr}\ln (r) + \omega _{hh}\ln (\alpha {\bar{r}}) - \omega _{\lnot h \lnot h}\ln ({\bar{\alpha }}{\bar{r}})) \end{aligned}$$

Substituting these derivatives in the inequality, we have:

$$\begin{aligned}&\omega _{hh}\omega _{rr}\ln (\alpha {\bar{r}}) - \omega _{rr}\omega _{hh}\ln (r) + 2\omega _{hh}\omega _{rh}\ln (\alpha {\bar{r}}) - 2\omega '_{rh}\omega _{hh}<0 \\&\omega _{rr}\ln (\alpha {\bar{r}}) - \omega _{rr}\ln (r) + 2\omega _{rh}\ln (\alpha {\bar{r}}) + \omega _{rr}\ln (r) + \omega _{hh}\ln (\alpha {\bar{r}}) - \omega _{\lnot h \lnot h}\ln ({\bar{\alpha }}{\bar{r}})< 0 \\&\omega _{rr}\ln (\alpha {\bar{r}}) + 2\omega _{rh}\ln (\alpha {\bar{r}}) + \omega _{hh}\ln (\alpha {\bar{r}}) - \omega _{\lnot h \lnot h}\ln ({\bar{\alpha }}{\bar{r}})<0 \\&(\omega _{rr} + 2r + 2\alpha {\bar{r}} - 1 - \omega _{rr} - \omega _{hh} + \omega _{\lnot h \lnot h} + \omega _{hh})\ln (\alpha {\bar{r}}) - \omega _{\lnot h \lnot h}\ln ({\bar{\alpha }}{\bar{r}})<0 \\&(2(r + \alpha {\bar{r}}) - 1)\ln (\alpha {\bar{r}}) + \omega _{\lnot h \lnot h}(\ln (\alpha {\bar{r}} -\ln ({\bar{\alpha }}{\bar{r}}) )<0 \\&(1 - 2{\bar{\alpha }}{\bar{r}})ln(\alpha {\bar{r}}) + ({\bar{\alpha }}{\bar{r}})^{1+\delta }ln(\frac{\alpha }{{\bar{\alpha }}}) <0 \end{aligned}$$

The last line being exactly the inequality claimed in Proposition 4.

$\square$