Variety of evidence and the elimination of hypotheses

Abstract

Varied evidence for a hypothesis confirms it more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys long-standing widespread intuitive support. Recent literature has raised serious doubts that the correlational approach of explicating the thesis can vindicate it. By contrast, the eliminative approach due to Horwich vindicates the Variety of Evidence Thesis but only within a relatively narrow domain. I investigate the prospects of extending the eliminative approach to a larger domain by considering a larger class of sensible explications of evidential variety. For a large subclass class of such explications I show how to construct cases in which the less diverse body of evidence for a hypothesis confirms more strongly. I hence argue that these prospects are dire since the eliminative approach widely fails to vindicate the thesis.

Appendix A:: Formal analysis

1.1 A.1 Varied evidence constraining less tightly

Here the formal analysis, in some detail. We set:

• 1. $\mathcal {E}_{N}:=[0,0.1]$ and $\mathcal {E}_{D}:=[3-\sqrt {4.59},1]\approx [0.86,1]$.

• 2. For the f_i we have f_i(y) := 0 for all y∉[0, 3] and for all x ∈ [0, 3]¹⁸

  $$ \begin{array}{@{}rcl@{}} f_{1}(x)&:=&2/3-2x/9\\ f_{2}(x)&:=&x\cdot f_{1}(x)=x\cdot (2/3-2x/9)=2x/3-2x^{2}/9\\ f_{3}(x)&:=&1/30+3x/5-x^{2}/5\\ f_{4}(x)&:=&1/12+x/2-x^{2}/6 . \end{array} $$

  The H_i are probabilistic hypotheses, $P(X\in [0,3])={{\int \limits }_{0}^{3}} f_{i}(x)dx=1$ for all i ∈{1, 2, 3, 4}.

We now check that the assumptions are satisfied:

• I_D is more diverse than I_N [from 1].

• H₁ is simpler than the other H_i: H₁ is affine-linear while the other H_i are parabola [from 2].

• The Ceteris Paribus clause (3) holds:

  $$ \begin{array}{@{}rcl@{}} P_{1}(\mathcal{E}_{N}|H_{1})&=&{\int}_{0}^{0.1} f_{1}(x)dx=[2x/3-x^{2}/9]_{0}^{0.1}=2/30-1/900 = 59/900\\ P_{1}(\mathcal{E}_{D}|H_{1})&=&{\int}^{1}_{3-\sqrt{4.59}} f_{1}(x)dx=[2x/3-x^{2}/9]_{3-\sqrt{4.59}}^{1}\\ &=&2/3-1/9-(2-\frac{2}{3}\sqrt{4.59})+1/9(9-6\sqrt{4.59}+4.59)\\ &=&5/9-1+ 459/900\\ &=&-4/9+ 459/900 = 59/900 . \end{array} $$

• $\mathcal {E}_{N},\mathcal {E}_{D}$ are both evidence for H₁: To show that $P(H_{1})<P(H_{1}|\mathcal E)$ for $\mathcal E\in \{\mathcal {E}_{N},\mathcal {E}_{D}\}$ we first note that $P(\mathcal E|H_{1})>P(\mathcal E|H_{i})$ for i ∈{2, 3, 4}, since for all x ∈ I_N ∪ I_D ∖{1} it holds that f₁(x) > f_i(x) > 0. We now find

  $$ \begin{array}{@{}rcl@{}} P(H_{1}|\mathcal E)>P(H_{1})&\Leftrightarrow& \frac{P(\mathcal E|H_{1})}{P(\mathcal E)}>1\\ &\Leftrightarrow& P(\mathcal E|H_{1})>{\sum}_{i=1}^{4} P(\mathcal E|H_{i})P(H_{i})\\ &\Leftrightarrow& P(\mathcal E|H_{1})(1-P(H_{1}))>{\sum}_{i=2}^{4} P(\mathcal E|H_{i})P(H_{i}) . \end{array} $$

To show that $\mathcal {E}_{N},\mathcal {E}_{D}$ are both evidence for H₁, it now suffices to note that

$$ \begin{array}{@{}rcl@{}} \sum\limits_{i=2}^{4} P(\mathcal E|H_{i})P(H_{i})&\leq&\max\limits_{2\leq k\leq 4}P(\mathcal{E}|H_{k})\sum\limits_{i=2}^{4}P(H_{i})\\ &=&(1-P(H_{1}))\max_{2\leq i\leq 4}P(\mathcal E|H_{i})\\ &<&(1-P(H_{1}))P(\mathcal{E}|H_{1}) . \end{array} $$

It only remains show that the more diverse body of evidence confirms less strongly, $P(H_{1}|\mathcal {E}_{D})<P(H_{1}|\mathcal {E}_{N})$:

$$ \begin{array}{@{}rcl@{}} P(H_{1}|\mathcal{E}_{D})<P(H_{1}|\mathcal{E}_{N})&\Leftrightarrow& \frac{P(\mathcal{E}_{D}|H_{1})}{{\sum}_{i=1}^{4}P(\mathcal{E}_{D}H_{i})}<\frac{P(\mathcal{E}_{N}|H_{1})}{{\sum}_{i=1}^{4}P(\mathcal{E}_{N}H_{i})} \end{array} $$

(4)

$$ \begin{array}{@{}rcl@{}} &\overset{3}{\Leftrightarrow}& {\sum\limits_{i=1}^{4}P(\mathcal{E}_{N}H_{i})}<{\sum\limits_{i=1}^{4}P(\mathcal{E}_{D}H_{i})}\\ &\overset{3}{\Leftrightarrow}& {\sum\limits_{i=2}^{4}P(\mathcal{E}_{N}|H_{i})}P(H_{i})<{\sum\limits_{i=2}^{4}P(\mathcal{E}_{D}|H_{i})P(H_{i})} . \end{array} $$

(5)

Since for all i ∈{2, 3, 4} it holds that $\sup _{x\in I_{N}}f_{i}(x)<\inf _{x\in I_{D}}f_{i}(x)$ and since |I_N| < |I_D| we indeed have

$$ \begin{array}{@{}rcl@{}} P(\mathcal{E}_{N}|H_{i})&=&{\int}_{I_{N}}f_{i}(x)dx\leq |I_{N}|\sup\limits_{x\in I_{N}}f_{i}(x)\\ &&<|I_{D}|\inf\limits_{x\in I_{D}}f_{i}(x)\leq {\int}_{I_{D}}f_{i}(x)dx=P(\mathcal{E}_{D}|H_{i}) \enspace. \end{array} $$

(6)

1.2 A.2 Varied Evidence constraining more tightly

The formal analysis for this case resembles the previous one and his hence given in a more condensed form. First put

• 1. I_N := [− 0.1, 0.1] and $I_{D}:=[3-\sqrt {4.59},1]\approx [0.86,1]$.

• 2. For i ∈{1, 2, 3, 4} the f_i are as above and for f₅ we have f₅(y) = 0 for all y < − 5 and y > 5 and for all x ∈ [− 5, 5]¹⁹

  $$ \begin{array}{@{}rcl@{}} f_{5}(x)&:=\left\{\begin{array}{ll}\frac{1}{25}+\frac{\sin(x)}{25} & x\in[-5,0)\\ \frac{4}{25}+\frac{\sin(x)}{25}& x\in[0,5] . \end{array}\right. \end{array} $$

• 3. Let P(H₅) be tiny but non-zero, $P(H_{5})=\frac {1}{10000}$ say.

Mutatis mutandis, the proof above (Appendix A1) applies here, too. After going through the calculations again we obtain $P(H_{1}|\mathcal {E}_{D})<P(H_{1}|\mathcal {E}_{N})$, since the prior of H₅ is tiny and all other hypotheses assign probability to zero to the points added to the left interval.

1.3 A.3 The general case

The two proofs above apply to a special case with fixed $\mathcal {E}_{D},\mathcal {E}_{N}$. I now show how to prove the general case for varied evidence constraining less tightly, i.e., arbitrary $\mathcal {E}_{D},\mathcal {E}_{N}$ satisfying |I_N| < |I_D|. This proof shows how to modify the functions f_i for the general case. The case for varied evidence constraining more tightly is all but analogous and is only sketched here.

In the general case of varied evidence constraining less tightly we have that |I_N| < |I_D|. The ceteris paribus condition $P_{1}(\mathcal {E}_{D})=P_{1}(\mathcal {E}_{N})$ is also assumed throughout.

Let us first assume that I_N and I_Dare disjoint. By flipping the x-axis (f_i(x)↦f_i(−x)) if necessary, we may assume that I_D is to the right of I_N.

Shifting along the x-axis (f_i(x)↦f_i(x + shift)) if necessary, we may assume that I_N = [0,α],I_D = [β,γ] with 0 < α <β< γ and γ−β > α.

If γ ≤ 1, then define the functions f_i for all i ∈{1, 2, 3, 4} as in the proof in Appendix A1 which then applies, mutatis mutandis. a and b hold. c holds by assumption, d holds since f_i(x) < f₁(x) for all x ∈ (0, 1) all i ∈{2, 3, 4}. $P(H_{1}|\mathcal {E}_{N})>P(H_{1}|\mathcal {E}_{D})$ follows.

If γ > 1, then put g_i(x) := γf_i(γx). Geometrically, this shrinks the x-axis by the factor γ while the y-axis is dilated by the factor γ. Hence, all ordinal comparisons are preserved for all $x\in \mathbb {R}$: f_i(γx) < f_j(γx), if and only if g_i(x) < g_j(x). It only remains to check that the g_i define probability functions; we hence calculate for all i ∈{1, 2, 3, 4}

$$ \begin{array}{@{}rcl@{}} {{\int}_{0}^{3}}g_{i}(x)dx&=&{{\int}_{0}^{3}}\gamma f_{i}(\gamma x) dx={\gamma{\int}_{0}^{3}} f_{i}(\gamma x) dx=\gamma[\frac{1}{\gamma} F_{i}(\gamma x)]_{0}^{3}\\ &=&F_{i}(3\gamma)-F_{i}(0)={\int}_{0}^{3\gamma}f_{i}(x)dx=1 , \end{array} $$

where F_i denotes a primitive function of f_i (uniquely defined up to an inconsequential additive constant) characterised by $\frac {d}{dx}F_{i}(x)=f_{i}(x)$.

I_N and I_D are not disjoint and I_N is not a proper subset of I_D, I_N ∩ I_D≠∅. We begin by putting $I_{D}^{\prime }:=I_{D}\setminus (I_{D}\cap I_{N})\neq \emptyset$ and $I_{N}^{\prime }:=I_{N}\setminus (I_{D}\cap I_{N})\neq \emptyset$. Now proceed with these bodies of evidence as above. The Ceteris Paribus Clause (3) again holds because

$$ \begin{array}{@{}rcl@{}} P_{1}(I_{N}^{\prime}|H_{1})&=&P_{1}(I_{N}|H_{1})-P_{1}(I_{N}\cap I_{D}|H_{1})\\ &=&P_{1}(I_{D}|H_{1})-P_{1}(I_{N}\cap I_{D}|H_{1})\\ &=&P_{1}(I_{D}^{\prime}|H_{1}) \enspace. \end{array} $$

It again follows that $P(H_{1}|I_{D}^{\prime })>P(H_{1})<P(H_{1}|I_{N}^{\prime })$ [because f₁(x) > f_i(x) on I_D ∪ I_N for all i ≥ 2] and eventually that $P(H_{1}|I_{N}^{\prime })>P(H_{1}|I_{D}^{\prime })$. Thus, $P(H_{1}|\mathcal {E}_{N}^{\prime })>P(H_{1}|\mathcal {E}_{D}^{\prime })$.

Finally, ifI_N is a proper subset of I_D, then by Ceteris Paribus Clause (3) it holds that P(I_D ∖ I_N|H₁) = 0 (almost everywhere). Now simply devise functions f_i for i ∈{2, 3, 4} which take large values on I_D ∖ I_N. Then, these H_i are more compatible with $\mathcal {E}_{D}$ than with $\mathcal {E}_{N}$. Hence, $\mathcal {E}_{N}$ confirms H₁ stronger than $\mathcal {E}_{D}$, $P(H_{1}|\mathcal {E}_{N})>P(H_{1}|\mathcal {E}_{D})$. Now proceed as above.

Here now the sketch for the case in which varied evidence constrains more tightly, |I_D| < |I_N|.

Again, first assume that I_N and I_D are disjoint. As above, by flipping and shifting, if necessary, we may assume that 0 is in the interior of I_N and that I_D is to the right of I_N. Next, shift the intervals again such that for the lower and upper bounds of I_N,I_D, l_N,l_D,u_N,u_D, it holds that u_N ⋅ 0.14 = (u_D − l_D) ⋅ 0.1. That is, the parts of the interval to the right of zero, stand in the same proportion as those in the example.

Now define a hypothesis H₅ in terms of $f_{5}:\mathbb {R}\to \mathbb {R}_{\geq 0}$ such that i) f₅(x) > 0 for all x ∈ I_N ∪ I_D, ii) f₅(x) is sufficiently small (so that we can ignore it in the remainder) relative to the other hypotheses where f₅ is non zero and iii) such that f₅ is a probability distribution, ${\int \limits }_{\mathbb {R}} f_{5}(x)dx=1$.

I_N and I_D are not disjoint and I_D is not a proper subset of I_N: The idea remains the same, pick functions f_i which vanish on the intersection I_D ∩ I_N. Proceed as in the case with |I_N| < |I_D| with those intervals outside the intersection.

The only not analogous case for varied evidence constraining more tightly occurs for the proper subset case, if I_D is a proper subset of I_N. In this case, Eq. 1 holds for at least one j ≥ 2 – otherwise the current frame of investigations collapses I_N to I_D. For all other hypotheses H_j we have P(I_D|H_j) ≤ P(I_N|H_j) since I_D ⊂ I_N. We are hence in a case in which all assumptions of the eliminative, including the caveats, apply (see Eq. 5). So, this is not a case in which could extend the eliminative approach, since it is already covered by it.