Coherence and Reliability in Judicial Reasoning

Abstract

Experimental studies of how juries reach their verdicts in court strongly suggest that coherence reasoning is ubiquitous in judicial reasoning. Under massive cognitive pressure to process large numbers of conflicting pieces of evidence and witness reports, jury members base their judgment on an assessment of the most coherent account of the events. From a normative perspective, the legitimacy of coherence reasoning in court hinges on the premise that such coherence is a plausible guide to justified belief. Unfortunately, this notion has been severely challenged by numerous recent studies in Bayesian formal epistemology. Bovens and Hartmann (2003) (Bayesian epistemology). New York/Oxford: Oxford University Press and Olsson (2005) (Against coherence: Truth, probability and justification). Oxford: Oxford University Press have shown that there is no way to measure coherence such that coherence is truth conducive in the sense that more coherence implies a higher likelihood of truth. This is so even under seemingly very weak boundary conditions. In previous work we have shown that (certain forms of) coherence can be reliability conducive in paradigmatic scenarios where such coherence fails to be truth conducive. In other words, more coherence can still be indicative of a higher probability that the witnesses are reliable. We have also argued that the connection between (certain forms of) coherence and probability of reliability may be what justifies our common reliance on coherence reasoning. While the link between coherence and reliability was found to be not completely general, our studies so far do support the contention that this link is stronger than that between coherence and truth. In this paper, we add credence to this conclusion by proving several new formal results connecting one prominent measure of coherence, the Shogenji measure, to witness reliability. The most striking of these results is that in a case where the witnesses’ degrees of reliability are maximally dependent of each other—i.e., where either all witnesses are reliable or all witnesses are unreliable—the Shogenji measure is reliability conducive. We also relate our approach to the Evidentiary Value tradition in Scandinavian legal theory.

Appendix

Proof of theorem 3:

$$ P\left( {{R_i}\left| {{E_1},\ldots,{E_n}} \right.} \right)=\frac{{{S_{{{A_i}n}}}+x{S_{{{A_i}n-1}}}+\ldots +{x^{n-2 }}{S_{{{A_i}2}}}+{x^{n-1 }}}}{{{S_n}+x{S_{n-1 }}+\ldots +{x^{n-2 }}{S_2}+n{x^{n-1 }}+{x^n}}} $$

Let ${\boldsymbol{R}_1},\ldots,{\boldsymbol{R}_n},{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n},{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}$ be propositional variables. Then:

$$ P\left( {{\boldsymbol{R}_1},\ldots,{\boldsymbol{R}_n},{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n},{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)= $$

$$ P\left( {{\boldsymbol{E}_1}\left| {{\boldsymbol{R}_1}}\right.,{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{E}_n}\left| {{\boldsymbol{R}_n}}\right.,{\boldsymbol{A}_n}} \right)\times P\left( {{\boldsymbol{R}_1},\ldots,{\boldsymbol{R}_n},{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)= $$

(iii)

$$ P\left( {{\boldsymbol{E}_1},{\boldsymbol{R}_1},{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{E}_n},{\boldsymbol{R}_n},{\boldsymbol{A}_n}} \right)\times \frac{{P\left( {{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)}}{{P\left( {{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{A}_n}} \right)}}= $$

(iv)

$$ \begin{gathered} P\left( {{\boldsymbol{A}_1}\left| {{\boldsymbol{E}_1},{\boldsymbol{R}_1}} \right.} \right)\times \ldots \times P\left( {{\boldsymbol{A}_n}\left| {{\boldsymbol{E}_n},{\boldsymbol{R}_n}} \right.} \right)\times \frac{{P\left( {{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)}}{{P\left( {{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{A}_n}} \right)}} \hfill \\ \times P\left( {{\boldsymbol{R}_1}\left| {{\boldsymbol{E}_1}} \right.} \right)\times P\left( {{\boldsymbol{E}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{R}_n}\left| {{\boldsymbol{E}_n}} \right.} \right)\times P\left( {{\boldsymbol{E}_n}} \right) \hfill \\ \end{gathered} $$

Using this derivation, we calculate the probability of $P\left( {{R_1}\left| {{E_1},\ldots,{E_n}} \right.} \right)$, which equals $P\left( {{R_i}\left| {{E_1},\ldots,{E_n}} \right.} \right)$, for any i. Let:

$$ P\left( {{A_i}} \right)={a_i} $$

$$ P\left( {{A_i},\ldots,{A_j}} \right)={a_{{i,\ldots,j}}} $$

$$ P\left( {{E_i}} \right)={e_i} $$

$$ P\left( {{R_i}\left| {{E_i}} \right.} \right)=m $$

Then:

$$ P\left( {{\boldsymbol{R}_1}\left| {{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n}} \right.} \right)=\frac{{P\left( {{\boldsymbol{R}_1},{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n}} \right)}}{{P\left( {{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n}} \right)}}= $$

$$ =\frac{{\sum\nolimits_{{{A_1},\ldots,{A_n},{R_2},\ldots,{R_n}}} {P\left( {{R_1},{R_2}\ldots, {R_n},{A_1},\ldots,{A_n},{E_1},\ldots,{E_n}} \right)} }}{{\sum\nolimits_{{{A_1},\ldots,{A_n},{R_1},\ldots,{R_n}}} {P\left( {{R_1},\ldots,{R_n},{A_1},\ldots,{A_n},{E_1},\ldots,{E_n}} \right)} }} $$

$$ P\left( {{R_1},{E_1},\ldots,{E_n}} \right)=\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^{n-1 } {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{b_k}} $$

(vii)

where ${b_k}=\sum\limits_{{1<{q_2}<\ldots <{q_r}\leq n,r=n-k}} {\frac{{{a_{{1,{q_2},\ldots,{q_r}}}}}}{{{a_1},{a_{{{q_2}}}},\ldots,{a_{{{q_r}}}}}}}$

$$ P\left( {{E_1},\ldots,{E_n}} \right)=\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^n {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{c_k}} $$

(vii)

where ${c_k}=\sum\limits_{{{q_1}<,\ldots,<{q_r}\leq n,r=n-k}} {\frac{{{a_{{{q_1},\ldots,{q_r}}}}}}{{{a_{{{q_1}}}},\ldots,{a_{{{q_r}}}}}}}$

Hence:

$$ \begin{gathered} P\left( {{R_1}\left| {{E_1},\ldots,{E_n}} \right.} \right) =\frac{{\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^{n-1 } {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{b_k}} }}{{\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^n {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{c_k}} }}= \end{gathered} $$

$$ \quad\quad\;\; =\frac{{\sum\limits_{k=0}^{n-1 } {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{b_k}} }}{{\sum\limits_{k=0}^n {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{c_k}} }} $$

$$ \quad\quad\;\; =\frac{{{S_{{{A_1}n}}}+x{S_{{{A_1}n-1}}}+\ldots +{x^{n-2 }}{S_{{{A_1}2}}}+{x^{n-1 }}}}{{{S_n}+x{S_{n-1 }}+\ldots +{x^{n-2 }}{S_2}+n{x^{n-1 }}+{x^n}}} $$

Proof of theorem 4:

Let $\boldsymbol{R},{\boldsymbol{E}}_{\textbf{1}},\ldots,{\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}}$ be propositional variables. Then:

$$ P\left( {\boldsymbol{R}},{\boldsymbol{E_1}},\ldots,{\boldsymbol{E}}_{\boldsymbol{n}},\boldsymbol{A}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)= $$

$$ P\left( {\boldsymbol{E}}_{\textbf 1}\left| {\boldsymbol{R}} \right.,{\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{E}}_{\boldsymbol{n}}\left| \boldsymbol{R} \right.,{\boldsymbol{A}}_{\boldsymbol{n}} \right)\times P\left( {\boldsymbol{R}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)= $$

(iii′)

$$ \begin{gathered} P\left( {\boldsymbol{E}}_{\textbf 1},{\boldsymbol{R}},{\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{R}},{\boldsymbol{A}}_{\boldsymbol{n}} \right)\times \hfill \\ \frac{{P\left( {\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)}}{{P\left( {\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{A}}_{\boldsymbol{n}} \right)}}\times \frac{1}{P{({\boldsymbol{R}})^{{\boldsymbol{n}}-{\textbf 1} }}}= \end{gathered} $$

(iv′)

$$ \begin{gathered} P\left( {\boldsymbol{A}}_{\textbf 1}\left| {{\boldsymbol{E}}_{\textbf 1},{\boldsymbol{R}}} \right. \right)\times \ldots \times P\left( {{\boldsymbol{A}}_{\boldsymbol{n}}\left| {{\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{R}}} \right.} \right)\times \frac{{P\left( {\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)}}{{P\left( {\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{A}}_{\boldsymbol{n}} \right)}}\times \hfill \\ \frac{{P\left( {\boldsymbol{R}}\left| {\boldsymbol{E}}_{\textbf 1} \right. \right)\times P\left( {\boldsymbol{E}}_{\textbf 1} \right)}}{P({\boldsymbol{R}})}\times \ldots \times \frac{{P\left( {\boldsymbol{R}}\left| {\boldsymbol{E}}_{\boldsymbol{n}} \right. \right)\times P\left( {\boldsymbol{E}}_{\boldsymbol{n}} \right)}}{P({\boldsymbol{R}})}\times P({\boldsymbol{R}}) \hfill \\ \end{gathered} $$

Then:

$$ P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)=P\left( {{A_1},\ldots,{A_n}} \right)\times \frac{{P\left( {{E_1}\left| R \right.} \right)}}{{P\left( {{A_1}} \right)}}\times \ldots \times \frac{{P\left( {{E_n}\left| R \right.} \right)}}{{P\left( {{A_n}} \right)}} $$

(i′)

$$ P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)=P\left( {{E_1}\left| {\neg R} \right.} \right)\times \ldots \times P\left( {{E_1}\left| {\neg R} \right.} \right) $$

(ii′)

Let:

$$ P(R)=r $$

$$ P\left( {R\left| {{E_i}} \right.} \right)=m $$

$$ \bar{p}=1-p, \hbox{for any variable}\ p. $$

Then:

$$ \begin{aligned}[b] &P\left( {R\left| {{E_1},\ldots,{E_n}} \right.} \right)=\\ &\frac{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{{m{e_1}}}{r}\times \ldots \times \frac{{m{e_n}}}{r}\times r}}{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{{m{e_1}}}{r}\times \ldots \times \frac{{m{e_n}}}{r}\times r+\frac{{\bar{m}{e_1}}}{\bar{r}}\times \ldots \times \frac{{\bar{m}{e_n}}}{\bar{r}}\times \bar{r}}} \\ &\quad =\frac{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{m}{r}\times \ldots \times \frac{m}{r}\times r}}{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{m}{r}\times \ldots \times \frac{m}{r}\times r+\frac{\bar{m}}{\bar{r}}\times \ldots \times \frac{\bar{m}}{\bar{r}}\times \bar{r}}} \end{aligned}$$

(vii′)

Thus P(R|E ₁,…,E _n ) is a strictly increasing function of C _Sh (A ₁,…, A _n ), given the assumptions of the scenario. Hence, the Shogenji measure is reliability conducive, in the present scenario.

Proofs of observation 3 and 4:

$$ {\boldsymbol{R}},{\boldsymbol{E}}_{\textbf 1},\ldots,{\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}}=P\left( {\boldsymbol{E}}_{\textbf 1}\left| {\boldsymbol{R}} \right.,{\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{E}}_{\boldsymbol{n}}\left| {\boldsymbol{R}} \right.,{\boldsymbol{A}}_{\textbf 1} \right)\\ \quad \times P\left( {\boldsymbol{R}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right) $$

(see proof of theorem 4)

Hence:

$$ P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)=P\left( {{A_1},\ldots,{A_n}} \right) $$

(a′)

$$ P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)=P\left( {{A_1}} \right)\times \ldots \times P\left( {{A_n}} \right) $$

(b′)

Proofs of observation 6 and 7:

See proof of theorem 4.

Proof of observation 9:

Assume for reductio that:

$$ P\left( {{R_1}\vee {R_2}\left| {{E_1},{E_2}} \right.} \right)<P\left( {{R_1}\left| {{E_1}} \right.} \right)+P\left( {{R_2}\left| {{E_2}} \right.} \right)-P\left( {{R_1}\left| {{E_1}} \right.} \right)P\left( {{R_2}\left| {{E_2}} \right.} \right) $$

Let P(H) = h. P(H) is the probability of what the witnesses agree upon.

$$ \frac{{P\left( {{R_1}\vee {R_2},{E_1},{E_2}} \right)}}{{P\left( {{E_1},{E_2}} \right)}}=\frac{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}} $$

(Schubert 2011, 273) together with condition (vii)

$$ P\left( {{R_1}\left| {{E_1}} \right.} \right)+P\left( {{R_2}\left| {{E_2}} \right.} \right)-P\left( {{R_1}\left| {{E_1}} \right.} \right)P\left( {{R_2}\left| {{E_2}} \right.} \right)=2m-{m^2} $$

$$ \frac{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}}<2m-{m^2} $$

$$ \Leftrightarrow {m^2}\frac{1}{h}+2m\left( {1-m} \right)<\left( {2m-{m^2}} \right)\left( {{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{{\left( {1-m} \right)}}^2}} \right) $$

$$ \Leftrightarrow {{\left( {1-m} \right)}^2}\left( {{m^2}\frac{1}{h}+2m\left( {1-m} \right)} \right)<\left( {2m-{m^2}} \right){{\left( {1-m} \right)}^2} $$

$$ \Leftrightarrow 1<h $$

But this contradicts condition (v), which says that 0 < h < 1. Hence $P\left( {{R_1}\vee {R_2}\left| {{E_1},{E_2}} \right.} \right)\geq P\left( {{R_1}\left| {{E_1}} \right.} \right)+P\left( {{R_2}\left| {{E_2}} \right.} \right)-P\left( {{R_1}\left| {{E_1}} \right.} \right)P\left( {{R_2}\left| {{E_2}} \right.} \right)$.

Proof of observation 10:

$$ P\left( {{R_1}\left| {{E_1},{E_2}} \right.,\neg {R_2}} \right)=\frac{{{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}\left( {1-m} \right)m+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}}=m $$

(Schubert 2011, 273) together with condition (vii)

Hence $P\left( {{R_1}\left| {{E_1},{E_2}} \right.,\neg {R_2}} \right)=P\left( {{R_1}\left| {{E_1}} \right.} \right)$

Proof of observation 11:

$$ \frac{{P\left( {{R_1},{E_1},{E_2}} \right)}}{{P\left( {{E_1},{E_2}} \right)}}=\frac{{{e_1}{e_2}{m^2}\frac{1}{h}+{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}} $$

(Schubert 2011, 273) together with condition (vii)

$$ =\frac{{{m^2}\frac{1}{h}+m\left( {1-m} \right)}}{{{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{{\left( {1-m} \right)}}^2}}} $$

Assume for reductio that $P\left( {{R_1}\left| {{E_1},{E_2}} \right.} \right)<P\left( {{R_1}\left| {{E_1}} \right.} \right)$

$$ \frac{{{m^2}\frac{1}{h}+m\left( {1-m} \right)}}{{{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{{\left( {1-m} \right)}}^2}}}<m $$

$$ \Leftrightarrow m\frac{1}{h}+\left( {1-m} \right)<{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{\left( {1-m} \right)}^2} $$

$$ \Leftrightarrow \frac{1}{h}-1<m\left( {\frac{1}{h}-1} \right) $$

But, since $1>m>0$ and $1>h>0$, this cannot hold. Hence $P\left( {{R_1}\left| {{E_1},{E_2}} \right.} \right)\geq P\left( {{R_1}\left| {{E_1}} \right.} \right)$.