Judgement Aggregation and Distributed Thinking

Abstract

In recent years, judgement aggregation has emerged as an important area of social choice theory. Judgement aggregation is concerned with aggregating sets of individual judgements over logically connected propositions into a set of collective judgements. It has been shown that even seemingly weak conditions on the aggregation function render it impossible to find functions that produce rational collective judgements from all possible rational individual judgements. This implies that the step from individual judgements to collective judgements requires trade-offs between different desiderata, such as universal domain, rationality, epistemological quality, and unbiasedness. These dilemmas challenge us to decide which conditions we should relax. The typical application for judgement aggregation is the problem of group decision making, with juries and expert committees as stock examples. However, the relevance of judgement aggregation goes beyond these cases. In this survey I review some core results in the field of judgement aggregation and social epistemology and discuss their implications for the analysis of distributed thinking.

Appendix

Bovens and Rabinowicz (2006) calculate the probabilities of the group being correct, conditional on the state. They define M ^pbp as the proposition ‘The premise-based procedure yields the correct result’ and calculate probabilities conditional on all 4 states:

$$ \begin{aligned}[c] P\bigl(M^{\mathit{pbp}}|\mathrm{S}1\bigr) =& P^{\mathit{CJT}}(n,p)^{2}, \\P\bigl(M^{\mathit{pbp}}|\mathrm{S}2\bigr) =& Pr^{\mathit{CJT}}(n,p)^{2} + P^{\mathit{CJT}}(n,p)\bigl(1 - P^{\mathit{CJT}}(n,p)\bigr)\\&{} + \bigl(1 - P^{\mathit{CJT}}(n,p)\bigr)^{2}, \\P\bigl(M^{\mathit{pbp}}|\mathrm{S}3\bigr) =& P\bigl(M^{\mathit{pbp}}|\mathrm{S}2\bigr), \\P\bigl(M^{\mathit{pbp}}|\mathrm{S}4\bigr) =& P^{\mathit{CJT}}(n,p)^{2} + 2P^{\mathit{CJT}}(n,p)\bigl(1 - P^{\mathit{CJT}}(n,p)\bigr). \end{aligned} $$

(3.2)

Note that one can arrive at the correct result even though some or even both collective judgements on the premises are wrong. Given the logical dependency between the propositions, we know that π(R)=π(P)π(Q). Summing up the conditional probabilities of being correct with the premise-based procedure, weighted by the probabilities that the different states obtain yields:

$$ \begin{aligned}[b] P\bigl(M^{\mathit{pbp}}\bigr) =& P\bigl(M^{\mathit{pbp}}|\mathrm{S}1\bigr)\pi(P)\pi(Q) + P\bigl(M^{\mathit{pbp}}|\mathrm{S}2\bigr)\bigl(1 - \pi (P)\bigr)\pi(Q) \\&{} + P\bigl(M^{\mathit{pbp}}|\mathrm{S}3\bigr)\pi(P)\bigl(1 - \pi(Q)\bigr)\\&{} + P\bigl(M^{\mathit{pbp}}|\mathrm{S}4\bigr)\bigl(1 - \pi(P)\bigr)\bigl(1 - \pi(Q)\bigr). \end{aligned} $$

(3.3)

Following Bovens and Rabinowicz’s exposition for the conclusion-based procedure, let V be the proposition that a single voter determines the conclusion correctly, and P(V) the probability the voter does so. Since each single voter applies deductive closure, we obtain the following probabilities for each single voter to be correct on the conclusion, based on their competence p:

$$ \begin{aligned}[c] P(V|\mathrm{S}1) &= p^{2} \\P(V|\mathrm{S}2) &= P(V|\mathrm{S}3) = p^{2} + p(1 - p) + (1 - p)^{2} \\P(V|\mathrm{S}4) &= p^{2} + 2p(1 - p). \end{aligned} $$

(3.4)

Each individual can reach the correct conclusion by being correct on both premises (probability p ²) but one can also be correct, even if one is wrong on one or even both of the premises. Let M ^cbp denote the proposition that the conclusion-based procedure yields the correct result. Conditional on the state, we can apply Eq. (3.1) to calculate the probability of a correct majority vote on the conclusion:

$$ P\bigl(M^{\mathit{cbp}}|\mathrm{S}i\bigr) = P^{\mathit{CJT}}\bigl(n,P(V| \mathrm{S}i)\bigr). $$

(3.5)

Summing up the probabilities weighted by the prior probabilities of the different states yields:

$$ \begin{aligned}[b] P\bigl(M^{\mathit{cbp}}\bigr) =& P\bigl(M^{\mathit{cbp}}|\mathrm{S}1\bigr)\pi(P)\pi(Q) + P\bigl(M^{\mathit{cbp}}|\mathrm{S}2\bigr)\bigl(1 - \pi(P)\bigr)\pi(Q) \\&{}+ P\bigl(M^{\mathit{cbp}}|\mathrm{S}3\bigr)\pi(P)\bigl(1 - \pi(Q)\bigr)\\&{} + P\bigl(M^{\mathit{cbp}}|\mathrm{S}4\bigr)\bigl(1 - \pi(P)\bigr)\bigl(1 - \pi(Q)\bigr). \end{aligned} $$

(3.6)

The results for the premise-based procedure in (3.3) and the conclusion-based procedure in (3.6) are based on the assumption that it does not matter whether the correct result is deduced from correct or incorrect judgements on the premises. If we want to be right “for the right reasons”, the cases where incorrect judgements lead to correct outcomes need to be removed. Let M ^pbp−rr denote the proposition that the group has arrived at the right judgement for the right reasons. This yields:

$$ P\bigl(M^{\mathit{pbp} - \mathit{rr}}\bigr) = P^{\mathit{CJT}}(n,p)^{2} . $$

(3.7)

Similarly, for the conclusion-based procedure one want to consider the probability that a majority of voters is correct for the right reasons:

$$P\bigl(M^{\mathit{cbp} - \mathit{rr}}\bigr) = P^{\mathit{CJT}}\bigl(n,p^{2}\bigr). $$