Comparing Probabilistic Accounts of Probability Judgments

Abstract

Bayesian theories of cognitive science hold that cognition is fundamentally probabilistic, but people’s explicit probability judgments often violate the laws of probability. Two recent proposals, the “Probability Theory plus Noise” (PT+N; Costello and Watts Psychological Review, 121, 463–480, 2014) and “Bayesian Sampler” (Zhu et al. Psychological Review, 127, 719–748, 2020) theories of probability judgments, both seek to account for these biases while maintaining that mental credences are fundamentally probabilistic. These models differ in their averaged predictions about people’s conditional probability judgments and in their distributional predictions about their overall patterns of judgments. In particular, the Bayesian Sampler’s Bayesian adjustment process predicts a truncated range of responses as well as a correlation between the average degree of bias and variability trial-to-trial. However, exploring these distributional predictions with participants’ raw responses requires a careful treatment of rounding errors and exogenous response processes. Here, I cast these theories into a Bayesian data analysis framework that supports the treatment of these issues along with principled model comparison using information criteria. Comparing the fits of both models on data collected by (Zhu et al. Psychological Review, 127(5), 719–748 2020), I find these data are best explained by an account of biases based on “noise” in the sample-reading process but in which conditional probability judgments are produced by a process of conditioning in the mental model of the events, rather than in a two-stage mental sampling process as proposed by the PT+N model.

Appendix:

For the trial-level response models participants’ rounded responses are modeled as discrete responses with a categorical (multinomial) distribution. For i ∈{0, 1,...,m} where m = 20 possible responses, define a set of cut points $a_{i} = \frac {i}{m}-\frac {1}{2m}$ and $b_{i} = \frac {i}{m}+\frac {1}{2m}$. Using x|_[0,1] to denote that x is restricted to the domain [0, 1], the probability of each response given μ and N is:

$$ \begin{array}{@{}rcl@{}} p_{i,5} &=& P([a_{i,5},b_{i,5}))\\ &=& B(a_{i,5}|_{[0,1]}, \mu N, (1-\mu)N) \\&&- B(b_{i,5}|_{[0,1]}, \mu N, (1-\mu)N) \end{array} $$

where B is the appropriate cumulative distribution function. To capture rounding to 10 we define $a_{i,10} = \frac {2i}{m}-\frac {1}{m}$ and $b_{i} = \frac {2i}{m}+\frac {1}{m}$, so that the probability of each response is:

$$ p_{i,10} = \left\{\begin{array}{ll} P([a_{i,10},b_{i,10})) & \text{i is even} \\ 0 & \text{i is odd} \end{array}\right. $$

(14)

Next defined a vector of mixture probabilities $\overrightarrow {\phi }$, with the zeroth index indicating a “contaminant” process. Combining these response processes, we can define the marginal probability of each response as:

$$ p_{i} = \frac{1}{21} \phi_{0} + p_{i,5} \phi_{1} + p_{i,10} \phi_{2} $$

And then responses themselves are distributed Categorical:

$$ y_{i} \sim Categorical(\overrightarrow{p}) $$

For the noise-based model, B is the incomplete Beta function, the CDF of the Beta distribution. The computations for the Bayesian Sampler model response probabilities are identical save that instead of B(x,α,β) we have $B(f_{BS}^{-1}(x), \alpha , \beta )$ when computing the probability of each response p_i, and we use N and $N^{\prime }$ where appropriate. To see this, let X be the Beta distribution success proportions from the mental sampling operations, ρ(A), and let Y be the distribution of resulting probabilities from the Bayesian sampler model. Then Y = g(X) where g is the function defined in equation 13 from the manuscript.

$$ \hat{P}_{BS}(A) = \frac{\rho(A)N}{N+2\beta} + \frac{\beta}{N+2\beta} $$

(15)

Letting F_X and F_Y be the CDF of X and Y respectively, we have that:

$$ \begin{array}{@{}rcl@{}} F_{Y}(y)= P(Y\leq y) &=& P(g(X)\leq y) \\ &=& P(X\leq g^{-1}(y)) = F_{X}(g^{-1}(y)) \end{array} $$

Putting this all together, define Z_NB as the function which calculates the probability of each categorical response under the noise-based model given the inputs of $\mu _{ijk}, d_{j}, d^{\prime }_{j}$ and ϕ. Here, $\mu _{ijk} = f_{\text {NB}}(\overrightarrow {\theta _{jk}}, d_{j}, d^{\prime }_{j}, x_{ijk})$ computes the expected probability according to the PT+N theory except that it treats conditional probability judgments like simple probability judgments.

Finally, define Z_BS as the function which calculate the probability of each categorical response under the Bayesian Sampler model. Note that, here, $\mu _{ijk} = f_{0}(\overrightarrow {\theta _{jk}}, x_{ijk})$ where the value of μ depends only on the underlying probabilities and query asked on a specific trial.