In this study, our goal was to test four key hypotheses in a manner that is described in detail below. For all hypotheses, we use the Bayes factor to quantify the degree of confirmation provided by the data (Jeffreys 1961); we will also provide the posterior distribution for the parameters of interest.
The registered analysis plan was carried out on the complete data set (subject to the outcome-blind decisions by the analyst; see the next section on the two-stage analysis process). In a second, exploratory analysis, we will test the hypotheses separately for the relatively homogeneous student group and the relatively heterogeneous non-student group.
Note that, with 1000 participants, we collected data that are sufficiently informative to pass Berkson’s “interocular traumatic test” (Edwards et al. 1963) such that the confirmatory hypothesis tests serve merely to corroborate what is immediate apparent from a cursory visual inspection of the data. Below, we provide a description of the hypotheses and analyses that is consistent with the original preregistration plan; as will become apparent later, the analyst executed some outcome-independent changes to this original plan.
Planned analysis of hypothesis 1: Worst performance rule
For each participant, we obtained a single WMC score from the WMC battery. Furthermore, for each participant we obtained the 1/6, 2/6, 3/6, 4/6, and 5/6 quantiles of correct RTs; it is possible to use more quantiles, but only at the cost of reducing the precision with which the mean RT within each bin is estimated. Hypothesis 1 states that the correlation between WMC and mean RT within each quantile is negative (i.e., higher WMC is associated with faster responding). More specifically, Hypothesis 1 states that the absolute magnitude of this correlation increases monotonically from the fastest to the slowest quantile (i.e., the WPR). Hypothesis 1a refers to the WPR for easy stimuli, and Hypothesis 1b refers to the WPR for difficult stimuli.
Both Hypothesis 1a and 1b are tested separately, in the following manner. Denote by ρ
i
the estimated Pearson correlation coefficient for quantile i. Then, the simplest linear version of the WPR predicts that ρ
i
= β
0 + β
1
I
i
, where I
i
indicates the quantile, β
0 is the intercept of the regression equation, and β
1 is the slope. We then use the Bayes factor (Jeffreys 1961; Kass and Raftery 1995) to quantify the support that the data provide for two competing hypotheses: the null hypothesis \(\phantom {\dot {i}\!}\mathcal {H}_{0}: \beta _{1} = 0\) versus the WPR alternative hypothesis \(\phantom {\dot {i}\!}\mathcal {H}_{1}: \beta _{1} < 0\). Under \(\phantom {\dot {i}\!}\mathcal {H}_{1}\), we assign each ρ
i
an independent uniform prior from −1 to 0, in order to respect the fact that all correlations are predicted to be negative. Furthermore, we assign a uniform prior to β
0 that ranges from −1 to 0, in order to respect the fact that even for the fastest RTs, the correlation is not expected to be positive. Finally, we assign a uniform prior to β
1 that ranges from its steepest possible value to 0. Specifically, since the quantiles are on the scale from zero to one, and the highest possible value of the intercept β
0 equals 0, the assumption of linearity across the scale implies that the steepest slope is −1. Hence, we assign β
1 a uniform prior from −1 to 0 (see the results section for an inconsistency in this model specification).
With the model specification in place, the Bayes factor between \(\phantom {\dot {i}\!}\mathcal {H}_{0}\!\!\!:\!\!\!\! \beta _{1}\!\!\!\!\!\! =\!\!\!\!\! 0\) versus \(\phantom {\dot {i}\!}\mathcal {H}_{1}\!\!\!\!: \beta _{1}\!\!\!\!\ \sim \!U[-1,0]\) can be obtained using an identity known as the Savage–Dickey density ratio (e.g., Dickey & Lientz, 1970; Wagenmakers, Lodewyckx, Kuriyal, & Grasman, 2010). Specifically, this involves focusing on parameter β
1 in \(\phantom {\dot {i}\!}\mathcal {H}_{1}\) and comparing the prior ordinate at β
1=0 to the posterior ordinate at β
1=0, that is, by computing \(\phantom {\dot {i}\!}\text {BF}_{10} = p(\beta _{1}=0 \mid \mathcal {H}_{1})/p(\beta _{1}=0 \mid y, \mathcal {H}_{1})\), where y denotes the observed data. Bayes factors higher than 1 favor \(\phantom {\dot {i}\!}\mathcal {H}_{1}\) and provide support for the WPR. All parameters will be estimated simultaneously using a hierarchical Bayesian framework and Markov chain Monte Carlo (MCMC, e.g., Lee & Wagenmakers, 2013).
Planned analysis of hypothesis 2: Stronger worst performance rule for more difficult stimuli
The WPR tested under Hypothesis 1 is predicted to be more pronounced for difficult stimuli than for easy stimuli. In the previous WPR model, ρ
i
= β
0 + β
1
I
i
; now denote β
1 for the difficult stimuli by β
1d
and denote β
1 for the easy stimuli by β
1e
. Hypothesis 2 holds that β
1e
>β
1d
. We multiply both parameters by −1 so that we obtain variables on the probability scale, and hence \(\phantom {\dot {i}\!}\beta ^{*}_{1d}>\beta ^{*}_{1e}\). We use a dependent prior structure (Howard 1998), apply a probit transformation, and orthogonalize the parameter space (Kass and Vaidyanathan 1992). Specifically, denoting the probit transformation by Φ−1, we write \(\phantom {\dot {i}\!}{\Phi }^{-1}(\beta ^{*}_{1d}) = \mu + \delta /2\) and \(\phantom {\dot {i}\!}{\Phi }^{-1}(\beta ^{*}_{1e}) = \mu - \delta /2\). We assign the probitized grand mean parameter μ an uninformative distribution, that is, μ∼N(0,1), and then use the Bayes factor to contrast two models: the null hypothesis \(\phantom {\dot {i}\!}\mathcal {H}_{0}: \delta =0\) versus the alternative hypothesis \(\phantom {\dot {i}\!}\mathcal {H}_{2}: \delta > 0\). We complete the model specification for \(\phantom {\dot {i}\!}\mathcal {H}_{2}\) by assigning the difference parameter δ a default folded normal prior defined only for positive values, that is, δ∼N(0,1)+. As before, parameter estimates are obtained from MCMC sampling in a hierarchical Bayesian model and Bayes factors will be computed using the Savage–Dickey density ratio test on parameter δ under \(\phantom {\dot {i}\!}\mathcal {H}_{2}\).
Planned analysis of hypothesis 3: Working memory capacity correlates positively with drift rate
We fit the diffusion model to the data using hierarchical Bayesian estimation (e.g., Wabersich & Vandekerckhove, 2014; Wiecki, Sofer, & Frank, 2013). This hierarchical method allows us to exploit the vast number of participants and estimate parameters even for participants whose data contain little information (for example due to a small number of errors, which are crucial for diffusion model parameter estimation). Hypothesis 3 holds that WMC correlates positively with drift rate. Hypothesis 3a refers to the positive correlation between WMC and drift rate for the easy stimuli, and Hypothesis 3b refers to the positive correlation between WMC and drift rate for the difficult stimuli. Both Hypothesis 3a and 3b will be tested separately, in the following manner.
First WMC is included within the hierarchical structure. WMC will then be correlated with drift rate estimates (Hypothesis 3a: for the easy stimuli; Hypothesis 3b: for the difficult stimuli) in a hierarchical structure. The null hypothesis holds that there is no correlation, \(\phantom {\dot {i}\!}\mathcal {H}_{0}: \rho = 0\), whereas the alternative hypothesis holds that the correlation is positive, \(\phantom {\dot {i}\!}\mathcal {H}_{3}: \rho > 0\). Specifically, we assign ρ a uniform prior from 0 to 1. Bayes factors can be obtained by a Savage–Dickey density ratio test on parameter ρ under \(\phantom {\dot {i}\!}\mathcal {H}_{3}\).
Planned analysis of hypothesis 4: Stronger correlation between working memory and drift rate for more difficult stimuli
Hypothesis 4 holds that WMC correlates more strongly with drift rates for difficult stimuli than with drift rates for easy stimuli. Denote by ρ
d
the WMC-drift rate correlation for the difficult stimuli, and by ρ
e
the WMC-drift rate correlation for the easy stimuli. Hypothesis 4 states that ρ
d
>ρ
e
. Moreover, both ρ
d
and ρ
e
are assumed to be positive, so that both are on the probability scale. Consequently, the proposed analysis mimics that of Hypothesis 2: We use a dependent prior structure, apply a probit transformation, and orthogonalize the parameter space. We write Φ−1(ρ
d
) = μ + δ/2 and Φ−1(ρ
e
) = μ−δ/2. We assign the probitized grand mean parameter μ an uninformative distribution, that is, μ∼N(0,1), and then use the Bayes factor to contrast two models: the null hypothesis \(\phantom {\dot {i}\!}\mathcal {H}_{0}: \delta =0\) versus the alternative hypothesis \(\phantom {\dot {i}\!}\mathcal {H}_{4}: \delta > 0\). We complete the model specification for \(\phantom {\dot {i}\!}\mathcal {H}_{4}\) by assigning the difference parameter δ a default folded normal prior defined only for positive values, that is, δ∼N(0,1)+. As before, parameter estimates are obtained from MCMC sampling in a hierarchical Bayesian model and Bayes factors will be computed using the Savage–Dickey density ratio test on parameter δ under \(\phantom {\dot {i}\!}\mathcal {H}_{4}\).