Experimental Philosophy of Free Will and the Comprehension of Determinism

Abstract

The experimental validity of research in the experimental philosophy of free will has been called into question. Several new, important studies (Murray et al. forthcoming; Nadelhoffer et al., Cognitive Science 44 (8): 1–28, 2020; Nadelhoffer et al., 2021; Rose et al., Cognitive Science 41 (2): 482–502, 2017) are interpreted as showing that the vignette-judgment model is defective because participants only exhibit a surface-level comprehension and not the deeper comprehension the model requires. Participants, it is argued, commit bypassing, intrusion, and fatalism errors. We respond in two ways: (1) we critique and improve existing methods for assessing deeper comprehension and (2) we develop videos to convey deterministic principles of change that succeed in significantly reducing participants’ bypassing, intrusion, and fatalism errors. Consequently, we have the best existing instrument for gauging folk intuitions about the relationship between free will and determinism.

Appendices

Appendix 1

For Study 3, a factor analysis with oblimin rotation was conducted on the 7-items assessing comprehension errors (intrusion, bypassing, fatalism) with the speculation of three underlying factors. The first factor accounted for 33.55% of the variance (Eigenvalue = 2.35), with the factor loading onto the two intrusion items (0.84, 0.95). The second factor accounted for an additional 26.07% of the variance (Eigenvalue = 1.83), with the factor loading onto Bypassing 1 and 2 (0.86, 0.79, respectively). The final factor accounted for an additional 13.83% of the variance, but had a low Eigenvalue of 0.97. The final factor loaded onto Fatalism-1 at 0.81, but weakly for Fatalism-2 at 0.37. Bypassing-3 did not load onto any factor above 0.18. We removed Bypassing-3 because it did not load onto any factor above 0.40. Similarly, we removed Fatalism-2 due to not loading above 0.40 onto any factor, having a negative zero-order correlation with Fatalism-1, and producing poor internal consistency.

Appendix 2

A better way to test for fatalism would be to explicitly include information about the fixity (or non-fixity) of the past. Interestingly, in MDN’s 2023 study, such a statement (Fatalism-4_ORIG) was included among the statements used to test for fatalism mistakes but is not included in their forthcoming study. Here are the four statements that were used in conjunction with Universe A/B:

Fatalism_ORIG.

• 1. In Universe A, there is no sense in which events could have unfolded differently than they did.

• 2. In Universe A, John would have ended up having French Fries no matter what he tried to do.

• 3. In Universe A, John will eat French Fries no matter what.

• 4. In Universe A, John’s eating French Fries had to happen, even if what happened in the past had been different.

MDN decided to keep Fatalism-2_ORIG and Fatalism-3_ORIG in their forthcoming study because of their strong correlation.

They report a combined score for all four statements, but, without examining their supplemental materials, it is impossible to see if the participants responded differently to the four different ways fatalism is presented. Most notably for our purposes, Fatalism-4_ORIG, unlike the other three statements, includes information about the past (namely that it is not fixed). Here is a summary of the means, standard deviations, and error rates for each individual Fatalism_ORIG statement from MDN’s 2023 study Table 8.

Table 8 Means, Standard Deviations, and Error Rates for Fatalism_ORIG statements from MDN’s 2023 study

Notice that there is less agreement with Fatalism-4_ORIG compared with the first three Fatalism_ORIG statements and that the difference in means is statistically significant (p < 0.01). This suggests that participants may answer differently when given explicit information about the fixity of the past.

While MDN performed an exploratory factor analysis on all four fatalism items, they combined the fatalism items across the Universe A/B and Supercomputer conditions. This assumes that the scale is assessing fatalism across both conditions, which was not verified. We performed an exploratory factor analysis on the four fatalism items with a varimax rotation, similar to MDN, however, we performed this EFA across each condition. For the Universe A/B condition, one factor was extracted from the data (Eigenvalue = 2.12) with 54.55% of the variance being explained. Results showed that Fatalism-1_ORIG (0.72), Fatalism-2_ORIG (0.69), and Fatalism-3_ORIG (0.89) loaded on this factor above the threshold of 0.40 for retaining items for a scale (Furr 2011). Fatalism-4_ORIG had a loading of 0.07, which did not load above the threshold. Further, a Cronbach’s alpha showed poor reliability for the four item scale (ɑ = 0.600) but good reliability for the three-item scale that omitted Fatalism-4_ORIG (ɑ = 0.804). The given results suggest that Fatalism-4_ORIG should be dropped when aggregating the items together.

We conducted an additional EFA with varimax rotation on the four items assessing fatalism in MDN’s 2022 study for participants who read Supercomputer. One factor was extracted from the data (Eigenvalue = 2.83) with 70.72% of the variance being explained. All four of the fatalism items loaded onto the factor above the threshold of 0.40 (0.57, 0.86, 0.89, and 0.79, respectively). A Cronbach’s alpha analysis also found good internal consistency for the four-item measure of fatalism (ɑ = 0.858). The given results suggest that aggregating the fatalism items into a single composite variable was appropriate for this data set.

Though our EFAs give us mixed results, the EFA for the Fatalism_ORIG statements under the Universe A/B condition suggests that Fatalism-4_ORIG should not be aggregated with the other three. This is reason to suspect that information about the fixity of the past can make a difference.

Appendix 3

We conducted a mediation analysis using PROCESS, a macro in SPSS that allows for multi-categorical independent variables. The independent variable was the manipulation, with Video set as the control condition and contrasted against each other vignette (Universe A/B, Supercomputer, Hybrid). The three mediators were Bypassing-combined, Intrusion-combined, and Fatalism-1_NEW. Bypassing-combined consisted of Bypassing-1_NEW and Bypassing-2_NEW (r_s = 0.640). The reverse-coded Bypassing-3_NEW was not included due to producing poor internal consistency in the measure. Intrusion-combined consisted of Intrusion-1_NEW and Intrusion-2_NEW (r_s = 0.865). Furthermore, Fatalism-2_NEW was not used because, as noted above, Fatalism-1_NEW and Fatalism-2_NEW are conceptually and statistically different and Fatalism-2_NEW does not include information about the fixity of the past. We conducted separate mediation analyses on free will and moral responsibility scores.

Results of the mediation analysis on free will scores showed that Bypassing-combined and Fatalism-1_NEW scores mediated the effect of Video (vs. Universe A/B) on free will scores; specifically, Video indirectly increased free will scores compared to Universe A/B due to lowering bypassing scores. Yet, simultaneously, Video indirectly lowered free will scores compared to Universe A/B due to lowering Fatalism-1_NEW scores. The mediation analysis also showed that Intrusion-combined and Fatalism-1_NEW scores mediated the effect of Video (vs. Supercomputer) on free will scores. Specifically, Video lowered free will scores compared to Supercomputer due to lowering intrusion and fatalism scores. These findings suggest that manipulations that change deeper comprehension errors will indirectly affect people’s attributions of free will Figures 7 and 8.

Fig. 7

Mediation Analysis of the Determinism Manipulation on Free Will Through Comprehension Items. Note: Unstandardized regression coefficients are presented in Fig. 1. *p < 0.05. **p < 0.01. ***p < 0.001

Results of the mediation analysis on moral responsibility scores showed a similar patterns of results. First, the mediation analysis on moral responsibility scores showed that Bypassing-combined scores mediated the effect of Video (vs. Universe A/B) on moral responsibility scores; specifically, Video indirectly increased moral responsibility scores compared to Universe A/B due to lowering bypassing scores. The mediation analysis also showed that Intrusion-combined scores mediated the effect of Video (vs. Supercomputer) on moral responsibility scores. Specifically, Video lowered moral responsibility scores compared to Supercomputer due to lowering intrusion scores. These findings suggest that, similar to above, manipulations that change deeper comprehension errors will indirectly affect people’s attributions of moral responsibility. See Table 9.

Fig. 8

Mediation Analysis of the Determinism Manipulation on Moral Responsibility Through Comprehension Items. Note: Unstandardized regression coefficients are presented in Fig. 1. *p < 0.05. **p < 0.01. ***p < 0.001

Table 9 Indirect Effects of the Determinism Manipulation on Free Will and Moral Responsibility Scores Through Comprehension Scores

These analyses help us understand why the free will and moral responsibility scores under Supercomputer and Universe A/B differ from the scores under Video. Participants under Supercomputer make more intrusion mistakes than those under Video. This means participants under Supercomputer have a greater tendency to think that Jeremy could have done otherwise. If participants tend to believe this then it’s natural for them to then think that Jeremy has free will and is morally responsible for his actions. This explains why free will (4.33) and moral responsibility (4.85) scores under Supercomputer are higher than the corresponding scores under Video. The scores under Video are more reliable as indicators of compatibilism or incompatibilism because Video, compared to Supercomputer, significantly reduces intrusion mistakes while maintaining deeper comprehension regarding bypassing and fatalism.

Participants under Universe A/B make more bypassing mistakes than those under Video. This means participants under Universe A/B tend to think that Jeremy’s mental states (e.g., his desires) make no difference to his subsequent behavior. If participants tend to think that Jeremy’s mental states are bypassed (and therefore irrelevant) in the production of his behavior then it seems natural for these participants to then think that Jeremy does not have free will and is not morally responsible for his actions. This explains why free will (3.16) and moral responsibility (3.88) scores under Universe A/B are lower than the corresponding scores under Video. The scores under Video are more reliable as indicators of compatibilism or incompatibilism because Video significantly reduces bypassing mistakes while maintaining deeper comprehension regarding fatalism and intrusion.

Interestingly, participants under Video make fewer mistakes on Fatalism-1_NEW than under Supercomputer or Universe A/B. Because this suggests that participants can properly distinguish determinism from fatalism one would think this leads to higher free will scores. The mediation analysis, however, shows that lowering scores on Fatalism-1_NEW indirectly lowers free will scores under Video. This is prima facie puzzling and may suggest directions for future research.