Procedures
We wanted to examine the (possibly differential) impact of luck on attributions of knowledge-how and knowledge-that, as well as on attributions of understanding-why and knowledge-that. To this end, we utilized a 1 × 2 study design in four separate pairs of vignettes. Two of the pairs concerned knowledge-how and knowledge-that in lucky and non-lucky conditions. And two of the pairs concerned understanding-why and knowledge-that in lucky and non-lucky conditions. In all of the pairs we were interested in the potential impact of luck regarding knowledge-how and knowledge-that, or regarding understanding-why and knowledge-that.
We were also interested in measuring the judgments of both philosophers and non-philosophers. So we recruited non-philosophers through Mechanical Turk, and we recruited philosophers via on-line advertisement on various philosophy blogs. Surveys were completed on Mechanical Turk or Qualtrics. 582 participants saw one of eight vignettes. 134 participants were excluded for failing comprehension questions or for failing to complete the survey. Of the 448 remaining participants, 293 were male, and the average age was 35.3.
Vignettes
The vignettes given to participants sought to measure the impact of the presence of epistemic luck on ascriptions of knowledge-how, knowledge-that, and understanding-why. In particular, we were interested in:
- (i)
whether the presence of epistemic luck impacts ascriptions of knowledge-that and knowledge-how equally, or not,
- (ii)
whether the presence epistemic luck impacts ascriptions of knowledge-that and understanding-why equally, or not.
In order to test for this, we designed four pairs of vignettes that varied the presence of epistemic luck. In two of the pairs, participants were asked to respond to statements that the lucky/non-lucky individual possessed knowledge-how and (next) knowledge-that. Possible responses fell on a scale, with 1 representing the strongest disagreement, 4 neither agreement nor disagreement, and 7 the strongest agreement. Here is an example of one of the case-pairs we usedFootnote 19:
[Knowledge-How: Luck] Charlie needs to learn how to change a lightbulb, and so he goes to the ‘how-to’ section in his local library. He finds a shelf full of identical looking books titled Home Repair. In each of these books are step-by-step instructions on the way to change a lightbulb—we’ll call the way the book describes way ‘w’. Unbeknownst to Charlie, all the copies of Home Repair on the shelf are fakes, except for one. Pranksters have placed these copies there, and these fake copies contain mistaken step-by-step instructions on the way to change a lightbulb. Since Charlie does not know this, he reaches up and grabs the copy of Home Repair nearest to him. By sheer luck, he selects the only copy in the entire library that contains genuine and reliable step-by-step instructions for changing a lightbulb, and he reads the correct step-by-step instructions on the way to change a lightbulb. Had Charlie picked up any of the other guides—which he so easily could have—he would have believed the mistaken instructions were correct.
[Knowledge-How: Control] Charlie needs to learn how to change a lightbulb, and so he goes to the ‘how-to’ section in his local library. He finds a shelf full of identical looking books titled Home Repair. In each of these books are step-by-step instructions on the way to change a lightbulb—we’ll call the way the book describes way ‘w’. Any of the copies of Home Repair on the shelf will give Charlie genuine and reliable step-by-step instructions. Charlie reaches up and grabs the copy of Home Repair nearest to him. So Charlie reads the correct step-by-step instructions on the way to change a lightbulb.
In two other pairs of cases, participants were asked respond to statements that the lucky/non-lucky individual possessed understanding-why and (next) knowledge-that. Again, Possible responses fell on a scale, with 1 representing the strongest disagreement, 4 neither agreement nor disagreement, and 7 the strongest agreement (see appendix for vignettes and questions). Here is an example of one of the case-pairs used:
[Understanding-Why: Luck] Kate is a scientist who seeks an explanation for why a certain chemical reaction occurred. She ordinarily uses a particular kind of instrument to conduct the relevant test, and there are several such instruments, indistinguishable from one another, on the table before her. Unbeknownst to Kate, however, all of the instruments in front of her have been tampered with, except one. If she uses any of the instruments that have been tampered with, they will tell her that it was the presence of hydrogen, rather than oxygen, that caused the reaction. Kate selects the instrument nearest to her, conducts the test, and as a result comes to believe that it was the presence of oxygen that caused the chemical reaction. As it turns out, by sheer luck, the instrument Kate selects is the only reliable instrument on the table, and it gives Kate the correct result. Had Kate picked up any of the other instruments—which she so easily could have—she would have believed falsely what caused the reaction.
[Understanding-Why: Control] Kate is a scientist who seeks an explanation for why a certain chemical reaction occurred. She ordinarily uses a particular kind of instrument to conduct the relevant test, and there are several such instruments, indistinguishable from one another, on the table before her. Further, all of the instruments on the table are reliable, such that conducting her test with any one of them will generate the same correct result. Kate selects the instrument nearest to her, conducts the test, and as a result comes to believe that it was the presence of oxygen that caused the chemical reaction. Since the instrument is reliable, it gives Kate the correct result.
We were also interested in the possibility that level of philosophical education or expertise might impact participant responses. We asked participants to rate their own expertise in epistemology on a 1–3 scale, with 1 representing ‘low degree of expertise in epistemology,’ 2 representing ‘moderate degree of expertise,’ and 3 indicating ‘high degree of expertise.’ We also asked participants their level of philosophical education on a 1–6 scale: no education in philosophy, some undergraduate courses, undergraduate major, some graduate courses, master’s degree in Philosophy, Ph.D. in Philosophy. Finally, we gathered information on gender, age, and nationality.
Results
We first tested for the impact of epistemic luck on ascriptions of knowledge or understanding in all four pairs of vignettes individually.
For the first knowledge-how/knowledge-that vignette (i.e., the one involving Charlie), a 1 × 2 ANOVA revealed a significant difference on ascriptions of knowledge-how (F(1, 114) = 7.242, p = .008, partial η2 = .060), but not on ascriptions of knowledge-that (F(1, 114) = 2.927, p = .09, partial η2 = .025). Interestingly, ascriptions of knowledge-how were significantly higher in the lucky condition (M = 5.80, SD = 1.55 vs M = 4.93, SD = 1.75). Ascriptions of knowledge-that were both well above the midline (M = 5.34, SD = 1.68 in the lucky condition, and M = 5.83, SD = 1.36 in the no luck condition).
For the second knowledge-how/knowledge-that vignette (involving Kenneth: see appendix), a one-way ANOVA revealed a significant difference on ascriptions of knowledge-how (F(1, 113) = 4.182, p = .043, partial η2 = .036), but not on ascriptions of knowledge-that (F(1, 113) = 1.717, p = .19, partial η2 = .017). As with the first knowledge-how vignette, ascriptions of knowledge-how were significantly higher in the lucky condition (M = 5.16, SD = 1.87 vs M = 4.41, SD = 2.01). Again, as with the first vignette, ascriptions of knowledge-that were both well above the midline (M = 5.47, SD = 1.65 in the lucky condition, and M = 5.88, SD = 1.61 in the no-luck condition).Footnote 20
For the first understand-why/knowledge-that vignette (i.e., the one involving Kate), a one-way ANOVA revealed that the luck condition made no difference for ascriptions of understanding-why (F(1, 115) = .107, p = .744, partial η2 = .001) or knowledge-that (F(1, 115) = 1.591, p = .210, partial η2 = .014). Ascriptions of understanding-why were high in luck and no luck conditions (M = 5.66, SD = 1.48 and M = 5.57, SD = 1.47 respectively). The same was true of ascriptions of knowledge-that (M = 5.28, SD = 1.78 and M = 5.68, SD = 1.58 respectively).
For the second understand-why/knowledge-that vignette (involving Fiona: see appendix), a one-way ANOVA revealed that the luck condition made no difference for ascriptions of understanding-why (F(1, 104 = .273, p = .603, partial η2 = .003) or knowledge-that (F(1, 104) = 1.112, p = .294, partial η2 = .011). Ascriptions of understanding-why were high in luck and no luck conditions (M = 5.26, SD = 1.65 and M = 5.42, SD = 1.41 respectively). The same was true of ascriptions of knowledge-that (M = 5.52, SD = 1.57 and M = 5.84, SD = 1.34 respectively).Footnote 21
If we pay no attention to differences in philosophical training, then, the results are clear and surprising. Ascriptions of knowledge-that and understanding-why are not sensitive to epistemic luck. Or, at the least, the null effect reported is consistent with, and seems to suggest, a lack of sensitivity.Footnote 22 In contrast with the null result regarding knowledge-that and understanding-why, ascriptions of knowledge-how are sensitive to epistemic luck, but in an odd way: knowledge-how ascriptions are higher in the lucky condition.
However, we were also interested in the impact of philosophical training. In order to measure this, we performed a few different kinds of tests. First, we split participants into two groups depending on their level of training. Our ‘low-training’ group had taken graduate courses or less. Our ‘high-training’ group had received a master’s degree or a Ph.D. in Philosophy. Admittedly, one might operationalize ‘low’ and ‘high’ training in different ways. We selected this way based upon the thought that completing a graduate degree in philosophy is a significant achievement, requiring a number of high-level courses. Given the wording of our question, the next level down was consistent with having taken one graduate course.
Next, we performed a 2 × 2 (Luck: Luck vs. No luck x Training: Low vs. High training) ANOVA for both knowledge-how and knowledge-that ascriptions in both pairs of vignettes (collapsed given the similar means for both pairs). We found a significant difference for Luck on ascriptions of knowledge-how (F(1, 228) = 9.72, p = .002, partial η2 = .042) as well as knowledge-that (F(1, 228) = 11.262, p = .001, partial η2 = .048). We also found a significant difference of Training on ascriptions of knowledge-how (F(1, 228) = 3.912, p = .049, partial η2 = .017) as well as knowledge-that (F(1, 228) = 8.800, p = .003, partial η2 = .038). Most importantly, while we found no interaction between Luck and Training for knowledge-how ascriptions (F(1, 228) = .452, p = .502, partial η2 = .002), we did find a significant interaction between Luck and Training for knowledge-that ascriptions (F(1, 228) = 9.409, p = .002, partial η2 = .040).
In the luck condition, High-training participants agreed at higher rates to the knowledge-that statement than did Low-training participantsFootnote 23 (M = 4.40, SD = 1.85 vs. M = 5.78, SD = 1.42 respectively); both groups agreed at roughly the same level in the no luck condition (M = 5.85, SD = 1.56 vs. M = 5.87, SD = 1.30 respectively). Regarding knowledge-how, High-training participants agreed at slightly lower levels than did Low-training participants in the lucky condition (M = 5.20, SD = 1.87 vs. M = 5.56, SD = 1.70 respectively) as well as in the no luck condition (M = 4.16, SD = 1.85 vs. M = 4.89, SD = 1.87 respectively). As is evident, both groups of participants agreed at lower levels regarding knowledge-how in the no luck condition.
We also performed a 2 × 2 (Luck: Luck vs. No luck x Training: Low vs. High training) ANOVA for both understanding-why and knowledge-that ascriptions in both pairs of vignettes (collapsed given the similar means for both pairs). We found no significant difference for Luck on ascriptions of understanding-why (F(1, 219) = .037, p = .847, partial η2 < .001); both groups agreed at roughly the same level (M = 5.51, SD = 1.55 in the Luck condition, M = 5.49, SD = 1.44 in No Luck). But we did find a significant difference for Luck on ascriptions of knowledge-that (F(1, 219) = 4.327, p = .039, partial η2 = .020). We found no significant difference of Training on ascriptions of understanding-why (F(1, 219) = 1.741, p = .188, partial η2 = .008); High Training participants agreed at the same level in Luck and No Luck conditions (M = 5.35 vs. M = 5.24 respectively), as did Low Training participants (M = 5.58 vs. M = 5.60 respectively). But we did find a significant difference of Training on ascriptions of knowledge-that (F(1, 219) = 8.300, p = .004, partial η2 = .037). In this case, there was no significant interaction for ascriptions of understanding-why (F(1, 219) = .070, p = .791, partial η2 < .001) or knowledge-that (F(1, 219) = 1.436, p = .232, partial η2 = .007). Overall, High-training participants agreed at lower levels regarding knowledge-that in both conditions (in the luck condition, M = 4.73, SD = 1.97 for High-training, and M = 5.67, SD = 1.48 for Low-training; in the no luck condition, M = 5.49, SD = 1.55 for High-training, and M = 5.88, SD = 1.41 for Low-training).
Next, we sought to examine the impact of self-reported expertise in epistemology on participant responses. We performed 2 × 3 ANOVA (Luck: Luck vs. No luck x Epistemology: Low vs. Moderate vs. High) for both knowledge-how and knowledge-that ascriptions in both pairs of vignettes (collapsed given the similar means for both pairs). We found a main effect for Luck on both knowledge-how ascriptions (F(2, 228) = 4.178, p = .042, partial η2 = .019) and knowledge-that ascriptions (F(2, 228) = 14.152, p < .001, partial η2 = .060). We found no effect for Epistemology on knowledge-how ascriptions (F(2, 228) = 2.903, p = .057, partial η2 = .026) or knowledge-that ascriptions (F(2, 228) = .298, p = .743, partial η2 = .003). There was a significant interaction for knowledge-that ascriptions (F(2, 228) = 5.641, p = .004, partial η2 = .049), and none for knowledge-how ascriptions (F(2, 228) = .109, p = .897, partial η2 = .001).
Regarding knowledge-how ascriptions: there was a general trend towards agreeing at lower levels regarding knowledge-how in the non-lucky condition. The means for low expertise participants (N = 155) were 5.65 (SD = 1.65) in the lucky condition, and 4.82 (SD = 1.93) in the non-lucky condition. The means for moderate expertise participants (N = 48) were 4.85 (SD = 2.06) in the lucky condition, and 4.14 (SD = 1.78) in the non-lucky condition. The means for high expertise participants (N = 24) were 5.33 (SD = 1.58) in the lucky condition, and 4.87 (SD = 1.73) in the non-lucky condition.
Regarding knowledge-that ascriptions: Low expertise participants (N = 155) agreed at the same level regarding knowledge in luck and no luck conditions (M = 5.68, SD = 1.53 vs. M = 5.67, SD = 1.62); Moderate expertise participants (N = 48) agreed at higher levels regarding knowledge in the no luck condition (M = 6.11, SD = 1.23 vs. M = 4.90, SD = 1.65); High expertise participants (N = 24) agreed at a higher level in the no luck condition (M = 6.47, SD = .64 vs. M = 4.56, SD = 2.19).
We then performed a 2 × 3 (Luck: Luck vs. No luck x Epistemology: Low vs. Moderate vs. High) ANOVA for both understanding-why and knowledge-that ascriptions in both pairs of vignettes (collapsed given the similar means for both pairs). We found no main effect for Luck on understanding-why ascriptions (F(2, 218) = .302, p = .583, partial η2 = .001), and a significant main effect for Luck on knowledge-that ascriptions (F(2, 218) = 6.019, p = .015, partial η2 = .028). We found significant main effects for Epistemology on understanding-why ascriptions (F(2, 218) = 4.225, p = .016, partial η2 = .038) as well as knowledge-that ascriptions (F(2, 218) = 10.009, p < .001, partial η2 = .086). We also found a significant interaction for both understanding-why ascriptions (F(2, 218) = 5.403, p = .005, partial η2 = .049), and knowledge-that ascriptions (F(2, 218) = 4.285, p = .015, partial η2 = .039).
Regarding understanding-why ascriptions: Low expertise participants (N = 133) agreed at the same level regarding understanding in luck and no luck conditions (M = 5.67, SD = 1.44 vs. M = 5.80, SD = 1.35); Moderate expertise participants (N = 61) agreed at a lower level in the no luck condition (M = 4.65, SD = 1.49 vs. M = 5.58, SD = 1.44); High expertise participants (N = 24) agreed at a higher level in the no luck condition (M = 5.85, SD = .90 vs. M = 4.64, SD = 2.01).
Regarding knowledge-that ascriptions: Low expertise participants (N = 133) agreed at roughly the same level regarding knowledge in luck and no luck conditions (M = 5.74, SD = 1.48 vs. M = 6.03, SD = 1.25); Moderate expertise participants (N = 61) agreed at roughly the same level regarding knowledge in luck and no luck conditions (M = 5.50, SD = 1.53 vs. M = 5.24, SD = 1.62); High expertise participants (N = 24) agreed at a higher level regarding knowledge in the no luck condition (M = 5.38, SD = 1.89 vs. M = 3.55, SD = 1.86) (Figs. 1, 2, 3, 4, 5, 6, 7 and 8).