Experimenter Philosophy: the Problem of Experimenter Bias in Experimental Philosophy
- 666 Downloads
It has long been known that scientists have a tendency to conduct experiments in a way that brings about the expected outcome. Here, we provide the first direct demonstration of this type of experimenter bias in experimental philosophy. Opposed to previously discovered types of experimenter bias mediated by face-to-face interactions between experimenters and participants, here we show that experimenters also have a tendency to create stimuli in a way that brings about expected outcomes. We randomly assigned undergraduate experimenters to receive two different hypotheses about folk intuitions of consciousness, and then asked them to design experiments based on their hypothesis. Specifically, experimenters generated sentences ascribing intentional and phenomenal mental states to groups, which were later rated by online participants for naturalness. We found a significant interaction between experimenter hypothesis and participant ratings indicating a general tendency for experimenters to obtain the result that they expected. These results indicate that experimenter bias is a real problem in experimental philosophy since the methods and design employed here mirror the predominant survey methods of the field as a whole. The bearing of the current results on Knobe and Prinz’s (Phenomenology and Cognitive Science 7(1):67–83, 2008) group mind hypothesis is discussed, and new methods for avoiding experimenter bias are proposed.
KeywordsMental State Sentence Type Experimenter Bias Feeling Condition Experimental Philosophy
We would like to thank Joshua Knobe, Chaz Firestone, Alex Shaw, Brandon Liverence, Hugo Mercier and Brian Scholl for useful comments and feedback. We would also like to thank Julie Van Dyke and Laurie Santos for help finding experimenters. Finally, we thank the Yale undergraduate experimenters for designing the experiments.
- Arico, A. 2007. Deregulating corporate consciousness: A critique of Knobe and Prinz’s intuitions about consciousness. Toronto: Poster presented at the Society for Philosophy and Psychology.Google Scholar
- Feltz, A., Harris, M., and Perez, A. (2012). Perspective in intentional action attribution. Philosophical Psychology.Google Scholar
- Ioannidis, J.P.A. 2005. Why most published research findings are false. Public Library of Science, Medicine 2: e124.Google Scholar
- Khemlani, S.S., A.B. Sussman, and D.M. Oppenheimer. 2011. Harry Potter and the sorcerer’s scope: latent scope biases in explanatory reasoning. Memory & Cognition 39(1).Google Scholar
- Knobe, J., Buckwalter, W., Nichols, S., Robbins, P., Sarkissian, H., and Sommers, T. (2011). Experimental philosophy. Annual Review of Psychology, 63.Google Scholar
- Phelan M., Arico A., and Nichols S. (2012). Thinking things and feeling things: On an alleged discontinuity in folk metaphysics of mind.Google Scholar
- Rosenthal, R., and I. Jacobson. 1968. Pygmalion in the classroom. New York: Holt, Rinehart and Winston.Google Scholar
- Scholl, B.J. 2008. Two kinds of experimental philosophy, and their methodological dangers. Talk given at the SPP Workshop on Experimental Philosophy. Philadelphia: University of Pennsylvania.Google Scholar