Abstract
One widely-endorsed argument in the experimental philosophy literature maintains that intuitive judgments are unreliable because they are influenced by the order in which thought experiments prompting those judgments are presented. Here, we explicitly state this argument from ordering effects and show that any plausible understanding of the argument leads to an untenable conclusion. First, we show that the normative principle is ambiguous. On one reading of the principle, the empirical observation is well-supported, but the normative principle is false. On the other reading, the empirical observation has only weak support, and the normative principle, if correct, would impugn the reliability of deliberative reasoning, testimony, memory, and perception, since judgments in all these areas are sensitive to ordering in the relevant sense. We then reflect on what goes wrong with the argument.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Throughout this paper, we will talk about intuitive judgments and the (propositional) contents of intuitive judgments. In our experience, the term “intuition” too easily slips back and forth between a mental state and the (propositional) content of a mental state. Readers who think that intuitions are non-propositional intellectual seemings, inclinations to make judgments, or something similar may make suitable substitutions in our arguments and in the arguments that we attribute to various experimental philosophers.
We are here treating “unreliable” as having normative content, in a sense similar to “untrustworthy.” Anyone bothered by this choice could rephrase the principle to indicate that judgments systematically affected by ordering should not be trusted or ought not to be relied on.
In their conclusion, Swain et al. write: “Specifically, we found that intuitions about the Truetemp Case vary depending on whether, and which, other cases are presented before it. Such variability calls into question the legitimacy of using the intuitions generated by the Truetemp Case as evidence against reliabilism. But it is unclear what about this case makes it susceptible to these effects, which raises questions about the reliance on intuitions about thought-experiments more generally, especially given that this is not the only case called into question by empirical research” (153, emphasis added). They go on to write: “We certainly do not take ourselves to have offered anything like a general proof of the unreliability of all intuitions (nor do we think that any such proof would be either possible or desirable). But we do take ourselves to have raised a serious empirical worry that philosophers need to begin deciding how to address.”
An anonymous referee worried that our choice of labels here is likely to introduce more confusion into the literature. In response, let us be very clear about our terminological choice. Consistent with standard usage in the field, we will use “order effect” to indicate any case where participants’ responses are systematically affected by the order in which stimuli are presented. The category of order effects then divides into two sub-types: the genuine or proper order effects and the updating effects. We retain the label “order effects” for one sub-type in part because that is consistent with the usage in psychology (for which see Hogarth and Einhorn 1992), in part because that sub-type is plausibly prima facie epistemically problematic in a way that the other species is not, and in part because we do not have any better, more evocative label to provide. The distinction is purely descriptive. There are two structurally different kinds of ordering effect. We label one of them “genuine ordering effects” and the other “updating effects.”
We are not assuming that judgments or cognitive processes that exhibit updating effects are reliable. What we are claiming is that the fact that a judgment or cognitive process exhibits an updating effect is not prima facie reason to think that judgment or cognitive process is not reliable. Put another way: If all one knows about a judgment is that it exhibits an updating effect, that doesn’t provide a reason to think that it is unreliable. It might be unreliable, but whether or not it is unreliable depends on much more than sensitivity to ordering. For example, perhaps a judgment is sensitive to an irrelevant feature. However, we think it is too hasty to conclude that prior cases are an irrelevant feature as we will discuss in the next few pages of the paper.
One point of interest is that if researchers have participants’ retrieve a source story from long term memory, and both the target and the source stories are particularly long, stories with superficial similarity are more likely to influence people’s judgments about the target story than stories that are structurally similar to each other. However, in the empirical studies of philosophical ordering effects, the thought experiments presented to people are both short and presented without delay between each thought experiment (see Hogarth and Einhorn’s meta-analysis for why length of vignette hinders ordering effects).
Hummel (personal communication) tells us: “There is no precise definition [of a schema]. Generally, it is a structured (i.e., explicitly relational; formally more powerful than a simple feature list) representation that summarizes the core/common properties of some domain of knowledge, especially the most important relations (including higher-order relations) governing that domain.” See Table 3 in Hummel and Holyoak (2003) for a short list of functional properties that schemas have.
See Horne et al. (2013) for a similar account.
However, at this point it is unknown whether people update their schemas explicitly or implicitly.
In typical cases, what we are calling the background belief will actually be a network of related beliefs.
The Control Case was the Coupon Case: the 9th non-moral dilemma from the supplementary materials of Greene et al. (2001).
In the Control Condition, the mean response was \(\hbox {M}=5.8, \hbox {SD}=1.2\).
In Condition 2, the mean response was \(\hbox {M}=5.1, \hbox {SD}= 1.56\). In Condition 3, the mean response was \(\hbox {M}=4.1, \hbox {SD}= 1.52\).
Here we note several other measures these researchers took to rule out alternative hypotheses for their results. In light of well-known difficulties with spontaneous recall of source problems (Gick and Holyoak 1983), we have every reason to believe these measures were effective. First, the scale participants responded on only had the endpoints labeled. Consequently, participants had very little information to go on about the degree of agreement they were assigning to their belief (e.g., 73 out of 100 agreement with the belief). For this reason, it would be extremely difficult for participants to recover the exact position of their response on the scale during the first phase of the study, especially because this belief was one of many controversial beliefs that participants had to consider. Finally, the moral dilemma used to manipulate people’s beliefs (i.e., the Transplant dilemma) was embedded among many other moral dilemmas, half of which were related to distractor beliefs noted above. In sum, the presence of belief distractors and dilemma distractors, as well as the scale Horne and colleagues used, make it much more likely that people merely reported their beliefs during phases 1 and 2 of the study, rather than remembering their rating in the first phase of the study and altering their agreement rating accordingly during the second phase of the study.
Whereas ordinary Bayesian conditionalization is given by the equation \(Cr_{New} (p)=Cr_{Old} (p|e)\), the simplest case of Jeffrey conditionalization is given by the equation \(Cr_{New} (p)=Cr_{Old} (p|e)\cdot Cr_{New} (e)+Cr_{Old} (p|\sim e)\cdot Cr_{New} (\sim e)\) to account for the fact that we might not be certain about our evidence e for the proposition p.
Although we will not focus on it here, the way that Lange describes the evidence also problematizes, we think, how to count the number of pieces of evidence one has. When a child sees her mother laugh and then scowl, does she see and experience a single thing (a laugh-to-scowl) or two things (a laugh and a scowl)? If she experiences two things, does she also experience temporal succession or is each event indexed by time? And if the time index matters to the nature of her experience, isn’t that just to sneak in the supposedly irrelevant information about the order of her pieces of evidence? We have no answers to these questions at present, but surely anyone interested in defending an order-invariance principle must answer them.
And that same response problematizes conclusions drawn from the psychological research we surveyed in Sect. 3 as well. Thanks to Conor Mayo-Wilson for pointing this out.
We are not here endorsing the view that differences in the judgments participants make are actually due to differences in how vividly they imagine the cases; our goal here is merely to give a how-possibly reconstruction. But as far as we know, the experimental evidence is consistent with such a view. For example, differences in vividness, as opposed to differences in personal contact, might be consistent with Schwitzgebel and Cushman ’s (2015) results respecting the Footbridge, Trapdoor, and Switch variants of the Trolley dilemma.
References
Allan, L. (1977). The time-order error in judgments of duration. Canadian Journal of Psychology, 31, 24–31.
Anderson, N. (1971). Integration theory and attitude change. Psychological Review, 78, 171–206.
Anderson, J., Bothell, D., Lebiere, C., & Matessa, M. (1998). An Integrated Theory of List Memory. Journal of Memory and Language, 38(4), 341–380.
Ashton, A., & Ashton, R. (1988). Sequential belief revision in auditing. The Accounting Review, 63, 623–641.
Bengson, J. (2013). Experimental Attacks on Intuitions and Answers. Philosophy and Phenomenological Research, 86(3), 495–532.
Cromwell, H. (1950). The relative effect on audience attitude of the first versus the second argumentative speech of a series. Speech Monographs, 17, 105–122.
Cullen, S. (2010). Survey-Driven Romanticism. Review of Philosophy and Psychology, 1(2), 275–296.
Davidson, D. (2001). “A Coherence Theory of Truth and Knowledge,” In Subjective and Intersubjective, Objective. Oxford: Clarendon Press.
Dean, M. (1980). Presentation order effects in product taste tests. The Journal of Psychology, 105, 107–110.
Döring, F. (1999). Why Bayesian psychology is incomplete. Philosophy of Science, 379-389.
Doumas, L. A., Hummel, J. E., & Sandhofer, C. M. (2008). A theory of the discovery and predication of relational concepts. Psychological Review, 115(1), 1–43.
Englund, M., & Hellström, Å. (2012). Presentation-order effects for aesthetic stimulus preference. Attention, Perception, & Psychophysics, 74, 1499–1511.
Eskine, K., Kacinik, N., & Prinz, J. (2011). A bad taste in the mouth: gustatory disgust influences moral judgment. Psychological Science, 22(3), 295–299.
Feigenbaum, E., & Simon, H. (1962). A theory of the serial position effect. British Journal of Psychology, 53(3), 307–320.
Feldman, R. (1988). Having Evidence. In D. Austin (Ed.), Philosophical Analysis. Dordrecht: Kluwer.
Feltz, A., & Cokely, E. (2011). Individual differences in theory-of-mind judgments: Order effects and side effects. Philosophical Psychology, 24, 343–355.
Feltz, A., & Cokely, E. (2012). The Philosophical Personality Argument. Philosophical Studies, 161(2), 227–246.
Gentner, D., Rattermann, M. J., & Forbus, K. D. (1993). The roles of similarity in transfer: Separating retrievability from inferential soundness. Cognitive psychology, 25(4), 524–575.
Gettier, E. (1963). Is justified true belief knowledge? Analysis, 23(6), 121–123.
Gick, M., & Holyoak, K. (1983). Schema Induction and Analogical Transfer. Cognitive Psychology, 15(1), 1–38.
Girotto, V., Mazzocco, A., & Tasso, A. (1997). The effect of premise order in conditional reasoning: a test of the mental model theory. Cognition, 63, 1–28.
Greene, J., et al. (2001). An fMRI Investigation of Emotional Engagement in Moral Judgment. Science, 293(5537), 2105–2108.
Haidt, J., et al. (1993). Affect, Culture, and Morality, or Is It Wrong to Eat Your Dog? Journal of Personality and Social Psychology, 65(4), 613–628.
Haarmann, H., & Usher, M. (2001). Maintenance of semantic information in capacity-limited item short-term memory. Psychonomic Bulletin & Review, 8, 568–578.
Hogarth, R., & Einhorn, H. (1992). Order Effects in Belief Updating: The Belief-Adjustment Model. Cognitive Psychology, 24(1), 1–55.
Holyoak, K., & Simon, D. (1999). Bidirectional Reasoning in decision making by constraint satisfaction. Journal of Experimental Psychology: General, 128, 3–31.
Horne, Z., Powell, D., & Spino, J. (2013). Belief Updating in Moral Dilemmas. Review of Philosophy and Psychology, 4(4), 705–714.
Horne, Z., Powell, D., & Hummel, J. (2015). A single counterexample leads to stable moral belief updating. Cognitive Science.
Howard, M., & Kahana, M. (1999). Contextual variability and serial position effects in free recall. Journal of experimental psychology Learning memory and cognition, 25(4), 923–941.
Hummel, J., & Holyoak, K. (2003). A Symbolic-Connectionist Theory of Relational Inference and Generalization. Psychological Review, 110(2), 220–264.
Jamieson, D., & Petrusic, W. (1975). Presentation order effects in duration discrimination. Perception & Psychophysics, 17, 197–202.
Lewis, D. (1996). Elusive Knowledge. Australasian Journal of Philosophy, 74, 549–567.
Liao, S. M., Wiegmann, A., Alexander, J., & Vong, G. (2012). Putting the trolley in order: Experimental philosophy and the loop case. Philosophical Psychology, 25(5), 661–671.
Machery, E., Mallon, R., Nichols, S., & Stich, S. P. (2004). Semantics, cross-cultural style. Cognition,92(3), B1–B12.
Markman, A. (1997). Constraints on analogical inference. Cognitive Science, 21(4), 373–418.
McDermott, M. (1995). Redundant causation. British Journal for the Philosophy of Science, 46(4), 523–544.
Murdock, B. (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64(5), 482–488.
Nadelhoffer, T., & Feltz, A. (2008). The Actor-Observer Bias and Moral Intuitions. Neuroethics, 1(2), 133–144.
Neta, R. (2008). What Evidence Do You Have? British Journal for the Philosophy of Science, 59, 89–119.
Newell, A., & Simon, H. (1972). Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall.
Nichols, S., & Knobe, J. (2007). Moral Responsibility and Determinism. Nous, 41(4), 663–685.
Oberauer, K., Hörnig, R., Weidenfeld, A., & Wilhelm, O. (2005). Effects of directionality in deductive reasoning: II. Premise integration and conclusion evaluation. The Quarterly Journal of Experimental Psychology, 58(7), 1225–1247.
Parducci, A. (1965). Category judgment: a range-frequency model. Psychological review, 72(6), 407–418.
Penn, D. C., Holyoak, K. J., & Povinelli, D. J. (2008). Darwin’s mistake: Explaining the discontinuity between human and nonhuman minds. Behavioral and Brain Sciences, 31(2), 109–130.
Pennington, D. (1982). Witnesses and Their Testimony: Effects of Ordering on Jurors Verdicts. Journal of Applied Social Psychology, 4, 318–333.
Petrinovich, L., & O’Neill, P. (1996). Influence of wording and framing effects on moral intuitions. Ethology and Sociobiology, 17(3), 145–171.
Prinz, J. (2007). The Emotional Construction of Morals. Oxford: Oxford University Press.
Quine, W. (1969). “Epistemology Naturalized,” In Ontological Relativity & Other Essays. New York: Columbia University Press.
Rachels, J. (1975). Active and Passive Euthanasia. New England Journal of Medicine, 292(2), 78–80.
Roediger, H., & Crowder, R. (1976). A serial position effect in recall of United States presidents. Bulletin of the Psychonomic Society, 8(4), 275–278.
Scarpi, D. (2004). Effects of Presentation Order on Product Evaluation: An Empirical Analysis. The International Review of Retail, Distribution and Consumer Research, 14, 309–319.
Schulz, E., Cokely, E. T., & Feltz, A. (2011). Persistent bias in expert judgments about free will and moral responsibility: A test of the expertise defense. Consciousness and Cognition, 20(4), 1722–1731.
Schwitzgebel, E., & Cushman, F. (2012). Expertise in Moral Reasoning? Mind & Language, 27(2), 135–153.
Schwitzgebel, E., & Cushman, F. (2015). Philosophers’ biased judgments persist despite training, expertise and reflection. Cognition, 141, 127–137.
Sinnott-Armstrong, W. (2008). Framing Moral Intuitions In Moral Psychology, Vol. 2: The cognitive science of morality: Intuition and diversity. Cambridge: MIT Press.
Sosa, E. (2007). Experimental Philosophy and Philosophical Intuition. Philosophical Studies, 132(1), 99–107.
Stevens, S. S., & Galanter, E. H. (1957). Ratio scales and category scales for a dozen perceptual continua. Journal of Experimental Psychology, 54(6), 377–411.
Swain, S., Alexander, J., & Weinberg, J. M. (2008). The instability of philosophical intuitions: Running hot and cold on truetemp. Philosophy and Phenomenological Research, 76(1), 138–155.
Sytsma, J. et al. (2015). Gödel in the Land of the Rising Sun. Review of Philosophy and Psychology, 6(2), 213–230.
Tobia, K., Buckwalter, W., & Stich, S. (2012). Moral intuitions: Are philosophers experts? Philosophical Psychology, 1-10.
Walker, L., Thibaut, J., & Andreoli, V. (1972). Order of presentation at trial. Yale Law Journal, 82, 216–226.
Weiss, D., & Anderson, N. (1969). Subjective averaging of length with serial presentation. Journal of Experimental Psychology, 82, 52–63.
Wiegmann, A., & Okan, J. (2012). Order effects in moral judgments. Searching for an explanation. In Miyake, N., Peebles, D., & Cooper, R.P. (Eds.), Proceedings of the \(34^{th}\) Annual Conference of the Cognitive Science Society (pp. 1143-1148). Austin, TX. Cognitive Science Society.
Wiegmann, A., Okan, J., & Nagel, J. (2012). Order effects in moral judgment. Philosophical Psychology, 1-24.
Wiegmann, A., & Waldmann, M. R. (2014). Transfer effects between moral dilemmas: A causal model theory. Cognition, 131(1), 28–43.
Williamson, T. (2000). Knowledge and its Limits. Oxford: Oxford University Press.
Williamson, T. (2004). Philosphical ‘Intuitions’ and Scepticism about Judgement. dialectica, 58(1), 109–153.
Williamson, T. (2008). The philosophy of philosophy. Oxford: John Wiley & Sons.
Williamson, T. (forthcoming). “Philosophical Criticisms of Experimental Philosophy,” In Sytsma and Buck-walter.
Wright, J. C. (2010). On intuitional stability: The clear, the strong, and the paradigmatic. Cognition, 115(3), 491–503.
Acknowledgments
Thanks to Josh Alexander, Wes Buckwalter, Greg Gandenberger, Balazs Gyenis, John Hummel, Josh Knobe, Dan Korman, Conor Mayo-Wilson, Derek Powell, David Rose, Jonah Schupbach, Eric Schwitzgebel, John Turri, Jonathan Waskan, Dan Malinsky, Shaun Nichols, and two anonymous referees for comments on earlier drafts.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Horne, Z., Livengood, J. Ordering effects, updating effects, and the specter of global skepticism. Synthese 194, 1189–1218 (2017). https://doi.org/10.1007/s11229-015-0985-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11229-015-0985-9