Abstract
Since it first appeared, there has been much research and critical discussion on the theory of optimal data selection as an explanation of Wason’s (1966, 1968) selection task (Oaksford & Chater, 1994). In this paper, this literature is reviewed, and the theory of optimal data selection is reevaluated in its light. The information gain model is first located in the current theoretical debate in the psychology of reasoning concerning dual processes in human reasoning. A model comparison exercise is then presented that compares a revised version of the model with its theoretical competitors. Tests of the novel predictions of the model are then reviewed. This section also reviews experiments claimed not to be consistent with optimal data selection. Finally, theoretical criticisms of optimal data selection are discussed. It is argued either that the revised model accounts for them or that they do not stand up under analysis. It is concluded that some version of the optimal data selection model still provides the best account of the selection task. Consequently, the conclusion of Oaksford and Chater’s (1994) original rational analysis (Anderson, 1990), that people’s hypothesis-testing behavior on this task is rational and well adapted to the environment, still stands.
Article PDF
Similar content being viewed by others
References
Adams, E. (1966). Probability and the logic of conditionals. In J. Hintikka & P. Suppes (Eds.),Aspects of inductive logic (pp. 265–316). Amsterdam: North-Holland.
Adams, E. (1975).The logic of conditionals: An application of probability theory to deductive logic. Dordrecht: Reidel.
Almor, A., &Sloman, S. A. (1996). Is deontic reasoning special?Psychological Review,103, 374–380.
Anderson, J. R. (1990).The adaptive character of thought. Hillsdale, NJ: Erlbaum.
Anderson, J. R., &Sheu, C.-F. (1995). Causal inferences as perceptual judgments.Memory & Cognition,23, 510–524.
Batchelder, W. H., &Riefer, D. M. (1999). Theoretical and empirical review of multinomial process tree modeling.Psychonomic Bulletin & Review,6, 57–86.
Beattie, J., &Baron, J. (1988). Confirmation and matching biases in hypothesis testing.Quarterly Journal of Experimental Psychology,40A, 269–297.
Berger, J. O. (1985).Statistical decision theory and Bayesian analyses. New York: Springer-Verlag.
Braine, M. D. S., &O’Brien, D. P. (1998).Mental logic. London: Erlbaum.
Byrne, R. M. J. (1989). Suppressing valid inferences with conditionals.Cognition,31, 1–21.
Chater, N. (1995). Neural networks: The new statistical models of mind. In J. P. Levy, D. Bairaktaris, J. A. Bullinaria, & P. Cairns (Eds.),Connectionist models of memory and language (pp. 207–227). London: UCL Press.
Chater, N., Crocker, M., &Pickering, M. (1998). The rational analysis of inquiry: The case of parsing. In M. Oaksford & N. Chater (Eds.),Rational models of cognition (pp. 441–468). Oxford: Oxford University Press.
Chater, N., &Oaksford, M. (1990). Autonomy, implementation and cognitive architecture: A reply to Fodor and Pylyshyn.Cognition,34, 93–107.
Chater, N., &Oaksford, M. (1999a). Information gain vs. decisiontheoretic approaches to data selection: Response to Klauer (1999).Psychological Review,106, 223–227.
Chater, N., &Oaksford, M. (1999b). The probability heuristics model of syllogistic reasoning.Cognitive Psychology,38, 191–258.
Chater, N., &Oaksford, M. (2001). Human rationality and the psychology of reasoning: Where do we go from here?British Journal of Psychology,92, 193–216.
Cheng, P. W., &Novick, L. R. (1990). A probabilistic contrast model of causal induction.Journal of Personality & Social Psychology,58, 545–567.
Chrostowski, J. J., &Griggs, R. A. (1985). The effects of problem content, instructions and verbalisation procedure on Wason’s selection task.Current Psychological Research & Reviews,4, 99–107.
Clocksin, W. F., &Mellish, C. S. (1984).Programming in Prolog. Berlin: Springer-Verlag.
Cohen, L. J. (1981). Can human irrationality be experimentally demonstrated?Behavioral & Brain Sciences,4, 317–370.
Comrie, B. (1986). Conditionals: A typology. In E. C. Traugott, A. terMeulen, J. S. Reilly, & C. A. Ferguson (Eds.),On conditionals (pp. 77–99). Cambridge: Cambridge University Press.
Cosmides, L. (1989). The logic of social exchange: Has natural selection shaped how humans reason? Studies with the Wason selection task.Cognition,31, 187–276.
Cummins, D. D., Lubart, T., Alksnis, O., &Rist, R. (1991). Conditional reasoning and causation.Memory & Cognition,19, 274–282.
Dennett, D. C. (1991).Consciousness explained. Boston: Little, Brown.
Earman, J. (1992).Bayes or bust? Cambridge, MA: MIT Press.
Evans, J. St. B. T. (1977). Toward a statistical theory of reasoning.Quarterly Journal of Experimental Psychology,29, 621–635.
Evans, J. St. B. T. (1983). Linguistic determinants of bias in conditional reasoning.Quarterly Journal of Experimental Psychology,35A, 635–644.
Evans, J. St. B. T. (1984). Heuristic and analytic processes in reasoning.British Journal of Psychology,75, 451–468.
Evans, J. St. B. T. (1989).Bias in human reasoning: Causes and consequences. Hillsdale, NJ: Erlbaum.
Evans, J. St. B. T. (1999). Rational analysis of illogical reasoning [Review ofRationality in a uncertain world: Essays on the cognitive science of human reasoning].Contemporary Psychology,44, 461–463.
Evans, J. St. B. T., Clibbens, J., &Rood, B. (1996). The role of implicit and explicit negations in conditional reasoning bias.Journal of Memory & Language,35, 392–409.
Evans, J. St. B. T., &Handley, S. J. (1999). The role of negation in conditional inference.Quarterly Journal of Experimental Psychology,52A, 739–770.
Evans, J. St. B. T., &Lynch, J. S. (1973). Matching bias in the selection task.British Journal of Psychology,64, 391–397.
Evans, J. St. B. T., &Over, D. E. (1996a).Rationality and reasoning. Hove, U.K.: Psychology Press.
Evans, J. St. B. T., &Over, D. E. (1996b). Rationality in the selection task: Epistemic utility versus uncertainty reduction.Psychological Review,103, 356–363.
Fedorov, V. V. (1972).Theory of optimal experiments. London: Academic Press.
Feeney, A., &Handley, S. J. (2000). The suppression ofq card selections: Evidence for deductive inference in Wason’s selection task.Quarterly Journal of Experimental Psychology,53A, 1224–1242.
Fiddick, L., Cosmides, L., &Tooby, J. (2000). No interpretation without representation: The role of domain-specific representations and inferences in the Wason selection task.Cognition,77, 1–79.
Fiedler, K., &Hertel, G. (1994). Content-related schemata versus verbal-framing effects in deductive reasoning.Social Cognition,12, 129–147.
Gebauer, G., &Laming, D. (1997). Rational choices in Wason’s selection task.Psychological Research,60, 284–293.
Gigerenzer, G., &Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats.Psychological Review,102, 684–704.
Girotto, V., Mazzocco, A., &Cherubini, P. (1992). Judgements of deontic relevance in reasoning: A reply to Jackson and Griggs.Quarterly Journal of Experimental Psychology,45, 547–575.
Gluck, M. A., &Bower, G. H. (1988). Evaluating an adaptive network model of human learning.Journal of Memory & Language,27, 166–195.
Goodman, N. (1954).Fact, fiction and forecast. Cambridge, MA: Harvard University Press.
Green, D. W. (1995a). The abstract selection task: Thesis, antithesis and synthesis. In S. E. Newstead & J. St. B. T. Evans (Eds.),Perspectives on thinking and reasoning: Essays in honour of Peter Wason (pp. 171–186). Hove, U.K.: Erlbaum.
Green, D. W. (1995b). Externalisation, counter-examples and the abstract selection task.Quarterly Journal of Experimental Psychology,48A, 424–446.
Green, D. W. (2000). [Review ofRationality in a uncertain world: Essays on the cognitive science of human reasoning].Quarterly Journal of Experimental Psychology,53A, 281–283.
Green, D. W., &Larking, R. (1995). The locus of facilitation in the abstract selection task.Thinking & Reasoning,1, 183–199.
Green, D. W., &Over, D. E. (1997). Causal inference, contingency tables and the selection task.Current Psychology of Cognition,16, 459–487.
Green, D. W., &Over, D. E. (1998). Reaching a decision: A reply to Oaksford.Thinking & Reasoning,4, 231–248.
Green, D. W., &Over, D. E. (2000). Decision theoretical effects in testing a causal conditional.Current Psychology of Cognition,19, 51–68.
Green, D. W., Over, D. E., &Pyne, R. A. (1997). Probability and choice in the selection task.Thinking & Reasoning,3, 209–236.
Griggs, R. A. (1984). Memory cueing and instructional effects on Wason’s selection task.Current Psychological Research & Reviews,3, 3–10.
Griggs, R. A., &Cox, J. R. (1982). The elusive thematic-materials effect in Wason’s selection task.British Journal of Psychology,73, 407–420.
Hardman, D. (1998). Does reasoning occur on the selection task: A comparison of relevance-based theories.Thinking & Reasoning,4, 353–376.
Hattori, M. (1999). The effects of probabilistic information in Wason’s selection task: An analysis of strategy based on the ODS model.Proceedings of the 16th Annual Meeting of the Japanese Cognitive Science Society,16, 623–626.
Hattori, M. (2002). A quantitative model of optimal data selection in Wason’s selection task.Quarterly Journal of Experimental Psychology,55A, 1241–1272.
Hoch, S. J., &Tschirgi, J. E. (1985). Logical knowledge and cue redundancy in deductive reasoning.Memory & Cognition,13, 453–462.
Horwich, P. (1982).Probability and evidence. Cambridge: Cambridge University Press.
Howson, C., &Urbach, P. (1989).Scientific reasoning: The Bayesian approach. La Salle, IL: Open Court.
Johnson-Laird, P. N. (1983).Mental models. Cambridge: Cambridge University Press.
Johnson-Laird, P. N., &Byrne, R. M. J. (1991).Deduction. Hove, U.K.: Erlbaum.
Johnson-Laird, P. N., &Wason, P. C. (1970). Insight into a logical relation.Quarterly Journal of Experimental Psychology,22, 49–61.
Kahneman, D., &Tversky, A. (1979). Prospect theory: An analysis of decisions under risk.Econometrica,47, 263–291.
Kirby, K. N. (1994). Probabilities and utilities of fictional outcomes in Wason’s four-card selection task.Cognition,51, 1–28.
Klauer, K. C. (1999). On the normative justification for information gain in Wason’s selection task.Psychological Review,106, 215–222.
Krauth, J. (1982). Formulation and experimental verification of models in propositional reasoning.Quarterly Journal of Experimental Psychology,34A, 285–298.
Laming, D. (1996). On the analysis of irrational data selection: A critique of Oaksford and Chater (1994).Psychological Review,103, 364–373.
Legrenzi, P. (1971). Discovery as a means to understanding.Quarterly Journal of Experimental Psychology,23, 417–422.
Lindley, D. V. (1956). On a measure of the information provided by an experiment.Annals of Mathematical Statistics,27, 986–1005.
Loehle, C. (2000).Global Optimization 4.0 [Computer program]. Naperville, IL: Loehle Enterprises.
Love, R. E., &Kessler, C. L. (1995). Focusing in Wason’s selection task: Content and instruction effects.Thinking & Reasoning,1, 153–182.
Mackie, J. L. (1963). The paradox of confirmation.British Journal for the Philosophy of Science,38, 265–277.
Manktelow, K. I., &Evans, J. St. B. T. (1979). Facilitation of reasoning by realism: Effect or non-effect.British Journal of Psychology,70, 477–488.
Marr, D. (1982).Vision. San Francisco: Freeman.
McClelland, J. L. (1998). Connectionist models and Bayesian inference. In M. Oaksford & N. Chater (Eds.),Rational models of cognition (pp. 21–53). Oxford: Oxford University Press.
McKenzie, C. R. M. (2000).Examining the rarity assumption and its implications. Unpublished manuscript, La Jolla, CA: University of California, San Diego, Department of Psychology.
McKenzie, C. R. M., Ferreira, V. S., Mikkelsen, L. A., McDermott, K. J., &Skrable, R. P. (2001). Do conditional statements target rare events?Organizational Behavior & Human Decision Processes,85, 291–309.
McKenzie, C. R. M., &Mikkelsen, L. A. (2000). The psychological side of Hempel’s paradox of confirmation.Psychonomic Bulletin & Review,7, 360–366.
Miller, D. (1994).Critical rationalism: A restatement and a defence. Chicago, IL: Open Court.
Morgan, C. L. (1894).An introduction to comparative psychology. London: Scott.
Moshman, D., &Geil, M. (1998). Collaborative reasoning: Evidence for collective rationality.Thinking & Reasoning,4, 231–248.
Myerson, J., &Miezen, F. M. (1980). The kinetics of choice: An operant systems analysis.Psychological Review,87, 160–174.
Myung, I. J., &Pitt, M. A. (1997). Applying Occam’s razor in modeling cognition: A Bayesian approach.Psychonomic Bulletin & Review,4, 79–95.
Nickerson, R. S. (1996). Hempel’s paradox and Wason’s selection task: Logical and psychological puzzles of confirmation.Thinking & Reasoning,2, 1–32.
Oaksford, M. (1989).Cognition and inquiry: The pragmatics of conditional reasoning. Unpublished doctoral thesis, University of Edinburgh, Centre for Cognitive Science.
Oaksford, M. (1998). Task demands and revising probabilities in the selection task.Thinking & Reasoning,4, 179–186.
Oaksford, M., &Chater, N. (1991). Against logicist cognitive science.Mind & Language,6, 1–38.
Oaksford, M., &Chater, N. (1993). Reasoning theories and bounded rationality. In K. I. Manktelow & D. E. Over (Eds.),Rationality (pp. 31–60). London: Routledge.
Oaksford, M., &Chater, N. (1994). A rational analysis of the selection task as optimal data selection.Psychological Review,101, 608–631.
Oaksford, M., &Chater, N. (1995). Theories of reasoning and the computational explanation of everyday inference.Thinking & Reasoning,1, 121–152.
Oaksford, M., &Chater, N. (1996). Rational explanation of the selection task.Psychological Review,103, 381–391.
Oaksford, M., &Chater, N. (1998a).Rationality in an uncertain world: Essays on the cognitive science of human reasoning. Hove, U.K.: Psychology Press.
Oaksford, M., &Chater, N. (1998b). A revised rational analysis of the selection task: Exceptions and sequential sampling. In M. Oaksford & N. Chater (Eds.),Rational models of cognition (pp. 372–398). Oxford: Oxford University Press.
Oaksford, M., &Chater, N. (2001). The probabilistic approach to human reasoning.Trends in Cognitive Sciences,5, 349–357.
Oaksford, M., Chater, N., &Grainger, B. (1999). Probabilistic effects in data selection.Thinking & Reasoning,5, 193–243.
Oaksford, M., Chater, N., Grainger, B., &Larkin, J. (1997). Optimal data selection in the reduced array selection task (RAST).Journal of Experimental Psychology: Learning, Memory, & Cognition,23, 441–458.
Oaksford, M., Chater, N., &Larkin, J. (2000). Probabilities and polarity biases in conditional inference.Journal of Experimental Psychology: Learning, Memory, & Cognition,26, 883–899.
Oaksford, M., &Stenning, K. (1992). Reasoning with conditionals containing negated constituents.Journal of Experimental Psychology: Learning, Memory, & Cognition,18, 835–854.
Oaksford, M., &Wakefield, M. (2003). Data selection and natural sampling: Probabilities do matter.Memory & Cognition,31, 143–154.
Oberauer, K., Wilhelm, O., &Rosas Diaz, R. (1999). Bayesian rationality for the Wason selection task? A test of optimal data selection theory.Thinking & Reasoning,5, 115–144.
Osman, M., &Laming, D. (2001). Misinterpretation of conditional statements in Wason’s selection task.Psychological Research,65, 128–144.
Over, D. E., &Evans, J. St. B. T. (1994). Hits and misses: Kirby on the selection task.Cognition,52, 235–243.
Over, D. E., &Jessop, A. (1998). Rational analysis of causal conditionals and the selection task. In M. Oaksford & N. Chater (Eds.),Rational models of cognition (pp. 399–414). Oxford: Oxford University Press.
Pearl, J. (1988).Probabilistic reasoning in intelligent systems. San Mateo, CA: Morgan Kaufman.
Pearl, J. (2000).Causality. Cambridge: Cambridge University Press.
Pirolli, P., &Card, S. (1999). Information foraging.Psychological Review,106, 643–675.
Pollard, P., &Evans, J. St. B. T. (1981). The effect of prior belief in reasoning: An associationist interpretation.British Journal of Psychology,72, 73–82.
Pollard, P., &Evans, J. St. B. T. (1983). The effect of experimentally contrived experience on reasoning performance.Psychological Research,45, 287–301.
Popper, K. R. (1959).The logic of scientific discovery. London: Hutchinson.
Read, T. R. C., &Cressie, N. A. C. (1988).Goodness-of-fit statistics for discrete multivariate data. Berlin: Springer-Verlag.
Rips, L. J. (1994).The psychology of proof. Cambridge, MA: MIT Press.
Sakamoto, Y., &Aikake, H. (1978). Analysis of cross-classif ied data by AIC.Annals of the Institute of Statistical Mathematics: Pt. B,30, 185–197.
Schroyens, W., Schaeken, W., Fias, W., &d’Ydewalle, G. (2000). Heuristic and analytic processes in conditional reasoning with negatives.Journal of Experimental Psychology: Learning, Memory, & Cognition,26, 1713–1734.
Schustack, M. W., &Sternberg, R. J. (1981). Evaluation of evidence in causal inference.Journal of Experimental Psychology: General,110, 101–120.
Shannon, C. E., &Weaver, W. (1949).The mathematical theory of communication. Urbana: University of Illinois Press.
Siegel, S., &Castellan, N. J., Jr. (1988).Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill.
Sperber, D., Cara, F., &Girotto, V. (1995). Relevance theory explains the selection task.Cognition,57, 31–95.
Stanovich, K. E., &West, R. F. (1998). Cognitive ability and variation in selection task performance.Thinking & Reasoning,4, 193–230.
Stanovich, K. E., &West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate.Behavioral & Brain Sciences,23, 645–726.
Stein, E. (1996).Without good reason. Oxford: Oxford University Press.
Stich, S. (1985). Could man be an irrational animal?Synthèse,64, 115–135.
Stich, S. (1990).The fragmentation of reason. Cambridge, MA: MIT Press.
Valentine, E. R. (1985). The effect of instructions on performance in the Wason selection task.Current Psychological Research & Reviews,4, 214–223.
Wason, P. C. (1966). Reasoning. In B. Foss (Ed.),New horizons in psychology (pp. 135–151). Harmondsworth, U.K.: Penguin.
Wason, P. C. (1968). Reasoning about a rule.Quarterly Journal of Experimental Psychology,20, 273–281.
Wason, P. C., &Green, D. W. (1984). Reasoning and mental representation.Quarterly Journal of Experimental Psychology,36A, 597–610.
Wason, P. C., &Johnson-Laird, P. N. (1970). A conflict between selecting and evaluating information in an inferential task.British Journal of Psychology,61, 509–515.
Wason, P. C., &Johnson-Laird, P. N. (1972).Psychology of reasoning: Structure and content. Cambridge, MA: Harvard University Press.
Wiener, N. (1948).Cybernetics. New York: Wiley.
Wolf, F. M. (1986).Meta-analysis: Quantitative methods for research synthesis. Newbury Park, CA: Sage.
Wolfram, S. (1999).Mathematica 4.0 [Computer program]. Cambridge: Cambridge University Press.
Yachanin, S. A. (1986). Facilitation in Wason’s selection task.Current Psychological Research & Reviews,5, 20–29.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Oaksford, M., Chater, N. Optimal data selection: Revision, review, and reevaluation. Psychonomic Bulletin & Review 10, 289–318 (2003). https://doi.org/10.3758/BF03196492
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03196492