Skip to main content
SpringerLink
Log in

Gigerenzer’s ‘external validity argument’ against the heuristics and biases program: an assessment

  • Published:
Mind & Society Aims and scope Submit manuscript

Abstract

Gigerenzer’s ‘external validity argument’ plays a pivotal role in his critique of the heuristics and biases research program (HB). The basic idea is that (a) the experimental contexts deployed by HB are not representative of the real environment and that (b) the differences between the setting and the real environment are causally relevant, because they result in different performances by the subjects. However, by considering Gigerenzer’s work on frequencies in probability judgments, this essay attempts to show that there are fatal flaws in the argument. Specifically, each of the claims is controversial: whereas (b) is not adequately empirically justified, (a) is inconsistent with the ‘debiasing’ program of Gigerenzer’s ABC group. Therefore, whatever reason we might have for believing that the experimental findings of HB lack experimental validity, this should not be based on Gigerenzer’s version of the argument.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. In the original Linda problem the list of possible outcomes contained more than two claims.

  2. For instance, Carruthers (2007) claims that the endorsement of massive modularity is a core feature of EP, but not of ABC. However, this does not seem to mark a real difference between them: after all, Gigerenzer and colleagues refer quite often to the massive modularity thesis (Gigerenzer 1995, 1997; Gigerenzer and Hug 1992).

  3. It should be noted that even authors who do not adhere to the ABC research program have defended the miscommunication view of reasoning (e.g. Politzer and Noveck 1991).

  4. Concerns about the isomorphism between the laboratory and the real environment have also been expressed by biologists. In the words of Boostaber and Langsam: ‘One might wonder if inconsistencies which appear between laboratory results and those in the field, or which seem counter to our general perceptions of human behaviour are in part the result of the experimenter failing to match in the laboratory setting the nature of the extended uncertainty which the subject must face in its normal habitat’ (1985, 178).

  5. While in Hertwig and Gigerenzer (1999) subjects were asked to give a frequency estimate, in Tentori et al. (2004) subjects were required to choose the group with the highest frequency. However, identifying the top frequency option seems to require the same type of comparative operations that are required for ranking.

  6. Girotto and Gonzalez (2001) introduced a representation in terms of number of chances as a way to translate natural frequencies into a language that looks like single-event probabilities. This has been the target of some criticism. For example, Hoffrage et al. (2002), who point out that ‘it is confusing that number of chances are called probabilities throughout the paper, because unlike probabilities, these are not single numbers in the interval [0,1] but natural frequencies disguised as probabilities’ (p. 350).

  7. Curiously, Lee points out that while ‘HB identifies the tasks in which human reasoning needs to be improved’, the scholars of ABC ‘identify the conditions that actually improve and debias judgments’ (2008, 64). This presentation seems misleading, for Richard Thaler and other scholars within the HB framework have suggested a list of debiasing methods (Thaler and Sunstein 2008).

  8. This point has been stressed by Stanovich (2003), who claims that it is often unclear whether Gigerenzer is referring to genes’ goals or rather to the individuals’ goals and that, because of this ambiguity, we do not know whether the environment is the EEA or the current environment.

  9. It should be mentioned that Charness et al.’s data (2010) seem to conflict with those reported by Stolarz-Fantino et al. (2003), who have shown that the conjunction fallacy occurred even when there were monetary incentives for making the correct answer. Yet, the two studies were different in that, whereas Stolarz-Fantino et al. (2003) told subjects that they would be entered a lottery if their answers were correct, Charness et al. (2010) informed subjects that there was a correct answer and that anyone who chose the correct answer would receive $ 4.

References

  • Berkeley D, Humphreys P (1982) Structuring decision problems and the bias heuristic. Acta Psychol 50(3):201–252

    Article  Google Scholar 

  • Binmore K (1999) Why experiment in economics? Econ J 109:16–24

    Article  Google Scholar 

  • Bonini N, Tentori K, Osherson D (2004) A different conjunction fallacy. Mind Lang 19(2):199–210

    Article  Google Scholar 

  • Bookstaber R, Langsam J (1985) On the optimality of coarse behaviour. J Theor Biol 116:161–193

    Article  Google Scholar 

  • Brunswik E (1955) Symposium on the probability approach in psychology. Psychol Rev 62(3):193–217

    Article  Google Scholar 

  • Buller DJ (2005) Adapting minds: evolutionary psychology and the persistent quest for human nature. MIT Press, Boston

    Google Scholar 

  • Bullock S, Todd PM (1999) Made to measure: ecological rationality in structured environments. Mind Mach 9:497–541

    Article  Google Scholar 

  • Camerer CF (2000) Prospect theory in the wild: evidence from the field. In: Kahneman D, Tversky A (eds) Choices, values and frames. Cambridge University Press, Cambridge, pp 288–300

    Google Scholar 

  • Campbell DT (1957) Factors relevant to the validity of experiments in social settings. Psychol Bull 54:297–312

    Article  Google Scholar 

  • Carruthers P (2007) Simple heuristics meet massive modularity. In: Carruthers P, Laurence S, Stich SP (eds) The innate mind: culture and cognition. Oxford University Press, Oxford

    Chapter  Google Scholar 

  • Charness G, Karni E, Levin D (2010) On the conjunction fallacy in probability judgment: new experimental evidence regarding Linda. Game Econ Behav 68:551–556

    Article  Google Scholar 

  • Chase VM, Hertwig R, Gigerenzer G (1998) Visions of rationality. Trends Cogn Sci 2(6):206–214

    Article  Google Scholar 

  • Cohen LJ (1981) Can human rationality be experimentally demonstrated? Behav Brain Sci 4:317–370

    Article  Google Scholar 

  • Cosmides L, Tooby J (1992) Cognitive adaptations for social exchange. In: Barkow JH, Cosmides L, Tooby J (eds) The adapted mind: evolutionary psychology and the generation of culture. Oxford University Press, New York, pp 163–228

    Google Scholar 

  • Cosmides L, Tooby J (1996) Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty. Cognition 58:1–73

    Article  Google Scholar 

  • Dhami M, Hoffrage K, Hertwig R (2004) The role of representative design in an ecological approach to cognition. Psychol Bull 130(3):959–988

    Article  Google Scholar 

  • Edwards W (1966) Nonconservative information processing systems. University of Michigan, Ann Arbor

    Google Scholar 

  • Edwards W (1983) Human cognitive capabilities, representativeness, and ground rules for research. In: Humphreys P, Svenson O, Vari A (eds) Analysing and aiding decision processes, vol 18. North-Holland, Amsterdam, pp 507–513

    Chapter  Google Scholar 

  • Fiedler K (1988) The dependence of the conjunction fallacy on subtle linguistic factors. Psychol Res 50(3):123–129

    Article  Google Scholar 

  • Fisk JE, Slattery R (2005) Reasoning about conjunctive probabilistic concepts in childhood. Can J Exp Psychol 59:168–178

    Article  Google Scholar 

  • Fodor J, Piattelli-Palmarini M (2010) What Darwin got wrong? Farrar, Straus and Giroux, New York

    Google Scholar 

  • Gigerenzer G (1984) External validity of laboratory experiments: the frequency–validity relationship. Am J Psychol 97(2):185–195

    Article  Google Scholar 

  • Gigerenzer G (1991) How to make cognitive illusions disappear. Eur Rev Soc Psychol 45(2):83–115

    Article  Google Scholar 

  • Gigerenzer G (1994) Why the distinction between single-event probabilities and frequencies is important for psychology? In: Wright G, Ayton P (eds) Subjective probability. Wiley, New York, pp 129–161

    Google Scholar 

  • Gigerenzer G (1995) The taming of content; some thoughts about domains and modules. Think Reason 1:324–335

    Article  Google Scholar 

  • Gigerenzer G (1996) On narrow norms and vague heuristics: a reply to Kahneman and Tversky. Psychol Rev 103(3):592–596

    Article  Google Scholar 

  • Gigerenzer G (1997) The modularity of social intelligence. In: Whiten A, Byrne R (eds) Machiavellian intelligence II: extensions and evaluations. Cambridge University Press, Cambridge, pp 264–288

    Chapter  Google Scholar 

  • Gigerenzer G (1998) Ecological intelligence: an adaptation for frequencies. In: Cummins D, Allen C (eds) The evolution of mind. Oxford University Press, New York, pp 9–29

    Google Scholar 

  • Gigerenzer G (2001) Ideas in exile: the struggles of an upright man. In: Hammond KR, Stewart TR (eds) The essential Brunswik: beginning, explications, applications. Oxford University Press, New York, pp 445–452

    Google Scholar 

  • Gigerenzer G (2002) Reckoning with risk: learning to live with uncertainty. Penguin Books, London

    Google Scholar 

  • Gigerenzer G (2006) Gut feelings: the intelligence of the unconscious. Penguin Books, London

    Google Scholar 

  • Gigerenzer G (2008) Moral intuition = fast and frugal heuristics? In: Sinnott-Armstrong W (ed) Moral psychology, vol 2: the cognitive science of morality: intuition and diversity. MIT Press, Cambridge, pp 1–26

    Google Scholar 

  • Gigerenzer G, Hug K (1992) Domain-specific reasoning: social contracts, cheating, and perspective Change. Cognition 43:127–171

    Article  Google Scholar 

  • Gigerenzer G, Sturm T (in press) How (far) can rationality be naturalized? Synthese

  • Gigerenzer G, Hoffrage U, Kleinbolting H (1991) Probabilistic mental models: a Brunswikian theory of confidence. Psychol Rev 98(4):506–528

    Article  Google Scholar 

  • Gilovich T, Vallone R, Tversky A (1985) The hot hand in basketball: on the misperception of random sequences. Cogn Psychol 17:295–314

    Article  Google Scholar 

  • Gilovich T, Griffin D, Kahneman D (2002) Heuristics and biases: the psychology of intuitive judgment. Cambridge University Press, New York

    Book  Google Scholar 

  • Girotto V, Gonzalez M (2001) Solving probabilistic and statistical problems: a matter of question form and information structure. Cognition 78:247–276

    Article  Google Scholar 

  • Girotto V, Gonzalez M (2008) Children’s understanding of posterior probability. Cognition 106:325–344

    Article  Google Scholar 

  • Grether DM, Plott CR (1979) Economic theory of choice and the preference reversals phenomenon. Am Econ Rev 69(4):623–638

    Google Scholar 

  • Harrison GW, List JA (2004) Field experiments. J Econ Lit 42:1009–1055

    Article  Google Scholar 

  • Hastie R, Dawes R (2001) Rational choice in an uncertain world: the psychology of judgment and decision making. Sage, Thousand Oaks

    Google Scholar 

  • Hertwig R, Chase VM (1998) Many reasons or just one: how response mode affects reasoning in the conjunction problem. Think Reason 4:319–352

    Article  Google Scholar 

  • Hertwig R, Gigerenzer G (1999) The conjunction fallacy revisited: how intelligent inferences look like reasoning errors. J Behav Decis Mak 12:275–305

    Article  Google Scholar 

  • Hoffrage U, Gigerenzer G, Krauss S, Martignon L (2002) Representation facilitates reasoning: what natural frequencies are and what they are not. Cognition 84:343–352

    Article  Google Scholar 

  • Hogarth RM (1981) Beyond discrete biases: functional and dysfunctional aspects of judgmental heuristics. Psychol Bull 90(2):197–217

    Article  Google Scholar 

  • Jones SK, Jones T, Frisch D (1995) Biases of probability assessment: a comparison of frequency and single-case judgments. 122 61(4):109–122

    Google Scholar 

  • Kahneman D, Tversky A (1973) On the psychology of prediction. Psychol Rev 80(2):237–251

    Article  Google Scholar 

  • Kahneman D, Tversky A (1979) Prospect theory: an analysis of decision under risk. Econometrica 47(2):263–292

    Article  Google Scholar 

  • Kahneman D, Tversky A (1996) On the reality of cognitive illusions. Psychol Rev 103(3):591–592

    Article  Google Scholar 

  • Koehler J (1996) On conveying the probative value of DNA evidence: frequencies, likelihood ratios, and error rates. U Colo Law Rev 67(3):859–886

    Google Scholar 

  • Lee JC (2008) Epistemology by applied cognitive psychology and the ‘strong replacement’ of normative psychology. Philos Soc Sci 38:55–75

    Article  Google Scholar 

  • Levitt SD, List JA (2009) Field experiments in economics: the past, the present and the future. Eur Econ Rev 53:1–18

    Article  Google Scholar 

  • List JA (2008) Homo experimentalis evolves. Science 321:207–208

    Article  Google Scholar 

  • Maciejovsky B, Budescu DV (2007) Collective induction without cooperation? Learning and knowledge transfer in cooperative groups and competitive auctions. J Pers Soc Psychol 92(5):854–870

    Article  Google Scholar 

  • Moro R (2009) On the nature of the conjunction fallacy. Synthese 171:1–24

    Article  Google Scholar 

  • Nilsson H, Andersson P (2010) Making the seemingly impossible appear possible: effects of conjunction fallacies in evaluations of bets on football games. J Econ Psychol 3:172–180

    Article  Google Scholar 

  • Politzer G (2004) Reasoning, judgement and pragmatics. In: Noveck IN, Sperber D (eds) Experimental pragmatics. Palgrave, Houndmills, pp 94–115

    Google Scholar 

  • Politzer G, Noveck IA (1991) Are conjunction rule violations the result of conversational rule violations? J Psycholinguist Res 20:83–103

    Article  Google Scholar 

  • Samuels R, Stich S, Bishop M (2002) Ending the rationality wars: how to make disputes about human rationality disappear? In: Renee R (ed) Common sense, reasoning and rationality. Oxford University Press, New York, pp 236–268

    Chapter  Google Scholar 

  • Scher SJ, Rauscher F (2003) Evolutionary psychology: alternative approaches. Kluwer, Boston

    Google Scholar 

  • Simon HA (1955) A behavioural model of rational choice. Q J Econ 69:99–118

    Article  Google Scholar 

  • Sinnott-Armstrong W, Young L, Cushman F (2010) Moral intuitions. In: Doris J (ed) The moral psychology handbook. Oxford University Press, Oxford

    Google Scholar 

  • Sloman SA, Over D, Slovak L, Stibel JM (2003) Frequency illusions and other fallacies. Organ Behav Hum Dec 29:296–309

    Article  Google Scholar 

  • Smith EA, Borgerhoff Mulder M, Hill J (2001) Controversies in the evolutionary social sciences: a guide for the perplexed. Trends Ecol Evol 16:128–135

    Article  Google Scholar 

  • Stanovich K (2003) Evolutionary versus instrumental goals: how evolutionary psychology misconceives human rationality? In: Over E (ed) Evolution and the psychology of thinking: the debate. Psychological Press, New York, pp 171–230

    Google Scholar 

  • Stein E (1996) Without good reason: the rationality debate in philosophy and cognitive science. Clarendon Press, Oxford

    Google Scholar 

  • Sterelny K, Griffiths PE (1999) Sex and death: an introduction to philosophy of biology. University of Chicago Press, Chicago

    Google Scholar 

  • Stevens JR (2010) Rational decision making in primates: the bounded and the ecological. In: Platt ML, Ghazanfar AA (eds) Primate neuroethology. Oxford University Press, Oxford, pp 96–116

    Google Scholar 

  • Stolarz-Fantino S, Fantino E, Zizzo DJ, Wen J (2003) The conjunction fallacy: new evidence for robustness. Am J Psychol 116:15–34

    Article  Google Scholar 

  • Sunstein CR (1997) Behavioral analysis of law. U Chic Law Rev 64:1175–1195

    Article  Google Scholar 

  • Téglás E, Girotto V, Gonzalez M, Bonatti L (2007) Intuitions of probabilities shape expectations about the future at 12 months and beyond. Proc Natl Acad Sci 104:19156–19159

    Article  Google Scholar 

  • Téglás E, Vul E, Girotto V, Gonzalez M, Tenenbaum JB, Bonatti L (2011) Pure reasoning in 12-month-old infants as probabilities inference. Science 332:1054–1059

    Article  Google Scholar 

  • Tentori K, Bonini N, Osherson D (2004) The conjunction fallacy: a misunderstanding about conjunction? Cogn Sci 28(2):467–477

    Google Scholar 

  • Thaler RH, Sunstein CR (2008) Nudge: improving decisions about health, wealth, and happiness. Yale University Press, New Haven

    Google Scholar 

  • Tversky A, Kahneman D (1983) Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment. Psychol Rev 90(4):293–315

    Article  Google Scholar 

  • Wedell DH, Moro R (2008) Testing boundary conditions for the conjunction fallacy: effects of response mode, conceptual focus and problem type. Cognition 107:105–136

    Article  Google Scholar 

Download references

Acknowledgments

I would like to thank Campbell Brown, Francesco Guala, Matteo Motterlini, Tillmann Vierkant and two anonymous referees for very helpful comments on earlier drafts of this paper. The usual disclaimer about remaining errors applies.

Author information

Authors and Affiliations

  1. Department of Philosophy, University of Edinburgh, Dugald Stewart Building, 3 Charles Street, George Square, Edinburgh, EH8 9AD, UK

    Andrea Polonioli

Authors
  1. Andrea Polonioli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrea Polonioli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Polonioli, A. Gigerenzer’s ‘external validity argument’ against the heuristics and biases program: an assessment. Mind Soc 11, 133–148 (2012). https://doi.org/10.1007/s11299-012-0098-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11299-012-0098-9

Keywords

Search

Navigation