Minds and Machines

, Volume 21, Issue 3, pp 389–410 | Cite as

Confirmation in the Cognitive Sciences: The Problematic Case of Bayesian Models

Article

Abstract

Bayesian models of human learning are becoming increasingly popular in cognitive science. We argue that their purported confirmation largely relies on a methodology that depends on premises that are inconsistent with the claim that people are Bayesian about learning and inference. Bayesian models in cognitive science derive their appeal from their normative claim that the modeled inference is in some sense rational. Standard accounts of the rationality of Bayesian inference imply predictions that an agent selects the option that maximizes the posterior expected utility. Experimental confirmation of the models, however, has been claimed because of groups of agents that “probability match” the posterior. Probability matching only constitutes support for the Bayesian claim if additional unobvious and untested (but testable) assumptions are invoked. The alternative strategy of weakening the underlying notion of rationality no longer distinguishes the Bayesian model uniquely. A new account of rationality—either for inference or for decision-making—is required to successfully confirm Bayesian models in cognitive science.

Keywords

Bayesian modeling Rationality Levels of explanation Methodology in cognitive science 

Notes

Acknowledgments

Numerous conversations with Josh Tenenbaum, Tom Griffiths, Noah Goodman, and Chris Lucas helped shape the arguments and ideas in this paper, though we doubt that they would endorse many (or any) of our conclusions. We also received valuable feedback from several anonymous reviewers. The first author was partially supported by a grant from the James S. McDonnell Foundation Causal Learning Collaborative. The second author was partially supported by a James S. McDonnell Foundation Scholar Award.

References

  1. Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Erlbaum.Google Scholar
  2. Bonawitz, E. B., Griffiths, T. L., & Schulz, L. E. (2006). Modeling cross-domain causal learning in preschoolers as Bayesian inference. In R. Sun & N. Miyake (Eds.), Proceedings of the 28th annual conference of the Cognitive Science Society. (pp. 89–94). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  3. Chater, N., & Oaksford, M. (2000). The rational analysis of mind and behavior. Synthese, 122, 93–131.CrossRefGoogle Scholar
  4. Chater, N., & Oaksford, M. (Eds.). (2008). The probabilistic mind: Prospects for Bayesian cognitive science. Oxford: Oxford University Press.Google Scholar
  5. Chater, N., Oaksford, M., Nakisa, R., & Redington, M. (2003). Fast, frugal, and rational: How rational norms explain behavior. Organizational Behavior and Human Decision Processes, 90, 63–86.CrossRefGoogle Scholar
  6. Chater, N., Tenenbaum, J. B., & Yuille, A. (2006). Probabilistic models of cognition: Conceptual foundations. Trends in Cognitive Sciences, 10(7), 287–291.CrossRefGoogle Scholar
  7. Courville, A. C., Daw, N. D., & Touretzky, D. S. (2006). Bayesian theories of conditioning in a changing world. Trends in Cognitive Sciences, 10(7), 294–300.CrossRefMathSciNetGoogle Scholar
  8. Danks, D. (2008). Rational analyses, instrumentalism, and implementations. In N. Chater & M. Oaksford (Eds.), The probabilistic mind: Prospects for rational models of cognition (pp. 59–75). Oxford: Oxford University Press.Google Scholar
  9. de Finetti, B. (1937). La prevision: Ses lois logiques, se sources subjectives. Annales De L’institut Henri Poincare, 7, 1–68.MathSciNetMATHGoogle Scholar
  10. Douven, I. (1999). Inference to the best explanation made coherent. Philosophy of Science, 66, S424–S435.CrossRefMathSciNetGoogle Scholar
  11. Doya, K., Ishii, S., Pouget, A., & Rao, R. P. N. (2007). Bayesian brain: Probabilistic approaches to neural coding. Cambridge, MA: The MIT Press.MATHGoogle Scholar
  12. Evans, J. S. B. T. (2009). Does rational analysis stand up to rational analysis? Behavioral and Brain Sciences, 32(1), 88–89.CrossRefGoogle Scholar
  13. Fiorina, M. P. (1971). A note on probability matching and rational choice. Behavioral Science, 16, 158–166.CrossRefGoogle Scholar
  14. Gaifman, H., & Snir, M. (1982). Probabilities over rich languages, testing, and randomness. Journal of Symbolic Logic, 47, 495–548.CrossRefMathSciNetMATHGoogle Scholar
  15. Gigerenzer, G., Czerlinski, J., & Martignon, L. (1999). How good are fast and frugal heuristics? In J. Shanteau, B. Mellers, & D. Schum (Eds.), Decision research from Bayesian approaches to normative systems (pp. 81–103). Norwell, MA: Kluwer.Google Scholar
  16. Griffiths, T. L., & Tenenbaum, J. B. (2005). Structure and strength in causal induction. Cognitive Psychology, 51(4), 334–384.CrossRefGoogle Scholar
  17. Heit, E. (1998). A Bayesian analysis of some forms of inductive reasoning. In M. Oaksford & N. Chater (Eds.), Rational models of cognition (pp. 248–274). New York: Oxford University Press.Google Scholar
  18. Juhl, C. F. (1993). Bayesianism and reliable scientific inquiry. Philosophy of Science, 60, 302–319.CrossRefMathSciNetGoogle Scholar
  19. Kelly, K. T., & Schulte, O. (1995). The computable testability of theories making uncomputable predictions. Erkenntnis, 43, 29–66.CrossRefMathSciNetGoogle Scholar
  20. Kemp, C., Perfors, A., & Tenenbaum, J. B. (2007). Learning overhypotheses with hierarchical Bayesian models. Developmental Science, 10(3), 307–321.CrossRefGoogle Scholar
  21. Kemp, C., & Tenenbaum, J. B. (2003). Theory-based induction. In Proceedings of the 25th annual conference of the Cognitive Science Society. Google Scholar
  22. Kersten, D., Mamassian, P., & Yuille, A. (2004). Object perception as Bayesian inference. Annual Review of Psychology, 55, 271–304.CrossRefGoogle Scholar
  23. Kersten, D., & Yuille, A. (2003). Bayesian models of object perception. Current Opinion in Neurobiology, 13, 1–9.CrossRefGoogle Scholar
  24. Khalil, E. L. (2009). Are stomachs rational? Behavioral and Brain Sciences, 32(1), 91–92.CrossRefGoogle Scholar
  25. Körding, K. P., & Wolpert, D. M. (2006). Bayesian decision theory in sensorimotor control. Trends in Cognitive Sciences, 10(7), 319–326.CrossRefGoogle Scholar
  26. Lee, T. S., & Mumford, D. (2003). Hierarchical Bayesian inference in the visual cortex. Journal of the Optical Society of America A, 20(7), 1434–1448.CrossRefGoogle Scholar
  27. Levi, I. (1988). The demons of decision. Monist, 70, 193–211.Google Scholar
  28. Levi, I. (2002). Money pumps and diachronic books. Philosophy of Science, 69, S235–S247.CrossRefGoogle Scholar
  29. Luce, R. D. (1959). Individual choice behavior: A theoretical analysis. New York: Wiley.MATHGoogle Scholar
  30. Luce, R. D. (1977). The choice axiom after twenty years. Journal of Mathematical Psychology, 15(3), 215–233.CrossRefMathSciNetMATHGoogle Scholar
  31. Maher, P. (1992). Diachronic rationality. Philosophy of Science, 59, 120–141.CrossRefMathSciNetGoogle Scholar
  32. Marr, D. (1982). Vision. San Francisco: W.H. Freeman.Google Scholar
  33. McAllester, D. A. (1999). PAC-Bayesian model averaging. In S. Ben-David, & P. Long (Eds.), Proceedings of the 12th annual conference on computational learning theory. (pp. 164–170). New York: ACM.Google Scholar
  34. McAllester, D. A. (2003). PAC-Bayesian stochastic model selection. Machine Learning, 51(1), 5–21.CrossRefMATHGoogle Scholar
  35. McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Ed.), Frontiers in econometrics (pp. 105–142). New York: Academic Press.Google Scholar
  36. McKenzie, C. R. M. (2004). Framing effects in inference tasks-and why they are normatively defensible. Memory & Cognition, 32(6), 874–885.CrossRefMathSciNetGoogle Scholar
  37. McKenzie, C. R. M., & Mikkelsen, L. A. (2007). A Bayesian view of covariation assessment. Cognitive Psychology, 54(1), 33–61.CrossRefGoogle Scholar
  38. Mozer, M. C., Colagrosso, M. D., & Huber, D. E. (2002). A rational analysis of cognitive control in a speeded discrimination task. In T. Dietterich, S. Becker, & Z. Ghahramani (Eds.), Advances in neural information processing systems 14 (pp. 51–57). Cambridge, MA: The MIT Press.Google Scholar
  39. Mozer, M. C., Colagrosso, M. D., & Huber, D. E. (2003). Mechanisms of long-term repetition priming and skill refinement: A probabilistic pathway model. In Proceedings of the 25th annual conference of the Cognitive Science Society. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  40. Nelson, J. D. (2009). Naive optimality: Subjects’ heuristics can be better motivated than experiments’ optimal models. Behavioral and Brain Sciences, 32(1), 94–95.Google Scholar
  41. Oaksford, M., & Chater, N. (Eds.). (1998). Rational models of cognition. New York: Oxford University Press.Google Scholar
  42. Oaksford, M., & Chater, N. (2007). Bayesian rationality: The probabilistic approach to human reasoning. Oxford: Oxford University Press.Google Scholar
  43. Oaksford, M., & Chater, N. (2009). Open peer commentary and authors’ response. Behavioral and Brain Sciences, 32(1), 85–120.Google Scholar
  44. Oaksford, M., Chater, N., Grainger, B., & Larkin, J. (1997). Optimal data selection in the reduced array selection task (RAST). Journal of Experimental Psychology. Learning, Memory, and Cognition, 23(2), 441–458.CrossRefGoogle Scholar
  45. Oaksford, M., Chater, N., & Larkin, J. (2000). Probabilities and polarity biases in conditional inference. Journal of Experimental Psychology. Learning, Memory, and Cognition, 26(4), 883–899.CrossRefGoogle Scholar
  46. Osherson, D. N., Stob, M., & Weinstein, S. (1988). Mechanical learners pay a price for Bayesianism. Journal of Symbolic Logic, 53, 1245–1251.CrossRefMathSciNetMATHGoogle Scholar
  47. Perfors, A., Tenenbaum, J., Griffiths, T. L., & Xu, F. (in press). A tutorial introduction to Bayesian models of cognitive development. Cognition.Google Scholar
  48. Ramsey, F. P. (1931). Truth and probability. In R. B. Braithwaite (Ed.), The foundations of mathematics and other logical essays (pp. 156–198). London: Harcourt, Brace & Co.Google Scholar
  49. Rao, R. P. N. (2005). Bayesian inference and attentional modulation in the visual cortex. Cognitive Neuroscience and Neuropsychology, 16(16), 1843–1848.Google Scholar
  50. Rao, R. P. N., Shon, A. P., & Meltzoff, A. N. (2004). A Bayesian model of imitation in infants and robots. In K. Dautenhahn & C. Nehaniv (Eds.), Imitation and social learning in robots, humans, and animals: Behavioral, social, and communicative dimensions. Cambridge: Cambridge University Press.Google Scholar
  51. Savage, L. J. (1972). Foundations of statistics. New York: Dover.MATHGoogle Scholar
  52. Schooler, L. J., Shiffrin, R. M., & Raaijmakers, J. G. W. (2001). A Bayesian model for implicit effects in perceptual identification. Psychological Review, 108(1), 257–272.CrossRefGoogle Scholar
  53. Schulte, O. (1999). The logic of reliable and efficient inquiry. Journal of Philosophical Logic, 28, 399–438.CrossRefMathSciNetMATHGoogle Scholar
  54. Schulz, L. E., Bonawitz, E. B., & Griffiths, T. L. (2007). Can being scared make your tummy ache? Naive theories, ambiguous evidence and preschoolers’ causal inferences. Developmental Psychology, 43(5), 1124–1139.CrossRefGoogle Scholar
  55. Seeger, M. (2003). PAC-Bayesian generalisation error bounds for Gaussian process classification. Journal of Machine Learning Research, 3, 233–269.CrossRefMathSciNetMATHGoogle Scholar
  56. Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233–250.CrossRefGoogle Scholar
  57. Shiffrin, R. M., & Steyvers, M. (1997). A model for recognition memory: REM-Retrieving effectively from memory. Psychonomic Bulletin & Review, 4, 145–166.CrossRefGoogle Scholar
  58. Sobel, D. M., & Kushnir, T. (2006). The importance of decision making in causal learning from interventions. Memory & Cognition, 34(2), 411–419.CrossRefGoogle Scholar
  59. Sobel, D. M., Tenenbaum, J. B., & Gopnik, A. (2004). Children’s causal inferences from indirect evidence: Backwards blocking and Bayesian reasoning in preschoolers. Cognitive Science, 28, 303–333.Google Scholar
  60. Stephens, D. W., & Krebs, J. R. (1986). Foraging theory. Princeton, NJ: Princeton University Press.Google Scholar
  61. Steyvers, M., & Griffiths, T. L. (2008). Rational analysis as a link between human memory and information retrieval. In N. Chater & M. Oaksford (Eds.), The probabilistic mind: Prospects for Bayesian cognitive science (pp. 329–349). Oxford: Oxford University Press.Google Scholar
  62. Steyvers, M., Griffiths, T. L., & Dennis, S. (2006). Probabilistic inference in human semantic memory. Trends in Cognitive Sciences, 10(7), 327–334.CrossRefGoogle Scholar
  63. Steyvers, M., Tenenbaum, J. B., Wagenmakers, E., & Blum, B. (2003). Inferring causal networks from observations and interventions. Cognitive Science, 27, 453–489.CrossRefGoogle Scholar
  64. Teller, P. (1973). Conditionalization and observation. Synthese, 26, 218–238.CrossRefMATHGoogle Scholar
  65. Teller, P. (1976). Conditionalization, observation, and change of preference. In W. L. Harper & C. A. Hooker (Eds.), Foundations of probability theory, statistical inference, and statistical theories of science (pp. 205–259). Dordrecht: Reidel.Google Scholar
  66. Tenenbaum, J. B., & Griffiths, T. L. (2001). Generalization, similarity, and Bayesian inference. Behavioral and Brain Sciences, 24, 629–641.Google Scholar
  67. Tenenbaum, J. B., Griffiths, T. L., & Kemp, C. (2006). Theory-Based Bayesian models of inductive learning and reasoning. Trends in Cognitive Sciences, 10(7), 309–318.CrossRefGoogle Scholar
  68. Tenenbaum, J. B., Griffiths, T. L., & Niyogi, S. (2007). Intuitive theories as grammars for causal inference. In A. Gopnik & L. E. Schulz (Eds.), Causal learning: Psychology, philosophy, and computation (pp. 301–322). Oxford: Oxford University Press.Google Scholar
  69. Tenenbaum, J. B., & Niyogi, S. (2003). Learning causal laws. In R. Alterman, & D. Kirsh (Eds.), Proceedings of the 25th annual conference of the Cognitive Science Society. (pp. 1152–1157). Mahwah, NJ: Erlbaum.Google Scholar
  70. van Fraassen, B. C. (1984). Belief and the will. Journal of Philosophy, 81(5), 235–256.CrossRefGoogle Scholar
  71. Vulkan, N. (2000). An economist’s perspective on probability matching. Journal of Economic Surveys, 14, 101–118.CrossRefGoogle Scholar
  72. West, R. F., & Stanovich, K. E. (2003). Is probability matching smart? Associations between probabilistic choices and cognitive ability. Memory & Cognition, 31(2), 243–251.CrossRefGoogle Scholar
  73. Xu, F., & Tenenbaum, J. B. (2005). Word learning as Bayesian inference: Evidence from preschoolers. In Proceedings of the 27th annual conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  74. Xu, F., & Tenenbaum, J. B. (2007). Sensitivity to sampling in Bayesian word learning. Developmental Science, 10(3), 288–297.CrossRefGoogle Scholar
  75. Yuille, A., & Kersten, D. (2006). Vision as Bayesian inference: Analysis by synthesis? Trends in Cognitive Sciences, 10(7), 301–308.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Department of PhilosophyWashington University in St LouisSt LouisUSA
  2. 2.Department of PhilosophyCarnegie Mellon UniversityPittsburghUSA
  3. 3.Florida Institute for Human and Machine CognitionPensacolaUSA

Personalised recommendations