Memory & Cognition

, Volume 45, Issue 2, pp 270–280 | Cite as

Physician Bayesian updating from personal beliefs about the base rate and likelihood ratio

Article

Abstract

Whether humans can accurately make decisions in line with Bayes’ rule has been one of the most important yet contentious topics in cognitive psychology. Though a number of paradigms have been used for studying Bayesian updating, rarely have subjects been allowed to use their own preexisting beliefs about the prior and the likelihood. A study is reported in which physicians judged the posttest probability of a diagnosis for a patient vignette after receiving a test result, and the physicians’ posttest judgments were compared to the normative posttest calculated from their own beliefs in the sensitivity and false positive rate of the test (likelihood ratio) and prior probability of the diagnosis. On the one hand, the posttest judgments were strongly related to the physicians’ beliefs about both the prior probability as well as the likelihood ratio, and the priors were used considerably more strongly than in previous research. On the other hand, both the prior and the likelihoods were still not used quite as much as they should have been, and there was evidence of other nonnormative aspects to the updating, such as updating independent of the likelihood beliefs. By focusing on how physicians use their own prior beliefs for Bayesian updating, this study provides insight into how well experts perform probabilistic inference in settings in which they rely upon their own prior beliefs rather than experimenter-provided cues. It suggests that there is reason to be optimistic about experts’ abilities, but that there is still considerable need for improvement.

Keywords

Bayesian reasoning Probabilistic reasoning Diagnosis 

References

  1. Barbey, A. K., & Sloman, S. A. (2007). Base-rate respect: From ecological rationality to dual processes. The Behavioral and Brain Sciences, 30(3), 241–254. doi:10.1017/S0140525X07001653. discussion 255–297.PubMedGoogle Scholar
  2. Bergus, G. R., Chapman, G. B., Gjerde, C., & Elstein, A. S. (1995). Clinical reasoning about new symptoms despite preexisting disease: Sources of error and order effects. Family Medicine, 27, 314–320.PubMedGoogle Scholar
  3. Bjerregaard, N. C., Tøttrup, A., Sørensen, H. T., & Laurberg, S. (2009). Detection of colorectal cancer in symptomatic outpatients without visible rectal bleeding: Validity of the fecal occult blood test. Clinical Epidemiology, 1, 119–124. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2943167&tool=pmcentrez&rendertype=abstract CrossRefPubMedPubMedCentralGoogle Scholar
  4. Casscells, W., Schoenberger, A., & Graboys, T. B. (1978). Interpretation by physicians of clinical laboratory tests. The New England Journal of Medicine, 299(18), 999–1001.CrossRefPubMedGoogle Scholar
  5. Castiglione, G., Ciatto, S., Mazzotta, A., & Grazzini, G. (1995). Sensitivity of screening sigmoidoscopy for proximal colorectal tumors. The Lancet, 345, 726–727. Retrieved from http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Sensitivity+of+screening+sigmoidoscopy+for+proximal+colorectal+tumors#0 CrossRefGoogle Scholar
  6. Christensen-Szalanski, J. J., & Beach, L. R. (1982). Experience and the base-rate fallacy. Organizational Behavior and Human Performance, 29(2), 270–278.CrossRefPubMedGoogle Scholar
  7. Christensen-Szalanski, J. J., & Bushyhead, J. B. (1981). Physicians’ use of probabilistic information in a real clinical setting. Journal of Experimental Psychology: Human Perception and Performance, 7(4), 928–935. doi:10.1037//0096-1523.7.4.928 PubMedGoogle Scholar
  8. Croskerry, P. (2002). Achieving quality in clinical decision making: Cognitive strategies and detection of bias. Academic Emergency Medicine : Official Journal of the Society for Academic Emergency Medicine, 9(11), 1184–1204.CrossRefGoogle Scholar
  9. Croskerry, P., Singhal, G., & Mamede, S. (2013). Cognitive debiasing 1: Origins of bias and theory of debiasing. BMJ Quality & Safety, 22, ii58–ii64. doi:10.1136/bmjqs-2012-001712 CrossRefGoogle Scholar
  10. Eddy, D. (1982). Probabilistic reasoning in clinical medicine: Problems and opportunities. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 249–267). Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  11. Edgell, S. E., Harbison, J. I., Neace, W. P., Nahinsky, I. D., & Lajoie, A. S. (2004). What is learned from experience in a probabilistic environment? Journal of Behavioral Decision Making, 17(3), 213–229. doi:10.1002/bdm.471 CrossRefGoogle Scholar
  12. Eva, K. W., & Cunnington, J. P. W. (2006). The difficulty with experience: Does practice increase susceptibility to premature closure? The Journal of Continuing Education in the Health Professions, 26(3), 192–198. doi:10.1002/chp CrossRefPubMedGoogle Scholar
  13. Evans, J. S. B. T., Handley, S. J., Over, D. E., & Perham, N. (2002). Background beliefs in Bayesian inference. Memory & Cognition, 30(2), 179–190.CrossRefGoogle Scholar
  14. Evans, J. S. B. T., Thompson, V. A., & Over, D. E. (2015). Uncertain deduction and conditional reasoning. Frontiers in Psychology, 6, 1–12. doi:10.3389/fpsyg.2015.00398 Google Scholar
  15. Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684–704. doi:10.1037//0033-295X.102.4.684 CrossRefGoogle Scholar
  16. Graber, M. L., Franklin, N., & Gordon, R. (2005). Diagnostic error in internal medicine. Archives of Internal Medicine, 165(13), 1493–1499. doi:10.1001/archinte.165.13.1493 CrossRefPubMedGoogle Scholar
  17. Graber, M. L., Kissam, S., Payne, V. L., Meyer, A. N. D., Sorensen, A., Lenfestey, N., . . . Singh, H. (2012). Cognitive interventions to reduce diagnostic error: A narrative review. BMJ Quality & Safety, 21, 535–557. doi:10.1136/bmjqs-2011-000149
  18. Hadjichristidis, C., Sloman, S. A., & Over, D. E. (2014). Categorical induction from uncertain premises: Jeffrey’s doesn’t completely rule. Thinking & Reasoning, 20(4), 405–431. doi:10.1080/13546783.2014.884510 CrossRefGoogle Scholar
  19. Hilbert, M. (2012). Toward a synthesis of cognitive biases: How noisy information processing can bias human decision making. Psychological Bulletin, 138(2), 211–237. doi:10.1037/a0025940 CrossRefPubMedGoogle Scholar
  20. Johnson, C., Mei-Hsiu Chen, M. M. M., Toledano, A. Y., Heiken, J. P., Dachman, A., Kuo, M. D., . . . Limburg, P. J. (2008). Accuracy of CT colonography for detection of large adenomas and cancers. The New England Journal of Medicine, 359(12), 1207–1217. Retrieved from http://www.nejm.org/doi/full/10.1056/nejmoa0800996
  21. Keren, G., & Thujs, L. J. (1996). The base rate controversy: Is the glass half-full or half-empty? Behavioral and Brain Sciences, 19, 26.CrossRefGoogle Scholar
  22. Kleiter, G. D. (1994). Natural sampling: Rationality without base-rates. In G. H. Fischer & D. Laming (Eds.), Contributions to mathematical psychology, psychometrics, and methodology (pp. 377–388). New York, NY: Springer-Verlag.Google Scholar
  23. Kleiter, G. D., Krebs, M., Doherty, M. E., Garavan, H., Chadwick, R., & Brake, G. (1997). Do subjects understand base rates? Organizational Behavior and Human Decision Processes, 72(1), 25–61. doi:10.1006/obhd.1997.2727 CrossRefGoogle Scholar
  24. Koehler, J. J. (1996). The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges. Behavioral and Brain Sciences, 19(01), 1. doi:10.1017/S0140525X00041157 CrossRefGoogle Scholar
  25. Krynski, T. R., & Tenenbaum, J. (2007). The role of causality in judgment under uncertainty. Journal of Experimental Psychology. General, 136(3), 430–450. doi:10.1037/0096-3445.136.3.430 CrossRefPubMedGoogle Scholar
  26. Lyman, G., & Balducci, L. (1993). Overestimation of test effects in clinical judgment. Journal of Cancer Education, 8(4). Retrieved from http://www.tandfonline.com/doi/full/10.1080/08858199309528246
  27. Lyman, G., & Balducci, L. (1994). The effect of changing disease risk on clinical reasoning. Journal of General Internal Medicine, 9(9), 488–495. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7996291 CrossRefPubMedGoogle Scholar
  28. Niv, Y., & Sperber, A. (1995). Sensitivity, specificity, and predictive value of fecal occult blood testing ( Hemoccult II ) for colorectal neoplasia in symptomatic patients : A prospective study with total colonoscopy. The American Journal of Gastroenterology, 90(11), 1974–1977. Retrieved from http://cat.inist.fr/?aModele=afficheN&cpsidt=2902479 PubMedGoogle Scholar
  29. Noguchi, Y., Matsui, K., Imura, H., Kiyota, M., & Fukui, T. (2002). Quantitative evaluation of the diagnostic thinking process in medical students. Journal of General Internal Medicine, 17(11), 839–844. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12406355 CrossRefPubMedGoogle Scholar
  30. Pauker, S., & Kassirer, J. (1980). The threshold approach to clinical decision making. The New England Journal of Medicine, 302(20), 1109–1117. Retrieved from http://europepmc.org/abstract/MED/7366635 CrossRefPubMedGoogle Scholar
  31. Reilly, J. B., Ogdie, A. R., Von Feldt, J. M., & Myers, J. S. (2013). Teaching about how doctors think: A longitudinal curriculum in cognitive bias and diagnostic error for residents. BMJ Quality & Safety, 22, 1044–1050. doi:10.1136/bmjqs-2013-001987 CrossRefGoogle Scholar
  32. Shafer, G. (1981). Jeffrey’s rule of conditioning. Philosophy of Science, 48, 337–362.CrossRefGoogle Scholar
  33. Singmann, H., Klauer, K. C., & Over, D. (2014). New normative standards of conditional reasoning and the dual-source model. Frontiers in Psychology, 5, 1–14. doi:10.3389/fpsyg.2014.00316 Google Scholar
  34. Talbott, W. (2015). Bayesian epistemology. Retrieved from http://plato.stanford.edu/entries/epistemology-bayesian/
  35. Voytovich, A. E., Rippey, R. M., & Suffredini, A. (1985). Premature conclusions in diagnostic reasoning. Journal of Medical Education, 60(4), 302–307. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/3981589 PubMedGoogle Scholar
  36. Warner, J. L., Najarian, R. M., & Tierney, L. M. (2010). Perspective: Uses and misuses of thresholds in diagnostic decision making. Academic Medicine : Journal of the Association of American Medical Colleges, 85(3), 556–563. doi:10.1097/ACM.0b013e3181ccd59b CrossRefGoogle Scholar
  37. Zauber, A., Lansdorp-Vogelaar, I., Knudsen, A. B., Wilschut, J., van Ballegooijen, M., & Kuntz, K. M. (2008). Evaluating test strategies for colorectal cancer screening: A decision analysis for the U.S. Preventive Services Task Force. Annals of Internal Medicine, 149(9), 659–668. Retrieved from http://annals.org/article.aspx?articleid=743580&atab=11 CrossRefPubMedPubMedCentralGoogle Scholar
  38. Zhao, J., & Osherson, D. (2010). Updating beliefs in light of uncertain evidence: Descriptive assessment of Jeffrey’s rule. Thinking & Reasoning, 16(4), 288–307. doi:10.1080/13546783.2010.521695 CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2016

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of PittsburghPittsburghUSA

Personalised recommendations