Does Prevalence Matter to Physicians in Estimating Post-test Probability of Disease? A Randomized Trial
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
The probability of a disease following a diagnostic test depends on the sensitivity and specificity of the test, but also on the prevalence of the disease in the population of interest (or pre-test probability). How physicians use this information is not well known.
To assess whether physicians correctly estimate post-test probability according to various levels of prevalence and explore this skill across respondent groups.
Population-based sample of 1,361 physicians of all clinical specialties.
We described a scenario of a highly accurate screening test (sensitivity 99% and specificity 99%) in which we randomly manipulated the prevalence of the disease (1%, 2%, 10%, 25%, 95%, or no information).
We asked physicians to estimate the probability of disease following a positive test (categorized as <60%, 60–79%, 80–94%, 95–99.9%, and >99.9%). Each answer was correct for a different version of the scenario, and no answer was possible in the “no information” scenario. We estimated the proportion of physicians proficient in assessing post-test probability as the proportion of correct answers beyond the distribution of answers attributable to guessing.
Most respondents in each of the six groups (67%–82%) selected a post-test probability of 95–99.9%, regardless of the prevalence of disease and even when no information on prevalence was provided. This answer was correct only for a prevalence of 25%. We estimated that 9.1% (95% CI 6.0–14.0) of respondents knew how to assess correctly the post-test probability. This proportion did not vary with clinical experience or practice setting.
Most physicians do not take into account the prevalence of disease when interpreting a positive test result. This may cause unnecessary testing and diagnostic errors.
- Phillips, B, Westwood, M (2009) Testing our understanding of tests. Arch Dis Child 94: pp. 178-179 CrossRef
- Ghosh, AK, Ghosh, K, Erwin, PJ (2004) Do medical students and physicians understand probability?. QJM 97: pp. 53-55 CrossRef
- Reid, MC, Lane, DA, Feinstein, AR (1998) Academic calculations versus clinical judgments: practicing physicians’ use of quantitative measures of test accuracy. Am J Med 104: pp. 374-380 CrossRef
- Richardson, WS (2007) We should overcome the barriers to evidence-based clinical diagnosis!. J Clin Epidemiol 60: pp. 217-227 CrossRef
- Lyman, GH, Balducci, L (1994) The effect of changing disease risk on clinical reasoning. J Gen Intern Med 9: pp. 488-495 CrossRef
- Puhan, MA, Steurer, J, Bachmann, LM, Riet, G (2005) A randomized trial of ways to describe test accuracy: the effect on physicians’ post-test probability estimates. Ann Intern Med 143: pp. 184-189
- Sox, CM, Doctor, JN, Koepsell, TD, Christakis, DA (2009) The influence of types of decision support on physicians’ decision making. Arch Dis Child 94: pp. 185-190 CrossRef
- Steurer, J, Fischer, JE, Bachmann, LM, Koller, M, Riet, G (2002) Communicating accuracy of tests to general practitioners: a controlled study. BMJ 324: pp. 824-826 CrossRef
- Attia, JR, Nair, BR, Sibbritt, DW (2004) Generating pre-test probabilities: a neglected area in clinical decision making. Med J Aust 180: pp. 449-454
- Richardson, WS (2002) Five uneasy pieces about pre-test probability. J Gen Intern Med 17: pp. 882-883 CrossRef
- Hoffrage, U, Gigerenzer, G (1998) Using natural frequencies to improve diagnostic inferences. Acad Med 73: pp. 538-540 CrossRef
- Young, JM, Glasziou, P, Ward, JE (2002) General practitioners’ self ratings of skills in evidence based medicine: validation study. BMJ 324: pp. 950-951 CrossRef
- Mosteller F, Tukey J. Data analysis and regression, a second course in statistics: Addison-Wesley publishing company; 1977.
- Efron B, Tibshirani R. An introduction to the Bootstrap: Chapman & Hall; 1993.
- R Development Core Team. R: A language and environment for statistical computing. http://www.R-project.org (Accessed on 3 October 2010). R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. 2008.
- Schwartz, WB, Gorry, GA, Kassirer, JP, Essig, A (1973) Decision analysis and clinical judgment. Am J Med 55: pp. 459-472 CrossRef
- Bianchi, MT, Alexander, BM (2006) Evidence based diagnosis: does the language reflect the theory?. BMJ 333: pp. 442-445 CrossRef
- Kurzenhauser, S, Hoffrage, U (2002) Teaching Bayesian reasoning: an evaluation of a classroom tutorial for medical students. Med Teach 24: pp. 516-521 CrossRef
- Fagan, TJ (1975) Letter: nomogram for Bayes’ theorem. N Engl J Med 293: pp. 257
- Noguchi, Y, Matsui, K, Imura, H, Kiyota, M, Fukui, T (2004) A traditionally administered short course failed to improve medical students’ diagnostic performance. A quantitative evaluation diagnostic thinking. J Gen Intern Med 19: pp. 427-432 CrossRef
- Gill, CJ, Sabin, L, Schmid, CH (2005) Why clinicians are natural Bayesians. BMJ 330: pp. 1080-1083 CrossRef
- Kahneman, D, Slovic, P, Tversky, A eds. (1982) Judgment under uncertainty: heuristics and biases. Cambridge University Press, Cambridge, UK
- Graber, M, Gordon, R, Franklin, N (2002) Reducing diagnostic errors in medicine: what’s the goal?. Acad Med 77: pp. 981-992 CrossRef
- Berner, ES, Graber, ML (2008) Overconfidence as a cause of diagnostic error in medicine. Am J Med 121: pp. S2-S23 CrossRef
- Norman GR, Eva KW. Diagnostic error and clinical reasoning. Med Educ. Jan;44(1):94-100.
- Croskerry, P (2003) The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med 78: pp. 775-780 CrossRef
- Kassirer, JP, Kopelman, RI (1989) Cognitive errors in diagnosis: instantiation, classification, and consequences. Am J Med 86: pp. 433-441 CrossRef
- Grijalva, CG, Poehling, KA, Edwards, KM (2007) Accuracy and interpretation of rapid influenza tests in children. Pediatrics 119: pp. e6-e11 CrossRef
- Veloski, J, Tai, S, Evans, AS, Nash, DB (2005) Clinical vignette-based surveys: a tool for assessing physician practice variation. Am J Med Qual 20: pp. 151-157 CrossRef
- Asch, DA, Jedrziewski, MK, Christakis, NA (1997) Response rates to mail surveys published in medical journals. J Clin Epidemiol 50: pp. 1129-1136 CrossRef
- Whiting, PF, Sterne, JA, Westwood, ME (2008) Graphical presentation of diagnostic information. BMC Med Res Methodol 8: pp. 20 CrossRef
- Ende, J, Bisoffi, Z, Puymbroek, H (2007) Bridging the gap between clinical practice and diagnostic clinical epidemiology: pilot experiences with a didactic model based on a logarithmic scale. J Eval Clin Pract 13: pp. 374-380 CrossRef
- Does Prevalence Matter to Physicians in Estimating Post-test Probability of Disease? A Randomized Trial
Journal of General Internal Medicine
Volume 26, Issue 4 , pp 373-378
- Cover Date
- Print ISSN
- Online ISSN
- Additional Links
- Bayes’ theorem
- predictive value of tests
- sensitivity and specificity
- risk assessment
- evidence-based medicine
- Industry Sectors