Journal of General Internal Medicine

, Volume 20, Issue 4, pp 334–339 | Cite as

Do physicians know when their diagnoses are correct?

Implications for decision support and error reduction
  • Charles P. Friedman
  • Guido G. Gatti
  • Timothy M. Franz
  • Gwendolyn C. Murphy
  • Fredric M. Wolf
  • Paul S. Heckerling
  • Paul L. Fine
  • Thomas M. Miller
  • Arthur S. Elstein
Original Articles

Abstract

OBJECTIVE: This study explores the alignment between physicians’ confidence in their diagnoses and the “correctness” of these diagnoses, as a function of clinical experience, and whether subjects were prone to over-or underconfidence.

DESIGN: Prospective, counterbalanced experimental design.

SETTING: Laboratory study conducted under controlled conditions at three academic medical centers.

PARTICIPANTS: Seventy-two senior medical students, 72 senior medical residents, and 72 faculty internists.

INTERVENTION: We created highly detailed, 2-to 4-page synopses of 36 diagnostically challenging medical cases, each with a definitive correct diagnosis. Subjects generated a differential diagnosis for each of 9 assigned cases, and indicated their level of confidence in each diagnosis.

MEASUREMENTS AND MAIN RESULTS: A differential was considered “correct” if the clinically true diagnosis was listed in that subject’s hypothesis list. To assess confidence, subjects rated the likelihood that they would, at the time they generated the differential, seek assistance in reaching a diagnosis. Subjects’ confidence and correctness were “mildly” aligned (k=.314 for all subjects, .285 for faculty, .227 for residents, and .349 for students). Residents were overconfident in 41% of cases where their confidence and correctness were not aligned, whereas faculty were overconfident in 36% of such cases and students in 25%.

CONCLUSIONS: Even experienced clinicians may be unaware of the correctness of their diagnoses at the time they make them. Medical decision support systems, and other interventions designed to reduce medical errors, cannot rely exclusively on clinicians’ perceptions of their needs for such support.

Key words

diagnostic reasoning clinical decision support medical errors clinical judgment confidence 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hersh WR. “A world of knowledge at your fingertips”: the promise, reality, and future directions of online information retrieval. Acad Med. 1999;74:240–3.PubMedCrossRefGoogle Scholar
  2. 2.
    Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000.Google Scholar
  3. 3.
    Bates DW, Gawande AA. Error in medicine: what have we learned? Ann Intern Med. 2000;132:763–7.PubMedGoogle Scholar
  4. 4.
    Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274:35–43.PubMedCrossRefGoogle Scholar
  5. 5.
    Wyatt JC. Clinical data systems, part 3: development and evaluation. Lancet. 1994;344:1682–7.PubMedCrossRefGoogle Scholar
  6. 6.
    Norman DA. Melding mind and machine. Technol Rev. 1997;100:29–31.Google Scholar
  7. 7.
    Chueh H, Barnett GO. “Just in time” clinical information. Acad Med. 1997;72:512–7.PubMedCrossRefGoogle Scholar
  8. 8.
    Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes. JAMA. 1998;280:1339–46.PubMedCrossRefGoogle Scholar
  9. 9.
    Miller RA. Medical diagnostic decision support systems—past, present, and future. J Am Med Inform Assoc. 1994;1:8–27.PubMedGoogle Scholar
  10. 10.
    Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Eng J Med. 1998;338:232–8.CrossRefGoogle Scholar
  11. 11.
    McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Inform. 1999;54:225–53.PubMedCrossRefGoogle Scholar
  12. 12.
    Wagner MM, Pankaskie M, Hogan W, et al. Clinical event monitoring at the University of Pittsburgh. Proc AMIA Annu Fall Symp. 1997;188–92.Google Scholar
  13. 13.
    Cimino JJ, Elhanan G, Zeng Q. Supporting infobuttons with terminological knowledge. Proc AMIA Annu Fall Symp. 1997;528–32.Google Scholar
  14. 14.
    Miller PL. Building an expert critiquing system: ESSENTIAL-ATTENDING. Methods Inf Med. 1986;25:71–8.PubMedGoogle Scholar
  15. 15.
    Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science. 1974;185:1124–31.CrossRefGoogle Scholar
  16. 16.
    Lichtenstein S, Fischhoff B. Do those who know more also know more about how much they know? Organ Behav Hum Perform. 1977;20:159–83.CrossRefGoogle Scholar
  17. 17.
    Christensen-Szalanski JJ, Bushyhead JB. Physicians’ use of probabilistic information in a real clinical setting. J Exp Psychol. 1981;7:928–35.Google Scholar
  18. 18.
    Tierney WM, Fitzgerald J, McHenry R, et al. Physicians’ estimates of the probability of myocardial infarction in emergency room patients with chest pain. Med Decis Making. 1986;6:12–7.PubMedCrossRefGoogle Scholar
  19. 19.
    Friedman CP, Elstein AS, Wolf FM, et al. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. JAMA. 1999;282:1851–6.PubMedCrossRefGoogle Scholar
  20. 20.
    Mann D. The relationship between diagnostic accuracy and confidence in medical students. Presented at the annual meeting of the American Educational Research Association, Atlanta, 1993.Google Scholar
  21. 21.
    Friedman C, Gatti G, Elstein A, Franz T, Murphy G, Wolf F. Are Clinicians Correct When They Believe They Are Correct? Implications for Medical Decision Support. Proceedings of the Tenth World Congress on Medical Informatics. London; 2000. Medinfo. 2001;10(PP. 1):454–8.PubMedGoogle Scholar
  22. 22.
    Swets JA, Pickett RM. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York, NY: Academic Press; 1982.Google Scholar
  23. 23.
    McCullagh P, Nelder JA. Generalized Linear Models. 2nd ed. New York, NY: Chapman and Hall; 1991.Google Scholar
  24. 24.
    Liang K, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73:13–22.CrossRefGoogle Scholar
  25. 25.
    Neter J, Kutner MH, Nachstsheim CJ, Wasserman W. Applied Linear Regression Models. Chicago, IL: Irwin; 1996.Google Scholar
  26. 26.
    SAS Institute Inc. SAS/STAT User’s Guide, Version 8. Cary, NC: SAS Institute Inc.; 1999.Google Scholar
  27. 27.
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.PubMedCrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2005

Authors and Affiliations

  • Charles P. Friedman
    • 8
  • Guido G. Gatti
    • 8
  • Timothy M. Franz
    • 1
  • Gwendolyn C. Murphy
    • 2
  • Fredric M. Wolf
    • 3
  • Paul S. Heckerling
    • 4
  • Paul L. Fine
    • 6
  • Thomas M. Miller
    • 7
  • Arthur S. Elstein
    • 5
  1. 1.Department of PsychologySt. John Fisher CollegeRochesterUSA
  2. 2.Division of Community HealthDuke UniversityDurhamUSA
  3. 3.Department of Medical Education and InformaticsUniversity of WashingtonSeattleUSA
  4. 4.Department of MedicineUniversity of Illinois at ChicagoChicagoUSA
  5. 5.Department of Medical EducationUniversity of Illinois at ChicagoChicagoUSA
  6. 6.Department of MedicineUniversity of MichiganAnn ArborUSA
  7. 7.Department of MedicineUniversity of North CarolinaChapel HillUSA
  8. 8.Center for Biomedical InformaticsUniversity of PittsburghPittsburgh

Personalised recommendations