Do physicians know when their diagnoses are correct?
- 162 Downloads
OBJECTIVE: This study explores the alignment between physicians’ confidence in their diagnoses and the “correctness” of these diagnoses, as a function of clinical experience, and whether subjects were prone to over-or underconfidence.
DESIGN: Prospective, counterbalanced experimental design.
SETTING: Laboratory study conducted under controlled conditions at three academic medical centers.
PARTICIPANTS: Seventy-two senior medical students, 72 senior medical residents, and 72 faculty internists.
INTERVENTION: We created highly detailed, 2-to 4-page synopses of 36 diagnostically challenging medical cases, each with a definitive correct diagnosis. Subjects generated a differential diagnosis for each of 9 assigned cases, and indicated their level of confidence in each diagnosis.
MEASUREMENTS AND MAIN RESULTS: A differential was considered “correct” if the clinically true diagnosis was listed in that subject’s hypothesis list. To assess confidence, subjects rated the likelihood that they would, at the time they generated the differential, seek assistance in reaching a diagnosis. Subjects’ confidence and correctness were “mildly” aligned (k=.314 for all subjects, .285 for faculty, .227 for residents, and .349 for students). Residents were overconfident in 41% of cases where their confidence and correctness were not aligned, whereas faculty were overconfident in 36% of such cases and students in 25%.
CONCLUSIONS: Even experienced clinicians may be unaware of the correctness of their diagnoses at the time they make them. Medical decision support systems, and other interventions designed to reduce medical errors, cannot rely exclusively on clinicians’ perceptions of their needs for such support.
Key wordsdiagnostic reasoning clinical decision support medical errors clinical judgment confidence
Unable to display preview. Download preview PDF.
- 2.Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000.Google Scholar
- 6.Norman DA. Melding mind and machine. Technol Rev. 1997;100:29–31.Google Scholar
- 12.Wagner MM, Pankaskie M, Hogan W, et al. Clinical event monitoring at the University of Pittsburgh. Proc AMIA Annu Fall Symp. 1997;188–92.Google Scholar
- 13.Cimino JJ, Elhanan G, Zeng Q. Supporting infobuttons with terminological knowledge. Proc AMIA Annu Fall Symp. 1997;528–32.Google Scholar
- 17.Christensen-Szalanski JJ, Bushyhead JB. Physicians’ use of probabilistic information in a real clinical setting. J Exp Psychol. 1981;7:928–35.Google Scholar
- 20.Mann D. The relationship between diagnostic accuracy and confidence in medical students. Presented at the annual meeting of the American Educational Research Association, Atlanta, 1993.Google Scholar
- 22.Swets JA, Pickett RM. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York, NY: Academic Press; 1982.Google Scholar
- 23.McCullagh P, Nelder JA. Generalized Linear Models. 2nd ed. New York, NY: Chapman and Hall; 1991.Google Scholar
- 25.Neter J, Kutner MH, Nachstsheim CJ, Wasserman W. Applied Linear Regression Models. Chicago, IL: Irwin; 1996.Google Scholar
- 26.SAS Institute Inc. SAS/STAT User’s Guide, Version 8. Cary, NC: SAS Institute Inc.; 1999.Google Scholar