While multiple theories exist to explain the diagnostic process, there are few available assessments that reliably determine diagnostic competence in trainees. Most methods focus on aspects of the process of diagnostic reasoning, such as the relation between case features and diagnostic hypotheses. Inevitably, detailed elucidation of aspects of the process requires substantial time per case and limits the number of cases that can be examined given a limited testing time. Shifting assessment to the outcome of diagnostic reasoning, accuracy of the diagnosis, may serve as a reliable measure of diagnostic competence and would allow increased sampling across cases. The present study is a retrospective analysis of 7 large studies, conducted by 3 research teams, that all used a series of brief written cases to examine the outcome of diagnostic reasoning—the diagnosis. The studies involved over 600 clinicians ranging from final year medical students to practicing emergency physicians. For 4 studies with usable reliability data, reliability for a 2 h test ranged from .63 to .94. On average speeded tests were more reliable (.85 vs. .73).To achieve a reliability of .75 required an average test time of 1.11 h for speeded tests and 1.99 for unspeeded tests. The measure was shown to be positively correlated with both written knowledge tests and measures of problem solving derived from OSCE performance tests. This retrospective analysis provides evidence to support the implementation of outcome-based assessments of clinical reasoning.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Al Qahtani, D. A., Rotgans, J. I., Mamede, S., ALAlwan, I., Magzoub, M. E. M., Altayeb, F. M., et al. (2016). Does time pressure have a negative effect on diagnostic accuracy? Academic Medicine,91(5), 710–716.
Barrows, H. S., Norman, G. R., Neufeld, V. R., & Feightner, J. W. (1982). The clinical reasoning of randomly selected physicians in general medical practice. Clinical and Investigative Medicine. Medecine Clinique et Experimentale,5(1), 49–55.
Burns, B. D. (2004). The effects of speed on skilled chess performance. Psychological Science,15(7), 442–447.
Charlin, B., Brailovsky, C., Leduc, C., & Blouin, D. (1998). The diagnosis script questionnaire: A new tool to assess a specific dimension of clinical competence. Advances in Health Sciences Education,3(1), 51–58.
Cook, D. A., Brydges, R., Ginsburg, S., & Hatala, R. (2015). A contemporary approach to validity arguments: A practical guide to Kane’s framework. Medical Education,49(6), 560–575.
Elstein, A. S., Shulman, L. S., & Sprafka, S. A. (1978). Medical problem-solving: An analysis of clinical reasoning. Cambridge: Harvard University Press.
Gagnon, R., Charlin, B., Lambert, C., Carrière, B., & Van der Vleuten, C. (2009). Script concordance testing: More cases or more questions? Advances in Health Sciences Education,14(3), 367–375.
Groves, M., O’Rourke, P., & Alexander, H. (2009). Clinical reasoning: The relative contribution of identification, interpretation and hypothesis errors to misdiagnosis. Medical Teacher,25(6), 621–625.
Gruppen, L. D., Wolf, F. M., Voorhees, C. Van, & Stross, J. K. (1988). The influence of general and case-related experience on primary care treatment decision making. Archives of Internal Medicine,148(12), 2657.
Hodges, B., Regehr, G., McNaughton, N., Tiberius, R., & Hanson, M. (1999). OSCE checklists do not capture increasing levels of expertise. Academic Medicine: Journal of the Association of American Medical Colleges,74(10), 1129–1134.
Ilgen, J. S., Bowen, J. L., McIntyre, L. A., Banh, K. V., Barnes, D., Coates, W. C., et al. (2013). Comparing diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytic thought. Academic Medicine,88(10), 1545–1551.
Ilgen, J. S., Eva, K. W., & Regehr, G. (2016). What’s in a label? Is diagnosis the start or the end of clinical reasoning? Journal of General Internal Medicine,31(4), 435–437.
Lineberry, M., Kreiter, C. D., & Bordage, G. (2013). Threats to validity in the use and interpretation of script concordance test scores. Medical Education,47(12), 1175–1183.
Lubarsky, S., Charlin, B., Cook, D. A., Chalk, C., & van der Vleuten, C. P. M. (2011). Script concordance testing: A review of published validity evidence. Medical Education,45(4), 329–338.
Mamede, S., Schmidt, H. G., Rikers, R. M., Penaforte, J. C., & Coelho-Filho, J. (2007). Breaking down automaticity: Case ambiguity and the shift to reflective approaches in clinical reasoning. Medical Education,41, 1185–1192.
McConnell, M. M., Regehr, G., Wood, T. J., & Eva, K. W. (2012). Self-monitoring and its relationship to medical knowledge. Advances in Health Sciences Education,17(3), 311–323.
Monteiro, S. D., Sherbino, J. D., Ilgen, J. S., Dore, K. L., Wood, T. J., Young, M. E., et al. (2015a). Disrupting diagnostic reasoning. Academic Medicine,90(4), 511–517.
Monteiro, S. D., Sherbino, J., Patel, A., Mazzetti, I., Norman, G. R., & Howey, E. (2015b). Reflecting on diagnostic errors: Taking a second look is not enough. Journal of General Internal Medicine,30(9), 1270–1274.
Norcini, J. J. (2002). The death of the long case? British Medical Journal,324, 408–409.
Norman, G., Sherbino, J., Dore, K., Wood, T., Young, M., Gaissmaier, W., et al. (2014). The etiology of diagnostic errors. Academic Medicine,89(2), 277–284.
Schmidt, H. G., & Mamede, S. (2015). How to improve the teaching of clinical reasoning: A narrative review and a proposal. Medical Education,49(10), 961–973.
Sherbino, J., Dore, K. L., Wood, T. J., Young, M. E., Gaissmaier, W., Kreuger, S., et al. (2012). The relationship between response time and diagnostic accuracy. Academic Medicine,87(6), 785–791.
Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measurement scales. Oxford: Oxford University Press.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Monteiro, S.D., Sherbino, J., Schmidt, H. et al. It’s the destination: diagnostic accuracy and reasoning. Adv in Health Sci Educ 25, 19–29 (2020). https://doi.org/10.1007/s10459-019-09903-7
- Diagnostic reasoning
- Written cases