Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

It’s the destination: diagnostic accuracy and reasoning

Abstract

While multiple theories exist to explain the diagnostic process, there are few available assessments that reliably determine diagnostic competence in trainees. Most methods focus on aspects of the process of diagnostic reasoning, such as the relation between case features and diagnostic hypotheses. Inevitably, detailed elucidation of aspects of the process requires substantial time per case and limits the number of cases that can be examined given a limited testing time. Shifting assessment to the outcome of diagnostic reasoning, accuracy of the diagnosis, may serve as a reliable measure of diagnostic competence and would allow increased sampling across cases. The present study is a retrospective analysis of 7 large studies, conducted by 3 research teams, that all used a series of brief written cases to examine the outcome of diagnostic reasoning—the diagnosis. The studies involved over 600 clinicians ranging from final year medical students to practicing emergency physicians. For 4 studies with usable reliability data, reliability for a 2 h test ranged from .63 to .94. On average speeded tests were more reliable (.85 vs. .73).To achieve a reliability of .75 required an average test time of 1.11 h for speeded tests and 1.99 for unspeeded tests. The measure was shown to be positively correlated with both written knowledge tests and measures of problem solving derived from OSCE performance tests. This retrospective analysis provides evidence to support the implementation of outcome-based assessments of clinical reasoning.

This is a preview of subscription content, log in to check access.

References

  1. Al Qahtani, D. A., Rotgans, J. I., Mamede, S., ALAlwan, I., Magzoub, M. E. M., Altayeb, F. M., et al. (2016). Does time pressure have a negative effect on diagnostic accuracy? Academic Medicine,91(5), 710–716.

  2. Barrows, H. S., Norman, G. R., Neufeld, V. R., & Feightner, J. W. (1982). The clinical reasoning of randomly selected physicians in general medical practice. Clinical and Investigative Medicine. Medecine Clinique et Experimentale,5(1), 49–55.

  3. Burns, B. D. (2004). The effects of speed on skilled chess performance. Psychological Science,15(7), 442–447.

  4. Charlin, B., Brailovsky, C., Leduc, C., & Blouin, D. (1998). The diagnosis script questionnaire: A new tool to assess a specific dimension of clinical competence. Advances in Health Sciences Education,3(1), 51–58.

  5. Cook, D. A., Brydges, R., Ginsburg, S., & Hatala, R. (2015). A contemporary approach to validity arguments: A practical guide to Kane’s framework. Medical Education,49(6), 560–575.

  6. Elstein, A. S., Shulman, L. S., & Sprafka, S. A. (1978). Medical problem-solving: An analysis of clinical reasoning. Cambridge: Harvard University Press.

  7. Gagnon, R., Charlin, B., Lambert, C., Carrière, B., & Van der Vleuten, C. (2009). Script concordance testing: More cases or more questions? Advances in Health Sciences Education,14(3), 367–375.

  8. Groves, M., O’Rourke, P., & Alexander, H. (2009). Clinical reasoning: The relative contribution of identification, interpretation and hypothesis errors to misdiagnosis. Medical Teacher,25(6), 621–625.

  9. Gruppen, L. D., Wolf, F. M., Voorhees, C. Van, & Stross, J. K. (1988). The influence of general and case-related experience on primary care treatment decision making. Archives of Internal Medicine,148(12), 2657.

  10. Hodges, B., Regehr, G., McNaughton, N., Tiberius, R., & Hanson, M. (1999). OSCE checklists do not capture increasing levels of expertise. Academic Medicine: Journal of the Association of American Medical Colleges,74(10), 1129–1134.

  11. Ilgen, J. S., Bowen, J. L., McIntyre, L. A., Banh, K. V., Barnes, D., Coates, W. C., et al. (2013). Comparing diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytic thought. Academic Medicine,88(10), 1545–1551.

  12. Ilgen, J. S., Eva, K. W., & Regehr, G. (2016). What’s in a label? Is diagnosis the start or the end of clinical reasoning? Journal of General Internal Medicine,31(4), 435–437.

  13. Lineberry, M., Kreiter, C. D., & Bordage, G. (2013). Threats to validity in the use and interpretation of script concordance test scores. Medical Education,47(12), 1175–1183.

  14. Lubarsky, S., Charlin, B., Cook, D. A., Chalk, C., & van der Vleuten, C. P. M. (2011). Script concordance testing: A review of published validity evidence. Medical Education,45(4), 329–338.

  15. Mamede, S., Schmidt, H. G., Rikers, R. M., Penaforte, J. C., & Coelho-Filho, J. (2007). Breaking down automaticity: Case ambiguity and the shift to reflective approaches in clinical reasoning. Medical Education,41, 1185–1192.

  16. McConnell, M. M., Regehr, G., Wood, T. J., & Eva, K. W. (2012). Self-monitoring and its relationship to medical knowledge. Advances in Health Sciences Education,17(3), 311–323.

  17. Monteiro, S. D., Sherbino, J. D., Ilgen, J. S., Dore, K. L., Wood, T. J., Young, M. E., et al. (2015a). Disrupting diagnostic reasoning. Academic Medicine,90(4), 511–517.

  18. Monteiro, S. D., Sherbino, J., Patel, A., Mazzetti, I., Norman, G. R., & Howey, E. (2015b). Reflecting on diagnostic errors: Taking a second look is not enough. Journal of General Internal Medicine,30(9), 1270–1274.

  19. Norcini, J. J. (2002). The death of the long case? British Medical Journal,324, 408–409.

  20. Norman, G., Sherbino, J., Dore, K., Wood, T., Young, M., Gaissmaier, W., et al. (2014). The etiology of diagnostic errors. Academic Medicine,89(2), 277–284.

  21. Schmidt, H. G., & Mamede, S. (2015). How to improve the teaching of clinical reasoning: A narrative review and a proposal. Medical Education,49(10), 961–973.

  22. Sherbino, J., Dore, K. L., Wood, T. J., Young, M. E., Gaissmaier, W., Kreuger, S., et al. (2012). The relationship between response time and diagnostic accuracy. Academic Medicine,87(6), 785–791.

  23. Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measurement scales. Oxford: Oxford University Press.

Download references

Author information

Correspondence to Sandra D. Monteiro.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Monteiro, S.D., Sherbino, J., Schmidt, H. et al. It’s the destination: diagnostic accuracy and reasoning. Adv in Health Sci Educ 25, 19–29 (2020). https://doi.org/10.1007/s10459-019-09903-7

Download citation

Keywords

  • Diagnostic reasoning
  • Assessment
  • Reliability
  • Written cases