Abstract
There is an ongoing debate regarding the cause of diagnostic errors. One view is that errors result from unconscious application of cognitive heuristics; the alternative is that errors are a consequence of knowledge deficits. The objective of this study was to compare the effectiveness of checklists that (a) identify and address cognitive biases or (b) promote knowledge retrieval, as a means to reduce errors in ECG interpretation. Novice postgraduate year (PGY) 1 emergency medicine and internal medicine residents (n = 40) and experienced cardiology fellows (PGY 4–6) (n = 21) were randomly allocated to three conditions: a debiasing checklist, a content (knowledge) checklist, or control (no checklist) to be used while interpreting 20 ECGs. Half of the ECGs were deliberately engineered to predispose to bias. Diagnostic performance under either checklist intervention was not significantly better than the control. As expected, more errors occurred when cases were designed to induce bias (F = 96.9, p < 0.0001). There was no significant interaction between the instructional condition and level of learner. Checklists attempting to help learners identify cognitive bias or mobilize domain-specific knowledge did not have an overall effect in reducing diagnostic errors in ECG interpretation, although they may help novices. Even when cognitive biases are deliberately inserted in cases, cognitive debiasing checklists did not improve participants’ performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
26 March 2019
Due to an unfortunate turn of events, Fig. 3 was omitted from the original publication.
References
Berner, E. S., Maisiak, R. S., Cobbs, C. G., & Taunton, O. D. (1999). Effects of a decision support system on physicians’ diagnostic performance. Journal of the American Medical Informatics Association, 6(5), 420–427. https://doi.org/10.1136/jamia.1999.0060420.
Blumenthal-Barby, J. S., & Krieger, H. (2015). Cognitive biases and heuristics in medical decision making. Medical Decision Making, 35(4), 539–557. https://doi.org/10.1177/0272989X14547740.
Christensen, C., Heckerling, P. S., Mackesy, M. E., Bernstein, L. M., & Elstein, A. S. (1991). Framing bias among expert and novice physicians. Academic Medicine, 66(9), S76–S78. https://doi.org/10.1097/00001888-199109000-00047.
Croskerry, P. (2003a). Cognitive forcing strategies in clinical decisionmaking. Annals of Emergency Medicine, 41(1), 110–120. https://doi.org/10.1067/MEM.2003.22.
Croskerry, P. (2003b). The importance of cognitive errors in diagnosis and strategies to minimize them. Academic Medicine, 78(8). Retrieved from https://journals.lww.com/academicmedicine/Fulltext/2003/08000/The_Importance_of_Cognitive_Errors_in_Diagnosis.3.aspx
Croskerry, P. (2009). A universal model of diagnostic reasoning. Academic Medicine, 84(8), 1022–1028. https://doi.org/10.1097/ACM.0b013e3181ace703.
Croskerry, P., Singhal, G., & Mamede, S. (2013a). Cognitive debiasing 1: Origins of bias and theory of debiasing. BMJ Quality & Safety, 22(Suppl 2), ii58 LP–ii64. Retrieved from http://qualitysafety.bmj.com/content/22/Suppl_2/ii58.abstract
Croskerry, P., Singhal, G., & Mamede, S. (2013b). Cognitive debiasing 2: Impediments to and strategies for change. BMJ Quality & Safety, 22(Suppl 2), ii65 LP–ii72. Retrieved from http://qualitysafety.bmj.com/content/22/Suppl_2/ii65.abstract
Dhaliwal, G. (2017). Premature closure? Not so fast. BMJ Quality & Safety, 26(2), 87–89. https://doi.org/10.1136/bmjqs-2016-005267.
Donaldson, M. S., Corrigan, J. M., & Kohn, L. T. (2000). To err is human: Building a safer health system. Washington: National Academies Press.
ECG Made Simple. (2017). Stepwise videos. Retrieved from: http://www.ecgmadesimple.com/stepwise-vids/story_html5.html. Accessed 16 Sept 2017.
Elstein, A. S., Friedman, C. P., Wolf, F. M., Murphy, G., Miller, J., Fine, P., et al. (1996). Effects of a decision support system on the diagnostic accuracy of users: A preliminary report. Journal of the American Medical Informatics Association, 3(6), 422–428. https://doi.org/10.1136/jamia.1996.97084515.
Ely, J. W., Graber, M. L., & Croskerry, P. (2011). Checklists to reduce diagnostic errors. Academic Medicine, 86(3), 307–313. https://doi.org/10.1097/ACM.0b013e31820824cd.
Eva, K. W., & Regehr, G. (2011). Exploring the divergence between self-assessment and self-monitoring. Advances in Health Sciences Education, 16(3), 311–329. https://doi.org/10.1007/s10459-010-9263-2.
Friedman, C. P., Elstein, A. S., Wolf, F. M., Murphy, G. C., Franz, T. M., Heckerling, P. S., et al. (1999). Enhancement of clinicians’ diagnostic reasoning by computer-based consultation. JAMA, 282(19), 1851. https://doi.org/10.1001/jama.282.19.1851.
Graber, M. L., Kissam, S., Payne, V. L., Meyer, A. N. D., Sorensen, A., Lenfestey, N., et al. (2012). Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Quality & Safety, 21(7), 535 LP–557. Retrieved from http://qualitysafety.bmj.com/content/21/7/535.abstract.
Haenssle, H. A., et al. (2018). Man against machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Annals of Oncology, 29(8), 1836–1842.
Hatala, R., Norman, G. R., & Brooks, L. R. (1999). Impact of a clinical scenario on accuracy of electrocardiogram interpretation. Journal of General Internal Medicine, 14(2), 126–129. https://doi.org/10.1046/j.1525-1497.1999.00298.x.
Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A.-H. S., Dellinger, E. P., et al. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360(5), 491–499. https://doi.org/10.1056/NEJMsa0810119.
Kahneman, D., & Egan, P. (2011). Thinking, fast and slow. New York: Farrar, Straus, and Giroux.
Mamede, S., & Schmidt, H. G. (2014). Reflection in diagnostic reasoning. Academic Medicine, 89(7), 959–960. https://doi.org/10.1097/ACM.0000000000000306.
Mamede, S., Schmidt, H. G., & Penaforte, J. C. (2008). Effects of reflective practice on the accuracy of medical diagnoses. Medical Education, 42(5), 468–475. https://doi.org/10.1111/j.1365-2923.2008.03030.x.
Mamede, S., van Gog, T., van den Berge, K., Rikers, R. M. J. P., van Saase, J. L. C. M., van Guldener, C., et al. (2010). Effect of availability bias and reflective reasoning on diagnostic accuracy among internal medicine residents. JAMA, 304(11), 1198. https://doi.org/10.1001/jama.2010.1276.
Marsteller, J. A., Sexton, J. B., Hsu, Y.-J., Hsiao, C.-J., Holzmueller, C. G., Pronovost, P. J., et al. (2012). A multicenter, phased, cluster-randomized controlled trial to reduce central line-associated bloodstream infections in intensive care units*. Critical Care Medicine, 40(11), 2933–2939. https://doi.org/10.1097/CCM.0b013e31825fd4d8.
Monteiro, S. D., Sherbino, J., Patel, A., Mazzetti, I., Norman, G. R., & Howey, E. (2015). Reflecting on diagnostic errors: Taking a second look is not enough. Journal of General Internal Medicine, 30(9), 1270–1274. https://doi.org/10.1007/s11606-015-3369-4.
Norman, G. R., Brooks, L. R., Colle, C. L., & Hatala, R. M. (1999). The benefit of diagnostic hypotheses in clinical reasoning: Experimental study of an instructional intervention for forward and backward reasoning. Cognition and Instruction, 17(4), 433–448. https://doi.org/10.1207/S1532690XCI1704_3.
Norman, G. R., & Eva, K. W. (2009). Diagnostic error and clinical reasoning. Medical Education, 44(1), 94–100. https://doi.org/10.1111/j.1365-2923.2009.03507.x.
Obermeyer, Z., & Emanuel, E. J. (2016). Predicting the future—Big data, machine learning, and clinical medicine. The New England Journal of Medicine, 375(13), 1216.
Reilly, J. B., Ogdie, A. R., Von Feldt, J. M., & Myers, J. S. (2013). Teaching about how doctors think: A longitudinal curriculum in cognitive bias and diagnostic error for residents. BMJ Quality & Safety, 22(12), 1044 LP–1050. Retrieved from http://qualitysafety.bmj.com/content/22/12/1044.abstract
Sherbino, J., Dore, K. L., Siu, E., & Norman, G. R. (2011). The effectiveness of cognitive forcing strategies to decrease diagnostic error: An exploratory study. Teaching and Learning in Medicine, 23(1), 78–84. https://doi.org/10.1080/10401334.2011.536897.
Sherbino, J., Dore, K. L., Wood, T. J., Young, M. E., Gaissmaier, W., Kreuger, S., et al. (2012). The relationship between response time and diagnostic accuracy. Academic Medicine, 87(6), 785–791. https://doi.org/10.1097/ACM.0b013e318253acbd.
Sherbino, J., Kulasegaram, K., Howey, E., & Norman, G. (2014). Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reasoning: A controlled trial. CJEM, 16(01), 34–40. https://doi.org/10.2310/8000.2013.130860.
Shimizu, T., Matsumoto, K., & Tokuda, Y. (2013). Effects of the use of differential diagnosis checklist and general de-biasing checklist on diagnostic performance in comparison to intuitive diagnosis. Medical Teacher, 35(6), e1218–e1229. https://doi.org/10.3109/0142159X.2012.742493.
Sibbald, M., de Bruin, A. B. H., & van Merrienboer, J. J. G. (2013). Checklists improve experts’ diagnostic decisions. Medical Education, 47(3), 301–308. https://doi.org/10.1111/medu.12080.
Sibbald, M., De Bruin, A. B. H., & van Merrienboer, J. J. G. (2014). Finding and fixing mistakes: Do checklists work for clinicians with different levels of experience? Advances in Health Sciences Education, 19(1), 43–51. https://doi.org/10.1007/s10459-013-9459-3.
Zwaan, L., Monteiro, S., Sherbino, J., Ilgen, J., Howey, B., & Norman, G. (2017). Is bias in the eye of the beholder? A vignette study to assess recognition of cognitive biases in clinical case workups. BMJ Quality & Safety, 26(2), 104 LP–110. Retrieved from http://qualitysafety.bmj.com/content/26/2/104.abstract.
Acknowledgements
The authors wish to thank Betty Howey for her technical support and facilitation of the study.
Funding
This study was supported by a Medical Education Grant from the Royal College of Physicians and Surgeons of Canada.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This study was approved by the Hamilton Integrated Ethics Board September 2017, #3856.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sibbald, M., Sherbino, J., Ilgen, J.S. et al. Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation. Adv in Health Sci Educ 24, 427–440 (2019). https://doi.org/10.1007/s10459-019-09875-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-019-09875-8