Advances in Health Sciences Education

, Volume 14, Supplement 1, pp 107–112

Next steps: envisioning a research agenda


    • VA Medical Center
    • Department of MedicineSUNY at Stony Brook

DOI: 10.1007/s10459-009-9183-1

Cite this article as:
Graber, M.L. Adv in Health Sci Educ (2009) 14: 107. doi:10.1007/s10459-009-9183-1


The topic of diagnostic error is a relatively new one in the academic arena and lacks an organized research agenda. Participants at “Diagnostic Error in Medicine— 2008” formally considered this issue and provided initial suggestions. Recommendations were made to standardize taxonomies and definitions, especially in regard to what constitutes a delay in diagnosis. Error discovery needs emphasis, especially with autopsies becoming a rarity. Developing standardized tools to study diagnostic error in laboratory-like conditions was viewed as a top priority. Many issues were reviewed in regard to error generation (what conditions are error-prone, what is the role of the clinicians expertise, etc.), and error reduction. These included both system-level and cognitive interventions, with specific suggestions for each dimension.


Health care qualityPatient safetyDiagnostic errorResearch


Mature areas of healthcare research (e.g. preventive medicine, quality improvement) draw upon a rich understanding of the field, derived from decades of research and scholarly consideration. At the other end of the spectrum lies the field of ‘diagnostic error’. The number of active investigators or funded projects addressing this problem can be counted on one or both hands and the studies that do exist are largely observational. Diagnostic error is just emerging as its own topic of interest within the larger field of patient safety. Given this immature state of affairs, one goal of this conference was to begin envisioning a research agenda that would advance the understanding of diagnostic error and how to minimize it. Investigators and funding agencies need to know what gaps exist, what areas need the most urgent attention, and where to direct energy and attention to move this field forward in an orderly manner.

To initiate such a discussion, conference participants attended two out of five different workshops where diagnostic error was discussed from the perspective of its site of origin—the Emergency Department, outpatient clinics, the clinical lab, Radiology, or Anatomic Pathology. After reviewing typical examples of diagnostic errors in this setting, participants were asked to brainstorm what ‘next steps’ should be taken to reduce the likelihood of error in terms of education, informatics interventions, cognitive interventions, patient-based solutions, and research needs. This report reviews the suggestions that focused specifically on research.


Participants recognized a need to standardize definitions used in the study of diagnostic error. What IS an error, exactly? A definition has been proposed by the Institute of Medicine based on James Reason’s original concepts of human error: “An error is defined as the failure of a planned action to be completed as intended (i.e. error of execution) or the use of a wrong plan to achieve an aim (i.e. error of planning)” (Reason 1990; Kohn et al. 1999). The group recognized this definition to be unwieldy and essentially unusable in a research setting, and the recommendation was made to use an operational definition, based on the original classification of the Australian Patient Safety Foundation which has now been applied in several recent studies: A diagnostic error is one where the diagnosis is unintentionally delayed (sufficient information was available earlier), wrong (another diagnosis was made before the correct one), or missed (no diagnosis ever made), as judged from the eventual appreciation of more definitive information (Graber et al. 2005).

Standardization would be especially helpful in defining delays in diagnosis—what is an acceptable timeframe for making a given diagnosis, and how does one account for the inevitable timeline over which information and test results needed for the correct diagnosis become available? Taxonomies of diagnostic error need attention as well—several have been proposed and no one standard has yet emerged (Graber et al. 2005; Pronovost et al. 2005; Battles and Shea 2001).

Error discovery

Error discovery was discussed as a weak link. How can clinicians become more aware of errors, especially their own? Autopsy is the gold standard for uncovering discrepancies between the clinical diagnosis and reality, but autopsies are vanishing from the medical scene and most trainees will graduate their residency programs without ever witnessing a post-mortem exam except on television. What can replace autopsies as a way to discover diagnostic errors? How can healthcare organizations find out about errors made on the wards, in the emergency department, in clinics, or during the diagnostic testing process? Can patients play a role in this endeavor, and how do we empower them to report errors? If we establish a system to follow-up with patients given a tentative diagnosis, when should this be done, who should do it, and will this erode or promote confidence in the healthcare system? (Wears and Schiff 2005) How can diagnostic error be incorporated into the occurrence screens that healthcare organizations use to identify quality lapses?

Diagnostic ‘tags’

Diagnostic ‘tags’ were discussed as a potentially useful tool. Anyone proposing a new diagnosis would tag it, identifying the diagnosis as a new one, along with information on who made the diagnosis and the degree of confidence or uncertainty associated with the diagnosis. Perhaps the confidence level could be estimated by referencing the actual evidence used in making the diagnosis. These tags would facilitate the study of diagnostic error by making it easier to identify any change in the diagnosis and by providing a way to establish feedback to the original diagnosticians.

Research tools

Access to laboratory-like settings for the study of diagnostic error was identified as a key research need. Studies using standardized patients, standardized case scenarios, and simulations illustrate the power of this approach. Further development of these tools and improving access to research investigators would enhance the quality of error research and permit hypothesis-driven approaches.

Error generation

The participants acknowledged the many gaps that exist in our understanding of how diagnostic errors arise. What practice conditions are most error-prone? Are there certain providers who are error prone? If so, how do we identify them, and can they be reformed or improved?

Access to experts

An interesting discussion centered on access to expertise as a factor in error generation. Are patients better off having quick and easy access to expert subspecialists, or is the primary care model better, where generalists deal with common issues and just refer the mysteries? Does the error rate change if physician extenders act as front-line diagnosticians? Do ‘curbside’ consults prevent or promote diagnostic error? Should we encourage patients to use the internet for self diagnosis? Do online resources lead to earlier diagnosis, or delays?


Substantial work remains to be done regarding the epidemiology of diagnostic errors. Although general types of error have been identified, it is not known how many errors are actually committed in practice, and it is unclear whether the estimates derived from autopsy studies and retrospective reviews are accurate. What is the actual prevalence of diagnostic error in practice? Perhaps the more relevant concern is: How often do initial diagnostic errors matter? Many conditions resolve on their own or an error has no impact for other reasons. Prospective studies in different settings are needed to more clearly define the ‘numerators’ and ‘denominators’ regarding diagnostic errors. A major hypothesis that could be clarified with such data is that we make relatively more mistakes diagnosing what we perceive as ‘simple’ problems (because automatic processes solve the problem without much cognitive monitoring) than we do with difficult cases, where rational cognition and comprehensive searching will more likely come into play from the start.

Cognitive issues

The cognitive underpinnings of clinical reasoning need more study. In particular, we do not know the best pathway to expertise. We understand that competency is largely context-specific, but we do not know how many examples are needed to develop competency, or how many variants of the prototypical example must be presented. Is it sufficient to read about these variants or must they be experienced?

The interplay between “System 1” (subconscious cognition) and “System 2” (conscious, deliberate review) also requires much more investigation. There are times when the urgent diagnosis is critical, but these instances are rare. We almost always have time for at least some reflection, but typically fail to invoke this powerful monitoring tool. How can we train doctors in the art of reflection and get them to use it routinely? (Singh et al. 2006; Mamede et al. 2007)


Overconfidence is recognized as a major factor contributing to diagnostic error, perhaps the dominant one (Berner and Graber 2008). How does this arise and are some clinicians more susceptible to this than others? In the dual-processing model, why is it that some problems solved with automatic, subconscious cognition go unchecked whereas others evoke some hint of a problem, at which point conscious monitoring can help identify any shortcomings and lead to better solutions?

Error reduction

Global issues

Medicine has never set a formal goal for reducing diagnostic error. Some errors are inevitable, but can we start to formulate a target, reducing errors by “x” percent over the next decade? Interventions exist that have the potential to reduce diagnostic errors in medicine, but we do not know which ones will work the best, or at all. The potential roles of decision support and enhanced access to expertise need much more study in terms of their impact on reducing error. Involving the patient was viewed as a prime strategy to help reduce errors, and this approach is ripe for clinical investigation.

System-level interventions

The current assumption is that we can reduce system-related error by attention to latent organizational flaws, but validated demonstrations are lacking. To improve the reliability of diagnosis, how should we optimize the physical setting in which diagnoses are made? Can we reduce distractions? Can we provide better information on test characteristics? Do we have the best resources in terms of expertise and decision support tools? How do you measure the aspects of safety culture that relate to diagnostic reliability and how do we optimize these?

Cognitive interventions

The more intriguing question is whether we can reduce cognitive errors. Can we somehow overcome overconfidence? The pathway to improved calibration involves focused, timely and relevant feedback—but how much feedback, when, by whom, and how do you do this? The central question in the cognitive domain is simple: We know that recognition-primed decision making is highly effective but potentially flawed. Will efforts to promote conscious, comprehensive review lead to fewer errors, or more?


Constructing an agenda for education to reduce diagnostic error is its own topic, but this area is also ripe for research. Assuming that educational interventions will be used extensively to help teach clinical reasoning and the pitfalls to avoid, many questions remain. Is there some better way to teach clinical reasoning? When should it be presented in the curriculum, and how should progressive sophistication in the clinical reasoning process be presented as students gain progressive bioscience competency? Which approach is more effective in reducing error: Presenting case examples with ever-more detail and complexity in an effort to enhance content-specific knowledge, or teaching students the inherent shortcomings of the various heuristics used in medical diagnosis and how to avoid these? (Croskerry 2003)

The costs

Interventions to reduce error will also have a cost. Even if it is simply the resources used for implementing the change, these resources will probably have been diverted from some other healthcare area. Unanticipated costs of safety interventions need to be more clearly defined. For example, if we exhort clinicians to be more comprehensive before deciding on a diagnosis, to what extent will this quest for certainty promote unnecessary testing, cascade effects, and the costs of ‘defensive’ medicine? (Hofer and Hayward 2002) We need to identify the point of equipoise where the costs of further pursuit are outweighed by the costs and false leads that would otherwise be investigated.

Caveats and conclusion

These discussions represent just a start in discussing our current gaps in understanding and reducing diagnostic error. The ideas presented are in no way represented as being complete. Additional discussion and introspection is needed to construct a comprehensive list of all the issues that need consideration, and to prioritize these. Nonetheless, it is somewhat remarkable that this endeavor has not been addressed before in the history of medicine, and this first attempt at constructing a research agenda therefore represents an overdue but important beginning.

Copyright information

© Springer Science+Business Media B.V. 2009