ARE MEDICAL ERROR DEATH STATISTICS A “MEME”?

What do articles in leading medical journals like The Lancet,1 the Journal of the American College of Cardiology,2JAMA Internal Medicine,3 and Nature Reviews Neurology4 have in common with popular television shows5 and documentaries,6 newspaper stories,7 textbooks,8 and politicians9? Recently, they have all uncritically reported that preventable medical errors lead to hundreds of thousands of annual patient deaths in the USA or represent its third leading cause of death.

Two claims about medical error deaths are commonly disseminated. First, that preventable medical errors cause up to 440,000 deaths annually, which originated from a 2013 manuscript by John James,10 and second, medical error is the third leading cause of death in the USA, touted in a commentary from 2016 by Martin Makary and Michael Daniel,11 which estimated preventable deaths at 251,454 per year.

The examples above are anecdotal, but by more objective metrics these statistics continue to spread throughout the academic and lay media. At the time of this writing, the James paper had been cited 407 times and the Makary and Daniel paper 467 times.12 The BMJ website still listed the Makary and Daniel piece among its top six most read articles.13 According to one analysis, their paper was the only scientific publication of 2016–2017 widely covered by the media that did not present a novel discovery.14 By comparison, a detailed, contemporaneous critique of these two estimates written by the editors-in-chief of BMJ Quality and Safety15 had been cited only 16 times.12

Why have these recent estimates been accepted so quickly and widely, despite presenting re-analyses of older data? One way to understand the success of these estimates is to view them as potent cultural “memes.” Biologist Richard Dawkins coined the term to describe ideas that like replicating genes, propagate “by leaping from brain to brain.”16 Memes, according to Dawkins, may be subject to many of the same evolutionary pressures as their genetic counterparts, and thus certain characteristics make ideas more fit to spread throughout the “meme pool” of society.

In this perspective, we explain why these popular medical error estimates are inaccurate, explore the potential adverse impact of their rapid dissemination, and consider how individual physicians can contribute nuance to the cultural meme pool around medical error deaths.

ARE MEDICAL ERROR DEATH ESTIMATES ACCURATE?

Others have pointed out that the methodology used to arrive at these medical error claims is imprecise.15, 17 We will briefly summarize the salient points of prior critiques. First, these medical error estimates were extrapolated by magnitudes, with only a few dozen purported medical error deaths closely reviewed in the underlying data. Second, most of the studies were performed without the intention of gathering a sample generalizable to the entire US population. Third, the figures seem to imply a causal relationship between medical errors and deaths, though the methods for determining whether an error led to a patient’s death and whether the error was truly preventable were not well-defined in the data. Fourth, neither estimate utilized any established meta-analytic methodology or guidelines for combining data from dissimilar studies, but rather reported simple or weighted arithmetic means. Finally, the largest estimate of 440,000 preventable deaths was only achieved by applying an arbitrary 200% multiplier to the author’s own calculation to account for presumed underreporting.

Assuming 440,000 were an accurate portrayal of annual preventable deaths that occur in hospitals, the context in which these studies were conducted and where about 715,000 people die annually,18 this implies 62% of all hospital deaths are caused by preventable medical errors. Taking the 251,454 estimate, almost 34% of hospital deaths would be due to medical errors. We do not believe most physicians could reconcile such a high percentage of hospital deaths being caused by preventable medical error. The estimates’ authors propose even these are fewer than the actual figures. Makary and Daniel believe their estimate “understates the true incidence of death due to medical error.” James doubled his estimate to account for hypothetical underreporting but claimed even this “is probably an underestimate,” suggesting a factor of three might be better, although doing so would likely have placed preventable errors as the leading cause of death in the USA. It has been said these calculations lead to a “bottomless well of medical error.”15

WHY DO INACCURATE ESTIMATES SPREAD?

In his explanation of the meme concept, Dawkins identified three features that allowed an idea to spread: longevity, fecundity, and copying-fidelity. These medical error death estimates tend toward longevity because they have been memorialized in the scientific literature—including in one of the top medical journals—which remains archived and accessible over a long period. Dawkins himself gave an example of “fecundity” as a scientific idea that is readily accepted and thus heavily cited by other scientists. This is certainly the case with these estimates. Perhaps most relevant to this issue is the third aspect of meme dissemination: copying-fidelity. Ideas are not fixed objects, and can change as they spread from person to person. A simple idea may be more easily transmitted unchanged. Consider the title of Makary and Daniel’s paper: “Medical error—the third leading cause of death in the US.” It is short, definitive, and provocative. The title has proven easy to repeat verbatim throughout the media.

There may be other human factors at play. One is innumeracy. Without understanding the total number of annual hospital deaths, the vast scale of the claims is not recognized. Another is the concept of “anchoring,” drawn from behavioral economics.19 As experts and laypeople alike repeat the popular statistics about medical error deaths, this anchor point becomes a tough one from which to deviate, and smaller estimates may be viewed with suspicion given the appropriate concern over adverse events.

These more recent estimates also suggest a narrative of change in medical error deaths, which can be more compelling than a static problem. In the seminal 1999 Institute of Medicine report, preventable medical error deaths were estimated at 44,000–98,000.20 As these newer estimates are much higher, they superficially suggest the problem is getting worse. These estimates, however, did not use a comparative methodology that could directly determine whether more people are historically dying from preventable medical errors. In fact, one of the studies underlying both estimates was designed to address this question and suggested rates of preventable harm were not significantly changing over time.21

While no estimates of medical error can fully account for underreporting, hindsight bias, and the subjectivity of the concept, better methods are available. Extensive sampling and review of medical records performed with the intention of estimating national trends can shed more light on this question. A rigorous study out of the UK suggested about 3.6% of hospital deaths were due to preventable medical error,22 while a similar one out of Norway estimated about 4.2%.23 Even in careful studies, however, reviewers often disagree about how preventable an error was. Extrapolating these estimates to 715,000 US hospital deaths, 25,740–30,030 people would die annually from preventable medical errors—significant but an order of magnitude fewer than more popular estimates. Estimates derived from forensic pathology data in New York City suggest that therapeutic complications are the tenth leading cause of death, not the third, and most complications could be attributed to predictable risks of medical treatment rather than unambiguous mistakes.24

DO INACCURATE ESTIMATES CAUSE HARM?

Some physicians may not perceive the importance of inaccurate estimates of medical error deaths being disseminated through the academic and popular media. Yet if estimates are presented as facts, they can be potent justification for medical, legal, and political activity. A national nursing union, for example, has used these estimates to advocate for new state nursing regulations.25 A local newspaper used the “third leading cause of death” claim to place a child’s tragic death as part of a widespread epidemic, possibly influencing the case’s ongoing litigation.7 Physicians—including the authors—may agree with some policies advocated through these estimates, but we do not personally feel the ends justify the means if it requires lending too much credibility to contested science. James, originator of the larger estimate, appears to disagree, concluding in his paper that “in a sense, it does not matter whether the deaths of 100,000, 200,000 or 400,000 Americans each year” are associated with adverse events because any of these estimates should prompt political action. We find such a view cavalier.

It is challenging to quantify the consequences of cultural activity. Nevertheless, we can examine comparable public narratives to see how the discussion of medical errors may contribute to an environment of decreasing public trust in medicine.26 A recent analysis by Kathleen Jamieson examined the public narrative about the scientific “reproducibility crisis.”14 Just as false positive results are part of science’s self-correcting process, so too is the potential for adverse events to some degree inherent to the process of providing any medical intervention—a therapeutic effect can also represent a side effect depending on pathophysiology and circumstance. Jamieson suggests that “by employing crisis language to add urgency to their case for change, scientists license such characterizations to the news.” If our scholarly literature repeats these implausible estimates of medical error deaths, we permit the popular media to do so. Jamieson argues that “defective narratives can enhance the capacity of partisans to discredit areas of science.” Those who have a financial or philosophical agenda to discredit physicians can bolster their arguments if they seem to originate from within the medical community.

Physician Esther Choo has pointed out that a creator of the popular television show “The Resident,” which is critical of the healthcare system, appears to be portraying physicians in an extreme and negative light because she believes “they are the third leading cause of death.”5, 27 A national dialogue that inaccurately overemphasizes medical error as a cause of death could paradoxically increase reticence to address error at the doctor-patient level if it heightens physician or hospital fear of litigation.

HOW CAN PHYSICIANS IMPROVE THE DISCUSSION?

While likely useful within specialist research, we do not believe refined estimates of the total number of preventable deaths are the primary way to improve the public conversation. Fundamentally, these statistics suggest no solutions and provide no nuance. In professional discussions, the popular term “medical error” is already considered too broad or vague a concept.

At first, it may seem the way to counteract an inaccurate meme is with an equally infectious but more accurate one. This is one option; however, we believe it does a disservice to doctors and patients to oversimplify these discussions. We advocate for finding ways to contextualize the issue for the public—to make it less of a meme entirely.

National-level estimates of medical error obscure the emergent nature of many mistakes. They erase the frequently winding narrative of adverse events. Many deaths in today’s hospitalized patients follow a variegated course marked by acute and chronic illness; invasive, wide-ranging interventions; fragmented care by multiple subspecialists; and a web of diagnostic testing and clinical reasoning unique to each patient. Within the hospital system, this narrative aspect of error is already embraced through root cause analyses, morbidity and mortality conferences, and other forms of individualized review, in addition to systematic approaches. The same nuance should be brought to bear on public-facing discussions. The most vocal public advocates of patient safety are often those whose professional work is focused on it. They are experts, but often have one specific viewpoint. Instead, we encourage “in the trenches” physicians who experience but do not specialize in error to contribute to the discussion—to share their diverse stories, concerns, and ideas for improvement. As “easy” as it is to cite a simple statistic to support a worthy cause, the public conversation will benefit from more nuance.

Memes are simple, long-lived, shareable ideas. Adverse medical events in contrast occur within complex biological and social systems. When it comes to discussing medical error deaths, we need fewer memes and more context.