Introduction

Evidence-based medicine has become increasingly popular to help establish guidelines in treating a myriad of health problems. An important aspect of this practice is to establish a hierarchy of evidence—including randomized controlled studies, case series, and case reports—that helps grade the relative strength of the evidence to support a study’s conclusions [1]. Though this argument is imperfect as any study has flaws and strengths, we must rely on the evidence available.

The treatment of poisoned patients has been established primarily upon recommendations based on human observational studies. It is difficult to conduct the gold standard of evidence-based studies—the randomized controlled study—in cases of poisonings. Few cases of life-threatening poisonings occur at any single poison center and the cost and complexity of multi-center randomized controlled trials have limited their use. This has not gone unnoticed. As early as 1990, Brent et al. noted that in the toxicology abstracts from the several national meetings between 1984 and 1989, the majority of studies presented was case series, followed by case reports, experimental studies, and then clinical controlled studies [2].

Textbooks are not considered an optimal source for identifying the “best evidence” in evidence-based medicine, primarily because they may not have the most recent information. However, textbooks generally reflect the standard of care and the evidence cited in textbook chapters in support of recommendations should reflect the “best evidence” at the time the chapter was written. As the overall quality of evidence for toxicology treatment recommendations has not been assessed, our objective was to determine the types of evidence used to justify treatment recommendations with the caveat that recommendations may change over time.

Methods

One author reviewed the acetaminophen (APAP), tricyclic antidepressant (TCA), calcium channel blocker (CCB), and any relevant antidote chapters in three textbooks: Goldfrank’s Toxicologic Emergencies (8th edition), Critical Care Toxicology–Diagnosis and Management of the Critically Poisoned Patient, and Medical Toxicology (3rd edition) [3–5] and identified statements that gave a treatment recommendation.

We recorded the citations for each treatment recommendation and determined the type of publication used to support the statement. We classified the citations as follows: no citation, general review article, in vitro study, animal study, case reports (n < 3), case series (n > 2), retrospective study, prospective observational study, and controlled clinical trial. The distinction between case series, retrospective study, and prospective observational study was made by either self-report of the study or publication type via PubMed. Publication type was determined by review of the original article, review of an article abstract, using PubMed’s classification of the source or evaluation via the source’s title. Articles that could not be classified and that could not be obtained were classified as unknown. Proportions for each type of citation with 95% confidence intervals were determined.

Results

Four hundred sixty-nine treatment recommendations were identified and there were 742 statements supporting these recommendations. Some references had no citations while others used multiple citations. We could not classify 57/742 citations as we could not locate the original article or abstract, there was no classification via PubMed, and we were unable to classify the article solely via the title. Overall, case reports and animal studies represented the most common publication type supporting a treatment statement (Table 1). Confidence intervals were calculated via the modified Wald method (http://www.graphpad.com/quickcalcs/ConfInterval1.cfm). However, there was significant variation among the three poisons.

Table 1 Type of citation used for statements suggesting a treatment for three common poisonings in three major toxicology textbooks
Full size table

Limitations

There are several limitations to this study, including the use of a single reviewer who went through all the texts and made the classifications, the lack of ability to definitively classify all resources via primary text, and the use of only selected chapters in three prominent toxicology textbooks. However, as we selected several common, life-threatening poisonings [6], we believe that the quality of evidence for rarer poisons would be even worse. Many citations could not be located in the PubMed index. However, if these studies could not be found in journals covered by PubMed, it is unlikely that they are of the highest quality, referenced often, or performed relatively recently. Indeed, PubMed covers articles as far back as 1951. It is also possible that some of the studies may have been misclassified by PubMed. In addition, some authors mentioned that they were told to limit the number of references listed for each chapter (personal communication). Based on this, one could assume that the authors would reference only the strongest levels of evidence available.

Discussion

Case reports and animal studies are commonly used as evidence to support treatment recommendations in medical toxicology textbooks. According to the hierarchy of strength of evidence for treatment decisions, these rank near the bottom, only above unsystematic clinical observations [1]. This does not mean we should disregard such evidence entirely. Indeed, under the right circumstances, a well done case report or animal study might deserve more weight than a poorly conducted randomized controlled trial [7]. As the physician and biostatistician Dr. Milos Jenicek pointed out, “case reports and case series may be the ‘lowest’ or the ‘weakest’ level of evidence, but they often remain the ‘first line of evidence.’ This is where everything begins” [8]. Case reports not only offer treatment recommendations, but they also describe newly recognized adverse reactions/interactions, generate and test hypotheses, demonstrate diagnostic techniques, elucidate mechanisms, educate others, and enable systematic reviews [9]. Indeed, weaker forms of evidence such as case reports have high sensitivity for detecting novelty, but may have less specificity for medical decision making [10].

Though there is a need for more systematic studies of poisoned patients in toxicology, it is often difficult do conduct these studies both for ethical reasons as well as a lack of subjects. Thus, a clinician is often left to base treatment decisions on anecdotal forms of evidence such as case reports and case series. As case reports are commonly used to support treatment recommendations, it is important that the case reports contain sufficient information to allow the reader to assess the validity of the case report. For example, it would be important that a case report describing the use of new treatment for a poison include information that would assure the reader that (1) the patient was truly intoxicated with the poison of interest, (2) that the therapy was responsible for the improvement (by discussing adjunct therapy and other alternative explanations), (3) that the effects of the treatment (both good and bad) are adequately described, and (4) that any potential limits to the generalizability of the findings are mentioned. Many case reports do not include information on all of these areas.

Guidelines exist via Consolidation Standards of Reporting Trials Statement for reporting randomized clinical trials, but no such accepted formal recommendations exist for case reports. Guidelines do exist for case reports in certain specific circumstances, such as for an adverse drug reaction [11–14]. Otherwise, ineffective interventions may appear useful—the so-called “Pollyanna” phenomenon [15]. Although the field of toxicology may favor retrospective data collection, it does not preclude an appropriate rigorous analysis [16]. Conclusions based on limited experiences should be made on valid cause-effect inferences rather than on accidental circumstance.

Similarly, treatment recommendations based solely on studies using animal models must be interpreted carefully. Many therapies that appear useful in animal models are not effective in treatment of human disease [17, 18]. While there has never been a systematic assessment to determine aspects of studies (such as design, model, or drug dose) that identify which experiments will ultimately translate to human disease, we believe some aspects should be considered. For example, studies that evaluate a clinically relevant outcome in a model that is similar to the human disease (e.g., reversal of hypotension) have more face validity that a study that measures a less direct outcome (e.g., developed pressure in an isolated heart preparation). Authors of book chapters should comment on how well the model actually reflects acute poisoning.

Conclusion

In the chapters describing three poisonings (APAP, TCA, and CCB) and their related antidotes in three major toxicology texts, many treatment recommendations were not referenced. Among those referenced, the majority were case reports and general review articles that are generally considered lower in the hierarchy of evidence. The types of studies for each poisoning revealed that APAP treatment recommendations had the highest level of evidence. Review articles reflect the available evidence and the personal opinion of the author(s), and thus, they may not be the optimal evidence-based sources. With stringent standards applied, case reports can be a valid source for treatment recommendations.