Avoid common mistakes on your manuscript.
INTRODUCTION
Randomized controlled trials (RCTs) are the most reliable form of evidence for evaluating drug safety and efficacy. Because clinicians rely on RCTs to inform clinical practice, accurate representation of clinical trial results is important to patient health and safety. Spin, defined as reporting that distorts results or misleads the reader,1 threatens accurate evidence interpretation and application by clinicians. Given that many clinicians obtain information from the study abstract only, spin in the abstract is concerning.2 We examined the prevalence of spin among abstracts in a random sample of trials focused on the efficacy of drugs and examined the association of spin with study characteristics.
METHODS
We searched Medline for all English-language RCTs examining drug efficacy published in 2013. Our search identified 2851 potential studies. After screening titles and abstracts, 1101 potential studies remained. We randomly selected 646 of this sample, of which 190 studies were focused on drug efficacy and met inclusion criteria. For each study, we extracted information on outcome, funding source, financial ties to the manufacturer, and design characteristics (Table 1). The methods are described in full in a prior publication.3
Identification of RCT Outcome
Trained abstractors used the results section of each study to determine whether the study reported positive or negative primary efficacy outcome. For superiority studies, if the drug of interest was statistically superior to the control (e.g., p-value < 0.05), the study outcome was defined as positive. For non-inferiority studies, if the drug of interest and the control had no significant difference, the study outcome was defined as positive. Study outcomes were assessed independently and in duplicate. Any disagreement on study outcome was resolved by discussion among the research team.
Outcome Measure
The main outcome variable considered was the presence or absence of spin in the abstract of the RCT. We considered spin present if the abstract outcome was positive or mixed and the study reported a negative primary efficacy outcome in the results section of the manuscript.1 Two clinician reviewers (SK, DK) evaluated all abstract conclusion sections, rating each conclusion as positive (in favor of study drug), negative (neutral or in favor of control), or mixed. A mixed rating meant that it was unclear in the abstract whether the study drug or the control was favored (i.e., if a subgroup analysis was emphasized over the primary outcome). The two clinicians remained blinded to the study outcome during this discussion.
Analysis
We report the prevalence of spin in a sample of RCTs. We examined the association between spin and study characteristics using a two-sided, 0.05-level χ2 test of significance. Statistical analysis was performed using SAS version 9 statistical software (SAS Institute Inc., Cary, NC).
RESULTS
Of the 190 RCTs identified, 59 had a negative primary outcome in the results. These 59 studies were evaluated for the presence of spin. Among the 59 studies, clinician reviewers rated 8 abstracts as having a positive outcome and 17 as having a mixed outcome, for a total of 25 (42%) abstracts with spin. Study characteristics were largely similar across studies with and without spin (Table 1). Overall, studies with spin had smaller samples (median: 201) than studies without spin (median: 352; p = 0.03). There was no relationship between any financial tie to the manufacturer and presence of spin in the abstract (p = 0.83).
DISCUSSION
Nearly half of abstracts of RCTs focused on drug efficacy that report negative results contain spin. We did not find an association between spin and financial ties to industry, but our study may be underpowered to detect this association. Many clinicians do not read beyond the abstract, and many readers of the literature may not have the skill to critically analyze a trial themselves to combat spin or other bias in the report.4,5,6 Given the widespread reliance on the abstract, the peer review process needs to be improved to reduce spin in abstracts. A simple prompt asking reviewers to comment on the presentation of the results of the study with a specific question about spin may help focus reviewer attention on this issue. Editors can also review for spin in the editorial decision process. These simple steps that allow more scrutiny of the abstract and provide feedback to authors may reduce, if not eliminate, spin in the literature.
References
Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and Interpretation of Randomized Controlled Trials With Statistically Nonsignificant Results for Primary Outcomes. JAMA. 2010; 303(20): 2058–64. doi: https://doi.org/10.1001/jama.2010.651
Saint S, Christakis DA, Saha S, et al. Journal reading habits of internists. J Gen Intern Med. 2000;15(12):881–4. doi:https://doi.org/10.1046/j.1525-1497.2000.00202.x
Ahn R, Woodbridge A, Abraham A, et al. Financial ties of principal investigators and randomized controlled trial outcomes: cross sectional study. BMJ. 2017;356:i6770. doi: https://doi.org/10.1136/bmj.i6770.
Caverly TJ, Matlock DD, Prochazka AV, Lucas BP, Hayward RA. Interpreting Clinical Trial Outcomes for Optimal Patient Care: A Survey of Clinicians and Trainees. J Grad Med Educ. 2016;8(1):57–62. doi: https://doi.org/10.4300/JGME-D-15-00137.1.
Johnson TV, Abbasi A, Schoenberg ED, et al. Numeracy among trainees: are we preparing physicians for evidence-based medicine? J Surg Educ. 2014;71(2):211–5. doi: https://doi.org/10.1016/j.jsurg.2013.07.013.
Korenstein D. Blinding Them With Science? Evidence-Based Medicine as a Barrier to Health Care Value. J Grad Med Educ 2016;8(1):106–8. doi: https://doi.org/10.4300/JGME-D-15-00570.1.
Funding
This project was not directly supported by any research funds. Dr. Keyhani is funded by grants from the NIH (grants RO1 HL116522-01A1, RO1 HL114563-01A1) and VA HSR&D (1IP1HX001994). Dr. Korenstein’s work on this paper was supported by a Cancer Center Support Grant from the National Cancer Institute to Memorial Sloan Kettering Cancer Center (award number P30 CA008748).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Prior Presentations
SGIM meeting April 21, 2017.
Conflict of Interest
All authors declare that they have no conflict of interest.
Ethical approval
Not needed.
Data sharing
Data set available from corresponding author on request.
Transparency
The manuscript’s guarantor (SK) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is noncommercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Rights and permissions
About this article
Cite this article
Woodbridge, A., Abraham, A., Ahn, R. et al. A Cross-Sectional Analysis of Spin in Randomized Controlled Trials. J GEN INTERN MED 33, 247–248 (2018). https://doi.org/10.1007/s11606-017-4252-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-017-4252-2