Skip to main content

Asymmetric Reporting of Harms and Benefits in Randomized Controlled Trials

INTRODUCTION

Physicians rely on accurate reporting of both efficacy and safety in randomized controlled trials (RCTs), reflecting benefits and harms of therapy. However, harms reporting, generally in the form of adverse events (AE), is often omitted or incomplete.1 “Framing” through asymmetrical reporting of harms vs. benefits can lead physicians to underestimate harms.2 Further, harms reporting language may be vague, which limits understanding of the prevalence of AEs and their impact on quality-of-life.3 These factors may impede optimal decision-making and worsen patient outcomes.

In response to concerns about inadequate safety reporting in RCTs, in 2004, the Consolidated Standards of Reporting Trials (CONSORT) Group published guidelines for complete and accurate reporting of harms.4 Implementation of these guidelines was initially poor.5 More recent adherence has not been described, though overall quantification of results remains suboptimal.6 We evaluated current patterns in RCT reporting of harms compared to benefits and determined if funding, region, specialty, or type of intervention influences AE reporting.

METHODS

In May 2019, we identified all RCTs published within the first 3 months of 2019 in the 10 highest impact medical journals according to Scimago Journal and Country Rank; the time frame was chosen as the most recent full quarter of a calendar year. We reviewed Tables of Contents of each journal and included phase III–IV RCTs of efficacy with sample size > 50, and at least 2 treatment arms. We classified RCTs by clinical area and treatment type (drug, therapeutic procedure, other) and collected information on primary region and funder(s). We recorded presentation method (absolute or relative numbers or both) of efficacy outcomes and AEs (harms) and completeness of safety reporting, defined as inclusion of the total number of participants with any AEs. Classifications were performed by one investigator (RY) and checked by a second (DK). We used chi-square and Fisher’s exact tests to examine univariate associations between AE reporting and study characteristics and multivariate logistic regression to examine the relationship between complete AE reporting and funder and clinical area, adjusting for geographic region and treatment type. We used SAS version 9.4 for all analyses.

RESULTS

We identified 121 eligible studies from 9 of the 10 journals. Most were from Europe (53%) or North America (40%); 64% evaluated drugs and 26% evaluated procedures. Funding and clinical area were mixed (Table 1). One hundred thirteen (93%) reported AE in the main paper and 26 (21%) specified a primary safety outcome. Safety reporting was complete in 17% of studies; 24% reported only severe AEs, 20% only common AEs, and 12% did not quantify AEs. Complete reporting differed by funder on univariate (Table 1) but not multivariate analysis (OR for any industry funding 0.41 [CI, 0.22–1.73]). In multivariate analysis, complete reporting was not associated with region or with clinical area: cardiology vs hematology/oncology OR 0.87 (CI, 0.18–4.32), infectious diseases vs hematology/oncology OR 1.17 (CI, 0.24–5.78), other vs hematology/oncology OR 0.72 (CI, 0.19–2.69).

Table 1 Characteristics of Included Studies (n=121)

Most studies (68%) reported AEs using absolute numbers only. In contrast, 96% reported efficacy in both absolute and relative numbers. Table 2 describes presentation and quantification of efficacy and safety outcomes, by study characteristics.

Table 2 Specification and Reporting of Efficacy and Safety Data (n=121)

DISCUSSION

Safety reporting in RCTs remains inconsistent and incomplete, regardless of funding, region, specialty, or intervention type, with complete safety reporting in fewer than 20% of RCT publications. Even when included, specific methods of reporting of potential harms may further impede clarity. Presentation of AE data often focuses on absolute event rates with efficacy data presented in both absolute and relative terms. CONSORT criteria call for reporting in absolute terms4; most studies in our sample presented harms in that way. However, the dual presentation of relative and absolute benefit contrasted with presentation of only absolute harms may bias clinical interpretation toward overestimates of potential benefit and underestimates of harm; poor numeracy may further bias interpretation and contribute to disparities.2 We call on authors and journals to quantify benefits and harms the same way within a given study, always including absolute numbers and adding relative numbers if appropriate.

To assess the appropriateness and value of health services, clinicians must weight potential benefits and harms for individual patients. More transparent safety reporting in RCTs, mirroring reporting of efficacy, is needed to enable clinicians to accurately perceive and describe the balance of benefits and harms and protect patient safety.

References

  1. Phillips, R., et al., Analysis and reporting of adverse events in randomised controlled trials: a review. BMJ Open, 2019. 9(2): e024537.

    Article  Google Scholar 

  2. Gigerenzer, G., Why does framing influence judgment? J Gen Intern Med, 2003. 18(11): p. 960-1.

    Article  Google Scholar 

  3. Sacks, C.A., P.W. Miller, and D.L. Longo, Talking about Toxicity - "What We've Got Here Is a Failure to Communicate". N Engl J Med, 2019. 381(15): p. 1406-1408.

    Article  Google Scholar 

  4. Ioannidis, J.P., et al., Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med, 2004. 141(10): p. 781-8.

    Article  Google Scholar 

  5. Pitrou, I., et al., Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med, 2009. 169(19): p. 1756-61.

    Article  Google Scholar 

  6. Elliott, M.H., et al., Characteristics and Reporting of Number Needed to Treat, Number Needed to Harm, and Absolute Risk Reduction in Controlled Clinical Trials, 2001-2019. JAMA Intern Med, 2021. 181(2): p. 282-284.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Deborah Korenstein.

Ethics declarations

Conflict of Interest

DK reports that her spouse serves on the Scientific Advisory Board and has equity interest in Vedanta Biosciences, and provides consulting for Opentrons and Fimbrion. No other authors report potential conflicts of interest. DK’s work on this project was supported in part by a Cancer Center Support Grant from the National Cancer Institute to Memorial Sloan Kettering Cancer Center (P30 CA008748).

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yeahia, R., Gennarelli, R.L., Morgan, D.J. et al. Asymmetric Reporting of Harms and Benefits in Randomized Controlled Trials. J GEN INTERN MED 37, 2113–2115 (2022). https://doi.org/10.1007/s11606-021-07056-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-021-07056-1