Background

Financial conflicts of interest (FCOIs) in drug trials can influence trial design, drug dosages and comparators, data analysis, interpretation of findings, and the likelihood that favorable results are reported [1,2,3,4,5,6,7]. Industry-sponsored trials are approximately 30% more likely to report favorable efficacy results and conclusions than non-sponsored trials, and this is not explained by other trial elements associated with risk of bias [6]. Similarly, trials conducted by principal investigators with FCOIs have higher odds of reporting positive outcomes than trials led by non-affiliated principal investigators, controlling for trial funding source [7].

Meta-analyses are highly cited [8] and are prioritized in the development of clinical practice guidelines and in setting research priorities [9,10,11]. A review of a sample of 29 meta-analyses of drug trials published in high-impact medical journals in 2009, however, reported that only 2 (6.9%) reported funding sources and none reported author FCOIs or industry employment from included trials [12]. A 2012 review of 151 Cochrane reviews of drug trials found that only 46 (30.5%) reported the funding source of some or all included trials; only 16 (10.6%) provided any information on author FCOIs or industry employment [13].

In 2012, the Cochrane Collaboration began to require that funding sources and author FCOIs be reported for all trials included in Cochrane reviews [14,15,16]. The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement, which was published in 2009 [17, 18], however, did not address reporting of trial funding and author-industry financial ties from included trials. International Committee of Medical Journal Editors (ICMJE) guidelines require that meta-analysis authors declare their own FCOIs but do not address reporting of study funding or author FCOIs of included trials [19].

The objectives of this study were to (1) investigate the extent to which pharmaceutical industry funding and author-industry financial ties and employment from drug trials synthesized in meta-analyses are reported in meta-analyses published in high-impact journals and (2) compare current reporting with results from 2009 [12].

Methods

Our study protocol was published on the Open Science Framework (https://osf.io/8xt5p/) and followed the methods of the previous Roseman et al. [12] study with 2 exceptions. First, whereas the previous study was limited to a 10-month period (January to October 2009), we extended the period to 22 months a priori (January 2017 to October 2018) to improve the likelihood that we could include the same number of meta-analyses from each journal or specialty area. Second, in addition to extracting reporting of trial information in meta-analyses, the previous study [12] reviewed FCOIs from all included trials. Since it is well-established that FCOIs are common in drug trials [1, 12, 20], we examined reporting in meta-analyses, but did not extract information from included trials.

Study selection

To be included, publications had to include at least 1 meta-analysis that (1) was part of a documented systematic review, (2) statistically combined results from ≥ 2 RCTs, (3) did not include non-RCTs, (4) evaluated the efficacy or harm of a drug or class of drugs, and (5) included at least 1 drug in any study arm under patent in the USA at the time of publication based on the electronic US Food and Drug Administration Orange Book [21]. Drugs were defined broadly to include biologics and vaccines but not nutritional supplements (e.g., vitamins) or medical devices without a drug component. Status of potential drug products not found in the FDA database, such as products registered as drugs only outside of the USA, was determined by review of team members and consensus. A drug was considered under patent if any aspect of the active ingredient (e.g., dosage, route, strength) was protected by an unexpired patent. We included the patent requirement to restrict the sample to drugs of potential economic importance to pharmaceutical companies. See Additional Methods 1 for the title/abstract and full-text eligibility coding guides.

Data sources and searches

We selected a sample of meta-analyses published from January 1, 2017, to October 25, 2018, in 3 categories of high-impact publications: (1) general medicine journals, (2) journals from 5 specialty medicine areas (oncology, cardiology, respiratory medicine, endocrinology, and gastroenterology), and (3) the Cochrane Database of Systematic Reviews [12].

In the previous study [12], among general medicine journals, we selected the 3 most recently published eligible meta-analyses from each journal with a 2008 impact factor ≥ 10 (New England Journal of Medicine, JAMA, Lancet, BMJ, Annals of Internal Medicine, PLoS Medicine) with fewer included if < 3 met eligibility criteria. Neither the New England Journal of Medicine nor PLoS Medicine had any eligible meta-analyses, and the Annals of Internal Medicine had only 2. Thus, in the present study, only meta-analyses from JAMA, Lancet, BMJ, and Annals of Internal Medicine were eligible, and we included only 2 meta-analyses from Annals of Internal Medicine.

In the previous study [12], we selected the top 5 specialty areas based on 2008 global pharmaceutical sales. For the present study, we included the same 5 specialty areas (oncology, cardiology, respiratory medicine, endocrinology, and gastroenterology). In each specialty area, we included the 3 most recently published eligible meta-analyses in the top impact factor journal in the specialty area based on the 2016 Clarivate Analytics Science Citation Index. If 3 eligible meta-analyses were not published in the top journal, we searched the journal with the next highest impact factor and continued in declining order of impact factor until 3 eligible meta-analyses were identified.

We searched MEDLINE via PubMed using limits of article type (“meta-analysis”) with journal names, supplemented by a manual search of each journal’s table of contents. Articles published online ahead of print, but not in the final format, as of the search date were not eligible. For articles published in the same journal issue, the one with the highest page number was considered most recent. In each Cochrane Database of Systematic Reviews issue, meta-analyses were reviewed in reverse sequence as the most recently published reviews are listed first.

Data extraction

We uploaded search results into the systematic review software DistillerSR® for inclusion and exclusion and result coding. Review of identified articles from each general medicine journal, specialty medicine area, and the Cochrane Database of Systematic Reviews was conducted independently by 2 reviewers, 1 article at a time, in reverse temporal sequence until the targeted number of eligible meta-analyses was obtained. Any disagreements were resolved by consensus, involving a third reviewer if necessary.

Two investigators independently reviewed all included meta-analyses, including disclosure statements, article texts and tables, author bylines and acknowledgments, and online journal supplements to identify (1) in the meta-analysis: disclosed funding sources, author-industry financial ties, author-industry employment, and whether a quality or risk of bias assessment was conducted, and (2) whether or not the meta-analysis reported trial funding, author-industry financial ties, and author-industry employment from included drug RCTs. See Additional Methods 2 for the Meta-Analysis Data Extraction dictionary. Additionally, in February 2020, we examined author instructions from all journals with included meta-analyses to determine if they included instructions on reporting of funding sources, author-industry financial ties, or author-industry employment for studies included in meta-analyses published in the journal.

Data synthesis and analysis

Study funding sources for meta-analyses were classified as pharmaceutical industry, non-industry (e.g., public granting agency, private not-for-profit granting agency), combined pharmaceutical industry and non-industry, no study funding, or not reported. Meta-analyses reported as funded “in part” by the pharmaceutical industry with no other indication of funding source or funded by a not-for-profit organization fully sponsored by pharmaceutical industry sources were coded as industry-funded. Industry funding was considered to be a provision of financial support, resources (e.g., statistical analyses), or study personnel.

Meta-analysis author financial ties to industry were defined per the ICMJE Uniform Disclosure Form for Potential Conflicts of Interest [19] and included current or former board membership, current or former consultancy, former industry employment, equity holdings (e.g., stock ownership, stock options), expert testimony, gifts, patents (planned, pending, issued), payment for manuscript preparation, other research funding, royalties, speaker fees/payment for presentation development, travel reimbursement, or other unspecified FCOIs, as disclosed in the article. If an article did not contain a disclosure statement, author-industry financial ties were coded as not reported. Authorship by persons employed by the pharmaceutical industry at the time of article publication was coded separately as “industry employment.”

For reporting in meta-analyses of trial funding, author-industry financial ties, and author-industry employment from included drug RCTs, for each category, we determined if the meta-analysis reported for all, some, or no included RCTs. When information was reported, we determined where information could be found in the meta-analysis (e.g., text, characteristics of studies table, risk of bias assessment, footnote).

Any discrepancies in data extraction were resolved by consensus, including consultation with a third reviewer if necessary.

We reported descriptive characteristics of included meta-analyses, their funding, and author-industry financial ties. To compare the proportion of meta-analyses that reported study funding, author financial ties, and author employment from included RCTs, we generated 95% confidence intervals (CIs) for the differences in proportions [22].

Role of the funding source

No funder had any role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication. Dr. Thombs had full access to all data in the study and had final responsibility for the decision to submit for publication.

Results

Article selection

A total of 176 publications were reviewed (37 from general medicine journals, 121 from specialty medicine journals, 18 from the Cochrane Database of Systematic Reviews) to obtain the 29 that were included in the review (Fig. 1) [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51]. As shown in Table 1, impact factors of journals with included publications ranged from 17.2 to 47.8 in general medicine, 11.9 to 24.0 in oncology, 19.3 to 19.9 in cardiology, 10.3 to 10.6 in respiratory medicine, 11.9 to 19.7 in endocrinology, 16.7 to 18.4 in gastroenterology, and 6.3 for the Cochrane Database of Systematic Reviews. The 29 selected meta-analyses evaluated a broad spectrum of pharmacological interventions, including 11 on treatment efficacy [24, 29, 33, 35, 36, 40, 42, 44,45,46,47], 2 on harms [31, 50], and 16 [23, 25, 28, 30, 32, 34, 37,38,39, 41, 43, 48, 49, 51] on both efficacy and harms. Between 2 and 522 RCTs were included in each meta-analysis. None of the journals with included meta-analyses mentioned reporting of funding or author FCOI of studies included in meta-analyses that are published in the journal.

Fig. 1
figure 1

PRISMA flowchart of included meta-analyses

Full size image
Table 1 Characteristics of included meta-analyses
Full size table

Study funding and author-industry financial ties of meta-analyses

As shown in Tables 1 and 2 of 29 (6.9%) included meta-analyses, both published in specialty journals [34, 43], reported receiving pharmaceutical industry funding, 11 (37.9%) reported non-industry funding [23, 26, 29,30,31,32, 35, 40, 49,50,51], 3 reported no study funding (10.3%) [28, 33, 46], and the funding source of 13 (44.8%) was not reported [24, 25, 27, 36,37,38,39, 41, 42, 44, 45, 47, 48]. Meta-analysis funding sources were reported for 8 of 11 meta-analyses from general medicine journals (72.7%) [23, 26, 28,29,30,31,32,33], 5 of 15 (33.3%) from specialty medicine journals [34, 35, 40, 43, 46], and all 3 (100%) Cochrane reviews [49,50,51].

Table 2 Financial ties to the pharmaceutical industry among authors of reviewed meta-analyses
Full size table

All 29 meta-analyses included author COI statements. In 15 of the 29 meta-analyses (51.7%), at least 1 author reported 1 or more financial ties to the pharmaceutical industry [24,25,26, 29, 34, 35, 37,38,39,40, 43, 45, 47, 49, 50], whereas all authors reported no financial ties in 14 (48.3%) [23, 27, 28, 30,31,32,33, 36, 41, 42, 44, 46, 48, 51]. In 5 of 29 (17.2%) meta-analyses [34, 35, 38, 39, 43], all published in specialty medicine journals and including the 2 meta-analyses with industry funding [34, 43], the majority of authors had financial ties to industry. Author-industry financial ties were present in 4 of 11 meta-analyses published in general medicine journals (36.4%) [24,25,26, 29], 9 of 15 (60.0%) in specialty medicine journals [34, 35, 37,38,39,40, 43, 45, 47], and 2 of 3 (66.7%) Cochrane reviews [49, 50]. Specific types of author ties to industry are shown in Table 2.

Reporting of trial funders and author FCOI from RCTs included in meta-analyses

As shown in Table 3, 13 of 29 (44.8%) meta-analyses reported the funding sources of included RCTs; 12 reported for all included RCTs [23, 24, 26, 27, 29, 30, 32, 34, 45, 49,50,51], whereas 1 reported for 5 of 7 included RCTs [39]. Funding sources of included RCTs were reported for 7 of 11 (63.6%) meta-analyses from general medicine journals [23, 24, 26, 27, 29, 30, 32], 3 of 15 (20.0%) from specialty medicine journals [34, 39, 45], and for all 3 (100%) Cochrane reviews [49,50,51]. The mean, median, and range of the number of RCTs in meta-analyses that reported trial funding sources were 52.9, 13, and from 2 to 522, respectively. For those that did not report funding sources, they were 29.2, 16, and from 3 to 236, respectively.

Table 3 Disclosure and reporting in meta-analyses of randomized controlled trial funding source, author financial ties to the pharmaceutical industry, and author employment by the pharmaceutical industry
Full size table

Only 2 of the 29 meta-analyses (6.9%) reported trial author-industry financial ties, including 1 from a general medicine journal [24] and 1 Cochrane review [49]. None of the 29 meta-analyses reported industry employment status of included RCT authors. See Fig. 2.

Fig. 2
figure 2

Percentage of included meta-analyses that reported trial funding, author financial conflict of interest, and author-industry employment for 2009 and 2017–2018

Full size image

Comparison of reporting of industry funding of included RCTs in 2017–2018 versus 2009

The overall percentage of meta-analyses of drug treatments that reported the funding source of included RCTs increased from 6.9% (2 of 29) in 2009 to 44.8% (13 of 29) in 2017–2018, a difference of 37.9% (95% CI, 15.7 to 56.3%). This included an increase from 0% (0 of 11) to 63.6% (7 of 11) in general medicine journals (difference 63.6%; 95% CI, 25.3 to 84.8%), an increase from 6.7% (1 of 15) to 20.0% (3 of 15) in specialty medicine journals (difference 13.3%; 95% CI, − 13.2 to 39.1%), and an increase from 1 of 3 (33.3%) to 3 of 3 (100%) among Cochrane reviews.

Discussion

The main finding was that the reporting of funding sources of drug trials included in meta-analyses in high-impact journals improved, from 2 of 29 (6.9%) in 2009 to 13 of 29 (44.8%) in 2017–2018, but it continues to be sub-optimal. Only 2 of 29 (6.9%) meta-analyses provided information on author-industry financial ties from included trials, and no meta-analyses reported if industry employees were involved in the trials.

In 2012, the Cochrane Collaboration began to require that trial funding sources and conflicts of interest of authors of included trials be reported in the “characteristics of included studies” table of all Cochrane reviews [15], and this is still mandatory [16]. We evaluated 3 Cochrane reviews, and all 3 reported trial funding sources, but only 1 of 3 provided information on author-industry financial ties from included trials. A recent study [52] that investigated the extent to which recently published meta-analyses reported trial funding, author-industry financial ties, and author-industry employment from included RCTs found that reporting of trial funding in Cochrane meta-analyses increased from 30% (46 of 151 reviews) in 2010 to 84% (90 of 107) in 2016–2018. Reporting of trial author-industry financial ties increased from 7% (11 of 151) in 2010 to 44% (47 of 107) in 2016–2018, which suggests that this could still improve. Non-Cochrane meta-analyses published in 2016–2018 reported funding sources of included studies 15% of the time (21 of 143) and author-industry financial ties from included trials 1% of the time (2 of 143).

Cochrane reviews are recognized for their rigor [53] and often seen as the standard for systematic reviews on the benefits and harms of health care interventions [54, 55]. Consistent with this, Cochrane reporting standards are highlighted on the PRISMA website [56]. Ideally, the Cochrane Collaboration would ensure that authors of reviews adhere to both the requirement to report funding of trials included in reviews and to report author-industry financial ties from those trials. Nonetheless, Cochrane provides an example of how institutional commitment can lead to change on a large scale, which suggests that other journals could achieve similar results, but that it would require explicit guidance from journal editors and enforcement of that guidance.

The original PRISMA statement, which was published in 2009, did not address reporting of the funding sources of studies included in systematic reviews and meta-analyses or the FCOIs of study authors [17, 18]. An updated PRISMA statement is forthcoming and, though not completed, based on a preliminary version will likely encourage, though not required, reporting of funding and author FCOI from studies included in systematic reviews and meta-analyses (personal communication, David Moher, February 13, 2020). It is possible that including encouragement to report on funding and FCOI in PRISMA could improve reporting, but the lack of a strong requirement and inclusion as an item in the checklist itself and the general low adherence to PRISMA [57] suggests that this may not have a strong effect.

It is not clear why general medicine journals improved in reporting between 2009 and the present study. It is unlikely related to PRISMA, since the existing PRISMA statement does not touch upon this issue. It is possible that this may be due to a more general awareness of these issues, that Cochrane added this requirement for its reviews, or a higher scrutiny by editors of these journals than previously or compared to specialty journals. None of the general medicine or specialty journals included in our review mention the reporting of funding sources or author FCOI from studies included in systematic reviews in their instructions to authors. Ideally, the forthcoming PRISMA checklist would include funding and FCOI of studies included in systematic reviews and meta-analyses as a dedicated item, which could support improvements in reporting, if adopted and enforced by reviewers and editors.

We previously recommended that the Cochrane Risk of Bias tool [58] be revised to include risk of bias due to industry sponsorship of trials and FCOIs of trial investigators [12]. This would be consistent with empirical evidence that has linked both sponsorship and other FCOIs to trial outcomes, controlling for other factors known to be associated with bias [6, 7, 14]. There is no, however, consensus on this approach [14, 59]. Currently, an alternative is being created to explicitly address risk of bias from industry sponsorship of trials and author-industry financial ties in Cochrane reviews, the Tool for Addressing Conflicts of Interest in Trials (TACIT) [60]. Once completed, TACIT will include a Conflicts of Interest Grid, which will facilitate a systematic collection of relevant information and allow for determination of when there is notable concern, which may then be integrated into an assessment of risk of bias. In the present study, only 3 meta-analyses, 1 of which was a Cochrane review, attempted to incorporate funding sources of included trials into an assessment of risk of bias.

In interpreting results from this study, there are limitations to consider. First, the focus of the study was on reporting of trial funding and trial author FCOIs, and it was not designed to assess whether these were associated with meta-analysis quality or with the results of meta-analyses. Second, in replicating the methods of the previous study from 2009 [12], we selected 29 meta-analyses from high-impact journals in general medicine and 5 specialty areas for review; thus, it is not known to what degree these results may be generalizable to other areas of medicine or to lower impact journals. Third, we examined only a relatively small number of meta-analyses and compared results to a similarly small sample from an earlier time period.

Conclusion

In summary, reporting of funding sources of included trials in meta-analyses of drug treatments published in high-impact journals has improved since 2009 but is still alarmingly low. Fewer than half of the meta-analyses we reviewed reported funding sources of included trials, and fewer than a third provided information on trial funding in the main meta-analysis report. Reporting of trial author FCOIs and industry employment is even more concerning. Only 2 studies reported trial author-industry financial ties, and none directly reported whether industry employees were authors of included trials. Confidence in medical research and the quality of care delivered by those who rely on evidence from meta-analyses depends on transparent reporting and the ability to evaluate the degree to which conflicts of interest may have influenced trial design, conduct, and outcomes. The forthcoming revised PRISMA statement will require transparent reporting of funding in trials included in systematic reviews and meta-analyses, and the new TACIT tool is being developed by the Cochrane Collaboration to supplement its risk of bias tool and to integrate considerations of FCOIs into bias assessment. We encourage uptake of both of these tools by journals and authors of systematic reviews and meta-analyses so that the potential influence of industry sponsorship and other author-industry ties can be considered by users of systematic reviews and meta-analyses.