Background

Clinical practice guidelines (CPGs) are „systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances“ [1]. Comparable to CPG development in many other countries, in Germany they are mainly developed by scientific medical societies. The development process is coordinated by the Association of the Scientific Medical Societies (AWMF). The AWMF consists of organisations that promote scientific work focused on medicine and related disciplines [2]. The majority of the members must be clinically active physicians, dentists or university graduates involved in academic research. Most CPGs developed through the AWMF involve a diverse field of groups and experts, including patient representatives and adjacent professions (e.g., physiotherapists, midwives, pharmacists). The quality of CPGs rests on two pillars: the evidence base and the consensus-building process. In Germany, the AWMF sets standards for the development of CPGs that are mandatory for all organisations developing and publishing CPGs through the AWMF [3]. CPGs are regularly updated on the AWMF homepage [2]. The AWMF grades guidelines based on their methodological rigour [4]:

S1: A representative expert group develops a guidance in an informal consensus-building process,

S2: Guidelines are based either on a systematic analysis of the scientific evidence (S2e) or on the structured consensus of a representative committee (S2k),

S3: Guidelines use both a systematic analysis of the available evidence and a structured consensus-building process.

CPGs have a broad visibility and can influence the decision making of physicians. However, CPGs can only maintain a high quality, if they are created with sufficient financial support (e.g., for the systematic literature search, consensus conferences) [5]. External funding of both the guideline development directly and of the guideline producing organisations indirectly may have an impact on the quality of guidelines [6]. In contrast to individual COI, where a recent Cochrane review suggests that financial COI are associated with favorable recommendations of drugs and devices in CPGs [7], there is limited evidence that funding of CPGs influences guideline recommendations [8, 9].

The AWMF Guidance Manual states that CPGs influenced by third party funding cannot be registered with the AWMF [3] and recommends the AGREE-II tool for guideline development [10]. However, currently there is no transparent reporting system for funding sources and expenses. This contrasts to the reporting of individual conflicts of interests (COI) in CPGs published by the AWMF, where strict and transparent guidelines are in place. These do not only require transparent COI statements from all guideline authors but also active measures to reduce the potential influence of individual COIs (e.g., abstentions during certain voting procedures or removal from the development process). Nevertheless, there is evidence of inadequate disclosure of COI [11] and stricter measures might be necessary to mitigate this problem (e.g., a “Physician Payments Sunshine Act for Germany”) [12].

Interestingly, Campsall et al. showed that only 1% (4/290) of the studied CPGs in the United States disclosed financial relationships between the scientific medical societies and biomedical companies, although 63% (60/95) of the organizations reported receiving funds from biomedical companies on their website or in response to a survey [13]. Elder et al. found that 14 (93%) of the 15 organisations disclosed funding sources from the industry. However, none disclosed this information in the CPGs [14].

Even though Campsall et al. and Elder et al. highlighted this mismatch between receiving funding and making this information available within CPGs, our understanding of general CPG funding practices is limited [5]. To assess the potential biases of the funding process, transparency is critical. In-depth knowledge of funding practices is also necessary to develop sustainable funding mechanisms to provide high quality guideline development in the future. The goal of this study, therefore, is to explore the accuracy and comprehensiveness of reporting on funding in German CPGs.

Methods

We selected all finalised CPGs published on the homepage of the AWMF [2]. All valid CPGs available on 10th of July 2020 were included in the main analysis. There were no other inclusion or exclusion criteria. We screened publicly accessible guideline texts (short, long and patient versions, reports) for statements on CPG funding and filed them into a database.

Different appraisal tools for CPGs have been developed and are commonly used [15]. These tools cover a range of quality dimensions, such as the evaluation of evidence (grading of evidence, consistency between evidence and recommendations) or the consideration of different perspectives (e.g., patient perspectives) [15]. For analysis, we utilised the German Instrument for Methodological Guideline Appraisal Version: 2005/2006 + Domain 8 (2008) (DELBI) [16] and assessed the structural dimension “independence” (“Guideline development organization and funding”) using Criterion 22. We decided to use the DELBI instrument in favor of the more widely used AGREE-II tool [10] as both tools are considered to be equally adequate for comprehensive guideline appraisal [15]. However, the DELBI rating (1–4, from lowest to highest) can be made from the information provided by the guideline authors in the accessible CPG documentation (Table 1).

Table 1 DELBI Criterion 22 (“The guideline is editorially independent of the funding organisation(s).”)
Full size table

We explored the categories derived from the DELBI criterion 22 (Table 1) and developed further categories to determine funding sources, expenses and the amount of funding provided. Funding sources were categorised using examples provided by the AWMF [2]. This included own contributions by CPG authors (“honorary office”). In most cases, guideline authors are not reimbursed for their work (excluding e.g., travel costs for consensus meetings). Although one could argue that this is not comparable to the other funding categories, the AWMF classifies author’s voluntary contributions as a component of guideline funding. It is important to assess the role of guideline authors to distinguish between the potential influence of the funding bodies and individual COIs on guideline recommendations. Because the aim of this study was to explore the accuracy and comprehensiveness of reporting on funding, individual COIs were intentionally left out of this analysis. Indirect funding of CPG producing organisations was also not assessed.

All categories were piloted by two authors in a selected subset of CPGs. Two authors carried out the data extraction independently. A third author was consulted if consensus could not be reached. Reviewers were excluded from the extraction of specific CPGs if they took part during their development process. Funding information of CPGs was considered inconclusive if no direct connection could be made between funding sources and expenses. For example, the statement “There was no external funding for the guideline.” is ambigious as to whether the CPG was funded by the guideline producing organisation or there was no actual funding at all. CPGs with missing or inconclusive information were not excluded from the analysis to provide an overall picture of the transparency of CPG funding. These CPGs received a DELBI rating of 1 (Table 1).

An exploratory data analysis was performed using R [17, 18]. Counts and percentages were used to report the data extracted from the CPGs. The non-parametric Kruskal-Wallis test was used to assess differences on DELBI scores between the different guideline tiers. Pairwise comparisons were done by using post hoc Wilcoxon test with Bonferroni-Holm correction for multiple testing. We performed a sensitivity analysis with a full dataset including expired CPGs since they are accessible older CPGs may be less transparent with respect to their funding. The study used only information based on routine-data and was performed in accordance with the Helsinki declaration and the International Committee of Medical Journal Editors [19].

Results

CPG selection

We found 763 CPGs through our search in the AWMF registry. After de-duplication of one (017–49/053 − 012), 762 CPGs could be assessed. 507 guidelines (66.5%, S1 = 175, S2e = 27, S2k = 171, S3 = 134) were published in 2015 or later. The remaining, expired CPGs (n = 255) were developed between 2006 and 2015 (n = 10 between 2006 and 2010). The complete dataset can be found in the Appendix (Additional file 1: Dataset).

Funding information provided by CPGs

315/507 (62.1%) of all CPGs provided conclusive information on guideline funding (Table 2a). There were stark differences between the different guideline tiers. While S2k/e and S3 CPGs provided information in > 80% of cases, only around 15% of S1 CPGs included conclusive information. Of all CPGs containing a statement 215/507 (42.4%) included a statement on funding types as well. A trend towards more transparent reporting from S1 to S3 tier could be seen (Fig. 1). While 144/507 (28.4%) of all CPGs declared that they were independent of external funding, statements to this effect were only found in 3/175 (1.7%) of S1 CPGs.

Fig. 1
figure 1

DELBI criterion 22 scores grouped by guideline grading

Full size image
Table 2 Funding of Clinical practice guidelines
Full size table

DELBI criterion 22 by guideline tier

DELBI scores stratified by the guideline tiers are displayed in Table 2a and Fig. 1. Only 23/507 (4.5%) of the CPGs achieved the highest score. A total of 192/507 (37.9%) received the lowest rating (1 of 4) not reporting any (135/507, 26.6%) or inconclusive (57/507, 11.2%) funding information. This was mainly due to S1 guidelines (DELBI (1) = 151/175, 86.3%). No CPG in this tier achieved the highest score.

A Kruskal-Wallis test to examine the differences of the DELBI score showed significant differences (χ2 (3) = 264, n = 507, p = 6.17*10− 53) between the different guideline tiers. Pairwise comparisons indicated that S1 scores were observed to be significantly different from all other groups (S2e: p = 4.28*10− 29, S2k: p = 3.24*10− 40, S3: p = 4.02*10− 47) and that S2k scores were observed to be significantly different from the S3 group (p = 6*10− 3). The other differences between the scores (S2e/S2k, S2e/S3) were not statistically significant.

Funding sources for CPGs with available information

We could assess funding sources of 315/507 (62.1%) CPGs (Table 2b). Scientific societies were most often (74.6%) mentioned with major differences between S1 (20.8%) and the other tiers (62.5–84.5%). Contributions by guideline authors were the second most common funding source mentioned overall. Other funding sources, such as hospitals and universities (as employers), independent foundations (e.g., charities, guideline programs (e.g., German Guideline Program in Oncology), self-regulatory bodies and the German Federal Government were almost exclusively mentioned in higher tiered CPGs. Funding through corporations in the healthcare sector (e.g., pharmaceutical industry, medical device manufacturers) was mentioned by five CPGs, once in an S2e and four times in S3 guidelines.

Funding types for CPGs with available information

Funding types of 215/507 (42.4%) CPGs could be assessed (Table 2c). Meeting costs were mentioned in 196/215 (91.2%). Specific funding types were stated in S2k and S3 CPGs due to the need to conduct consensus conferences and the necessity of systematic literature appraisal. Independent methodological support for systematic literature appraisal through independent bodies (e.g., Institute for Quality and Efficiency in Health Care, IQWIG), an assumed quality measure, was mentioned in 13/215 6% (6%) of CPGs, mainly in S3 CPGs. Administrative costs weere only addressed in around 30% of CPGs although these are expected in all guideline tiers.

Amount of funding provided for CPGs with available information

The amount of funding provided for CPG development was rarely documented and could therefore not be analysed. We could not establish whether the amount of external funding, or the different types of funding had an impact on the amount of funding. The available sums can be found in the raw dataset (Additional file 1: Dataset). However, these represent only approximations because commonly, only aggregate or indirect information was available (e.g., workload of people involved, paygrades).

Sensitivity analysis

For the sensitivity analysis we also included expired CPGs. A total of 762 guidelines could be analysed (S1 = 283, S2e = 39, S2k = 249, S3 = 191). The analysis did not show any differences compared to the main analysis where expired CPGs were excluded (Additional file 2: Sensitivity analysis) indicating that the accuracy and completeness of reporting on funding of “out-of-date” CPGs is comparable to valid CPGs and there is no trend towards better reporting of newer CPGs.

Discussion

Main findings

This is the first structured description of the accuracy and comprehensiveness of reporting on funding within German clinical practice guidelines. It can thereby help to inform the public, guideline users, guideline producing scientific societies and funding bodies on strengths and shortcomings of funding practices as well as potential improvements. The funding processes of CPGs in Germany were not reported transparently as shown by rating all 507 valid AWMF guidelines with Criterion 22 of the DELBI tool. The guideline tier had an influence on the transparency scores and funding sources reported. The information presented here suggests that there is incomplete reporting of funding types.

Context of other work

There is evidence from observational studies that individual COI are associated with prescribing patterns. A systematic review and meta-analysis by Brax et al. included 19 studies with different types of interactions, including detailing, industry-funded continuing medical education, and free gifts. They found moderate quality evidence that physicians’ interactions with pharmaceutical companies are associated with their prescribing patterns (OR = 2.52) [20]. A study among 41,257 French General Practitioners found better drug prescription efficiency indicators (e.g., prescription of generic drugs) and less costly drug prescriptions for GPs who did not receive gifts from pharmaceutical companies compared to GPs who received gifts [21].

A Cochrane review [7] that assessed the association of conflicts of interests in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews with recommendations supports these findings (RR: 1.26, 95% CI: 1.09 to 1.44). However, there was no significant difference for CPGs (4 studies with 86 CPGs, RR: 1.26, 95% CI: 0.93 to 1.69), probably related to the small sample size. Norris et al. included twelve studies that target the impact of COI on guideline quality [9, 22] with very limited evidence on the extent of commercial influence [23, 24].

There is also only limited information on the potential impact of guideline funding as shown by Campsall et al. (290 CPGs by 95 organizations) [13]. In this study, more comprehensive COI policies were associated with fewer positive and more negative recommendations regarding patented biomedical products. However, there were no disclosures for financial relationships of organizations although 63% (60/95) reported receiving funds from a biomedical company [13]. As shown above, Elder et al. [14] also revealed a mismatch between the organizational funding of medical societies from the industry and the disclosures in the CPGs. Besides non-transparent reporting, insufficient funding might also influence CPG quality.

Boyd et al. [6] recommend (1) public funding for guideline development, (2) development without commercial support, (3) no COI related to the subject matter and (4) no single source sponsorship. More transparent reporting might help to better understand funding mechanisms and costs of guideline development. This could inform policy makers on how to improve funding practices to ensure high quality CPG development and regulate funding mechanisms to enable timely updates in the future. The AWMF has called for sustainable and independent funding of high-quality guidelines [25]. Steps were already taken in Germany by implementing funding schemes for the systematic analysis of the scientific evidence through the Innovationsfonds (Innovation fund). In addition, independent guideline programs targeting oncology (Leitlinienprogramm Onkologie) and highly prevalent diseases (Nationale VersorgungsLeitlinien, NVL) exist. These programs should be further expanded and used as models for the implementation of other structured programs. However, independence of guideline development and recommendations always needs to be ensured.

Limitations and strengths

A strength of our study is that it relied on a comprehensive listing of German guidelines, rather than a selected sample. However, we cannot generalise these results to guidelines not published in the registry but frequently used in the German speaking context (e.g., CPGs produced by the European Society of Cardiology, other international societies), or other healthcare systems (e.g., UK or USA). This study may not only raise awareness for more transparent reporting of funding practices but, more importantly, challenge the status quo and enable more sustainable and independent funding practices in the future. The development of categories to properly define funding expenses and interpretation of texts provided by the guideline authors was difficult. We neither assessed indirect funding of guideline developing organisations nor individual COI. Mandatory reporting with clear rules and templates, as in the handling of conflicts of interests, could overcome this limitation in future research.

Policy implications

Transparency of guideline funding could be achieved by making it mandatory to publish information for all guidelines. For that purpose, a standardised form and guidance should be developed by the AWMF. This should contain information on the funding bodies and their COI related to the subject matter (e.g., grants from the pharmaceutical industry towards scientific medical societies) [6], expenses and the amount of funding. However, we need to find the right balance between the need for transparency and feasibility. In our own experience, guideline development is complex and, in most cases, the exact costs of an individual’s contribution cannot be calculated. The benefits from publishing details on guideline funding must therefore outweigh the bureaucratic burden. All information should relate to the specific guideline and contain an explicit statement on the independence of the body that develops the guideline from all funding bodies [16]. In case of CPGs exclusively or primarily developed through the contributions of the guideline authors, individual conflicts of interests play a big role. Therefore, funding practices and individual COIs both need to be assessed to get a complete picture.

Conclusion

In recent years, steps were taken in the right direction towards more transparent reporting and handling of potential conflicts of interests by guideline authors in Germany [26]. In the context of guideline funding, this process is still in its infancy. The development of standard forms – as already used for reporting of individual COI - could enhance transparent reporting of funding mechanisms and thereby trust in clinical practice guidelines. Germany could take a proactive role globally and thereby influence other countries to take the same steps.