Guidance for guidelines

In 2011 the Institute of Medicine (IOM) [1] updated and extended its definition of clinical practice guidelines (CPGs). The current definition explicitly refers to evidence-based medicine, and asks for different options of care to be outlined: ‘Clinical Practice Guidelines are statements that include recommendations intended to optimise patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.’ [1].

While several organisations, such as the WHO, National Institute for Health and Clinical Excellence (NICE) and Scottish Intercollegiate Guidelines Network (SIGN) have suggested standards for developing CPGs, the recently published IOM ‘Standards for developing rigorous, trustworthy clinical practice guidelines’ [1] are considered gold standard practice [2]. According to the IOM [1], trustworthy CPGs should be ‘based on a systematic review of the existing evidence; be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups; consider important patient subgroups and patient preferences, as appropriate; be based on an explicit and transparent process that minimises distortions, biases, and conflicts of interest (COIs); provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations; and be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.’ The eight categories of IOM standards are listed in Text box 1.

The feasibility and practicability of the IOM standards have been questioned, and the Guidelines International Network (G-I-N) argues for minimal standards [3]. No doubt the gold standard is aspirational, but how can it be achieved? What supportive structures and skills are needed to guarantee feasible evidence-based guideline development?

In this review we discuss selected methodological issues related to CPG development and suggest procedures for pooling resources and preparing CPGs for informed shared decision making with patients.

Methods

PubMed was searched in August 2012 and then in December 2012 for updates, for reviews on the quality of guidelines and on generic issues of guideline development. In order to avoid missing the most important contributions to the discussion about the quality of CPGs we used a rather open search strategy combining the Medical Subject Headings (MeSH) term ‘Practice Guidelines as Topic’[MeSH] with (method* OR develop*) as text words. The search was limited to the last 5 years and English and German language and generated 11,775 references. The title and abstract of each reference was assessed by one author (GM), and 22 papers were judged as informative for our paper. In addition, selected websites of organisations dealing with guideline development and critical appraisal were screened (e.g. US Agency for Healthcare Research and Quality [AHRQ], G-I-N, IOM, German Institute for Quality and Efficiency in Health Care [IQWIG], NICE, SIGN). Since we were not intending to present a systematic review, the inclusion of papers was not driven by strict inclusion and exclusion criteria.

Variance in quality of diabetes guidelines

IQWiG regularly performs routine reviews of international guidelines to update German disease management programmes, and has high-quality methodological reviewing standards [4]. Recent systematic reviews of guidelines for type 1 and 2 diabetes [5–7] highlight the methodological shortcomings of present diabetes guidelines. Use of the Appraisal of Guidelines for Research and Evaluation (AGREE) I [8] instrument revealed low quality in the ‘rigorousness of the development process’ domain.

In the IQWiG reviews, diabetes guidelines [5–7] appear to have a rather low score compared with guidelines on asthma/chronic obstructive pulmonary disease [9], obesity [10], coronary heart disease [11] and heart failure [12].

A recent review of guidelines on oral medication for type 2 diabetes mellitus has also reported substantial variance in the methodological quality of guidelines for diabetes care [13]. Differences in rigour of development do not necessarily result in different recommendations [14]. Thus, CPGs that fulfil the best methodological quality standards according to AGREE do not necessarily provide the most valid recommendations and vice versa. The role of evidence vs expert opinion remains obscure in many guidelines. This troubling phenomenon has previously been described for guidelines on other clinical topics [15, 16].

Table 1 summarises key findings of two reviews of type 2 diabetes guidelines, one by IQWiG and the other by Bennett et al [7, 13]. The IQWiG review [7] excluded some guidelines from the outset because they did not meet even basic criteria for evidence-based guidelines. Minimal criteria for rating a guideline as evidence-based were: (1) systematic literature search; (2) grading the level of evidence and/or recommendations; and (3) linking recommendations with evidence.

Table 1 Methodological appraisal of guidelines on type 2 diabetes using the AGREE instrument [8] in two recent systematic reviews [7, 13]
Full size table

It is noteworthy that the two reviews tend to disagree in terms of quality domains of individual guidelines even though both have used the same AGREE instrument; significant inter-rater variability is a well known limitation of this instrument [17]. The AGREE instrument mainly addresses the methods used for developing the CPGs and the quality of the reporting. The evidence base of the guideline recommendations and the completeness of the body of evidence is not appraised [16, 18], a problem that has not been solved by the publication of AGREE II [17, 19].

ADA/EASD statements on type 2 diabetes treatment

Since 2006 the ADA and EASD have published joint statements on the treatment of hyperglycaemia in patients with type 2 diabetes [20]. The publications are consensus statements by a selected group of medical diabetes experts rather than CPGs. In particular, they lack a detailed and transparent section on the methodological process and handling of conflicts of interest. Nor do they include patient representatives or patient views. This also holds true for the most recent publication, in 2012 [21]. ADA/EASD consensus statements are rated low on rigour of development (Table 1) and have been criticised as being non-evidence-based [22]. In fact, the statements had to be revised in 2008 [23], 2009 [24] and 2012 [21] as a result of emerging evidence from randomised controlled trials reporting patient-relevant clinical endpoints. Based on opinion leaders rather than on available external evidence, targets for blood glucose had been continuously decreased. In 2008, according to the ADA, the overall glycaemic objective for persons with type 2 diabetes was ‘to achieve and maintain glycaemic levels as close to the non-diabetic range as possible and to change interventions at as rapid a pace as titration of medications allow’ [25]. In contrast, the most recent ADA/EASD statement suggests that treatment goals should be personalised and that higher than normal HbA1c targets should be considered [21]. The ADA/EASD 2006 recommendation to include rosiglitazone as a step 2 treatment had to be withdrawn only 1.5 years later [25].

How are guidelines put together?

The objective of improving the quality of CPGs has been driven by the principle that the greater the strength of the evidence incorporated, the greater the quality of the CPGs, and the greater their potential to improve quality of healthcare delivery [26]. Emphasis was placed on validity, reliability, reproducibility, clinical applicability, clarity, multidisciplinary process, scheduled review and documentation [26]. In the present paper we focus on two aspects that are particularly critical and contemporary from the perspective of evidence-based medicine: patient centredness and shared decision making, and COI.

Patient centredness and shared decision making

Traditional guideline development, including the 2012 ADA/EASD position statement [21], targets healthcare providers. Consumer centredness is a dominant paradigm of modern healthcare systems and a core element of evidence-based medicine [27]. The involvement of patients or consumer representatives is claimed to ensure transparency, to some extent, and to help ensure that guidelines do not serve special interests beyond patients’ needs and interests. Consumers of health refers to miscellaneous individuals/groups, i.e. individual patients, formal and informal caregivers, members of the public, voluntary and community organisations representing the interests of patients and their caregivers and patient advocacy groups [28]. Patients might provide perspectives that clinicians and scientists lack and might act to safeguard against any COIs the clinical and scientific experts might have [1].

Caregivers contribute with proxy values and own values. Taking patients’ and caregivers’ values and opinions into account is likely to make CPGs more meaningful to them and might therefore support the applicability of CPG recommendations. Public views might conflict with patients’ perspectives, which is likely to further enrich CPG development [28].

Various techniques have been suggested to get patients and consumers involved [28–30]. However, a solid evidence base for decision making about the most effective and cost-effective ways to include patient and consumer values in CPGs is currently missing [28, 30]. Knowledge is based on descriptive and qualitative studies only or policy papers by CPG-developing organisations [29]. Controlled clinical trials about consumer involvement in CPG development are needed [29, 31].

The values and opinions of consumers could be determined through systematic literature review on qualitative and quantitative studies, and direct elicitation through participation in the guideline development group as an integrated member or as part of the patient panel informing the CPG development group, through focus groups, individual interviews and questionnaires. Consumers might also contribute within a consultation phase of the CPGs [28].

The IOM standards [1] emphasise that CPGs should be relevant to and accessible to consumers [2]. As a prerequisite, the IOM requests sophisticated patient and public involvement at all stages of guideline development. A current or former patient should be engaged in clinical question formulation and review of the draft CPGs. A patient advocate or patient/consumer organisation representative should be a member of the guideline development group.

The IOM acknowledges that the involvement of laypersons in CPG development may be problematic owing to a lack of scientific literacy and training in guideline development [1], and therefore states that effective facilitating strategies should be used, including training in appraisal of evidence [1].

The minimal standards defined by G-I-N [3] do not explicitly outline how patients and consumer representatives are to be involved in the guideline development process. Here the guideline group should ideally, but not necessarily, include healthcare consumers. The authors state that ‘consumers’ views about the quality of life experienced with different medical conditions and interventions can be valuable […] However, lack of training in evidence-based medicine and limited scientific literacy can hinder an evidence-based process. If consumers are included as voting members of guideline panels, consumers may need training and support to fulfil their role.’ These concerns appear to reflect a rather paternalistic attitude, as medical and other healthcare professionals cannot be assumed to possess critical health literacy skills and therefore might ‘hinder’ an evidence-based process to the same extent. For example, in an earlier study we have demonstrated that diabetes physicians and nurse educators are prone to misinterpret the effects of diabetes prevention studies depending on the format of the presentation of the results [32, 33]. Overall, science literacy is low among physicians [34]. Thus, not only patients and consumer representatives need training, but training on critical appraisal skills must be a prerequisite for all guideline group members.

Training courses in evidence-based medicine skills for patients and consumer representatives are offered by various organisations, e.g. the Cochrane Consumer Network [35], and have been shown to be feasible and well appreciated [36]. A recent literature review concluded that the majority of studies concerned with patient participation in guideline development are in favour of active patient involvement in the development process despite the lack of evidence confirming that patient input is useful [37].

Involving patients in the guideline development process creates the danger that not as much attention will be paid to patients’ preferences at the individual level. Co-opting patients or consumer representatives and using other techniques of active patient involvement, such as focus group interviews and surveys on patients’ problems, attitudes and values, as well as a literature search, does not replace individual patient preferences in clinical decision-making processes. Given that the key principle of evidence-based medicine is the integration of best research evidence with clinical expertise and patient values [27], the implementation of guideline recommendations would require the adaptation of decisions to individual values [38]. Guidelines are under suspicion of counteracting shared decision making because patient preferences could be at odds with CPG recommendations, and patient preferences are generally not elicited [39]. Closer links between CPGs and patient decision aids through collaborative development of both have recently been demanded [22, 40]. Patients want to be involved in medical decision making [41, 42]. Evidence-based patient information requires data on the personal risks, benefits and harms of various options, including the option not to intervene. These data could be extracted from publications used for guideline development. The recent ADA/EASD 2012 position statement recommends that treatment should be personalised and that patients should be involved in medical decision making including the utilisation of decision aids [21]. However, implementation of shared decision making in diabetes care will require the provision of evidence-based patient information and decision aids, as well as training of physicians to share decisions with their patients [43].

Conflict of interest

The IOM report ‘Conflict of interest in medical research, education, and practice’ defines COI as ‘a set of circumstances that creates a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest’ [44].

COI between clinicians and industry has been a topic of concern for the medical profession for several decades. This particularly holds true for the development of CPGs [45]. Since CPGs aim to standardise care towards evidence-based practice, their freedom from bias is of utmost importance [46]. Not all organisations producing CPGs have set up COI policies. Existing policies vary widely concerning the types of COI addressed, those from whom COIs are collected and COI management approaches [47].

Simple disclosure of a COI has been accused of being insufficient to protect the guideline from biased recommendations [38]. Therefore, some organisations have gone further and exclude authors with a COI from relevant decision making [48]. Guyatt and colleagues [38, 49] suggest that methodologists without any COI should lead guideline development and have the primary responsibility. Experts with important financial or intellectual conflicts of interest can collect and interpret evidence, but only panel members without important conflicts should be involved in developing the recommendation for a specific question.

The IOM standards [1] request that all individuals being considered as guideline group members should provide a written disclosure of all interests and activities potentially resulting in a COI with development group activity: ‘Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial proportion of income), noncommercial, intellectual, institutional, and patient–public activities pertinent to the potential scope of the CPG.’ [1].

Prior to onset of work, all potential group members should present and discuss their COI within the prospective guideline development group and reflect how the identified COI could influence the CPG development process or specific recommendations.

Whenever possible, CPG development group members should not have any COI. The IOM acknowledges that this might not always be possible, but members with a COI should represent only a minority of the guideline development group. The chair and co-chair should be persons without a COI. The IOM requests that funders should have no role in CPG development [1].

Although the IOM standards have recently been criticised because they do not address the accessibility of COI policies and the optimal presentation of disclosures [48], they go far beyond the minimal standards defined by G-I-N [3]. The latter do not precisely define a management of COI policy; the focus is on disclosure of financial and nonfinancial COIs rather than on the procedures of disclosure, the COI threshold in a CPG, the role of funders and absence of COIs for chairs of the CPG.

The reality of guideline development is far from the IOM standards. A recent study [46] that analysed 14 CPGs on screening, treatment or both for hyperlipidaemia or diabetes found that the vast majority of guideline panels reviewed had financial COIs. Under-reporting of panel members was common. Twelve of the 14 panels had chairs, and half of these had COIs.

The AGREE instrument [19], the most widely accepted approach to assessing the quality of guidelines, is not likely to prospectively contribute to an improvement in COI policies and management in CPG development. Even the recently updated AGREE instrument does not reflect the current best evidence approaches of COI management. AGREE comprises only two items referring to COI, which have apparently low sensitivity: ‘The views of the funding body have not influenced the content of the guideline.’ (item no 22) and ‘Competing interests of guideline development group members have been recorded and addressed.’ (item no 23). As a result, guidelines without reliable COI disclosure and management techniques will receive a high quality rating.

What guidelines can and cannot do for clinicians

CPGs can contribute to the definition of best practice standards in a specific medical discipline. They are increasingly used in place of medical textbooks. CPGs help healthcare professionals assimilate and apply the exponentially expanding body of knowledge [50], are teaching tools for medical students and knowledge of them is a prerequisite for medical licensure; they serve as basis for the evaluation and improvement of quality of care in a population and even act as a basis for reimbursement [51].

High quality CPGs have the potential to promote the use of effective clinical measures and services, to decrease practice variation and to reduce the use of outdated, ineffective measures of questionable value. They are intended to increase the use of effective but underused practices, and target services to those subgroups that might benefit most [47]. CPGs have been shown to improve the process and structure of care. However, the effects of CPGs on patient health outcomes have been poorly investigated [52].

CPGs share some limitations with traditional medical textbooks. CPGs often cannot answer highly specific questions. Their applicability to patients with comorbid conditions is limited [53]. CPGs are temporally delayed instruments; an update cannot easily be conducted and new evidence not flexibly taken into account. CPG development is time consuming and expensive, for example, a NICE guideline costs approximately £400,000. The cost-effectiveness of CPG development is unknown [54]. CPGs are prone to be outdated and updates are often scheduled irregularly [15]. Thus, use of a guideline is not a replacement for a systematic search in biomedical databases or the use of other more flexible information systems and tools.

CPGs are not a substitute for clinical expertise. In the context of evidence-based medicine clinical expertise implies the careful approval of whether a guideline’s recommendation is applicable to the individual patient’s clinical situation or a specific healthcare system. In addition, according to evidence-based medicine, clinical expertise includes ‘the ability to integrate research evidence and patients’ circumstances and preferences to help patients arrive at optimal decisions’ [55]. CPGs cannot replace deliberation about individual treatment goals depending on the specific conditions of a patient. On the other hand, deliberation about a patient’s individual treatment goals and treatment options in the context of evidence-based medicine takes place against a background of best evidence recommendations.

Guidelines are usually specific to a certain healthcare system and cannot be transferred to another country or healthcare system without adaption.

What could be improved?

There is a lot of room for improvement through network transnational activities in the field of CPG development. Experts and healthcare providers worldwide use clinical trials and other studies to compile evidence and to inform decisions. Institutions such as Cochrane Collaboration, NICE, AHRQ, SIGN, Haute Autorité de Santé (HAS), IQWiG and others collect and process a huge amount of information on individual studies, information that is not readily accessible to clinicians or patients. Experts developing regional or national CPGs and Health Technology Assessment reports generally go back to the original studies when undertaking such work. They spend weeks and months identifying and critically appraising the evidence base. A large part of this time is spent repeating work already done by colleagues [22, 56].

The EASD could provide a clearing house for the collection and dissemination of information from systematic reviews, clinical studies and other sources of evidence. This could be in the form of a database providing complete information (supplemented as needed by unpublished information available to, for example, Cochrane reviewers) on the individual studies. Experts in methodology would design a template for the presentation of information from such studies, and gaps in knowledge should be clearly identified. Handling of COIs of contributors would be indispensable for credibility and acceptability.

What should be provided is a description of the features of a study according to the standards of evidence-based medicine. The strengths and limitations of the study protocol, performance of the trial and reporting of results should be compiled. Deviations from the study protocol, and apparent or evolving biases, should be reported. The study profile could be continuously updated to include data that were hidden by intention but uncovered later on and information not originally published [22]. Interpretation should be separated from data presentation.

The information would be disseminated in scientific Wiki fashion, which would mean that it would be openly available and constantly updated (in monitored fashion) by those qualified to do so. The Cochrane Metabolic and Endocrine Disorders Group would be an excellent partner in this outstanding and exemplary project. A comparable suggestion has been published recently on an open, collaborative Internet-based systematic review data repository [56].

A further important aim would be to develop and present information in a way that can be shared with patients, allowing them—if they so wish—full participation in informed decision making about their own condition and treatment. This includes providing information on individual risk assessment, benefit and harm related to patient relevant outcome measures or positive and negative predictive values of diagnostic interventions [57].

As a scientific association, the EASD could provide an exceptional contribution to the information technology-driven ‘third health revolution’ [58] by setting up such a database. As a result CPG development processes should be easier, and it would represent a major step towards the simultaneous development of CPGs and topic-related patient decision aids. Finally, this strategy might help to close the gap between minimal standards and best practice in CPG development.