Background

The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 14th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.

Guidelines are formal advisory statements that should be robust enough to meet the unique circumstances and constraints of the specific situation to which they are being applied [1]. The basic nature and intent of guidelines have been variously labelled as guidance, guides, guiding principles, recommendations, policies, protocols, best practice, algorithms, consensus statements, expert committee recommendations, integrated care pathways, manuals, tool kits, handbooks, model lists, technical updates and principles [1, 2]. Whatever they are called, rigorously developed guidelines, can translate complicated research findings into actionable recommendations. They are an important step in moving from research to action and ensuring that the best available research evidence informs decisions and actions [3]. However, for users of guidelines to be able to apply criteria to assess whether guidelines have been rigorously developed and are likely to be valid and applicable [3–6], the information needed to make these judgements must be reported [7]. Unfortunately, critical information is often absent from published guidelines [5, 7, 8].

In this paper we address the following questions:

  • What standard types of recommendations or reports should WHO use?

  • How should recommendations be formulated and reported?

We address questions about reporting systematic reviews [9] and dissemination [10] in other papers in this series.

What WHO is doing now

Although the Guidelines for WHO Guidelines recommends "that a uniform, readily-recognizable printing format be developed for WHO guidelines," there are, as yet, no standard formats for WHO policies, recommendations or guidelines. A survey of WHO guidelines published in 2005 found that WHO publishes a large number of recommendations of many different types, in many different formats [2], and a review of WHO documents did not find any standards for reporting WHO recommendations [11].

What other organisations are doing

In a recent survey of organisations that produce clinical practice guidelines, all 31 organisations that responded (response rate 86%), and 46 of 57 (81%) of units that support the use of research evidence by governments in developing health policy, reported producing full versions of guidelines with references and notes [12]. Several organisations use different formats for different types of recommendations, and a majority produce different versions of guidelines, such as executive summaries, summaries of take-home messages, separate versions for different target users, and tools for application (e.g., algorithms or flow charts). Many guideline producers have standard formats that they use and some organisations, such as the U.S. National Guidelines Clearing House, have developed standard formats for reporting guidelines produced by other organisations [13].

The UK National Center for Health and Clinical Excellence (NICE) states that recommendations should be clear and concise, but should contain sufficient information that they can be understood without reference to other supporting material [14]. This is particularly important where recommendations are published in isolation from the background details in the full guideline. Any terminology included in the recommendations therefore needs to be clearly defined and unambiguous.

Methods

The methods used to prepare this review are described in the introduction to this series [15]. Briefly, the key questions addressed in this paper were vetted amongst the authors and the ACHR Subcommittee on the Use of Research Evidence (SURE). We did not conduct a full systematic review. We searched PubMed and three databases of methodological studies (the Cochrane Methodology Register [16], the US National Guideline Clearinghouse [17], and the Guidelines International Network [18]) for existing systematic reviews and relevant methodological research that address these questions. The answers to the questions are our conclusions based on the available evidence, consideration of what WHO and other organisations are doing, and logical arguments.

For this review we searched PubMed using [the MeSH terms 'Documentation/standards' and 'Practice Guidelines/standards'] and related articles; the Cochrane Methodology Register using [the key words 'CMR: Review methodology – applicability & recommendations' and the text words (format or reporting)] and ['Levels of evidence and strength of recommendations']; the National Guidelines Clearinghouse annotated bibliography using the terms format, reporting and structure; and checked the reference lists of retrieved articles.

Findings

What standard types of recommendations or reports should WHO use?

Given the wide variety of different types of recommendations that are made by WHO, there is likely to be a need for several standard types of recommendations. Systematically developed clinical practice guidelines can take 18 months or more and as many as 15 meetings [19]. Systematically developed public health guidelines also require substantial resources and time [20]. Given the time and resources required to produce guidelines, many organisations, particularly HTA organisations, have developed rapid assessment processes [21–25]. There is variation in the scope, methods, time to complete assessments, and the formats used to report rapid assessments. Another type of recommendation or guideline that is receiving increasing attention, also because of the resources and time required to develop guidelines systematically, are guidelines developed by other organisations that have been adapted or endorsed [26–28]. Another approach being taken in several countries is to create databases or clearinghouses of clinical practice guidelines with the aim of facilitating their evaluation and adaptation for local use by health care organizations [29].

Systematically developed clinical recommendations, public health recommendations, and health systems recommendations all require similar processes to ensure their quality. Decision makers also require similar types of information to be able to critically appraise whether guidelines have been rigorously developed and are likely to be valid [3, 4, 7, 20, 30, 31].

How should recommendations be formulated and reported?

The Conference on Guideline Standardization (COGS) used a two-stage modified Delphi process to develop standards for reporting clinical practice guidelines [7]. Representatives of 22 organisations active in guideline development reviewed the proposed items and commented favourably. The items were consolidated into 18 topics (Table 1) to create the COGS checklist, which provides a framework to support comprehensive documentation of guidelines. While it is possible that some guideline developers may not include content for every item, it is suggested that they should address explicitly whether the guideline development team considered that item.

Table 1 The COGS checklist for reporting clinical practice guidelines (from Shiffman et al. [7])*
Full size table

While many organisations have their own standard formats for reporting guidelines, this is the only consensus standard for reporting guidelines across organisations. We have, however, summarised the key items included in guidelines for guidelines, which provides the basis for a similar, but more comprehensive checklist for conducting or reporting guidelines [32]. In addition, there are a number of instruments for evaluating clinical practice guidelines that can also be used as checklists for reporting [1, 4–6, 33]. The content used in the National Guideline Clearing house also represents a standard for reporting imposed on organisations that want their guidelines included in that database [13]. It includes 52 items under the following headings: scope, methodology – including rating scheme and cost analysis, recommendations, evidence supporting the recommendations, benefits/harms of implementing the recommendations, contraindications, qualifying statements, implementation of the guideline, Institute of Medicine (IOM) national healthcare quality report categories, identifying information and availability, and disclaimer; in addition to indexing attributes.

Similarly, some journals have standard formats for reporting clinical practice guidelines, including structured abstracts with the following headings [34]:

Objective

a succinct statement of the objective of the guideline, including the targeted health problem, the targeted patients and providers, and the main reason for developing recommendations concerning this problem for this population.

Options

principal practice options that were considered in formulating the guideline.

Outcomes

significant health and economic outcomes identified as potential consequences of the practice options.

Evidence

methods used to gather, select, and synthesize evidence, and the date of the most recent evidence obtained.

Values

persons and methods used to assign values (relative importance) to potential outcomes of alternative practice options.

Benefits, harms, and costs

the type and magnitude of the main benefits, harms, and costs that are expected to result from guideline implementation.

Recommendations

a brief and specific list of key recommendations.

Validation

the results of any external review, comparison with guidelines developed by other groups, or clinical testing of guideline use.

Sponsors

key persons or groups that developed, funded, or endorsed the guideline.

While many organisations have standards for how recommendations are formulated, we are not aware of any consensus standards for how recommendations should be formulated. Most guidelines development groups now grade the quality of evidence and the strength of recommendations, but a variety of different grading systems are used [12, 35].

Shekelle and colleagues evaluated the effect of different levels of specificity of recommendations on clinicians test ordering behaviour using clinical vignettes [36]. They found that clinicians receiving the non-specific recommendations ordered fewer indicated tests for appropriate clinical vignettes than did physicians receiving specific recommendations. Furthermore, compared to physicians receiving non-specific guidelines, physicians receiving specific guidelines ordered significantly more appropriate tests for corresponding vignettes and significantly fewer tests for inappropriate vignettes. The authors concluded that the clarity and clinical applicability of a guideline might be important attributes that contribute to the effects of practice guidelines. We did not find any other comparisons of different ways of formulating recommendations, and it is likely that the way in which recommendations are formulated may need to be adapted to the specific characteristics of a guideline. However, there is a consensus that recommended actions should be stated precisely [7].

Both checklists for evaluating guidelines and for reporting guidelines include items that may be dependent on the specific setting in which a guideline is developed or used. There are a number of international organisations, including WHO, that develop guidelines that are intended to be used in a variety of settings around the world. We did not find any published papers that addressed methods for taking into account setting specific factors in international guidelines, although several groups are working on methods for adapting guidelines developed in one setting for use in another [27, 28, 37–39].

Discussion

While the content of WHO guidelines, recommendations and policies will vary, depending on the topic, it would be desirable to have standard formats across different topics to facilitate recognition and use by decision makers and ensure that all the information needed to judge the quality of a guideline, determine its applicability and, if needed, adapt it is reported. There is likely a need for at least three standard formats: full systematically developed guidelines that are sponsored by WHO, rapid assessments, and guidelines that are endorsed by WHO. Standards such as those advocated by COGS should form the basis for developing a uniform format for full guidelines developed by WHO. Although the COGS standards were developed for clinical practice guidelines, the same considerations are relevant to public health and health systems recommendations. Further consideration is needed regarding the inclusion of additional items that need to be considered in WHO guidelines, such as applicability to different settings, equity, and scaling up. In particular, work is needed to develop a template for decision-making frameworks when different conditions are likely to lead to different decisions in different settings [40, 41]. In addition, different versions of guidelines should be developed for different target audiences, including a structured executive summary and key messages [31, 34].

A different format for rapid assessments would help to distinguish these from full guidelines and could be designed to reduce the work and time necessary to complete a report. Rapid assessments should, nonetheless, include the same information as full guidelines, indicating explicitly what the group preparing the guideline did not do, as well as the methods that were used.

WHO has limited resources and capacity for developing guidelines. At the same time, low and middle-income (LMIC) countries also have limited resources and capacity, and a core function of WHO is to provide its member states, particularly LMIC, with technical advice that is informed by the best available research evidence. Through collaborating with other organisations and establishing standards for reporting, and possibly endorsing guidelines developed by other organisations, WHO may be able to expand its potential for supporting access to guidelines that are appropriate for LMIC or can easily be adapted to conditions in LMIC.

Given WHO's role as the world's leading public health agency and its mandate to provide evidence-informed guidance, it is of major importance for WHO to address and develop methods for formulating and reporting recommendations that are developed internationally, but need to be adapted and implemented in specific settings.

Further work

As part of the methodological work that is needed to addresses the challenges of developing international guidelines, specific attention should be given to the development of standard templates that provide decision makers with the relevant global evidence that is needed to inform a decision and offers practical methods for incorporating the context specific evidence and judgements that are needed [40, 41].

In addition to ensuring that standard formats are used for WHO guidelines to ensure complete reporting, attention should be paid to ensuring that the format that is used, and derivative versions, are understandable and useful to the intended target audiences.