Background

According to the World Health Organization (WHO), standard clinical practice guidelines (hereafter referred to as "standard guidelines") must be developed clearly and transparently following a rigorous methodology to obtain unbiased and high-quality recommendations and minimize the influence of conflicts of interest. The development of standard guidelines however will require on average about six months to two years to complete [1, 2]. Due to time constraints and the enormous input needed, developing standard guidelines is not the best way to assist decision-makers to rapidly make evidence-based decisions in a short period of time in case of public health emergencies. Thus, in 2006, the WHO introduced the concept and methodology of rapid advice guideline (RAG), which is characterized by short, scientific and standardized way of development. RAG aims to respond to potential public health emergencies or urgencies [3]. RAG has been proven successful with applications in several infectious diseases such as H5N1, HIV, and Ebola [4,5,6].

Since December 2019, a novel infectious disease (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection has swept through the world, posing significant challenges to public health safety and healthcare delivery systems worldwide [7]. In such public health emergencies, guideline developers face great challenges in making urgent reasonable recommendations. Therefore, the development of a RAG is one of the effective ways to address this issue [1, 2, 8, 9]. On January 28, 2020, the development of the first international rapid advice guideline for management of children with COVID-19 (hereinafter referred to as the "RAG for children with COVID-19"), following the WHO RAG methodology and led by the National Clinical Research Center for Child Health and Disorders, was launched. RAG for children with COVID-19 was initiated by the contributions of the WHO Collaborating Center for Guideline Implementation and Knowledge Translation. It took three months from approving RAG to its publication [10, 11]. This guideline focuses on the management of children younger than 18 years infected with SARS-CoV-2, and covers screening, diagnosis, treatment, and patient education. The guideline has so far been translated into 20 languages and accepted into the Guidelines International Network library [12]. Based on the practical experience from the "RAG for children with COVID-19", the aim of this article is to summarize the methodological processes and experiences in the development of the RAG, and provides suggestions and reflections on the future development of RAGs.

Assessment of the need for the RAG

The WHO suggests to consider five factors when deciding to develop a RAG rather than standard guideline, or to delay the development of guideline. If all these five aspects are met, the development of a RAG is considered necessary. Therefore, prior to the official launch of the RAG for children with COVID-19, the core members (CMs: Liu E, Smyth RL, Chen Y, Luo Z, Li W, Zhou Q and Ren L) of the RAG working group assessed the need and rationale for the development of the RAG for children with COVID-19 based on these five aspects suggested by the WHO [1]. Having found that all five criteria were met, they concluded that it was necessary to develop a RAG for children with COVID-19 (Table 1).

Table 1 Results of the assessment of the need for a guideline for children with COVID-19
Full size table

Selection of development manuals

There are currently more than 60 manuals for the development of standard guidelines, including manuals issued by the Institute of Medicine (IOM), Canadian Medical Association (CMA), Scottish Intercollegiate Guidelines Network (SIGN), World Health Organization (WHO), American College of Physicians (ACP), and National Institute for Health and Care Excellence (NICE) [1, 26,27,28,29,30]. It generally takes one to two years to develop a standard guideline by following the methodology of these manuals [1, 26]. In response to public health emergencies, we need manuals that can shorten the time of guideline development. The GIN-McMaster guideline development checklist and the WHO RAG development handbook can help guideline developers complete a guideline in a short time [3, 31,32,33]. We compared the two manuals and finally chose the WHO RAG handbook for the following reasons: 1) The WHO RAG handbook has been successfully applied in the development of the RAGs of childhood tuberculosis treatment and HIV prevention and treatment in children and adolescents [34,35,36]; while GIN-McMaster has not yet been implemented in the development of RAGs; 2) We have recruited experts from WHO, who can provide us with support for the development of a RAG, and experts from WHO also recommended us to use the WHO RAG handbook; and 3) Both the WHO RAG handbook and the GIN-McMaster guideline development checklist were co-developed by experts from McMaster University, and their development processes were likely similar.

The methodology and experience from the development process

Formulation of the RAG Working Group

The RAG was initiated on January 28, 2020 by the National Clinical Research Center for Child Health and Disorders. On February 3, 2020, one of the CMs (Li W) sent invitation letters to selected experts in the relevant disciplines. Sixty-seven members from 11 countries were invited and indicated their willingness to participate in the development of the guideline. Three working groups were formed (see Additional file 1): (I) A Guideline Development Group (GDG), which comprised 39 panelists from various disciplines, including pediatricians, infectious disease physicians, pulmonologists, epidemiologists, clinical pharmacists, methodologists, nurse practitioners, health economists, general practitioners, legal experts and global health researchers. The main responsibilities of the GDG were to draft the proposal, identify clinical questions, reach consensus on the recommendations with use of a Delphi approach, and approve the final version of the guidelines; (II) A Rapid Review Group (RRG), which comprised 26 evidence synthesis methodologists and pediatricians. The primary responsibilities of the RRG were to collect clinical questions, develop rapid reviews, assess the quality of evidence, prepare recommendation decision forms and take meeting minutes; and (III) Patient Representatives (PR), which comprised of two guardians of children with COVID-19. We did not recruit children or adolescent themselves for the following reasons:1) at that time, no policy for recruitment of children with COVID-19 existed and the ethics committee of Children’s Hospital of Chongqing Medical University did not permit us to recruit children with COVID-19; 2) at the time of the development of the RAG, only few children had been diagnosed with COVID-19 and the guardians tended to be reluctant to involve their children in research [37, 38]; and 3) children, especially those under 6 years of age, are also not necessarily able to express themselves clearly enough [39]. The PRs were involved in the voting process for recommendations and providing feedback on the full text of the guideline.

Declaration and management of conflicts of interest

To ensure clarity and transparency in the development of the RAG, all participants were required to declare any potential conflicts of interest that may be relevant to the RAG [40, 41]. One CM (Li W) collected the conflicts of interest declaration forms among all members participating in the development of the RAG through email. Participants were required to declare any financial, professional and other interests that may be related to the RAG. The declared conflict of interests was reviewed by the CMs (Liu E, Chen Y, Li W, Zhou Q). Based on the assessment of conflict of interests, we limited the participation of experts with potential conflict of interests in the core work or excluded them from the overall development of the guideline. All disclosures and conflicts of interest management decisions have been publicly reported [42, 43].

Registration of the guideline and publication of the protocol

The RAG was registered on the International Practice Guidelines Registry Platform (registration No. IPGRP-2020CN008). The protocol was published on March 26 of 2020 [11].

Collection and prioritization of clinical questions

Four CMs (Liu E, Chen Y, Li W, Zhou Q) conducted a scoping review of the published literatures related to COVID-19, and developed the initial clinical questions related to children with COVID-19 [44]. After two rounds of discussions among three CMs (Liu E, Smyth RL, Li W) and clinical experts working in the frontline of infectious disease prevention, 26 clinical questions were identified for further assessment of their importance (Table 2).

Table 2 List of clinical questions
Full size table

On February 2, 2020, one CM (Li W) sent a clinical questionnaire to 33 GDG members by email. All experts scored each question from 1 to 7 according to its importance (1: the question is not important at all and should not be included in the guideline; 7: the question is extremely important and must be included in the guideline). Twenty-seven (81.8%) of the invited 33 GDG members completed the questionnaire. On February 7, 2020, two CMs (Li W, Zhou Q) prioritized the clinical questions according to the questionnaire scores and modified the questions based on the feedback from the GDG. After considering the score ranking and discussing the feedback, six CMs (Liu E, Smyth RL, Chen Y, Luo Z, Li W, Zhou Q) reached an agreement and included ten clinical questions (Table 2).

Evidence retrieval, assessment and synthesis

Based on the ten selected clinical questions, the RRG conducted 11 rapid reviews [45,46,47,48,49,50,51,52,53,54,55]. Each rapid review was assigned specific inclusion and exclusion criteria, based on the amount of available direct or indirect evidence on COVID-19 in children. In the event of sufficient available studies, we included only direct evidence from COVID-19. If there were no studies on COVID-19 in children or indirect evidence from adults, we also included indirect evidence from other coronavirus respiratory infections such as Severe Acute Respiratory Syndrome (SARS) or Middle East Respiratory Syndrome (MERS) to answer the clinical questions.

The RRG systematically searched the following databases: Cochrane library, MEDLINE (via PubMed), Embase, Web of Science, CBM (China Biology Medicine), CNKI (China National Knowledge Infrastructure) and Wanfang Data. The search terms were COVID-19, 2019-CoV, Novel coronavirus, 2019-nCoV, Middle East Respiratory Syndrome Coronavirus, MERS, MERS-CoV, Severe Acute Respiratory Syndrome, SARS, SARS-CoV, and their synonyms, as well as terms related to specific the topic of each review. We developed the search strategy in cooperation with an experienced medical information retrieval specialist [5, 56]. The first searches were completed on 28 February 2020. Later, due to the length of the development process of the RAG, the searches were updated on March 31, 2020. In addition, the RRG also manually searched SSRN (https://www.ssrn.com/index.cfm/en/), medRxiv (https://www.medrxiv.org/) and bioRxiv (https://www.biorxiv.org/) preprint platforms and Google Scholar, and tracked references of the included studies.

Since another high-quality rapid review on the same topic was published, the rapid review on quarantine to control COVID-19 was discontinued on 8 April 2020 [45, 57]. Finally, the RRG completed 13 rapid reviews (see Additional file 2). The GRADE method was used to grade the evidence from the results of the rapid reviews [45, 58]. The detailed grading results were reported in the published articles. The questions and supporting evidence from the rapid reviews is shown in Table 3.

Table 3 The questions and supporting evidence from rapid reviews
Full size table

Formulation of recommendations

Based on the ten clinical questions and supporting evidence from rapid reviews, six CMs (Liu E, Smyth RL, Chen Y, Luo Z, Zhou Q, Li W) initially drafted ten recommendations. The relevant materials and questionnaires were sent to the GDG members by email in electronic format, and also to the two children’s guardians in paper versions. The preferences and values of the two children’s guardians and clinical experts were consistent. Two rounds of Delphi survey were conducted on February 24, 2020 and February 28, 2020, respectively, and we voted on the recommendations. The consensus criterion was that at least 70% of the invited panelists agreeing on the recommendation [45, 60]. After each round of Delphi survey, the core group discussed the feedback, revised the recommendations and also disclosed the feedback to the panel members before the next round. Detailed expert feedback can be found in Additional file 3.

A total of nine recommendations reached consensus in the first round of Delphi survey, leaving only one recommendation to be determined (Recommendation #3: CT scan should not be used routinely in the diagnosis of COVID-19 in children, although it may be helpful in monitoring children who develop severe respiratory symptoms). Six CMs (Liu E, Smyth RL, Luo Z, Chen Y, Zhou Q, Li W) revised the wording and rationale of the recommendations based on the first round of Delphi survey. Afterwards, a second round of Delphi survey was conducted for two recommendations: recommendation #3 for which consensus was not reached in the first round, and recommendation #6 (Recommendation #6: systemic glucocorticoids should not be used routinely for children with COVID-19. Only low-dose and short-duration systemic glucocorticoid therapy can be used for children with severe COVID-19 in the context of clinical trials). The recommendation was divided into two sub-recommendations (a. We recommend against using systemic glucocorticoids for children with COVID-19 routinely; b. We suggest a low dose and a short duration for severe COVID-19 children only when over inflammatory reaction or in the context of clinical trials) after feedback from the first round. Consensus were reached for both recommendations, resulting in a total of ten recommendations being finally included in the RAG. The panelists made a total of 181 comments during the two rounds of Delphi survey (Table 4).

Table 4 Results of the two rounds of Delphi survey
Full size table

Draft of RAG and external review

According to the Reporting Items for Practice Guidelines in Healthcare (RIGHT) reporting checklist, the RAG development process, clinical questions, recommendations, evidence summary and research gaps, two CMs (Zhou Q, Li W) drafted the first version of the RAG [45, 61]. Four other CMs (Liu E, Luo Z, Smyth RL, Chen Y) made secondary revisions to the content and expression. The revised version of the RAG was reviewed by four invited experts. We selected the external reviewers to ensure a balanced representation of different specialties (one external reviewer was specialized in health policy and guideline methodology, two in pediatric respiratory medicine, and one in pulmonary and critical care medicine), 2geographical regions (one reviewer from the WHO, UK, Chinese mainland and Hong Kong, China each) and genders (one female, three male). All external experts were requested to complete a conflicts of interest form, to identify any factual errors and to comment on the clarity of the language and contextual issues. Finally, the core members discussed and revised the guidelines based on the external reviewers’ comments.

In summary, the development of the RAG by a group of 67 researchers from 11 countries took 50 days from the official commencement of the work (January 28, 2020) to submission (March 17, 2020). The process took another 50 days from submission to acceptance, resulting in a total duration of 100 days (Table 5). In addition, to ensure real-time communication, the development of the RAG was intensively discussed in online meetings almost every week from February 1st to March 7th. A total of 21 meetings between the CM and the RRG were held throughout the RAG development process with a total duration of 48 h (average 2.3 h per meeting) and an average of 16.5 participants attending (Table 6).

Table 5 Actual versus planned completion time of the development of RAG
Full size table
Table 6 Meetings between Core Members and Rapid Review Group during the RAG development
Full size table

Dissemination and implementation

We have translated this guideline into the following 20 languages: English, Chinese, Japanese, Russian, German, French, Italian, Vietnamese, Thai, Spanish, Arabic, Portuguese, Polish, Czech, Romanian, Burmese, Hungarian, Hebrew, Hindi, Turkish and Malay. The guideline is indexed in the International Practice Guidelines Registry Platform (IPGRP, http://www.guidelines-registry.org/news/141), the Guidelines International Network (GIN, https://g-i-n.net/get-involved/resources/), and the Emergency Care Research Institute (ECRI, https://guidelines.ecri.org/profile/1868) databases. In addition, we contacted the editor-in-chief in advance and discussed publication strategies to ensure a rapid publication (e.g., taking the fast track option, and fast recruitment of peer-reviewers). Considering the differences in health policies and systems, resources, feasibility and equity across the countries, through above strategies, we will assist countries and regions to adopt or adapt the guidelines into their local context.

Updating plan of the RAG

There is still a lack of effective manuals on how to update the RAG to the standard guideline. Therefore, our working group has proposed the following update plan: 1) According to the WHO guideline handbook [1], when a public health event lasts longer than six months, it should be considered to develop a standard version of the guidelines, and we therefore will update the RAG; 2) The evidence evaluation group will continuously monitor the new evidence from the field of COVID-19 (especially related to children with COVID-19), summarize the new evidence and submit the summaries to the CMs; 3) The CMs will evaluate whether potential new clinical questions should be added in the updated version and the whether the original recommendations are out of date according to the principles proposed by Shekelle et al. [62]; 4) We will invite new clinical experts and methodologists to participate in the updating of the guideline if necessary; 5) We will monitor the updating of existing RAG on COVID-19 and use their experience to guide the updating of our guideline; and 6) The updated guideline will be reported in accordance with the CheckUp checklist [63].

Challenges in the development process of the RAG

Difficulties in collecting and prioritizing clinical questions

As shown in Table 5, it took 50 days to complete the entire development process of the RAG from launch of the guideline to submission to a medical journal. This was 20 days longer than the planned schedule (one month). Two weeks of this time was spent on the identification of clinical questions (proposing clinical questions, selecting the initial clinical questions, prioritizing questions, and formatting them in PICO format).

Because COVID-19 is an emerging infectious disease that was poorly understood in children and even adults when this work was started, it was extremely challenging to select the top ten clinical questions at the beginning of the development of the RAG. First, there was a severe paucity of literature in children on COVID-19 at that time. Second, the limited number of pediatric cases were all contacts of infected adults, and the transmission routes and dynamics among the children themselves or through other sources (e.g. contaminated surfaces) were unknown. Third, the RAG development groups had no frontline clinicians with relevant clinical experience of the natural history of the disease. Fourth, as an international RAG, the clinical questions needed to be applicable to a wide and heterogeneous target population around the world.

The health systems and policies of each country need to be taken into consideration in the process of identifying clinical issues, and the importance of clinical issues vary from country to country. For example, during the survey of clinical questions, the experts proposed the question "who should be tested for nucleic acid at the beginning of an outbreak?". Health policies in some countries have been very clear on this issue (China's policy is to provide free nucleic acid testing to all persons who show clinical signs and have had close contact with a confirmed case [64]). But in most countries, only a limited number of people could be tested in the early stages of the pandemic because of the lack of nucleic acid testing kits.

Limited of direct evidence

The RAG was supported by 13 rapid reviews that incorporated evidence from studies of COVID-19, SARS and MERS. Table 3 shows only two of the ten recommendations were fully supported by direct evidence for COVID-19, three recommendations were supported by indirect evidence only, and the proportion of COVID-19 studies among the body of evidence in the remaining five recommendations ranged between 10 and 83%. Moreover, only one of the recommendations was fully supported by direct evidence from children with COVID-19, and for seven recommendations there was almost no direct evidence from children with COVID-19.

Non-peer-reviewed evidence

The RAG aims to provide instructional recommendations in the short term, and therefore studies available as preprints were used to support the evidence. Table 3 shows that six of the ten recommendations used COVID-19 preprints as evidence support, and up to 50% of the studies with direct evidence on COVID-19 were preprints. The RAG recommendations may have been influenced by the variable quality of the underpinning evidence, which may challenge the reliability of the guideline.

Rapid evolution of evidence

COVID-19 is a global pandemic and the research evidence is growing rapidly. Some of the original studies included in the rapid reviews were retracted for various reasons, resulting in the data being updated again during the production of the rapid review. For example, two studies included in the rapid review for the clinical question #4 (“Should Antiviral drugs such as ribavirin, interferon, remdesivir (GS-5734), lopinavir /ritonavir or oseltamivir be used to treat children with COVID-19?") were withdrawn, a controlled study on favipiravir was temporarily removed from Engineering on April 1 [65], and another non-randomized controlled trial of hydroxychloroquine was withdrawn from the International Journal of Antimicrobial Agents on April 3rd due to not meeting the industry consensus criteria [66]. Because of the withdrawal of these studies, the rapid reviews had to be updated and revised.

Strengths in the development process of the RAG

Prospective registration and publication of the guideline protocol

The registration and publication of the protocol determines in advance the steps and methodology to be followed in developing the RAG according to the WHO handbook. Thorough development and documentation of the methodology, key issues and outcomes reduces the bias in the formulation of recommendations and minimizes the risk of inducing randomness into decision-making. In addition to the above, the registration of the guideline helps to increase the transparency of the development process, avoid duplication, enhance the credibility of the guidelines, and facilitate their dissemination and implementation.

Considering parental preferences and values

The collection and integration of parents’ preferences and values to form recommendations will help families fully understand the pros and cons of various treatment options, so that they have a stronger sense of acceptance and willingness to participate in clinical decision-making. Consideration of families’ preferences and values can thus improve patient compliance, maximize patient benefits, and improve their clinical outcomes [67]. Therefore, the RAG conducted a recommendation survey process that incorporated two guardians of children scoring our recommendations and providing relevant feedback.

Collection and prioritization of clinical questions

Clinical questions are the starting point for guideline development that determine the scope of evidence to be retrieved and evaluated at a later stage. Clinical questions also determine the content of the final recommendations [68]. Due to the urgency of the COVID-19 public health situation, solving all clinical problems at once was not possible, so we conducted a Delphi survey to rank clinical problems by a representative group of experts, and finalized 10 key clinical questions for clinicians fighting the epidemic in the frontline.

Evidence-based recommendations based on rapid review

The development of guidelines must consider the best currently available research evidence to ensure that the recommendations are comprehensive and objective. Rapid reviews can synthesize the available research evidence on the same topic in a short duration of time, and thus reduce the risk of bias and ultimately improve the reliability and accuracy of decision making. Our guideline was accompanied by 13 rapid reviews to support the development of recommendations.

Advantages of following the manual and protocol

It is well known that strict adherence to guideline development manuals and the protocol can enhance the transparency and clarity and ensure the quality and credibility of the guideline [69]. In this particular case, we did not use other standard handbooks due to time constraints. We registered a protocol, and followed the protocol and the WHO RAG manual in every step of the development process. The manual is specifically intended for rapid advice guidelines, and contains several aspects that help to carry out the development in a rapid and efficient manner. The recommendations of the WHO RAG handbook differs from the standard guideline development in the following ways:1) The method of rapid systematic review production is used instead of the traditional development process of systematic reviews in evidence synthesis, which helps to save a lot of time in document screening, evaluation and synthesis; 2) Online meetings are used instead of face-to-face meetings, which saves time, administrative workload and resources of both the organizers and the attending experts; and 3) the WHO RAG handbook requires all members of the expert group to prioritize the preparation of the guidelines, postpone other non-urgent matters, and provide efficient feedback around the rapid development of the guidelines, which guarantees a rapid turnaround time of thefeedback and response.

Conclusion

In order to respond to public health emergencies, the development of a RAG requires a clear and transparent formulation process, and usually uses a large amount of indirect and non-peer-reviewed evidence to support the formation of recommendations. Strictly following the WHO RAG handbook does not only enhance the transparency and clarity of the guideline, but also speeds up the guideline development process, thereby saving time and resources.