Background

Randomised controlled trials (RCTs) are challenging to conduct in paediatric critical care yet they are crucial for guiding clinical practice. Barriers to conducting these RCTs include difficulties obtaining informed consent [1], resistance to randomisation [2] and limited funding opportunities [3]. Paediatric critical-care RCTs tend to be fewer in number, single-centred and have smaller sample sizes than adult RCTs [4]. Due to these challenges, careful background work is necessary to determine disease estimates, target populations, sample sizes and whether equipoise on the study question exists prior to conducting these trials. Surveys and retrospective chart reviews are often used for this background work [5, 6]. Surveys are inexpensive, easy to administer and may be completed in a timely manner. However, survey results may be difficult to interpret due to potential sampling bias, low response rates and response bias [7]. Retrospective cohort studies provide actual patient data, but usually require funding and are more labour intensive and subject to confounding and misclassification bias [6]. Although the relative merits of these two methodologies are well known, the utility of the specific information obtained using each of these methods in planning a prospective clinical trial has not been evaluated.

Corticosteroid use in the treatment of paediatric septic shock has been debated in the literature for over 40 years [8]. Despite numerous editorials [9], reviews [8, 10] and a limited number of small RCTs [11], the role of corticosteroids for treating paediatric septic shock remains unclear. Furthermore, despite this lack of evidence, it is unclear whether individual physicians have equipoise on corticosteroid use for paediatric septic shock [12]. As such, it is uncertain whether an RCT on the use corticosteroids in the treatment of paediatric septic shock would be feasible. Therefore, in preparation for a future trial, both a survey and retrospective cohort study were conducted to assess current beliefs and practices with regard to corticosteroid use for paediatric septic shock. Our objective was to compare the specific information obtained from each of these respective study designs to determine the need to conduct both types of studies as background work for future RCTs in paediatric critical care.

Methods

We examined responses from questions of a previously published physician survey conducted between February and March 2012 [6] and data obtained from a previously published retrospective cohort study conducted between January 2010 and June 2011 [7]. The original survey included physicians from all academic centres across Canada. However, for this study, only survey responses obtained from paediatric intensive-care physicians at the four academic tertiary care centres that participated in the retrospective chart review were included. The survey included demographic questions, questions on physicians’ practice for patients with septic shock and questions regarding their beliefs relating to a RCT on the use of hydrocortisone versus placebo in the treatment of paediatric septic shock. Each non-demographic question within the survey included selections from choices and, where appropriate, an ‘other’ category where free-text comments could be inserted.

The original cohort study included data from 327 patients with moderate to severe shock from all causes. For this study, we extracted data from the septic shock sub-group, which included 81 patients from birth to 17 years of age with moderate to severe septic shock. The survey and retrospective study both defined moderate shock as hypotension despite a minimum of 60 cm3/kg of fluid and at least one vasoactive infusion. Retrospective data spanned a 1-year period for all but one centre, for which data was collected for 3 months due to its significantly larger patient population. Descriptive statistics are presented for the surveys and retrospective data. A comparison of proportions was performed using a chi-squared test or ANOVA for multiple comparisons between centres (SPSS version 24, IBM Corporation, New York).

Results

The overall participant and patient flow diagram is shown in Fig. 1. All 23 physicians included in the survey were also responsible for patient care during the period of the retrospective cohort study.

Fig. 1
figure 1

Flow diagram of survey respondents and cohort study patients for current study

Full size image

Eligible patient estimates

Less than half of surveyed physicians (11/23, 48%) correctly estimated the number of patients with moderate to severe shock treated at their site per year, with incorrect estimates being both higher and lower than the observed numbers. The retrospective cohort study, however, provided precise numbers of patients with mild, moderate and severe septic shock.

Determination of the primary outcome for a trial

In total, 61% of respondents (14/23) stated that time to discontinuation of all vasoactive infusions was the most feasible and clinically important primary outcome for an RCT on the use corticosteroids in the treatment of paediatric septic shock. The retrospective chart review provided mean values with standard deviations for time on vasoactive infusions for those who did and did not receive hydrocortisone, from which we could estimate sample sizes for a future trial. None of the physicians surveyed chose mortality as the most feasible and clinically important primary outcome. In addition, the retrospective chart review found a baseline mortality rate of 11.7% for paediatric septic shock. As previously shown, this would require a sample size of 7682 patients to demonstrate an absolute mortality reduction of 2% (relative reduction of 17%) or 1764 patients for an absolute mortality reduction of 4% (relative reduction of 34%) [13]. Neither sample size is likely to be feasible given that the largest paediatric septic shock trial required pharmaceutical industry funding and 104 sites in 18 countries to recruit 477 patients in 3 years [14].

Target population for corticosteroid administration in septic shock

When physicians were asked how likely they were to administer corticosteroids for septic shock, their responses varied within and between each centre. Eight physicians stated that they often or always administered corticosteroids (35%, 8/23) and nine physicians stated that they infrequently or never did (9/23, 39%). The observed frequency of corticosteroid use for septic shock did not vary significantly between centres (36%, 6/17; 50%, 10/20; 57%, 17/30; 71%, 10/14; P = 1.00) and there was an overall observed rate of 53% (43/81). A comparison of stated and observed practices regarding corticosteroid administration in septic shock is shown in Fig. 2. Despite the low response rate from site 2, the diversity of responses is similar in all centres.

Fig. 2
figure 2

Comparison of reported versus observed use of corticosteroids in septic shock patients. aThe frequency of corticosteroid use is reported as the number of septic shock patients for whom corticosteroids were administered divided by the number of patients with septic shock

Full size image

The percentage of physicians who stated they would administer corticosteroids, would be willing to randomise patients into an RCT and would administer open-label steroids at each of three levels of vasotropic support is shown in Table 1. The percentage of patients who received corticosteroids at each level of vasotropic support is also shown in Table 1. A statistically higher percentage of physicians stated they would administer corticosteroids to patients who were on two or more vasoactive drugs compared to those on one high dose or one low dose of a vasoactive drug (95.7%, 22/23 vs 52.2%,12/23 vs 8.7%, 2/23; p <  0.0001). However, this is also the population of physicians who were least likely to randomise patients in a trial of corticosteroids for paediatric septic shock (34.8%, 8/23 vs 69.6%, 16/23 vs 65.2%, 15/23; p <  0.0001) and were most likely to administer open-label steroids (87%, 20/23 vs 30.4%, 7/23 vs 4.3%, 1/23; p <  0.0001).

Table 1 Determination of target population for an RCT on the use of corticosteroids for paediatric septic shock
Full size table

Although 95.7% (22/23) of physicians stated they would administer corticosteroids to patients requiring two or more vasoactive medications, only 50.8% (41/81) of such patients received corticosteroids (p <  0.0001). These results were unchanged if the largest centre, which had the lowest survey response rate, was removed from the analysis (100%, 16/16; p = 1.0 and 47.1%, 24/51; p = 1.0).

Existence of equipoise on corticosteroid administration for septic shock

A summary of stated and observed corticosteroid administration practices is shown in Table 1. Only 35% of physicians said they would be willing to randomise septic shock patients who had received 60 cm3/kg of fluid and were on at least two vasoactive infusions (the population of interest) into an RCT of corticosteroids versus placebo. In addition, 87% of respondents stated that they would administer open-label steroids to patients meeting these criteria if they were randomised. Interestingly however, only 50% of such patients actually received corticosteroids in the retrospective review. Of physicians, 70% stated they would be willing to randomise slightly less sick patients who had received 60 cm3/kg of fluid and were on one high dose of vasoactive infusion but 30% of those surveyed said they would still consider open-label corticosteroids for this group of randomised patients. In the retrospective study, 24% of such patients received corticosteroids. The combined information obtained from the survey and the retrospective study suggests that enrolment of slightly less sick patients may be feasible but that strategies will still have to be considered and implemented to prevent open-label corticosteroid use.

Dose and duration of corticosteroid therapy

Altogether, 65% of survey respondents reported using the equivalent of 1 mg/kg/dose of hydrocortisone q6h and 26.1% used 2 mg/kg/dose q6h. In the retrospective study, 56.3% of patients received 1 mg/kg/dose q6h and 26.7% received 2 mg/kg/dose q6h providing strong evidence for consistency between beliefs and practices for hydrocortisone dosing for paediatric septic shock. There was significant variation in stated and observed practices within and between centres, suggesting that there is no standard duration of hydrocortisone therapy for paediatric septic shock. Therefore, the RCT protocol would require a physiologic rationale for duration of therapy rather than one based on an accepted standard of care.

Feasibility of adrenal function testing of septic shock patients

In total, 78% of physicians (18/23) stated that they sometimes, often or always conducted adrenal axis testing prior to giving corticosteroids to patients in shock, but only 5% (4/78) of patients who received corticosteroids actually underwent testing. In each of the four centres, equal percentages of physicians reported using cortisol levels and adrenocorticotropic hormone (ACTH) testing for patients with shock. However, a larger proportion of patients who underwent adrenal function testing had cortisol levels measured compared to ACTH testing (73.3%, 11/15 versus 26.7%, 4/15; P = 0.012). The discrepancy between stated and observed rates of adrenal function testing prompted our team to explore potential barriers to adrenal testing rather than mandating its inclusion in our protocol.

Information gained in the survey and cohort studies

A summary of the information gained from each of the respective study designs and its effect on the ultimate design of the future RCT are shown in Table 2.

Table 2 Summary of information gained from the survey and cohort study
Full size table

Discussion

We used a specific research question to examine the information obtained from a survey and a retrospective cohort study to evaluate their respective utility in designing a prospective RCT. We observed that both types of study were needed to determine the numbers of eligible patients, justify the primary outcome measure, calculate the sample size, select the target population and establish the feasibility of the selected protocol.

Estimating numbers of potentially eligible patients is one of the most important tasks when planning an RCT. Due to the limited literature on paediatric critical care [3], there is heavy reliance on surveys when deriving estimates [15,16,17,18]. We found that the majority of physicians incorrectly estimated the number of patients admitted with shock at their centre, suggesting that reliance on survey data alone may not provide accurate estimates of eligible patients and that despite the additional time and costs involved, observational studies are needed [16]. In addition, the inaccuracy of the patient estimates obtained from the survey suggest that it may be wise to omit such questions from future surveys to shorten the surveys and limit survey fatigue [19].

Choosing an appropriate primary outcome for an RCT is another important component of RCT study design and is crucial for sample size calculations [20], determination of feasibility and interpretation of the relevance of a trial [21]. The choice of primary outcome is often justified based on the available literature. However, this approach may not always provide the most appropriate primary outcome. A systematic review of previous paediatric septic shock trials found that mortality was the most commonly used primary outcome [13] but its use may be limited by low baseline mortality rates in developed countries [22]. Our survey also supported the limited utility of mortality as a primary outcome in paediatric septic shock trials and provided a formalised method for stakeholders to suggest alternative clinically important outcomes. The survey results suggested time to haemodynamic stability as the most clinically important and feasible primary outcome and the retrospective cohort study allowed us to determine means and standard deviations for this outcome measure. Methodologists have highlighted the need for rigour when determining the minimal clinically important difference [23] and our retrospective study provided an estimate of the difference in time to haemodynamic stability between those who did and did not receive hydrocortisone. Thus, both study designs contributed to the justification of haemodynamic stability as a possible primary outcome. Interestingly, none of the respondents from our survey suggested a patient-centred measure as the most important outcome for an RCT. However, a more recent survey found quality of life to be the most commonly chosen primary outcome for a future RCT on using steroids in the treatment of paediatric septic shock [24], which may reflect a growing focus of granting agencies on patient-centred outcomes.

The next step in designing an RCT is to determine the target population. Inclusion criteria are often chosen using a balance between scientific rationale and feasibility and researchers have often used surveys to establish clinical criteria for RCTs in paediatric critical care [15, 17, 18]. Physiologic evidence suggests there is a potential benefit of prescribing corticosteroids to haemodynamically unstable patients [25,26,27,28], whom our survey respondents defined as those who had received 60 cm3/kg of fluid and were on two or more vasoactive infusions. Our survey showed that only a third of physicians would be willing to randomise such patients into an RCT and that 87% of physicians would administer open-label corticosteroids to such patients if randomised. This suggests a lack of equipoise amongst individual physicians, which could pose a significant threat to the feasibility of a trial if not specifically addressed. Interestingly though, the cohort study showed that almost exactly half of such patients actually received corticosteroids, suggesting that there may, in fact, be community equipoise on the use of corticosteroids for paediatric septic shock. The lack of clear evidence for benefit or harm from corticosteroids in treating septic shock and the presence of community equipoise together provide the ethical mandate for recruitment into an RCT, further supporting the additional value of the cohort study in preparing for this RCT.

The last step in the design of an RCT is finalising the details of the actual protocol. The survey and cohort study were used to explore the duration of hydrocortisone therapy to be used in the protocol. However, the significant variation in both stated and actual practice showed that there is no accepted standard of care, which provides good justification for the use of limited physiologic data and adult studies to determine the duration of hydrocortisone for the RCT. Finally, the majority of physicians surveyed stated that they would conduct adrenal axis testing prior to corticosteroid administration, but the retrospective data revealed that such testing was rarely performed. The survey results suggested that physicians are aware of the recommendations for adrenal axis testing [29], but the cohort study found that these recommendations were not followed. This discrepancy between stated and observed practices led us to explore potential barriers to adrenal axis testing. These included the cost of ACTH, the lack of real-time availability of test results [30] and difficulty in interpreting the results [31]. Based on these findings, adrenal testing was not included in the protocol.

Finally, it is important to acknowledge the potential limitations of surveys and retrospective cohort studies. Surveys are subject to sampling and response bias, which may lead to a misinterpretation of true beliefs [32]. In addition, concerns have been raised about survey fatigue, which may limit response rates and the quality of information obtained [6]. However, surveys are inexpensive, easy to administer and most importantly, still provide a formalised structure by which to obtain expert opinions. In our specific example, the survey provided valuable information on choice of primary outcome, inclusion criteria, existence of equipoise and protocol details. Retrospective studies may be subject to misclassification bias and incomplete data but provide an important source of preliminary estimates [19] and have played an important role in the design of adult critical-care trials [33]. Our chart review provided us with estimates of eligible patients, justification for the minimal clinically important difference and actual practice patterns.

The strengths of this study include the inclusion of multiple centres and the novel comparison of the contributions of information obtained from a survey versus a retrospective study in the planning of a paediatric critical-care RCT. There are, however, several limitations to this study. Survey non-respondents may have cared for patients within the retrospective study, thus affecting observed values but not influencing reported practices. However, this limitation applies only to one site as the other sites had 100% survey response rates. Lastly, we compared only one survey and only one retrospective study of a single topic in a specialised group of physicians and therefore, our observations may not be applicable to other studies or settings.

Conclusions

The survey provided valuable information on physician preferences for the choice of primary outcome, identification of the target population and individual physician equipoise. The retrospective cohort study provided estimates of eligible patient numbers, the minimal clinically important difference, evidence of community equipoise and actual physician practice patterns. A comparison of the information obtained from each of the two study methods provided necessary insights into the feasibility of recruiting our target population and of conducting adrenal function testing. Strong consideration should be given to performing both types of study prior to conducting an RCT in a paediatric critical-care environment.