Introduction

When designing a clinical trial, investigators make several fundamental decisions that impact the feasibility to conduct the study in a timely fashion, and the significance and generalizability of the results. In 2009, the United States National Heart, Lung, and Blood Institute (NHLBI) convened a workshop entitled “Beyond Mortality: Future Clinical Research in Acute Lung Injury” to examine specific issues related to the conduct of clinical trials including the selection of outcome variables [1]. Relevant to this pro-con debate, the panel of experts concluded that mortality continues to be the most critical outcome in ARDS clinical trials [1]. Despite these recommendations, many ARDS clinical trials continue to use non-mortality variables as their primary outcome. In this pro/con debate, I present four reasons and provide supporting evidence for why mortality should be the preferred primary outcome variable for ARDS clinical trials.

Therapies have been identified that improve ARDS mortality

A recent review of 146 critical care clinical trials concluded that many studies are powered to identify likely unrealistic treatment effects, particularly when using mortality as the primary outcome variable [2]. For the 40 clinical trials that used mortality as the primary outcome, only 4 (10 %) were positive. Similarly, the ARDS literature is filled with clinical trials of promising therapies that eventually yielded negative results. However, some ARDS clinical trials have demonstrated mortality benefits including: low tidal volume ventilation, higher amounts of positive end-expiratory pressure (PEEP), early neuromuscular blockade, and prone positioning [3–8]. These positive ARDS clinical trials demonstrate several important points. First, there are interventions that actually can reduce the mortality in ARDS patients. Second, not all ARDS clinical trials require extremely large sample sizes. The Amato lung protective ventilation study and the Talmor PEEP titration study were both positive studies with sample sizes of less than 100 [4, 8]. Other times, meta-analysis of several clinical trials are necessary to demonstrate a positive treatment effect [5]. Third, it is essential to enroll patients that will most likely respond to the invention. By including heterogeneous patients with either hypoxemic acute respiratory failure or patients with less severe ARDS, the initial prone positioning studies were potentially destined to yield negative results [9, 10]. The recent Proseva prone positioning study narrowed their inclusion criteria to include patients who were most likely to respond to the intervention; those with severe hypoxemia [6]. This restriction in the inclusion criteria is one potential reason why the study identified a positive effect of prone positioning on ARDS mortality. Previous prone positioning studies may have also been negative because of an inadequate duration in the prone position [9, 10].

Valid intermediate outcome variables do not currently exist for ARDS clinical trials

The use of intermediate outcomes variables is advantageous for several reasons. For most studies, they occur more frequently than the more definitive outcome. They may be categorized as a continuous as opposed to a dichotomous outcome, enabling a reduction in the estimated sample size. Intermediate outcome variables may also develop more rapidly than more definitive outcomes. Therefore, clinical trials that use intermediate outcomes can often decrease the sample size, length, and ultimately the cost of the study. However, the validity of these study results assumes that the intermediate outcome is clinically or biologically relevant.

Other diseases are fortunate to have well-established intermediate outcome variables that correlate with more definitive outcomes. For example, treatments that reduce hemoglobin A1c levels in diabetic patients also improve definitive outcomes such as diabetes-associated mortality [11]. Unfortunately, intermediate outcome variables do not currently exist for ARDS studies. Treatments that improve oxygenation do not uniformly result in improvements in ARDS mortality [3, 12]. Similarly, changes in cytokine concentrations are not currently useful intermediate outcome measures for ARDS [13]. An intervention that reduces the concentration of a certain pro-inflammatory cytokine may be scientifically intriguing. However, physicians would be remiss to change their clinical practice based on the ability of an intervention to reduce the concentration of a deleterious cytokine.

The inconsistent use of non-mortality outcomes may impact external validity

By accounting for competing risks of death and duration of mechanical ventilation, ventilator-free days (VFDs) have become a popular outcome for ARDS clinical trials [14]. However, the diagnostic criteria used to define VFDs need to be applied consistently; specifically regarding the definition of mechanical ventilation start and end points [15]. For example, some studies measure the end of mechanical ventilation at the time of extubation, and others at the time the patient no longer requires non-invasive ventilation. Similarly, a variety of long-term assessment tools may be combined if they measure comparable outcomes. Performance-based tests such as the 6-min walk have been reported to measure different constructs than self-report measures such as health-related quality of life questionnaires [16]. Heterogeneity in the diagnostic criteria of these non-mortality outcome assessments may impact the external validity of the trial and affect the interpretation of subsequent meta-analyses [14].

ARDS trials that use of non-mortality variables have their own inherent problems

Non-mortality outcomes have their own issues that may complicate the conduct of ARDS clinical trials. Because they are often collected in a unblinded fashion, non-mortality outcomes are prone to ascertainment bias; investigators have predispositions about the utility of the intervention and can alter the interpretation of the outcome assessment [17]. Non-blinded assessors are also prone to optimism error and may detect fewer treatment failures when compared to blinded assessors. The use of blinded outcomes assessors is relatively uncommon in clinical trials, raising issues regarding the validity of the study’s results [17]. In addition, when specific outcomes are not clearly delineated during the design phase, investigators may be prone to report only the specific outcomes that are positive (selective outcome bias).

Conclusion

While the selection of non-mortality outcomes may facilitate the conduct of less costly and shorter trials, these studies may not yield useful information about how well an intervention impacts definitive outcomes such as mortality. The ARDS community is in dire need of clinically useful and effective intermediate outcome variables. Until valid intermediate outcome variable are identified, studies that use a definitive outcome measure such as mortality provide the best evidence of the true efficacy of an intervention.