Do trials that report a neutral or negative treatment effect improve the care of critically ill patients? Yes

Introduction

Over the last half-century, the emergence and evolution of critical care has made possible the conduct of incredibly complex lifesaving surgery and the recovery of untold thousands of critically ill medical patients who previously had no chance of survival [1]. Despite this success, most interventions delivered to critically ill patients were adopted based on physiological theory or “borrowed” from other settings, e.g., positive pressure ventilation from the operating room and fluid resuscitation from the infirmaries and battlefields of the world wars. While this approach was entirely appropriate in the early days of our specialty, it is now clear that many standard practices of the past, and some new ones, harmed the very patients they were designed to help. We know this predominantly because academic researchers have designed and conducted high-quality, robust, pragmatic randomised clinical trials (RCTs); many of the trials that have improved the care of our patients have reported neutral or negative treatment effects.

Clinical trials in critical care

The conduct of RCTs in critical care is challenging; patients are rarely able to give or withhold consent, diagnosis may be unclear early in the clinical course, making complex inclusion criteria difficult to apply, interventions are often time-critical, and the natural trajectory of critical illness is incredibly variable. Although clinical trial methodology is continually evolving with exciting new methods such as adaptive and platform trials being brought to bear [2, 3], most high-quality evidence comes from traditional individual or cluster RCTs, which, simplistically, can be divided into efficacy and effectiveness trials [4]. Efficacy trials are designed to answer the question, “Does this intervention work in the ideal patient population in the ideal circumstances?” Efficacy trials typically use complex inclusion and exclusion criteria and are the appropriate design when investigating the likely impact of a new intervention [4]. Efficacy trials sacrifice the ability to apply their results to real-world practice in pursuit of maximal internal validity. In contrast, effectiveness trials, also called pragmatic trials, are designed to determine the effect of an intervention when it is used in more a diverse population or typical clinical settings [5]. Notable features include simplified inclusion and exclusion criteria that seek to maximise generalisability and thus to understand the true impact of new and established treatments on outcomes important to patients [5].

The impact of trials that demonstrate harm

On this background it is both rational and essential to test as many of the interventions used in critical care as possible. Priority should be given to interventions that are used for to many patients, interventions that are costly or labour intensive and those for which there is insufficient evidence to reliably estimate the balance between benefit and harm. That the risk of us causing harm to our patients is real is confirmed by a recent systematic review that reported on RCTs in critical care in which the intervention studied significantly affected mortality; mortality was increased by half the interventions studied [6]. Notably critical trials that report harm are generally of higher quality, being more likely to be multi-centred and blinded and have larger sample sizes [7]. Importantly, many interventions shown to be harmful in high-quality trials were in regular clinical use at the time of testing, including high-frequency oscillatory ventilation [8], hydroxyethyl starch [9] and intensive glucose control [10]. In reaction to these results, clinicians, guideline writers and regulators have taken actions to reduce the use of harmful interventions and thus to improve outcomes for our patients.

The impact of trials that report a neutral treatment effect

A positive, a neutral and a negative result for a given test of intervention X vs. intervention Y will be highly informative for clinicians, those writing clinical practice guidelines and policy-makers and healthcare funders provided the research is of high quality and free from significant risk of bias. The results of robust pragmatic trials allow everyone to change clinical practice with confidence. Some of the many RCTs with neutral results that have allowed us to provide better and more cost-effective care are listed in Table 1.

Table 1 Randomised clinical trials done in the critical care setting where neutral results allowed changing clinical practice guidelines and/or clinical practice

Simplifying critical care

Trials reporting neutral or negative treatment effects are important in the process of simplifying critical care as they show us what not to do. As many standard critical care interventions and therapeutic targets are being challenged, simplifying care becomes increasingly rational from the patient, organisational and financial perspective. Doing less may improve patients’ outcomes, reduce the number of drug interactions and adverse events and save money. Doing less allows us to focus our efforts on what is important to patients, notably to reduce pain, anxiety, thirst, breathlessness and other distressing symptoms. Additionally, simplification will harmonise care, which will facilitate staff training. Simple care will form a cleaner baseline for observational and interventional research and thereby increase the likelihood of developing new diagnostics, risk scores and interventions that will be useful for future patients.

Summary

While it is tempting to be disappointed when an RCT reports that a new or established treatment does not have demonstrable beneficial effects, or even harms our patients, such information is critical and has undoubtedly contributed to the improved outcomes now experienced by critically ill patients. We must stop characterising such results as “negative trials” and instead celebrate the knowledge they provide and encourage all critical care practitioners to incorporate that knowledge into their decision making at the bedside.