Introduction

Medical knowledge increases rapidly, making it increasingly difficult for clinicians to update their practice to allow the incorporation of new advances in care. Arguably the largest deficiency in modern health care is the frequent failure to adhere to evidence-based best practices [1]. These are practices related to the prevention, diagnosis and treatment of disease that have been demonstrated to improve clinically relevant outcomes. Both the government [2] and the healthcare industry allocate the majority of their resources towards the discovery of new interventions. In the United States the annual budget of the Agency for Healthcare Research and Quality, the federal agency responsible for research devoted to improve fidelity, is 1% of the budget for the National Institutes of Health [3], where the majority of money is allocated to research towards the discovery of new and more effective interventions. The assumption that new interventions are effectively adopted by healthcare systems may partially explain this disparity. However, studies have shown that as many as 30 to 50% of patients do not receive recommended evidence-based care and 20 to 30% of patients receive unnecessary interventions [1, 4].

The problem is partly related to the lack of generalizable knowledge on how to effectively perform knowledge translation. Recent trials and meta-analysis demonstrate only modest, or even a lack of improvement, in evidence-based practices through the use of multifaceted programs to improve quality, including educational efforts, "bundle" tools, and audit and feedback [59]. Filling this gap in the implementation of available evidence, thus increasing system fidelity, can be a better opportunity to improve care than investigating for more effective interventions [10]. One of the approaches adopted in the industry to increase fidelity is the ongoing monitoring of processes of care, coupled with actions to improve quality even when minimal deviations from a standard are observed. This is usually performed with the aid of statistical process control charts, where deviations from a standard are differentiated from variation due to chance [11]. These charts are constructed with continued audits of "products" (in healthcare the "product" can be viewed as compliance with a process of care or the measurement of unwanted outcomes, such as nosocomial infections) and plotting of results over time. Depending on the characteristics of the variable (continuous or categorical) charts are constructed showing the variation over time and the standard error (SE) of the point estimates are used to create control lines at ± 2 SE and ± 3 SE, equivalent to 95% and 99% confidence intervals (CIs). Several rules are used to detect variation that is not due to chance, the most common ones being 1 point above or below the 3 SE control or 2 out of 3 points above or below the 2 SE control [12]. While large deviations from practice can trigger interventions in healthcare, there are scarce data to support a similar approach when the deviation is minimal, such as when there is reasonably high compliance with a process of care.

Strategies to limit the amount of sedation in critically ill patients, either by performing a daily interruption [13], or by minimizing the use of continuous infusions [14] reduce time on mechanical ventilation without causing harm. In one trial, benefits of duration of mechanical ventilation were clinically relevant, with a mean reduction of 2.4 days [13] and up to 4 extra ventilator-free days in a subsequent trial [14]. It is one of the most compelling examples of evidence-based medicine in critical care, and guidelines endorse the use of minimal sedation [15]. Ideally, every patient undergoing mechanical ventilation should have sedation minimized, as long as sedatives are not being used as treatment for neurological disorders. We have previously identified that in our unit, compliance with this evidence-based process was 80.4% (95% CI: 66.9 to 90.2). We undertook a quality improvement project with a goal of increasing compliance to 95% and we sought to investigate whether this small increment in compliance would result in further decreases in the duration of mechanical ventilation.

Given the limited efficacy of most interventions to improve quality of care [7] we sought first to understand barriers to current practice and used an interprofessional team to customize solutions to the barriers identified locally. The interprofessional group focused on theoretical approaches that have been demonstrated to increase compliance with evidence-based interventions:

  1. (1)

    Reminders: Even when clinicians are aware of recommendations, they may forget to provide evidence-based interventions. A good example of this is the use of prophylaxis for deep venous thrombosis (DVT). This is an intervention where there is robust evidence and no disagreement. It is simple, not time consuming, and clinicians are aware of its importance and willing to use it. However, they may still forget to prescribe it. A randomized controlled trial of electronic reminders demonstrated a higher than two-fold increase in the use of DVT prophylaxis in hospitalized patients [16]. Reminders may also help with decreasing inertia, which is defined as lack of adoption of a process even in the face of knowledge and acceptance [17]. The presence of a reminder may increase compliance by reducing inertia in up to 40% of encounters [18].

  2. (2)

    Simplification: simpler interventions have a lower threshold for adoption. Simpler interventions allow clinicians to easily understand their meaning and the rationale behind them. Also, clinicians perceive simple interventions to reduce their workload, which is an important barrier for implementation [19]. Almost 40% of physicians classified specific guidelines as "inconvenient" or difficult to use [20]. A systematic review identified guideline complexity, defined as when the average clinician perceives the guideline to be difficult to understand or to acquire the necessary skills, as responsible for 30% of the variation in compliance [21].

  3. (3)

    Academic detailing: 'one-on-one' teaching by clinician champions seems to be particularly helpful. Two large systematic reviews support academic detailing as one of the most effective tools for behavioral change [22, 23]. Academic detailing at the point of care may be particularly useful, as specific questions are answered and an immediate action is taken when the champion is available to help provide care.

Materials and methods

Setting

Sunnybrook Health Sciences Centre is a tertiary teaching hospital, affiliated with the University of Toronto. It has six critical care units including three level-3 units (where patients can be ventilated), namely a trauma-medical-surgical level-3 unit, a burn unit and a cardiovascular unit, and three level-2 units (where patients do not undergo invasive mechanical ventilation), namely a neuro-trauma unit, a medical-surgical unit and a coronary unit. Data for this study are based on admissions to the level-3 trauma-medical-surgical unit, which has 20 beds, the majority of patients cared for with a nurse ratio of 1:1. The nursing care delivery model is total patient care, which includes basic and advanced critical care core competencies. No other changes in the organization, relevant processes of care, or structure of the ICU took place at the same time.

Quality Improvement Process

In April 2009 a protocol for sedation interruption was introduced in the level-3 units. In February 2010 we conducted an audit of compliance with the process, which revealed that 80.4% (95% CI: 66.9 to 90.2) of patients received an interruption of sedation or were given only boluses of sedatives as opposed to continuous infusions. An interprofessional team, consisting of physicians, nurses, respiratory therapists and pharmacists, was assembled to improve this process. The first step was to identify barriers and solutions from the end-users. We interviewed bedside nurses and respiratory therapists and identified 4 main barriers: (1) lack of knowledge of the protocol; (2) complexity of the protocol; (3) time to start sedation interruption considered unsafe and not realistic; (4) lack of accountability.

Based on the suggestions given by end-users, we designed a multi-faceted strategy to improve compliance with minimal sedation:

(1) Protocol design: the interprofessional team re-designed and simplified the protocol, outlining exclusion criteria and parameters for clinical assessment; the suggested time to interrupt sedation was also changed. We also clarified the misconception that a sedation interruption was only needed when patients were ready for extubation. Protocol redesign was a key step in addressing barriers 2 to 4 as the new protocol is simpler, more in line with accepted nursing practice, and establishes the bedside nurse as the clinician responsible for interrupting sedation in the morning.

(2) Reminder: we included a sedation interruption order in the admission pre-printed order set. In the first month we also used sticky-notes in the patients' charts to remind the team of patients who were receiving a continuous infusion of sedation. Reminders are important to address lack of knowledge, as they steer the clinician's attention to a required practice and also overcome inertia.

(3) Education: staff received in-service training, electronic material by e-mail and available on the intranet, training during staff meetings and at the point of care. During the first month of implementation, the critical care fellow, the respiratory therapist and the nurse in charge of the unit identified all eligible patients during the night shift. The next morning, the nurse educator, advanced practice nurse and/or nurse manager discussed the protocol directly with the bedside nurse. Questions were answered and if the nurse felt uncomfortable proceeding with the sedation interruption, the educator, advanced practice nurse or nurse manager encouraged the nurse and stayed at the bedside for troubleshooting. This interprofessional approach to identifying patients helped ensure that sedation was discussed at night and that patients were 'cleared' by physicians and respiratory therapists before the interruption. Education and especially the academic detailing provided at the point of care were instrumental in improving knowledge and accountability. The discussion of inclusion and exclusion criteria and troubleshooting the protocol at the bedside facilitates learning by using real situations.

Ethics

As this project was undertaken to improve the quality of an already existing evidence-based process and benchmarking databases, the Research Ethics Board at Sunnybrook Health Sciences Centre waived the need for its review.

Data Collection

The primary outcome was the length of mechanical ventilation. Data on length of mechanical ventilation were retrieved from a locally held quality benchmark database that has been in place since 2009 and has not undergone changes in data entry for this variable since its inception. We collected data from May 2009 through April 2011. The intervention took place in May 2010, yielding data for analysis from 12 months before and 12 months after the intervention. We included data only for the first ICU admission of each patient. Data for duration of mechanical ventilation were collected at the individual patient level.

Compliance with the evidence-based process: a trained data collector collected data on all ventilated patients every other day during weekdays, on the use of no sedation, boluses of sedation and continuous infusions of sedatives. Inclusion and exclusion criteria for sedation interruption were based on the protocol, as used in two previous randomized controlled trials [13, 24]. If the patient was receiving a continuous infusion of sedatives and had no contraindications we recorded whether or not his sedation was interrupted. To calculate the compliance with the process we included in the denominator all patients who were ventilated and had no contraindications to have minimal sedation, and in the numerator all those who satisfied our definition of minimal sedation. We define minimal sedation as the use of boluses of sedatives without a continuous infusion or as an interruption in the continuous infusion of sedatives. Analgesics are not considered for sedation interruption and are titrated according to pain scales.

Data on severity of illness and type of admission was retrieved from an acute physiology and chronic health evaluation (APACHE II) database, in place for benchmarking before the quality improvement project was initiated.

Statistical Analysis

Data are presented as mean and standard deviation, median and interquartile range (IQR) or proportions, with a 95% CI. Our outcome of interest, time of mechanical ventilation, was log transformed to normalize the error terms, as it was positively skewed [25]. For ease of interpretation Figure 1 shows the exponentiated results, which represent the geometric mean.

Figure 1
figure 1

Length of mechanical ventilation over time. LOMV, Length of Mechanical Ventilation.

Before- and after-studies may be biased by secular trends, and this bias may either show differences that are not real, or obscure real differences that exist. To control for secular trends we used an interrupted-time series with adjustment for autocorrelation using an autoregressive integrated moving average (ARIMA) model as described elsewhere [26]. Individual, log-transformed, patient data were aggregated by month to allow for ARIMA modeling. All regression models used maximum likelihood estimates and we controlled for autocorrelation by correcting for autoregressive effects. The interrupted time series is the strongest, quasi-experimental design to evaluate effects of time-delimited interventions [27]. Briefly, this involves a multilinear regression with two variables for time (β1 = time since inception of the cohort and β2 = time since the intervention, being 0 for all time-points before the intervention) and a variable for group (β3 = 0 before the intervention and 1 after the intervention). Interpretation of the regression involves looking at both the immediate change in level (β3) and the change in trend over time (β2). We checked for autocorrelation by visual inspection of autocorrelograms and partial autocorrelograms of the series and its residuals, and with the Box-Pierce Q statistic [28]. Time-series analysis requires a stationary process, which implies that the mean and variance do not change over time. When this assumption is not met, statistically significant results can be generated when no real signal exists. ARIMA models can use a differentiating function to create a stationary process. We checked the process was stationary using both the Phillips-Perron and the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) tests [29, 30].

We calculated the expected length of mechanical ventilation in each month by the end of the intervention period with data from the intercept, and β1 (time since cohort inception), and calculated the fitted value for the post-intervention months with the intercept, β1, β2 (time after the intervention), and β3 (level after intervention). The relative difference in these two points represents the effect of the intervention in each month. We have multiplied this difference by the number of patients in each month post-intervention to provide an estimate on the total amount of ventilator-days gained with the intervention. We assessed the fit of the model by examining residuals around the predicted lines. Normally distributed residuals that follow no observable pattern over time indicate that assumptions of the linear model are met [26].

All analysis was performed using STATA/SE 10.0 for Windows (StataCorp LP, TX, USA).

Results

In this 2-year time period 1,826 patients were admitted to the level-3 medical-surgical unit. We excluded 108 patients who were never ventilated and 162 re-admissions, for a total of 1,556 patients included in the study (753 pre-intervention, 803 post-intervention). The median number of ventilated patients per month was 65 (IQR = 61 to 69). Baseline characteristics of the two groups are shown in Table 1. After the intervention there were more patients admitted under the trauma diagnostic category and fewer under the respiratory category. Otherwise, the two cohorts were similar in age, source of admission, APACHE II scores and mortality.

Table 1 Baseline characteristics.

Compliance with the quality improvement intervention

During the 12 months after the intervention we observed that patients received minimal sedation in 1,921 patient-days, from a total of 1,997 eligible patient-days (96.2%, 95% CI 95.2 to 97.0).

ARIMA Modeling

The Phillips-Perron and KPSS tests demonstrated stationarity of the series (Phillips-Perron, P = 0.0000, rejecting the null hypothesis on non-stationarity; KPSS, P > 0.1, confirming the null hypothesis of stationarity), therefore no differentiating function was introduced in the ARIMA model. Neither the visual inspection of correlograms nor the Box-Pierce Q statistic suggested autocorrelation. An ARIMA model with zero lags of autocorrelation, however, demonstrated one lag of autocorrelation in the residuals. Therefore we modeled our data with one lag of autocorrelation. The point estimates of the main effects did not change between models with or without autocorrelation adjustment, but the residuals of the model with one lag of autocorrelation showed no more autocorrelation.

Length of Mechanical Ventilation

There was an immediate effect of the intervention (regression coefficient = -0.129, P < 0.001) and the secular trend (β1) was a determinant of length of mechanical ventilation (regression coefficient = 0.010, P = 0.004). The trend post-intervention (β2) was not significant (regression coefficient = 0.004, P = 0.380) (Figure 1).

The median relative change in the fitted length of mechanical ventilation was 14.5% (IQR 13.8 to15.8) and the total expected decrease in mechanical ventilation days was 502.7 days (95% CI 300.9 to 729.1) over one year.

Due to the differences observed in the diagnostic categories between the two time periods we reanalyzed the data restricting the observations to respiratory/non-respiratory admission and to trauma/non-trauma admissions (Table 2). In the model excluding respiratory admissions we could not find any signal of a secular trend or changes in trend and level after the intervention (full model, P = 0.2), while the analysis limited to respiratory admissions demonstrated an immediate effect of the intervention (regression coefficient = -0.383, P < 0.03). The model excluding trauma admissions also demonstrated an immediate effect of the intervention (regression coefficient = -0.215, P < 0.001), a secular trend towards an increase in length of mechanical ventilation (regression coefficient = 0.011, P < 0.001), and a post-intervention trend towards an increase in length of mechanical ventilation (regression coefficient = 0.013, P < 0.032), while the model limited to trauma admissions demonstrated no signal (full model, P = 0.29).

Table 2 Changes in length of mechanical ventilation.

Discussion

Our results support the conclusion that a quality improvement intervention can reduce the length of mechanical ventilation even when baseline compliance is already high. Our cohort demonstrated a trend towards an increased length of mechanical ventilation over time (reflected by a positive coefficient on β1). We can only speculate on the possible reasons for this increase: if clinicians are changing their practice towards being more conservative in making decisions to withhold or withdraw life support [31], such as after neurological trauma or cardiac arrest, or in patients with cancer [32], we can expect to see an increase in time on ventilation. Another possibility is a change in severity of illness over time, which we could not demonstrate from our limited data with APACHE scores. Interestingly, two reports of time trends in mechanical ventilation suggest a similar increase in duration of mechanical ventilation over past years [33, 34].

We were able to demonstrate the effectiveness of our intervention: there was a change in level (negative coefficient for β3), implying an immediate effect of the intervention, which did not reverse after the intervention (reflected by a non-significant coefficient on β2). We interpret these results as an indication that there was a persistent effect of the intervention over time. If, as is the case with so many quality improvement interventions, these effects were transient, we would expect to see a positive and statistically significant coefficient on analysis of trends post-intervention (β2). Our study may have been underpowered to detect this return to baseline, but we can conclude that at least over a period of 12 months, the intervention was effective in reducing the length of mechanical ventilation.

One of the limitations in our study is the difference in admission categories between the two time periods. While we could not formally adjust the ARIMA models for the types of admissions, we report results for the same modeling for the main diagnostic categories of respiratory illness and trauma. The cohort of patients with respiratory problems shows the same immediate effect of our intervention, without a return to baseline, and there was no identifiable effect in the trauma cohort. While we can offer no data to address the reasons for these differences, one may consider that patients admitted due to respiratory causes were more likely to be over-sedated and benefited from the intervention, while trauma patients may have two distinctive patterns of sedation use: the need for sedation to control intracranial pressure or the use of minimal amounts of sedatives for a short period of mechanical ventilation. In both cases it is possible that a protocol for sedation interruption would have no effect, as patients with intracranial hypertension do not undergo sedation interruption and patients who undergo a short period of mechanical ventilation may already be using minimal doses of sedatives. Importantly, given that an effect is seen in the respiratory group, a lower number of admissions of this group in the second part of the study should bias our results towards the null hypothesis, as there would be fewer patients in whom the intervention is effective. This imbalance therefore, strengthens our results for the whole cohort. We would caution against the interpretation that the intervention was only effective in respiratory cases, as the statistical model may not perform as well with the smaller numbers of patients in the subgroups that were analyzed.

An important limitation of our findings relates to the environment where this study took place. In Ontario, most critical care units capable of caring for ventilated patients are staffed to provide the majority of patients with one-to-one nursing care, which may not be the reality in other healthcare settings [35]. However, nurses in Ontario have responsibilities, such as bathing, mobilizing, suctioning and dialyzing patients, that may be performed by different clinicians or non-clinicians in other ICUs. Given this perspective, it is difficult to understand whether units with less intensive staffing will be able to safely reach high compliance with minimal sedation. Also, our unit has 20 beds with a high occupancy (> 100%) and mainly cares for acute care, ventilated patients (those transferred to a step-down unit when they no longer require invasive mechanical ventilation). Therefore, the exposure to the new protocol was facilitated by a process that occurs frequently (on average 3 to 4 patients were using continuous infusions of sedation every morning in the initial implementation month); in smaller and less acute units it may be difficult to have such intense exposure to a new process and the implementation period can be much longer. One of the main limitations of this paper is its use of a customized strategy to improve compliance. Our approach focused on understanding the barriers at the local level and applying methodologies that were tailored directly to address the local barriers. This is clearly not generalizable, but yet, may also be one of the strongest components of this paper: quality improvement projects may need to be customized to organizations. Consider that two different patients with similar symptoms (dyspnea) may have different diseases (congestive heart failure and asthma, for example), requiring different treatments; a trial that investigates a drug to treat dyspnea is unlikely to show benefit. Similarly, two organizations with the same problem (lack of compliance with minimal sedation) may have different causes (insufficient human resources or a strong predisposition against minimizing sedation) that may require different approaches [36].

Although we would not consider our results generalizable, the components of our approach may be transportable to other environments: reminders are a simple and effective tool in several different settings, ranging from DVT prophylaxis [16] to checklist prompts during rounds [37]; simplification of processes is used in the industry to improve efficiency and reduce errors; and academic detailing is suggested to be one of the few relevant knowledge translation methods in the current literature [22, 23]. Although the use of these processes was customized to our unit (how to simplify the process, how to create the reminders and the daily process of academic detailing), their basic characteristics can be easily customized to most critical care units.

Our study also highlights another important aspect of research in quality improvement. Had we taken an approach to simply measure length of mechanical ventilation before and after we might have concluded that there was no effect of the intervention. Because the trend was towards an increased length of mechanical ventilation, any benefits would have been upset by this trend. Interrupted-time series analysis has been recommended to avoid identifying associations that are not real and only due to secular trends, but it can also be helpful to identify effective interventions when the trend is in the opposite direction. Interrupted time-series is a stronger research design than before- and after-studies to assess the effect of quality improvement interventions. However they are frequently poorly reported or analyzed in the literature. A review of 58 studies using interrupted time-series demonstrated that two-thirds were unable to rule out significant threats to internal validity. Thirty-three of these studies could be reanalyzed, and 8 were found to have no real effect [38]. Our study meets all quality criteria as described in this review: (1) intervention occurred independently of other changes over time; (2) sources of data collection were the same in both periods; (3) the primary outcome is objective and reliable (duration of mechanical ventilation); (4) all patients are included in the study; (5) there is good evidence that the intervention is linked to the outcome; (6) there are enough data points to detect a significant decay after the intervention and (7) we tested and adjusted for autocorrelation.

Although our unit was already compliant in more than 80% of patients, our small increase in compliance, just slightly above 15%, led to important gains in efficiency for the healthcare system: considering only our unit, we decreased the amount of mechanical ventilation by the equivalent of more than 500 days. Considering that most ICU costs are fixed (staffing, equipment, infra-structure costs) [39], this ICU would be able to care for an extra 125 patients (assuming an average 4 days of mechanical ventilation), with minimal changes in cost, after the intervention.

Conclusions

A quality improvement intervention to increase compliance with minimal sedation is effective in reducing the length of mechanical ventilation. Monitoring and redesigning processes of care can increase compliance above 95%. Gains for the healthcare system were achieved in a unit that was already compliant with this evidence-based process. Further research should concentrate in identifying the most effective strategies to quickly implement and monitor evidence-based processes.

Key messages

  • A multifaceted strategy improved compliance with minimization of sedation

  • Small improvements in compliance with minimizing sedation led to reduced duration of mechanical ventilation

  • Quality assurance monitoring is an important component of quality improvement and ICUs should monitor its' key quality processes on a continued basis