Surgical-site infections represent a major morbidity and mortality burden, with an estimated 500,000 infections and 8,000 deaths worldwide per year costing the healthcare system upwards of $7 billion annually.1 Approximately 40-60% of these infections are thought to be preventable.2 Furthermore, as perioperative prophylactic use of antibiotics reduces rates of surgical-site infections,3–5 both the Joint Commission Surgical Care Improvement Project6 and the World Health Organization Guidelines for Safe Surgery7 suggest that appropriately selected and timed antibiotic administration is crucial for safe surgery.

Despite the importance of antibiotic prophylaxis, anesthesia clinicians often fail to comply with guidelines.8 Studies suggest that appropriately timed antibiotic administration occurs in 12-100% of cases.8 The electronic medical record (EMR) can be a tool for encouraging compliance for two reasons. First, as anesthesia clinicians interact with the EMR during their cases, it can serve as an interface through which to educate clinicians regarding current guidelines. Additionally, as the EMR serves as the official record for how medications are administered during the case, oversight organizations rely on the EMR to determine rates of compliance with guidelines.

A retrospective audit at our institution suggested that 85% of first doses of antibiotics but only 40-50% of second and third doses were administered at the correct time. Other groups have created interventions in the EMR to increase clinician compliance with the guidelines for the first antibiotic dose,9–11 but there is a lack of studies examining compliance for repeated doses. As longer operations (due to either an inherently involved procedure or patient factors prolonging the procedure) carry increased morbidity and mortality,12–14 these doses represent a key opportunity to protect particularly vulnerable patients.

Cefazolin is an important antibiotic to study in the perioperative setting. It is the most commonly used perioperative antibiotic15 with a broad spectrum of bactericidal activity.16 Cefazolin does not require dose reduction in patients with renal or hepatic impairment, so there is little reason to deviate from dosing guidelines.15 Thus, it is a useful marker of clinician compliance with guidelines.

Based on the aforementioned audit, a quality improvement initiative sought to utilize the EMR to improve compliance with antibiotic guidelines. In a survey of anesthesia clinicians at our institution, 88% of respondents cited either not knowing hospital guidelines or forgetting to administer/chart antibiotics as their primary reason for not administering repeated doses. Thus, we created two interventions to the EMR in our hospital system to address these two key reasons for clinician noncompliance with guidelines. Based on initial positive clinician responses, we formally investigated the effectiveness of the interventions. We used a historical cohort before-and-after study (i.e., before, immediately after, and three to seven months after intervention implementation) to examine clinician compliance with perioperative antibiotic guidelines regarding the administration of cefazolin. We hypothesized that these interventions would significantly improve the proportion of correctly administered repeat doses of cefazolin, defined a priori as at least a 15% absolute improvement. Additionally, we hypothesized that, if improvements occurred, we would see no clinically significant decline in the compliance rate, also defined a priori as at least a 5% absolute decrease in compliance between the immediate and the delayed post-intervention periods.

Methods

Ethics approval

We obtained approval from the Washington University Institutional Review Board (Study Number 201602006, approved February 1, 2016) to obtain the times and doses of antibiotic administration for the appropriate cases as well as the necessary information to verify that doses were given correctly, including patient weight and time of surgical incision. Our study was granted a waiver of consent as this was a retrospective analysis of a previously implemented quality improvement intervention and no changes were made to patient care for the purpose of this study.

Electronic medical record interventions

The study intervention consisted of two changes to the MetaVision EMR (iMDsoft®, Needham, MA, USA), an integrated part of the Anesthesia Information Management System. A decision support tool showed clinicians the recommended dose, redosing interval, and weight-based dosing alterations for commonly used antibiotics (Figs 1 and 2). This information was displayed on the buttons that clinicians clicked to document a dose of antibiotic. These recommendations were drawn from our hospital’s perioperative antibiotic dosing guidelines.13 The reminder portion of the intervention consisted of a pink bar displayed alongside medications and fluids in the Gantt view of the EMR to signify that the patient was covered by the most recently administered dose of antibiotic (Fig. 3). This reminder was generated automatically when an antibiotic was administered, and its duration was defined by the antibiotic used. The bar closed 15 min before the antibiotic was due to be redosed, cueing the clinician to administer the antibiotic. Neither intervention created extra steps to chart a dose of antibiotic.

Fig. 1
figure 1

Decision support portion of the electronic medical record intervention

Full size image
Fig. 2
figure 2

Comparison of pre- and post-intervention antibiotic administration buttons

Full size image
Fig. 3
figure 3

Redosing reminder portion of the electronic medical record intervention

Full size image

Study design and case selection

In this historical cohort before-and-after study of a previously executed quality improvement intervention, cases were drawn from procedures performed on the main campus of a 1,158-bed adult teaching hospital (and Level 1 trauma centre) in a United States urban setting. In 2015, 34,288 cases were performed (with 2,614 lasting > 240 min), including all major specialties except obstetrics. The anesthesia staffing model in our institution is comprised of approximately 75% medically directed certified registered nurse anesthetists (median length of service = five years), 20% residents/fellows, and 5% attending anesthesiologists who are the primary anesthesia caregivers. Inclusion criteria for cases were the use of cefazolin as the perioperative antibiotic from the start of the case, case duration > 240 min, the redosing interval for cefazolin per our hospital guidelines, and the anesthesia clinician documented a pre-incision safety time-out, which forced the clinician to document whether or not perioperative antibiotics were indicated for the procedure. We excluded cases involving cardiopulmonary bypass (CPB) as our routine practice is to redose cefazolin immediately after terminating CPB. We also excluded pediatric cases as pediatric and adult antibiotic dosing guidelines differ. Finally, we excluded cases in which the anesthesia clinician documented a change in the antibiotic dosing schedule because recently administered antibiotics provided satisfactory coverage for part of the case or because no antibiotics were initially deemed necessary.

We planned the immediate post-intervention period to cover only the first month after implementation of the EMR interventions in April 1, 2015. We planned a minimum of a three-month period from any previous quality improvement interventions to the start of our sampled cases to be sure that the effects of any such interventions would have stabilized. The pre-intervention and delayed post-intervention periods occurred during the same calendar months to control for variability in clinician experience as the academic year progresses at a teaching hospital. Given these parameters, we sampled cases meeting the inclusion and exclusion criteria from three time periods: July 1, 2014 to November 28, 2014 (pre-intervention group); April 1-25, 2015 (immediate post-intervention group); and July 1, 2015 to November 28, 2015 (delayed post-intervention group).

Case analysis

For each case, we determined whether each of the first three doses of cefazolin was correctly dosed for weight and timed. Our hospital guidelines recommend that patients weighing < 120 kg receive 2 g of cefazolin every four hours and that patients weighing ≥ 120 kg receive 3 g. A first dose of antibiotic was to be administered 0-60 min before incision. Second and third doses were to be administered 225-255 min after the previous dose or earlier in the setting of ≥1 L of estimated blood loss. These cut-offs were pre-determined prior to extracting any information from the case database.

Statistical methods

There were two co-primary outcomes of this study. First, we examined whether the interventions increased the proportion of correctly administered second dose of cefazolin in the immediate and delayed post-intervention periods. We defined a clinically significant improvement as an absolute increase of 15% in the proportion of correctly administered doses. Second, we examined whether improvements seen in the immediate post-intervention period declined by the delayed post-intervention period. We defined a clinically significant decrease in compliance as a 5% absolute decrease in the proportion of correctly administered doses. As secondary outcomes, we compared the immediate and delayed post-intervention periods with regard to changes in the proportion of correctly administered first dose, third dose, and all repeated doses as well as the stability of these changes.

The sample size for each period was dictated by the aforementioned limitations of when each period was required to start and end. We conducted post hoc power analysis based on correctly administering a 50% proportion of the pre-intervention second dose and correctly administering a minimum expected 65% proportion of repeated doses in each of the post-intervention periods. The results suggested that our study included a sufficient number of repeated doses in both the immediate and delayed post-intervention periods to have 80% power to show a minimum of 15% absolute increase or decrease in the proportion of correctly administered repeated doses of antibiotics, with α = 0.05 (two-tailed).

Comparisons were made using the Chi square test, except for situations where the expected frequency of an event was ≤ five, in which case, Fisher’s exact test was used. In all cases, P < 0.05 (two-tailed) was used for statistical significance. Bonferroni correction was used to account for multiple comparisons. SAS® version 9.4 (SAS Institute Inc., Cary, NC, USA) was used for all statistical analyses.

Results

Case enrolment is displayed in Fig. 4. We initially identified 604 cases during the pre-intervention period, 127 cases during the immediate post-intervention period, and 629 cases during the delayed post-intervention period. We excluded four cases from analysis during the pre-intervention period, zero cases during the immediate post-intervention period, and two cases during the delayed post-intervention period because the cases were pediatric patients undergoing surgery at the adult hospital. A further three cases were excluded during the pre-intervention period, zero cases during the immediate post-intervention period, and three cases during the delayed post-intervention period because no weight was documented for the patient. Thus, 1,348 total cases were analyzed, including 597 from the pre-intervention period, 127 from the immediate post-intervention period, and 624 from the delayed post-intervention period. Eighty, 15, and 85 cases in the pre-intervention, immediate post-intervention, and delayed post-intervention groups, respectively, merited a third dose of cefazolin. As such, there were 1,528 opportunities to administer repeated doses of cefazolin correctly, including 677, 142, and 709 in the pre-intervention, immediate post intervention, and delayed post-intervention groups, respectively.

Fig. 4
figure 4

Case enrolment flowchart

Full size image

Proportions of correctly administered second doses of cefazolin are shown by month in Fig. 5. Though month-to-month variability existed, there was a largely stable pre-intervention baseline level of compliance with the guidelines. Proportions of correctly administered doses of cefazolin are shown by dose and time period in Table 1 and Fig. 6. Differences between the groups are shown in Table 2. In the pre-intervention period, 51.4% of second doses of cefazolin were correctly administered, and in the immediate and delayed post-intervention periods, 68.5% and 73.3% were correctly administered, respectively. These results represent a clinically significant absolute improvement of 17.1% (95% confidence interval [CI], 8.1 to 26.1) and 21.9% (95% CI, 16.6 to 27.2), respectively (P < 0.001 compared with pre-intervention in both cases). In the delayed post-intervention period, there was a 4.8% absolute increase in the proportion of correctly administered second doses of cefazolin compared with the immediate post-intervention period (95% CI, −4.0 to 13.6; P = 0.27).

Fig. 5
figure 5

Proportion of second doses of cefazolin correctly administered by month. Red line represents date of implementation of electronic medical record interventions. Error bars represent 95% confidence intervals

Full size image
Table 1 Proportions of correctly administered antibiotic by time period and dose
Full size table
Fig. 6
figure 6

Proportion of doses of cefazolin correctly administered by time period and dose. Error bars represent 95% confidence intervals

Full size image
Table 2 Absolute differences between groups in the proportion of repeated antibiotic doses correctly administered
Full size table

The first dose of antibiotic was correctly administered in 91.1%, 92.1%, and 94.1% of cases in the pre-intervention, immediate post-intervention, and delayed post intervention periods, respectively. Where applicable, the third dose of antibiotic was correctly administered in 47.5%, 86.7%, and 70.6% of cases in the pre-intervention, immediate post-intervention, and delayed-post intervention periods, respectively. Combined, second and third doses of antibiotic were correctly administered in 51.0%, 70.4%, and 72.9% of cases in the pre-intervention, immediate post-intervention, and delayed post-intervention periods, respectively. These results represent a significant difference between groups with regard to proportions of correctly administered third doses (P = 0.001) and all repeated doses (P < 0.001).

Regarding the proportions of correctly administered first dose of antibiotic, there was no significant difference between the pre-intervention group and either the immediate or delayed post-intervention groups (P = 0.72 and 0.05, respectively). Regarding the proportions of correct administration of the third dose of antibiotic and all repeated doses of antibiotic, there was also a significant increase in both the immediate pre-intervention period (P = 0.005 and P < 0.001, respectively) and the delayed post-intervention period (P = 0.003 and P < 0.001, respectively). There was no significant difference in the proportion of correctly administered first, second, third, or all repeated doses between the immediate and delayed post-intervention periods (P = 0.40, 0.27, 0.20, and 0.55, respectively).

Discussion

This study investigated the effectiveness of interventions to the EMR designed to increase the proportion of correctly administered repeated doses of perioperative antibiotics. We a priori defined a 15% absolute improvement in compliance as being clinically significant. There was no significant improvement in correct administration of the first dose of antibiotic. The improvements for the second and third doses, however, were both statistically significant and clinically meaningful (17-22% and 23-39% absolute increases in the proportions of correctly administered second and third doses, respectively). Importantly, antibiotic dosing compliance should approach 100%. These results suggest that it might be difficult to achieve perfect compliance with repeat antibiotic doses.

Previous studies attempted to address the problem of failure to administer perioperative antibiotics properly,17 but preliminary reviews of EMR interventions aimed at repeated dosing have only recently been reported. While the proportion of correctly administered pre-intervention first dose of antibiotic was already > 90%, our proportion of correctly administered subsequent doses was only around 50%. Therefore, the current interventions focus on a time when clinicians are more prone to error.

Other previous studies used means beyond the EMR to improve clinician compliance with guidelines. Such methods included pre-surgical checklists or safety timeouts18–20 and training sessions for perioperative personnel.21–23 By comparison, our interventions are inexpensive; do not require clinician training, continuing manpower, or financial investment to maintain; do not increase the duration of surgery; and do not interfere with the clinicians’ activities. As an inexpensive nonintrusive intervention, the current approach has the advantage of achieving increased perioperative safety without significant drawbacks. While the current interventions were introduced in the MetaVision EMR, their principles can likely be carried over to other EMRs.

While our interventions led to meaningful improvement, they still fell noticeably shy of ideal levels of proper antibiotic administration. Though conclusions from studies targeting first doses of antibiotics may not apply completely to subsequent doses, lessons from these studies may guide the design of future interventions. In general, successful interventions all involved real-time reminders to clinicians.9–11 Importantly, one study9 achieved close to 100% compliance for first dose of antibiotic. This intervention included more intrusive alerts and was notable for its multi-tiered approach, which previous research on human error suggests is important.24 Our interventions were as nonintrusive as possible by design, as they occurred while surgery was ongoing. More intrusive reminders may increase compliance, but they might also impair how clinicians perform other duties.

This study examined the effect of the interventions on the administration of only one antibiotic, i.e., cefazolin. There were two reasons for our choice. First, cefazolin needed to be adjusted only for patient weight, and its dose does not change for different types of procedures, patient age, or patient comorbidities. As such, clinician deviation from established guidelines was unlikely to represent appropriate clinical judgement. Additionally, cefazolin is the most commonly used perioperative antibiotic.15 As such, clinicians know well how to dose cefazolin properly, and they are better accustomed with properly remembering to repeat doses. For these reasons, cefazolin administration was less likely to show improvement by our interventions compared with administration of other antibiotics, which may have underestimated the benefit of the study interventions.

The before-and-after design of the study prevents us from definitively concluding that our interventions caused the improvements in guideline compliance. That said, the epochs studied control for changes in trainee experience during the calendar year and were distant from major changes to the EMR, hospital antibiotic guidelines, or similar quality improvement initiatives. The stable pre-intervention baseline, abrupt improvement in clinician compliance starting April 2015, and sustained improvement after the interventions (Fig. 5) suggest a strong temporal relationship between the interventions and the improved clinician compliance.

This study did not examine patient outcomes such as rates of wound infection, hospital length of stay, or mortality. Previous studies have repeatedly shown that improvement in perioperative antibiotic administration translates to decreased rates of surgical-site infection.2 The number needed to treat to prevent one infection varied by type of surgery, ranging from 58 for breast and hernia surgery, to 3-9 for gastrointestinal surgery, to 2-3 for cardiac surgery.7 By including only those cases with a minimum duration of four hours, our study was biased toward longer cases that likely included more involved procedures or sicker patients in whom infection would be particularly devastating. Given that the interventions in this study showed a number needed to treat of five repeated doses of antibiotics to administer one additional correct dose, it is reasonable to infer that these interventions would have significant impact on patient outcomes.

Conclusions

In summary, these straightforward inexpensive non-intrusive modifications to the EMR resulted in enduring improvement in anesthesia clinician compliance with guidelines for administration of repeated doses of perioperative antibiotics. The nature of the interventions is such that these advantages can be gained with a minimum of drawbacks. Secondary analyses suggest that the effect size and sustainability of the improvements come from both second and third doses of antibiotics, though the small number of cases requiring third doses of intraoperative antibiotics limits this analysis.