Feedback of performance data has been shown to improve professional practice and outcomes.1-3 Examples include improved antibiotic administration,4,5 reduced drug cost,6 and improved operating room booking efficiency.7 Vital signs, drug infusion, and therapy device data are not routinely included in quality improvement (QI) research because of their limited availability. That is, they are not routinely captured, are captured only on paper, or are not captured at a sufficiently high time resolution. Nonetheless, these data can provide a wealth of potentially useful information. They could also fulfill the requirement that, for feedback to improve performance effectively, it needs to be of local relevance and come from a trusted, credible source.8 Anesthesia information management systems (AIMSs) may present significant opportunities to provide individualized feedback.4,9 Unfortunately, many institutions (including ours) do not have an AIMS. Hence, we have explored retrospective vital signs data analysis as an alternative solution.

Why use temperature metrics as an example?

Hypothermia (core temperature < 36 °C)10 during surgery has been associated with numerous deleterious effects, including surgical site infections and bleeding.10-12 General anesthesia profoundly reduces core body temperature, which is primarily due to core-to-periphery redistribution of body heat.13 Although hypothermia is a risk in all anesthetized patients, children are especially susceptible because of their high surface area/body mass ratio. Most heat loss in the operating room occurs soon after induction of anesthesia,14 so early monitoring is indicated to guide the warming therapy. Temperature monitoring during anesthesia has intrinsic value regarding safety.15 Hence, the sooner the temperature is measured, the sooner the benefit of this additional safety mechanism is realized. As anesthesiologists are unable to estimate their patients’ thermal status reliably,16 a measurement technique is needed to titrate the various warming methods effectively. A recent editorial advocated routine, continuous, electronic measurement of the core temperature15 as economic benefits outweigh the risks of not monitoring.

Despite widespread awareness of the consequences of hypothermia and a National Institute for Health and Care Excellence guideline on perioperative normothermia,10 a 2011 National Confidential Enquiry into Patient Outcome and Death report found that many hospitals do not have a policy for preventing perioperative hypothermia.17 Temperature monitoring is not typically prioritized along with other vital signs after induction because there are many conflicting priorities requiring the anesthesiologist’s attention. We speculated that providing feedback on temperature-monitoring delays might help keep temperature monitoring and management a priority in the anesthesiologist’s mind and may act as a reminder to do so proactively. Improving early awareness of core temperature was identified as a key driver to minimize intraoperative hypothermia in our original preoperative warming work.18

Study aim

The purpose of this QI exercise was to investigate the impact of individualized and group feedback using intraoperative physiological data. Specifically, we aimed to investigate whether feedback on thermoregulation metrics, without changes in policy, could reduce temperature-monitoring delays at the start of scoliosis correction surgery.

Methods

This endeavor was a QI study. The University of British Columbia and Children’s & Women’s Health Centre of British Columbia Research Ethics Board does not review quality assurance or QI studies, in accordance with Article 2.5 of the Tri-Council Policy Statement 2. The Research Ethics Board, however, did approve (H15-01795) the collection of anonymized vital signs data from all operating rooms in our tertiary pediatric centre for the purpose of research and QI. The study was presented and approved a priori by all spine team anesthesiologists, who understood and agreed explicitly to the collection and sharing of team and anonymized individual performance data.

Data extraction and processing

Electronic vital signs data from S/5 patient monitors (GE Healthcare Canada, Mississauga, ON, Canada) were captured in real time for all surgical spine cases. Children underwent spine surgery in a single dedicated spine operating room with a standardized setup. Esophageal temperature probes were placed in approximately two-thirds of cases with nasopharyngeal temperature probes placed in the other one-third of cases. From our vital signs databank, temperature values were extracted at a resolution of 0.1 Hz using a previously described method.18 In short, MATLAB (Mathworks Inc., Natick, MA, USA) was used to plot temperature trends, which were manually inspected for artefacts. Next, episodes of hypothermia and their duration were identified. Finally, the temperature-measurement delay was calculated as the difference between the case starting time (when the pulse oximeter is connected to the patient) and the time of the first valid temperature measurement.

Data visualization and feedback mechanism

Anesthesiologists were introduced to the interpretation of run charts and box-and-whisker plots during presentations of these data at research rounds and quality-of-care rounds. Next, each anesthesiologist was assigned a unique identifier for QI purposes that was known only to him or her. Subsequently, spine team anesthesiologists were provided with team and individualized feedback on two variables: (1) percentage of time their patients were hypothermic (%CSH) and (2) the time delay until instituting temperature monitoring. Data were shared by the quality lead for the spine team (S.W.) using e-mail every six months, both as run charts19 for the entire group (see Fig. 1 for an example) and as anonymized (coded) box plots, overlaid by dot plots, for each anesthesiologist (see Fig. 2 for examples).

Fig. 1
figure 1

Run charts for the percentage of time patients were hypothermic (top subplot) and for the temperature delay (bottom subplot). Box plots indicate interquartile ranges for a given month. Group medians (black dots) are overlaid. The green bars indicate the median over each intervention window. Arrows indicate the times of starting the two main initiatives used to reduce the duration of intraoperative hypothermia

Full size image
Fig. 2
figure 2

Individual performance for temperature delay. Box plots indicate interquartile ranges. Dot plots of raw data are superimposed. The overall median (solid line spanning the entire plot width) is overlaid. Each anesthesiologist is assigned an integer identifier (x-axis) that is valid throughout the study period. Note that the composition of the spine anesthesia team changed three times: in July 2013, January 2014, and January 2015

Full size image

Data analysis

Run chart data were assessed qualitatively (by visual inspection of trends) and quantitatively. To assess the impact of the feedback, statistical comparisons of both feedback variables were performed between the reference period (March 2011 to June 2012) and the latest six-month feedback period (July to December 2015). The Wilcoxon rank sum test was used to compare the temperature delays and %CSH values. Pseudo-medians and non-parametric 95% confidence intervals (CIs) were obtained using R software (R Foundation for Statistical Computing, Vienna, Austria).

Results

Data from 556 anesthetic performances by ten anesthesiologists were included in the personalized feedback analysis. Each attending anesthesiologist performed a median [interquartile range (IQR)] of 55 [48-75] cases.

Feedback of temperature data to attending anesthesiologists reduced the group median [IQR] temperature delay from 39.0 [18.7-61.5] min during the 16 months prior to July 2012 to 14.4 [10.8-22.9] min for the July to December 2015 window (see Fig. 1). It amounted to a median reduction of 21.8 min (95% CI, 14.9 to 28.2 min; P < 0.001). Although some anesthesiologists improved their performance after being presented with their first data feedback, a personalized reminder was needed to improve performance for two outlying anesthesiologists in December 2014, at which point they also improved their performance to the group median (see Fig. 2). Interestingly, the starting performances of two new anesthesiologists joining the spine team were similar to that of the overall group’s reduced performance.

The percentage of time that patients were hypothermic decreased from a median of 33% prior to the introduction of preoperative warming to a median of 12% after preoperative warming became the standard of care.18 No subsequent improvements that could have been associated with feedback were detected (see Fig. 1). The median reduction between the reference period and the latest feedback window was 0% (95% CI, 0% to 5.0%; P = 0.13).

Discussion

Although performance feedback can improve professional practice and outcomes,1,2 the efficacy of its use in the field of anesthesia has not been widely reported. In this study, we demonstrated how routinely captured physiological data can be used effectively to provide feedback that improves temperature-monitoring compliance by anesthesiologists during spine surgery. Here, group and anonymous individualized feedback of temperature-monitoring data reduced the median temperature-monitoring delay by half (median reduction of 21 min) and substantially reduced the variability within and between team members (see Fig. 2). No changes in hypothermia durations were observed despite the feedback provided, presumably because all available patient warming methods were applied as quickly as possible in all of the cases. Thus, our previous prewarming initiative18 may have minimized intraoperative hypothermia to the greatest extent possible with our available warming technologies.

Clinical relevance and translation to other metrics

Although reducing the temperature-monitoring delay during pediatric scoliosis correction surgery probably has a limited impact on surgical or patient-centric outcomes, such an approach can be generalized to other settings, including the conduct of anesthesia in high-risk populations, such as neonates. In fact, we are planning to expand our methods to evaluate the conduct of neonatal anesthesia with respect to avoiding hypotension and hypocarbia.20,21 We are also expanding this approach to provide individualized hypothermia and hypotension feedback as part of our involvement in the American College of Surgeons National Surgical Quality Improvement Program Pediatrics.22

We believe that the demonstrated reduction in temperature-monitoring delay is more clinically relevant in the broad context of human factors and performance than in the narrow context of improving perioperative care during pediatric scoliosis correction surgery. Although clinicians believe they are delivering the best care to their patients, they cannot really know how well they are doing or where changes might be needed until given objective, trustworthy data.8 In a clinical environment replete with policies, guidelines, standards of care, clinical practice advisories, checklists, and care pathways, engaging high-performing individuals’ desire to do better is not always readily achieved by repeatedly reiterating these methods. Neither does highlighting suboptimal care at a composite group level (e.g., the whole department) because we each tend to believe that we are not among the individuals contributing to that suboptimal performance. Furthermore, even good group performance can mask substantial variability.23

Rolling feedback1,2 helps provide and maintain focus. Breaking down performance to the individual level allows comparison with other group members and with the group as a whole. It provides a powerful incentive, harnesses innate competitive instincts, and reduces variability in performance by removing ‘high outliers’ and driving the overall standard of care upward. The power of anonymized comparative performance feedback to deliver improved quality of care can be generalized to other perioperative metrics and is worthy of further study.

Warming feedback effects

Preoperative warming, which increases the patient’s thermal mass, has been shown to decrease the percentage of operations during which the patient is hypothermic by a median of 22% in children undergoing spine surgery.18 It was also associated with a reduction in the number of allogenic blood transfusions.24 Prewarming alone, however, does not prevent hypothermia. To decrease the incidence of intraoperative hypothermia effectively in all patients, additional measures are required, including forced air warming, fluid warming, or increasing the operating room temperature.25 A consistent approach to intraoperative temperature monitoring is vital for the guidance of best-practice medicine.

Comparative performance feedback improved performance and reduced variability with respect to temperature-monitoring delay. This improvement, however, did not translate into additional reductions in the percentage of time that patients were hypothermic (median 12%) (Fig. 1) despite the fact that our institution has made recent strides in improving perioperative temperature regulation through prewarming, heightened vigilance, and active temperature management. An additional systemic change in perioperative management appears to be necessary to reduce it further, although the relevant intervention remains to be identified.

Limitations

Although the data collection methodology presented was robust - and free of biases and confounders - the audit and feedback cycle of six months is quite long. The reason for this extended period is that a minimum number of cases must be captured to allow comparisons. For this particular patient population, four spine surgery cases are performed each week in our institution. For institutions in which a larger case sample is generated more rapidly, more frequent review would be feasible and would provide more contemporaneous feedback.

It is possible that a Hawthorne effect contributed to the improved performance of some (or all) of the anesthesiologists. That is, the awareness that temperature metrics were being measured was responsible for the improvement in performance, rather than the feedback itself. In terms of QI, this distinction is moot. The initiative of using personalized feedback significantly reduced temperature-monitoring delays, possibly for different reasons for different anesthesiologists. It is also possible that other factors confounded a causal relation between feedback of data and individuals’ performances. If implemented in other centers, one could evaluate the difference between personalized feedback, as described here, and a setting in which anesthesiologists are told that metrics are being studied but no feedback is given.

Even in institutions with an AIMS, analyzing data in a manner similar to that described herein would require an analyst to generate reports that aggregate vital signs data and then transform them into comprehensible presentations. Although it would somewhat limit its generalizability, the process would be significantly simplified as the temperature data and attending anesthesiologist would already be available in an accessible form.

Conclusions

Feedback of intraoperative thermoregulation data to pediatric spine team anesthesiologists was able to improve group and individual performances, as measured by significant, sustained reductions in temperature-monitoring delays during pediatric scoliosis correction surgery. This example of group and anonymized individual comparative performance feedback illustrates how intraoperative vital signs data could be used to encourage improved quality and reduce variability of care in anesthetic practice, which in turn should contribute to improved perioperative outcomes for patients.