Using physiological monitoring data for performance feedback: an initiative using thermoregulation metrics
- 552 Downloads
Feedback of performance data can improve professional practice and outcomes. Vital signs are not routinely used for quality improvement because of their limited access. Intraoperative hypothermia has been associated with deleterious effects, including surgical site infections and bleeding. We speculated that providing feedback could help keep temperature monitoring and management a priority in the anesthesiologist’s mind, thereby improving perioperative temperature management. We hypothesized that feedback on thermoregulation metrics, without changes in policy, could reduce temperature-monitoring delays at the start of scoliosis correction surgery.
Although our tertiary pediatric centre does not have an anesthesia information management system, vital signs for all surgical cases are recorded in real time. Temperature data from children undergoing spine surgery are extracted from a vital signs databank and analyzed using MATLAB. Spine team anesthesiologists are provided with both team and individualized feedback regarding two variables: the percentage of time that patients are hypothermic and the time delay from the start of the case to the first temperature monitoring (our primary outcome). These data are shared every six months as run charts for the entire group and as anonymized (coded) box-and-whisker plots for each anesthesiologist.
This feedback of temperature-delay data reduced the median [interquartile range] delay from 39.0 [18.7-61.5] min to 14.4 [10.8-22.9] min (median reduction, 21.8 min; 95% confidence interval, 14.9 to 28.2; P < 0.001). It did not, however, further reduce the percentage of time patients remained hypothermic beyond the improvements already achieved with prewarming.
Feedback of intraoperative thermoregulation management improved both group and individual performances as measured by significant, sustained reductions in temperature-monitoring delays. Thus, intraoperative vital signs data may improve the quality of, and reduce the variability in, anesthetic practice.
KeywordsTemperature Monitoring Individualize Feedback Personalized Feedback Anesthesia Information Management System Quality Improvement Study
Utilisation des données de monitorage physiologique pour les rétroactions sur la performance: une initiative basée sur les mesures de la thermorégulation
Les retours de performance peuvent améliorer la pratique professionnelle et les pronostics. Habituellement, les signes vitaux ne sont pas utilisés pour l’amélioration de la qualité en raison de leur accès limité. L’hypothermie peropératoire a été associée à des effets délétères, notamment à des infections du site chirurgical et des saignements. Nous avons émis l’hypothèse que des renvois d’information aideraient l’anesthésiste à prioriser le monitorage et la prise en charge de la température et ainsi à améliorer la prise en charge de la température périopératoire. Selon notre hypothèse, des renvois des mesures de la thermorégulation, sans modification des politiques de prise en charge, pourraient réduire les retards dans le monitorage de la température en début de chirurgie de correction de scoliose.
Bien que notre centre tertiaire de pédiatrie ne dispose pas d’un système de prise en charge des renseignements anesthésiques, les signes vitaux sont enregistrés en temps réel pour tous les cas chirurgicaux. Les données de température des enfants subissant une chirurgie de la colonne vertébrale ont été extraites d’une banque de données de signes vitaux et analysées à l’aide de MATLAB. On a présenté aux anesthésiologistes de l’équipe rachidienne des renvois d’information d’équipe et individuelle concernant deux variables : le pourcentage de temps pendant lequel les patients sont en hypothermie et le laps de temps entre le début du cas et le premier monitorage de température (notre critère d’évaluation principal). Ces données sont partagées tous les six mois sous forme de graphique de séquences pour le groupe complet et sous forme de diagramme des quartiles dépersonnalisé (codé) pour chaque anesthésiologiste.
Ce renvoi des données de retard dans la prise de température a réduit le retard moyen [écart interquartile] de 39,0 [18,7-61,5] min à 14,4 [10,8-22,9] min (réduction médiane, 21,8 min; intervalle de confiance 95 %, 14,9 à 28,2; P < 0,001). Toutefois, ce renvoi d’information n’a pas permis de réduire davantage le pourcentage de temps durant lequel les patients étaient en hypothermie au-delà des améliorations déjà apportées grâce au préchauffage.
Les renvois d’information sur la prise en charge de la thermorégulation peropératoire ont amélioré les performances du groupe et des individus telles que mesurées par des réductions significatives et durables des retards dans le monitorage de la température. Par conséquent, les données de signes vitaux peropératoires peuvent améliorer la qualité et réduire la variabilité de la pratique anesthésique.
Feedback of performance data has been shown to improve professional practice and outcomes.1-3 Examples include improved antibiotic administration,4,5 reduced drug cost,6 and improved operating room booking efficiency.7 Vital signs, drug infusion, and therapy device data are not routinely included in quality improvement (QI) research because of their limited availability. That is, they are not routinely captured, are captured only on paper, or are not captured at a sufficiently high time resolution. Nonetheless, these data can provide a wealth of potentially useful information. They could also fulfill the requirement that, for feedback to improve performance effectively, it needs to be of local relevance and come from a trusted, credible source.8 Anesthesia information management systems (AIMSs) may present significant opportunities to provide individualized feedback.4,9 Unfortunately, many institutions (including ours) do not have an AIMS. Hence, we have explored retrospective vital signs data analysis as an alternative solution.
Why use temperature metrics as an example?
Hypothermia (core temperature < 36 °C)10 during surgery has been associated with numerous deleterious effects, including surgical site infections and bleeding.10-12 General anesthesia profoundly reduces core body temperature, which is primarily due to core-to-periphery redistribution of body heat.13 Although hypothermia is a risk in all anesthetized patients, children are especially susceptible because of their high surface area/body mass ratio. Most heat loss in the operating room occurs soon after induction of anesthesia,14 so early monitoring is indicated to guide the warming therapy. Temperature monitoring during anesthesia has intrinsic value regarding safety.15 Hence, the sooner the temperature is measured, the sooner the benefit of this additional safety mechanism is realized. As anesthesiologists are unable to estimate their patients’ thermal status reliably,16 a measurement technique is needed to titrate the various warming methods effectively. A recent editorial advocated routine, continuous, electronic measurement of the core temperature15 as economic benefits outweigh the risks of not monitoring.
Despite widespread awareness of the consequences of hypothermia and a National Institute for Health and Care Excellence guideline on perioperative normothermia,10 a 2011 National Confidential Enquiry into Patient Outcome and Death report found that many hospitals do not have a policy for preventing perioperative hypothermia.17 Temperature monitoring is not typically prioritized along with other vital signs after induction because there are many conflicting priorities requiring the anesthesiologist’s attention. We speculated that providing feedback on temperature-monitoring delays might help keep temperature monitoring and management a priority in the anesthesiologist’s mind and may act as a reminder to do so proactively. Improving early awareness of core temperature was identified as a key driver to minimize intraoperative hypothermia in our original preoperative warming work.18
The purpose of this QI exercise was to investigate the impact of individualized and group feedback using intraoperative physiological data. Specifically, we aimed to investigate whether feedback on thermoregulation metrics, without changes in policy, could reduce temperature-monitoring delays at the start of scoliosis correction surgery.
This endeavor was a QI study. The University of British Columbia and Children’s & Women’s Health Centre of British Columbia Research Ethics Board does not review quality assurance or QI studies, in accordance with Article 2.5 of the Tri-Council Policy Statement 2. The Research Ethics Board, however, did approve (H15-01795) the collection of anonymized vital signs data from all operating rooms in our tertiary pediatric centre for the purpose of research and QI. The study was presented and approved a priori by all spine team anesthesiologists, who understood and agreed explicitly to the collection and sharing of team and anonymized individual performance data.
Data extraction and processing
Electronic vital signs data from S/5 patient monitors (GE Healthcare Canada, Mississauga, ON, Canada) were captured in real time for all surgical spine cases. Children underwent spine surgery in a single dedicated spine operating room with a standardized setup. Esophageal temperature probes were placed in approximately two-thirds of cases with nasopharyngeal temperature probes placed in the other one-third of cases. From our vital signs databank, temperature values were extracted at a resolution of 0.1 Hz using a previously described method.18 In short, MATLAB (Mathworks Inc., Natick, MA, USA) was used to plot temperature trends, which were manually inspected for artefacts. Next, episodes of hypothermia and their duration were identified. Finally, the temperature-measurement delay was calculated as the difference between the case starting time (when the pulse oximeter is connected to the patient) and the time of the first valid temperature measurement.
Data visualization and feedback mechanism
Run chart data were assessed qualitatively (by visual inspection of trends) and quantitatively. To assess the impact of the feedback, statistical comparisons of both feedback variables were performed between the reference period (March 2011 to June 2012) and the latest six-month feedback period (July to December 2015). The Wilcoxon rank sum test was used to compare the temperature delays and %CSH values. Pseudo-medians and non-parametric 95% confidence intervals (CIs) were obtained using R software (R Foundation for Statistical Computing, Vienna, Austria).
Data from 556 anesthetic performances by ten anesthesiologists were included in the personalized feedback analysis. Each attending anesthesiologist performed a median [interquartile range (IQR)] of 55 [48-75] cases.
Feedback of temperature data to attending anesthesiologists reduced the group median [IQR] temperature delay from 39.0 [18.7-61.5] min during the 16 months prior to July 2012 to 14.4 [10.8-22.9] min for the July to December 2015 window (see Fig. 1). It amounted to a median reduction of 21.8 min (95% CI, 14.9 to 28.2 min; P < 0.001). Although some anesthesiologists improved their performance after being presented with their first data feedback, a personalized reminder was needed to improve performance for two outlying anesthesiologists in December 2014, at which point they also improved their performance to the group median (see Fig. 2). Interestingly, the starting performances of two new anesthesiologists joining the spine team were similar to that of the overall group’s reduced performance.
The percentage of time that patients were hypothermic decreased from a median of 33% prior to the introduction of preoperative warming to a median of 12% after preoperative warming became the standard of care.18 No subsequent improvements that could have been associated with feedback were detected (see Fig. 1). The median reduction between the reference period and the latest feedback window was 0% (95% CI, 0% to 5.0%; P = 0.13).
Although performance feedback can improve professional practice and outcomes,1,2 the efficacy of its use in the field of anesthesia has not been widely reported. In this study, we demonstrated how routinely captured physiological data can be used effectively to provide feedback that improves temperature-monitoring compliance by anesthesiologists during spine surgery. Here, group and anonymous individualized feedback of temperature-monitoring data reduced the median temperature-monitoring delay by half (median reduction of 21 min) and substantially reduced the variability within and between team members (see Fig. 2). No changes in hypothermia durations were observed despite the feedback provided, presumably because all available patient warming methods were applied as quickly as possible in all of the cases. Thus, our previous prewarming initiative18 may have minimized intraoperative hypothermia to the greatest extent possible with our available warming technologies.
Clinical relevance and translation to other metrics
Although reducing the temperature-monitoring delay during pediatric scoliosis correction surgery probably has a limited impact on surgical or patient-centric outcomes, such an approach can be generalized to other settings, including the conduct of anesthesia in high-risk populations, such as neonates. In fact, we are planning to expand our methods to evaluate the conduct of neonatal anesthesia with respect to avoiding hypotension and hypocarbia.20,21 We are also expanding this approach to provide individualized hypothermia and hypotension feedback as part of our involvement in the American College of Surgeons National Surgical Quality Improvement Program Pediatrics.22
We believe that the demonstrated reduction in temperature-monitoring delay is more clinically relevant in the broad context of human factors and performance than in the narrow context of improving perioperative care during pediatric scoliosis correction surgery. Although clinicians believe they are delivering the best care to their patients, they cannot really know how well they are doing or where changes might be needed until given objective, trustworthy data.8 In a clinical environment replete with policies, guidelines, standards of care, clinical practice advisories, checklists, and care pathways, engaging high-performing individuals’ desire to do better is not always readily achieved by repeatedly reiterating these methods. Neither does highlighting suboptimal care at a composite group level (e.g., the whole department) because we each tend to believe that we are not among the individuals contributing to that suboptimal performance. Furthermore, even good group performance can mask substantial variability.23
Rolling feedback1,2 helps provide and maintain focus. Breaking down performance to the individual level allows comparison with other group members and with the group as a whole. It provides a powerful incentive, harnesses innate competitive instincts, and reduces variability in performance by removing ‘high outliers’ and driving the overall standard of care upward. The power of anonymized comparative performance feedback to deliver improved quality of care can be generalized to other perioperative metrics and is worthy of further study.
Warming feedback effects
Preoperative warming, which increases the patient’s thermal mass, has been shown to decrease the percentage of operations during which the patient is hypothermic by a median of 22% in children undergoing spine surgery.18 It was also associated with a reduction in the number of allogenic blood transfusions.24 Prewarming alone, however, does not prevent hypothermia. To decrease the incidence of intraoperative hypothermia effectively in all patients, additional measures are required, including forced air warming, fluid warming, or increasing the operating room temperature.25 A consistent approach to intraoperative temperature monitoring is vital for the guidance of best-practice medicine.
Comparative performance feedback improved performance and reduced variability with respect to temperature-monitoring delay. This improvement, however, did not translate into additional reductions in the percentage of time that patients were hypothermic (median 12%) (Fig. 1) despite the fact that our institution has made recent strides in improving perioperative temperature regulation through prewarming, heightened vigilance, and active temperature management. An additional systemic change in perioperative management appears to be necessary to reduce it further, although the relevant intervention remains to be identified.
Although the data collection methodology presented was robust - and free of biases and confounders - the audit and feedback cycle of six months is quite long. The reason for this extended period is that a minimum number of cases must be captured to allow comparisons. For this particular patient population, four spine surgery cases are performed each week in our institution. For institutions in which a larger case sample is generated more rapidly, more frequent review would be feasible and would provide more contemporaneous feedback.
It is possible that a Hawthorne effect contributed to the improved performance of some (or all) of the anesthesiologists. That is, the awareness that temperature metrics were being measured was responsible for the improvement in performance, rather than the feedback itself. In terms of QI, this distinction is moot. The initiative of using personalized feedback significantly reduced temperature-monitoring delays, possibly for different reasons for different anesthesiologists. It is also possible that other factors confounded a causal relation between feedback of data and individuals’ performances. If implemented in other centers, one could evaluate the difference between personalized feedback, as described here, and a setting in which anesthesiologists are told that metrics are being studied but no feedback is given.
Even in institutions with an AIMS, analyzing data in a manner similar to that described herein would require an analyst to generate reports that aggregate vital signs data and then transform them into comprehensible presentations. Although it would somewhat limit its generalizability, the process would be significantly simplified as the temperature data and attending anesthesiologist would already be available in an accessible form.
Feedback of intraoperative thermoregulation data to pediatric spine team anesthesiologists was able to improve group and individual performances, as measured by significant, sustained reductions in temperature-monitoring delays during pediatric scoliosis correction surgery. This example of group and anonymized individual comparative performance feedback illustrates how intraoperative vital signs data could be used to encourage improved quality and reduce variability of care in anesthetic practice, which in turn should contribute to improved perioperative outcomes for patients.
Conflicts of interests
Matthias Görges helped design the study, conduct the study, analyze the data, and write the manuscript. He has seen the original study data, reviewed the analysis of the data, and approved the final manuscript. Nicholas West helped conduct the study and helped write the manuscript. He approved the final manuscript. Simon Whyte helped design the study, helped conduct the study, oversaw the data analysis, and helped write the manuscript. He has seen the original study data and approved the final manuscript.
This submission was handled by Dr. Philip M. Jones, Associate Editor, Canadian Journal of Anesthesia.
- 3.Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012; 6: CD000259.Google Scholar
- 10.National Collaborating Centre for Nursing and Supportive Care. The Management of Inadvertent Perioperative Hypothermia in Adults. NICE Clinical Guidelines, No. 65. London, United Kingdom; 2008.Google Scholar
- 17.Findlay GP, Goodwin AP, Protopapa KL, Smith NC, Mason M; NCEPOD. Knowing the Risk: a Review of the Peri-operative Care of Surgical Patients. London, United Kingdom; 2011.Google Scholar
- 19.Woodall WH, Adams BM, Benneyan JC. The use of control charts in healthcare. In: Faltin FW, Kenett RS, Ruggeri F (Eds). Statistical Methods in Healthcare. Chichester, UK: John Wiley & Sons, Ltd; 2012: 251-67.Google Scholar
- 20.Lauder G, Görges M, West N, Karlsdóttir E. Neonatal Vital Sign Deviations During General Anaesthesia: Developing Consensus On What Represents Significant Change. Proceedings of the 2015 Association of Paediatric Anaesthetists of Great Britain & Ireland Annual Scientific Meeting. Aberdeen, UK: Association of Paediatric Anaesthetists of Great Britain and Ireland; 2015: 108.Google Scholar
- 22.Bedford JE, Görges M, Whyte SD. Anonymised Individual Feedback on Management of Intraoperative Thermoregulation in Children - a NSQIP-P Quality Improvement Project. Proceedings of the 2015 ACS NSQIP National Conference. Chicago, IL: American College of Surgeons; 2015: ACSNSQIP20150192.Google Scholar