Advertisement

Using physiological monitoring data for performance feedback: an initiative using thermoregulation metrics

  • Matthias Görges
  • Nicholas C. West
  • Simon D. Whyte
Reports of Original Investigations

Abstract

Background

Feedback of performance data can improve professional practice and outcomes. Vital signs are not routinely used for quality improvement because of their limited access. Intraoperative hypothermia has been associated with deleterious effects, including surgical site infections and bleeding. We speculated that providing feedback could help keep temperature monitoring and management a priority in the anesthesiologist’s mind, thereby improving perioperative temperature management. We hypothesized that feedback on thermoregulation metrics, without changes in policy, could reduce temperature-monitoring delays at the start of scoliosis correction surgery.

Methods

Although our tertiary pediatric centre does not have an anesthesia information management system, vital signs for all surgical cases are recorded in real time. Temperature data from children undergoing spine surgery are extracted from a vital signs databank and analyzed using MATLAB. Spine team anesthesiologists are provided with both team and individualized feedback regarding two variables: the percentage of time that patients are hypothermic and the time delay from the start of the case to the first temperature monitoring (our primary outcome). These data are shared every six months as run charts for the entire group and as anonymized (coded) box-and-whisker plots for each anesthesiologist.

Results

This feedback of temperature-delay data reduced the median [interquartile range] delay from 39.0 [18.7-61.5] min to 14.4 [10.8-22.9] min (median reduction, 21.8 min; 95% confidence interval, 14.9 to 28.2; P < 0.001). It did not, however, further reduce the percentage of time patients remained hypothermic beyond the improvements already achieved with prewarming.

Conclusion

Feedback of intraoperative thermoregulation management improved both group and individual performances as measured by significant, sustained reductions in temperature-monitoring delays. Thus, intraoperative vital signs data may improve the quality of, and reduce the variability in, anesthetic practice.

Keywords

Temperature Monitoring Individualize Feedback Personalized Feedback Anesthesia Information Management System Quality Improvement Study 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Utilisation des données de monitorage physiologique pour les rétroactions sur la performance: une initiative basée sur les mesures de la thermorégulation

Résumé

Contexte

Les retours de performance peuvent améliorer la pratique professionnelle et les pronostics. Habituellement, les signes vitaux ne sont pas utilisés pour l’amélioration de la qualité en raison de leur accès limité. L’hypothermie peropératoire a été associée à des effets délétères, notamment à des infections du site chirurgical et des saignements. Nous avons émis l’hypothèse que des renvois d’information aideraient l’anesthésiste à prioriser le monitorage et la prise en charge de la température et ainsi à améliorer la prise en charge de la température périopératoire. Selon notre hypothèse, des renvois des mesures de la thermorégulation, sans modification des politiques de prise en charge, pourraient réduire les retards dans le monitorage de la température en début de chirurgie de correction de scoliose.

Méthode

Bien que notre centre tertiaire de pédiatrie ne dispose pas d’un système de prise en charge des renseignements anesthésiques, les signes vitaux sont enregistrés en temps réel pour tous les cas chirurgicaux. Les données de température des enfants subissant une chirurgie de la colonne vertébrale ont été extraites d’une banque de données de signes vitaux et analysées à l’aide de MATLAB. On a présenté aux anesthésiologistes de l’équipe rachidienne des renvois d’information d’équipe et individuelle concernant deux variables : le pourcentage de temps pendant lequel les patients sont en hypothermie et le laps de temps entre le début du cas et le premier monitorage de température (notre critère d’évaluation principal). Ces données sont partagées tous les six mois sous forme de graphique de séquences pour le groupe complet et sous forme de diagramme des quartiles dépersonnalisé (codé) pour chaque anesthésiologiste.

Résultats

Ce renvoi des données de retard dans la prise de température a réduit le retard moyen [écart interquartile] de 39,0 [18,7-61,5] min à 14,4 [10,8-22,9] min (réduction médiane, 21,8 min; intervalle de confiance 95 %, 14,9 à 28,2; P < 0,001). Toutefois, ce renvoi d’information n’a pas permis de réduire davantage le pourcentage de temps durant lequel les patients étaient en hypothermie au-delà des améliorations déjà apportées grâce au préchauffage.

Conclusion

Les renvois d’information sur la prise en charge de la thermorégulation peropératoire ont amélioré les performances du groupe et des individus telles que mesurées par des réductions significatives et durables des retards dans le monitorage de la température. Par conséquent, les données de signes vitaux peropératoires peuvent améliorer la qualité et réduire la variabilité de la pratique anesthésique.

Feedback of performance data has been shown to improve professional practice and outcomes.1-3 Examples include improved antibiotic administration,4,5 reduced drug cost,6 and improved operating room booking efficiency.7 Vital signs, drug infusion, and therapy device data are not routinely included in quality improvement (QI) research because of their limited availability. That is, they are not routinely captured, are captured only on paper, or are not captured at a sufficiently high time resolution. Nonetheless, these data can provide a wealth of potentially useful information. They could also fulfill the requirement that, for feedback to improve performance effectively, it needs to be of local relevance and come from a trusted, credible source.8 Anesthesia information management systems (AIMSs) may present significant opportunities to provide individualized feedback.4,9 Unfortunately, many institutions (including ours) do not have an AIMS. Hence, we have explored retrospective vital signs data analysis as an alternative solution.

Why use temperature metrics as an example?

Hypothermia (core temperature < 36 °C)10 during surgery has been associated with numerous deleterious effects, including surgical site infections and bleeding.10-12 General anesthesia profoundly reduces core body temperature, which is primarily due to core-to-periphery redistribution of body heat.13 Although hypothermia is a risk in all anesthetized patients, children are especially susceptible because of their high surface area/body mass ratio. Most heat loss in the operating room occurs soon after induction of anesthesia,14 so early monitoring is indicated to guide the warming therapy. Temperature monitoring during anesthesia has intrinsic value regarding safety.15 Hence, the sooner the temperature is measured, the sooner the benefit of this additional safety mechanism is realized. As anesthesiologists are unable to estimate their patients’ thermal status reliably,16 a measurement technique is needed to titrate the various warming methods effectively. A recent editorial advocated routine, continuous, electronic measurement of the core temperature15 as economic benefits outweigh the risks of not monitoring.

Despite widespread awareness of the consequences of hypothermia and a National Institute for Health and Care Excellence guideline on perioperative normothermia,10 a 2011 National Confidential Enquiry into Patient Outcome and Death report found that many hospitals do not have a policy for preventing perioperative hypothermia.17 Temperature monitoring is not typically prioritized along with other vital signs after induction because there are many conflicting priorities requiring the anesthesiologist’s attention. We speculated that providing feedback on temperature-monitoring delays might help keep temperature monitoring and management a priority in the anesthesiologist’s mind and may act as a reminder to do so proactively. Improving early awareness of core temperature was identified as a key driver to minimize intraoperative hypothermia in our original preoperative warming work.18

Study aim

The purpose of this QI exercise was to investigate the impact of individualized and group feedback using intraoperative physiological data. Specifically, we aimed to investigate whether feedback on thermoregulation metrics, without changes in policy, could reduce temperature-monitoring delays at the start of scoliosis correction surgery.

Methods

This endeavor was a QI study. The University of British Columbia and Children’s & Women’s Health Centre of British Columbia Research Ethics Board does not review quality assurance or QI studies, in accordance with Article 2.5 of the Tri-Council Policy Statement 2. The Research Ethics Board, however, did approve (H15-01795) the collection of anonymized vital signs data from all operating rooms in our tertiary pediatric centre for the purpose of research and QI. The study was presented and approved a priori by all spine team anesthesiologists, who understood and agreed explicitly to the collection and sharing of team and anonymized individual performance data.

Data extraction and processing

Electronic vital signs data from S/5 patient monitors (GE Healthcare Canada, Mississauga, ON, Canada) were captured in real time for all surgical spine cases. Children underwent spine surgery in a single dedicated spine operating room with a standardized setup. Esophageal temperature probes were placed in approximately two-thirds of cases with nasopharyngeal temperature probes placed in the other one-third of cases. From our vital signs databank, temperature values were extracted at a resolution of 0.1 Hz using a previously described method.18 In short, MATLAB (Mathworks Inc., Natick, MA, USA) was used to plot temperature trends, which were manually inspected for artefacts. Next, episodes of hypothermia and their duration were identified. Finally, the temperature-measurement delay was calculated as the difference between the case starting time (when the pulse oximeter is connected to the patient) and the time of the first valid temperature measurement.

Data visualization and feedback mechanism

Anesthesiologists were introduced to the interpretation of run charts and box-and-whisker plots during presentations of these data at research rounds and quality-of-care rounds. Next, each anesthesiologist was assigned a unique identifier for QI purposes that was known only to him or her. Subsequently, spine team anesthesiologists were provided with team and individualized feedback on two variables: (1) percentage of time their patients were hypothermic (%CSH) and (2) the time delay until instituting temperature monitoring. Data were shared by the quality lead for the spine team (S.W.) using e-mail every six months, both as run charts19 for the entire group (see Fig. 1 for an example) and as anonymized (coded) box plots, overlaid by dot plots, for each anesthesiologist (see Fig. 2 for examples).
Fig. 1

Run charts for the percentage of time patients were hypothermic (top subplot) and for the temperature delay (bottom subplot). Box plots indicate interquartile ranges for a given month. Group medians (black dots) are overlaid. The green bars indicate the median over each intervention window. Arrows indicate the times of starting the two main initiatives used to reduce the duration of intraoperative hypothermia

Fig. 2

Individual performance for temperature delay. Box plots indicate interquartile ranges. Dot plots of raw data are superimposed. The overall median (solid line spanning the entire plot width) is overlaid. Each anesthesiologist is assigned an integer identifier (x-axis) that is valid throughout the study period. Note that the composition of the spine anesthesia team changed three times: in July 2013, January 2014, and January 2015

Data analysis

Run chart data were assessed qualitatively (by visual inspection of trends) and quantitatively. To assess the impact of the feedback, statistical comparisons of both feedback variables were performed between the reference period (March 2011 to June 2012) and the latest six-month feedback period (July to December 2015). The Wilcoxon rank sum test was used to compare the temperature delays and %CSH values. Pseudo-medians and non-parametric 95% confidence intervals (CIs) were obtained using R software (R Foundation for Statistical Computing, Vienna, Austria).

Results

Data from 556 anesthetic performances by ten anesthesiologists were included in the personalized feedback analysis. Each attending anesthesiologist performed a median [interquartile range (IQR)] of 55 [48-75] cases.

Feedback of temperature data to attending anesthesiologists reduced the group median [IQR] temperature delay from 39.0 [18.7-61.5] min during the 16 months prior to July 2012 to 14.4 [10.8-22.9] min for the July to December 2015 window (see Fig. 1). It amounted to a median reduction of 21.8 min (95% CI, 14.9 to 28.2 min; P < 0.001). Although some anesthesiologists improved their performance after being presented with their first data feedback, a personalized reminder was needed to improve performance for two outlying anesthesiologists in December 2014, at which point they also improved their performance to the group median (see Fig. 2). Interestingly, the starting performances of two new anesthesiologists joining the spine team were similar to that of the overall group’s reduced performance.

The percentage of time that patients were hypothermic decreased from a median of 33% prior to the introduction of preoperative warming to a median of 12% after preoperative warming became the standard of care.18 No subsequent improvements that could have been associated with feedback were detected (see Fig. 1). The median reduction between the reference period and the latest feedback window was 0% (95% CI, 0% to 5.0%; P = 0.13).

Discussion

Although performance feedback can improve professional practice and outcomes,1,2 the efficacy of its use in the field of anesthesia has not been widely reported. In this study, we demonstrated how routinely captured physiological data can be used effectively to provide feedback that improves temperature-monitoring compliance by anesthesiologists during spine surgery. Here, group and anonymous individualized feedback of temperature-monitoring data reduced the median temperature-monitoring delay by half (median reduction of 21 min) and substantially reduced the variability within and between team members (see Fig. 2). No changes in hypothermia durations were observed despite the feedback provided, presumably because all available patient warming methods were applied as quickly as possible in all of the cases. Thus, our previous prewarming initiative18 may have minimized intraoperative hypothermia to the greatest extent possible with our available warming technologies.

Clinical relevance and translation to other metrics

Although reducing the temperature-monitoring delay during pediatric scoliosis correction surgery probably has a limited impact on surgical or patient-centric outcomes, such an approach can be generalized to other settings, including the conduct of anesthesia in high-risk populations, such as neonates. In fact, we are planning to expand our methods to evaluate the conduct of neonatal anesthesia with respect to avoiding hypotension and hypocarbia.20,21 We are also expanding this approach to provide individualized hypothermia and hypotension feedback as part of our involvement in the American College of Surgeons National Surgical Quality Improvement Program Pediatrics.22

We believe that the demonstrated reduction in temperature-monitoring delay is more clinically relevant in the broad context of human factors and performance than in the narrow context of improving perioperative care during pediatric scoliosis correction surgery. Although clinicians believe they are delivering the best care to their patients, they cannot really know how well they are doing or where changes might be needed until given objective, trustworthy data.8 In a clinical environment replete with policies, guidelines, standards of care, clinical practice advisories, checklists, and care pathways, engaging high-performing individuals’ desire to do better is not always readily achieved by repeatedly reiterating these methods. Neither does highlighting suboptimal care at a composite group level (e.g., the whole department) because we each tend to believe that we are not among the individuals contributing to that suboptimal performance. Furthermore, even good group performance can mask substantial variability.23

Rolling feedback1,2 helps provide and maintain focus. Breaking down performance to the individual level allows comparison with other group members and with the group as a whole. It provides a powerful incentive, harnesses innate competitive instincts, and reduces variability in performance by removing ‘high outliers’ and driving the overall standard of care upward. The power of anonymized comparative performance feedback to deliver improved quality of care can be generalized to other perioperative metrics and is worthy of further study.

Warming feedback effects

Preoperative warming, which increases the patient’s thermal mass, has been shown to decrease the percentage of operations during which the patient is hypothermic by a median of 22% in children undergoing spine surgery.18 It was also associated with a reduction in the number of allogenic blood transfusions.24 Prewarming alone, however, does not prevent hypothermia. To decrease the incidence of intraoperative hypothermia effectively in all patients, additional measures are required, including forced air warming, fluid warming, or increasing the operating room temperature.25 A consistent approach to intraoperative temperature monitoring is vital for the guidance of best-practice medicine.

Comparative performance feedback improved performance and reduced variability with respect to temperature-monitoring delay. This improvement, however, did not translate into additional reductions in the percentage of time that patients were hypothermic (median 12%) (Fig. 1) despite the fact that our institution has made recent strides in improving perioperative temperature regulation through prewarming, heightened vigilance, and active temperature management. An additional systemic change in perioperative management appears to be necessary to reduce it further, although the relevant intervention remains to be identified.

Limitations

Although the data collection methodology presented was robust - and free of biases and confounders - the audit and feedback cycle of six months is quite long. The reason for this extended period is that a minimum number of cases must be captured to allow comparisons. For this particular patient population, four spine surgery cases are performed each week in our institution. For institutions in which a larger case sample is generated more rapidly, more frequent review would be feasible and would provide more contemporaneous feedback.

It is possible that a Hawthorne effect contributed to the improved performance of some (or all) of the anesthesiologists. That is, the awareness that temperature metrics were being measured was responsible for the improvement in performance, rather than the feedback itself. In terms of QI, this distinction is moot. The initiative of using personalized feedback significantly reduced temperature-monitoring delays, possibly for different reasons for different anesthesiologists. It is also possible that other factors confounded a causal relation between feedback of data and individuals’ performances. If implemented in other centers, one could evaluate the difference between personalized feedback, as described here, and a setting in which anesthesiologists are told that metrics are being studied but no feedback is given.

Even in institutions with an AIMS, analyzing data in a manner similar to that described herein would require an analyst to generate reports that aggregate vital signs data and then transform them into comprehensible presentations. Although it would somewhat limit its generalizability, the process would be significantly simplified as the temperature data and attending anesthesiologist would already be available in an accessible form.

Conclusions

Feedback of intraoperative thermoregulation data to pediatric spine team anesthesiologists was able to improve group and individual performances, as measured by significant, sustained reductions in temperature-monitoring delays during pediatric scoliosis correction surgery. This example of group and anonymized individual comparative performance feedback illustrates how intraoperative vital signs data could be used to encourage improved quality and reduce variability of care in anesthetic practice, which in turn should contribute to improved perioperative outcomes for patients.

Notes

Funding

None.

Disclosures

None.

Conflicts of interests

None declared.

Author contributions

Matthias Görges helped design the study, conduct the study, analyze the data, and write the manuscript. He has seen the original study data, reviewed the analysis of the data, and approved the final manuscript. Nicholas West helped conduct the study and helped write the manuscript. He approved the final manuscript. Simon Whyte helped design the study, helped conduct the study, oversaw the data analysis, and helped write the manuscript. He has seen the original study data and approved the final manuscript.

Editorial responsibility

This submission was handled by Dr. Philip M. Jones, Associate Editor, Canadian Journal of Anesthesia.

References

  1. 1.
    Benn J, Arnold G, Wei I, Riley C, Aleva F. Using quality indicators in anaesthesia: feeding back data to improve care. Br J Anaesth 2012; 109: 80-91.CrossRefPubMedGoogle Scholar
  2. 2.
    Benn J, Arnold G, D’Lima D, et al. Evaluation of a continuous monitoring and feedback initiative to improve quality of anaesthetic care: a mixed-methods quasi-experimental study. Health Serv Deliv Res 2015; 3: 1-248.CrossRefGoogle Scholar
  3. 3.
    Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012; 6: CD000259.Google Scholar
  4. 4.
    Nair BG, Newman SF, Peterson GN, Wu WY, Schwid HA. Feedback mechanisms including real-time electronic alerts to achieve near 100% timely prophylactic antibiotic administration in surgical cases. Anesth Analg 2010; 111: 1293-300.CrossRefPubMedGoogle Scholar
  5. 5.
    Sutherland T, Beloff J, Lightowler M, et al. Description of a multidisciplinary initiative to improve SCIP measures related to pre-operative antibiotic prophylaxis compliance: a single-center success story. Patient Saf Surg 2014; 8: 37.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Malapero RJ, Gabriel RA, Gimlich R, et al. An anesthesia medication cost scorecard-concepts for individualized feedback. J Med Syst 2015; 39: 48.CrossRefPubMedGoogle Scholar
  7. 7.
    Gabriel RA, Gimlich R, Ehrenfeld JM, Urman RD. Operating room metrics score card-creating a prototype for individualized feedback. J Med Syst 2014; 38: 144.CrossRefPubMedGoogle Scholar
  8. 8.
    D’Lima DM, Moore J, Bottle A, Brett SJ, Arnold GM, Benn J. Developing effective feedback on quality of anaesthetic care: what are its most valuable characteristics from a clinical perspective? J Health Serv Res Policy 2015; 20(1 Suppl): 26-34.CrossRefPubMedGoogle Scholar
  9. 9.
    Epstein RH, Dexter F, Patel N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesth Analg 2015; 121: 678-92.CrossRefPubMedGoogle Scholar
  10. 10.
    National Collaborating Centre for Nursing and Supportive Care. The Management of Inadvertent Perioperative Hypothermia in Adults. NICE Clinical Guidelines, No. 65. London, United Kingdom; 2008.Google Scholar
  11. 11.
    Linam WM, Margolis PA, Staat MA, et al. Risk factors associated with surgical site infection after pediatric posterior spinal fusion procedure. Infect Control Hosp Epidemiol 2009; 30: 109-16.PubMedGoogle Scholar
  12. 12.
    Kurz A, Sessler DI, Lenhardt R. Perioperative normothermia to reduce the incidence of surgical-wound infection and shorten hospitalization. Study of Wound Infection and Temperature Group. N Engl J Med 1996; 334: 1209-15.CrossRefPubMedGoogle Scholar
  13. 13.
    Sessler DI. Temperature monitoring and perioperative thermoregulation. Anesthesiology 2008; 109: 318-38.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Sun Z, Honar H, Sessler DI, et al. Intraoperative core temperature patterns, transfusion requirement, and hospital duration in patients warmed with forced air. Anesthesiology 2015; 122: 276-85.CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Shafer SL, Dexter F, Brull SJ. Deadly heat: economics of continuous temperature monitoring during general anesthesia. Anesth Analg 2014; 119: 1235-7.CrossRefPubMedGoogle Scholar
  16. 16.
    Arkiliç CF, Akça O, Taguchi A, Sessler DI, Kurz A. Temperature monitoring and management during neuraxial anesthesia: an observational study. Anesth Analg 2000; 91: 662-6.CrossRefPubMedGoogle Scholar
  17. 17.
    Findlay GP, Goodwin AP, Protopapa KL, Smith NC, Mason M; NCEPOD. Knowing the Risk: a Review of the Peri-operative Care of Surgical Patients. London, United Kingdom; 2011.Google Scholar
  18. 18.
    Görges M, Ansermino JM, Whyte SD. A retrospective audit to examine the effectiveness of preoperative warming on hypothermia in spine deformity surgery patients. Paediatr Anaesth 2013; 23: 1054-61.CrossRefPubMedGoogle Scholar
  19. 19.
    Woodall WH, Adams BM, Benneyan JC. The use of control charts in healthcare. In: Faltin FW, Kenett RS, Ruggeri F (Eds). Statistical Methods in Healthcare. Chichester, UK: John Wiley & Sons, Ltd; 2012: 251-67.Google Scholar
  20. 20.
    Lauder G, Görges M, West N, Karlsdóttir E. Neonatal Vital Sign Deviations During General Anaesthesia: Developing Consensus On What Represents Significant Change. Proceedings of the 2015 Association of Paediatric Anaesthetists of Great Britain & Ireland Annual Scientific Meeting. Aberdeen, UK: Association of Paediatric Anaesthetists of Great Britain and Ireland; 2015: 108.Google Scholar
  21. 21.
    Görges M, West NC, Karlsdóttir E, Ansermino JM, Cassidy M, Lauder GR. Developing an objective method for analyzing vital signs changes in neonates during general anesthesia. Paediatr Anaesth 2016; 26: 1071–81.CrossRefPubMedGoogle Scholar
  22. 22.
    Bedford JE, Görges M, Whyte SD. Anonymised Individual Feedback on Management of Intraoperative Thermoregulation in Children - a NSQIP-P Quality Improvement Project. Proceedings of the 2015 ACS NSQIP National Conference. Chicago, IL: American College of Surgeons; 2015: ACSNSQIP20150192.Google Scholar
  23. 23.
    Selby JV, Schmittdiel JA, Lee J, et al. Meaningful variation in performance: what does variation in quality tell us about improving quality? Med Care 2010; 48: 133-9.CrossRefPubMedGoogle Scholar
  24. 24.
    Görges M, West N, Cheung W, Zhou G, Miyanji F, Whyte S. Preoperative warming and undesired surgical and anesthesia outcomes in pediatric spinal surgery - a retrospective cohort study. Paediatr Anaesth 2016; 26: 866-75.CrossRefPubMedGoogle Scholar
  25. 25.
    Bissonnette B. Temperature monitoring in pediatric anesthesia. Int Anesthesiol Clin 1992; 30: 63-76.PubMedGoogle Scholar

Copyright information

© Canadian Anesthesiologists' Society 2016

Authors and Affiliations

  1. 1.Department of Anesthesiology, Pharmacology & TherapeuticsThe University of British ColumbiaVancouverCanada
  2. 2.Research InstituteBC Children’s HospitalVancouverCanada

Personalised recommendations