Introduction

The successful transition of care from the operating room to the post-anesthesia care unit (PACU) requires comprehensive communication between anesthesia, nursing, and surgical teams in the form of verbal hand offs as well as orders placed by the anesthesia team for necessary PACU therapies. Computerized provider order entry (CPOE) systems are frequently used for electronic placement of such orders and provide an opportunity for implementation of automated electronic reminders to encourage timely order placement. Since delays in PACU order placement can lead to delays in patient care and resulting adverse events or costs to the healthcare system, this type of reminder holds potential for improving patient outcomes.

Numerous studies have demonstrated the positive implications of CPOE in different clinical practice environments, showing that effective implementation can result in reduced errors, direct savings to the healthcare system, improved patient safety and outcomes, and decreased response and turnaround times [1,2,3,4,5,6,7,8,9,10,11,12]. However, limited data are available to study the implications of CPOE systems in the PACU setting, and specifically how reminder systems might provide value in supporting compliance with timely PACU order placement.

To mitigate delays in PACU order placement, a mobile electronic health record (EHR) reminder system was implemented at our institution in 2021. This study's aim was to estimate the association between the PACU mobile reminder system and the timeliness of PACU order placement and the degree of delay in care. We hypothesized that implementation of this reminder system would result in a greater proportion of patients having orders placed prior to arrival in PACU and more rapid order entry for those who did not have orders placed prior to arrival. Secondarily, we also studied the association of late order entry with various PACU outcomes.

Methods

Study Design/Sample

We performed a retrospective observational analysis, including subjects with an anesthetic care record from our institution’s main operating rooms who received a qualifying postprocedure order from January 1, 2019, to May 31, 2023.It is possible that anesthesia providers anticipated ICU transfers for sicker patients with higher case urgency, and thus the normal clinical flow for these cases was altered. For this reason, we excluded subjects admitted directly from the operating room (OR) to the intensive care unit (ICU). We also excluded those whose primary anesthesia attending provider participated in the pilot testing of the reminder system, and those with missing covariate data. This study was approved with a waiver for written, informed consent from the Human Subjects Research Protections Program at Vanderbilt University Medical Center and adheres to the guidelines provided in the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) statement [13].

Variables

Mobile reminder system usage was determined based on the EHR audit log of 'OR Closing’ push notifications sent to providers opted into the system and was defined for each case as “positive” (i.e., a reminder was sent for this case) if the anesthesia provider received a relevant reminder on the same day as the surgical case. The alerts were implemented using Epic System’s Haiku application (Verona, WI) and were designed to deliver a push notification to the attending anesthesiologist’s smartphone or tablet if PACU orders were not placed by the time that surgical closure began. Providers could opt out of push notifications by adjusting their application settings, or by choosing to not install the Haiku application on their personal smartphone or tablet. The study's main outcomes were (1) the incidence of patients arriving in the PACU without orders and (2) the degree of delay in order placement for those patients without orders present on arrival. To analyze the latter, we created a two-part survival variable to define (1) did patient have their order placed within 90 min of PACU start (Yes/No) and (2) minutes from PACU start to order placement. Patients who did not have their order placed within the first 90 min of their stay in PACU were considered negative for PACU order placement (i.e., lost to follow-up). This follow-up period was selected based on the distribution of time from PACU start to order placement for cases with late orders. Secondary outcomes were defined as (1) total PACU length of stay (LOS) in minutes, (2) ≥ 2 administrations of PACU antiemetics (promethazine, ondansetron, or haloperidol), and (3) peak PACU pain score.

Data Collection Methods

Data on the relevant exposure, outcomes, and covariates were electronically extracted from Clarity, a relational database populated nightly from the EHR. Covariates included patient demographics (patient age, sex, self-reported race, height, and weight), procedural variables (American Society of Anesthesiologists’ [ASA] physical status classification, case level, and anesthesia duration), and three separate provider-based variables indicating if (1) the primary attending anesthesiologist ever changed, (2) the in-room provider ever changed, or (3) an in-room provider eligible to place orders was ever present during the case.

Data Analyses

Demographic and procedural variables were summarized with median and interquartile range for continuous variables and with counts and percentages for categorical variables. Univariate groupwise comparisons were performed using the Chi-Squared test for categorical variables and Kruskal-Wallis Rank Sum test for continuous variables.

We first tested the univariate association between the PACU mobile reminder system and the incidence of patients arriving in the PACU with orders. We then estimated this association, adjusting for patient, procedural, and provider-based covariates, using a logistic regression. Next, we filtered out patients who did not have their order placed prior to PACU start and used a Cox Proportional Hazards survival model to analyze the association between the reminder system and degree of additional delay in order placement. Odds ratios (ORs) with 95% confidence intervals (CIs) quantified the effect of PACU mobile reminder system usage on the odds of arrival in PACU with orders already placed. Hazard ratios (HR) with 95% CI were reported for the association between the mobile reminder and the hazard, or the instantaneous rate of order placement. For numeric covariates, odds/hazard ratios were calculated by taking the difference in odds/hazard between the 25th and 75th percentile of the variable. Survival curves were plotted to visualize the difference in survival rates across patients who did and did not utilize the reminder system. ANOVA was utilized to test the significance of model terms. The significance level for model terms was set at 5%. Optimism-corrected concordance statistics were reported as metrics of discriminative performance of each respective model (i.e., ability of model to accurately distinguish between patients with and without the outcome of interest). Calibration plot assessed the agreement between predicted and observed outcomes, corrected for optimism (i.e., adjustment for overfitting). Restricted cubic splines were utilized to relax the linearity assumption of continuous covariates. All analyses were performed with R (Version 4.1.2; R Core Team; Vienna, Austria) and libraries used include tidyverse, knitr, Hmisc, rms, tableone, RODBC, DBI, odbc, pROC, survival, survminer, ggpubr, car, and plotly.

Post-hoc Secondary Analysis

As a post-hoc secondary analysis, we assessed the association between order placement prior to PACU admission and secondary PACU outcomes (PACU duration, administration of antiemetics, and peak pain score). Due to its distribution, we performed a linear regression of a logarithmic transformation of the total PACU length of stay variable. Logistic regression was utilized to study the second administration of antiemetics, and a proportional odds logistic regression was used to study the ordered peak PACU pain score outcome.

Results

Of the 149,607 records that met our inclusion criteria, 11,257 cases went straight from the OR to the ICU, 10,841 cases had a primary attending who was involved in the initial pilot testing of the reminder system, and 662 cases were excluded for missing data (Fig. 1). There were thus 126,847 cases that were eligible for analysis. In 12.5% (n = 15,919) of cases, the primary attending provider received a push notification reminder on the day of surgery (Table 1). Overall, 88 (44.9%) attending providers received at least one ‘OR Closing’ push notification. Use of the mobile reminder system was associated with a reduced incidence of missing PACU orders on admission (12.6% vs. 17.5%; p < 0.001) and orders that were placed sooner on average (p < 0.001) (Table 2). The rate of order placement prior to PACU start was 83.2% (n = 105,485). Compared to those whose orders were late, cases with timely order placement had on average higher rates of mobile reminder system usage by the attending provider (13.2% vs. 9.4%; p < 0.001) (Table 3).

Fig. 1
figure 1

Inclusion-exclusion flow chart

Full size image
Table 1 Sample descriptive statistics
Full size table
Table 2 Sample descriptive statistics, stratified by exposure
Full size table
Table 3 Sample descriptive statistics, stratified by primary outcome
Full size table

Association Between PACU Mobile Reminder Usage and the Incidence of Timely PACU Order Placement

Order reminder system usage had a univariate association with increased rate of order placement prior to PACU admission (p < 0.001). Adjusting for patient, procedural, and provider-based variables, we found that PACU mobile reminder system usage was associated with 51% increased odds (OR: 1.51; 95% CI: 1.43, 1.58; p < 0.001) of timely PACU order placement (Table 4). Overall, there was no evidence of model overfit, but it displayed poor discriminative ability (optimism-corrected concordance statistic = 0.58).

Table 4 Primary adjusted analysis of association between incidence of arrival in PACU with order already placed and PACU mobile reminder system usage
Full size table

Association Between PACU Mobile Reminder Usage and Degree of Delay in Care

The records of 21,362 patients whose orders were not placed prior to PACU admission were analyzed to assess their degree of delay in order placement. There were 2,001 (9.4%) cases where the primary attending received a push notification on the day of surgery. Within this set of patients, usage of the PACU mobile reminder system was associated with 10% more rapid placement of a patient’s orders (HR: 1.10; 95% CI 1.05, 1.15; p < 0.001) (Table 5). Figure 2 shows that the delivery of a push notification reminder to the primary attending provider on the day of surgery was associated with reduced survival probability and, thus, were more likely to have order placement sooner than cases in which the primary attending did not receive a notification. The survival model displayed low corrective optimism adjustment for model performance (i.e., no evidence of overfit model) but displayed poor discriminative ability (optimism-corrected c-statistic = 0.54).

Table 5 Primary adjusted analysis of degree of delay in care in patients whose order was placed after PACU admission
Full size table
Fig. 2
figure 2

Survival curve

Full size image

Post-Hoc Sensitivity Analysis

Order placement prior to PACU admission was associated with 3% reduced PACU duration (p =  < 0.001), 8% decreased odds of multiple administrations of antiemetics (p = 0.02), and 28% reduced odds of having a higher pain score (p < 0.001) (Table 6).

Table 6 Secondary analyses
Full size table

Discussion

In a large retrospective sample of patients admitted to the PACU at a single academic medical center, we demonstrated an association between a mobile electronic PACU order reminder system and more timely order entry. Use of the system was associated with a greater percentage of patients arriving to the PACU with orders entered and, in those without preexisting orders, was associated with more rapid order entry during the PACU stay. In a post-hoc sensitivity analysis, order placement prior to PACU admission was associated with shorter PACU duration and decreased odds of both higher PACU pain scores and the incidence of multiple administrations of antiemetics.

Interestingly, while the reminder system was associated with increased odds of timely order placement, and on-time order placement was associated with shorter PACU durations, we found that patients whose attending utilized the system had longer median PACU durations than those who didn’t. However, this discrepancy is likely related to an interaction with one or more confounding variables. For example, we found that, other than on-time order placement, PACU length of stay was also associated with sex, race, height, weight, ASA class, case level, anesthesia duration, and provider-specific variables.

In the logistic regression analysis, the following provider-level factors were additionally associated with the placement of PACU orders prior to PACU admission: attending change during the case, in-room provider change during the case, and in-room provider with ordering privileges present during the case. Interestingly, these factors had variable effects on the outcome. A change of attending or the presence of an in-room ordering provider were both associated with greater odds of order placement, possibly because the oncoming ordering provider used the handoff as a cue to place the orders at that time. Conversely, a change in the in-room provider was associated with decreased odds of order placement. Each of these findings may be due to the distribution of responsibility for order placement amongst a group of providers, each of whom may have assumed orders were placed by another team member.

To account for other factors which may have affected the likelihood of timely PACU order placement, we additionally adjusted for anesthesia duration, case urgency, and patient ASA physical status. As expected, sicker patients and higher case urgency were associated with a decreased odds of order placement prior to PACU admission, while longer anesthesia time was associated with increased odds.

Over recent decades, the evidence supporting CPOE and electronic alerts and reminders in varying healthcare environments has grown [5, 6]. In the operating rooms, alerts embedded in anesthesia information management systems (AIMS) have been shown to improve compliance with antibiotic administration, treatment of hyperglycemia [5], and more. The transition to PACU, however, offers an additional opportunity where electronic alerts may provide operational or patient benefit. The added challenge of alerting a provider who is not actively working within the AIMS was overcome, in this case, via mobile device push notifications. Our study demonstrates a proof-of-concept that such technology can successfully reach clinicians and impact their behavior, with the potential to translate this method to other areas of healthcare.

It is also important to note that frequent push notifications can contribute to provider fatigue, leading to increased likelihood of being ignored. We attempted to mitigate these issues by allowing clinicians to add personalized user preferences and opt out of specific types of reminders. While data on changes to user preferences over time are not available, anecdotally our attending anesthesiologists have found this functionality useful.

This study has significant strengths including the inclusion of a large, diverse surgical population at a tertiary medical center operating room. The large sample size allowed increased power to detect significant differences between cases in which an order reminder system was or was not used for PACU order placement. Furthermore, the robust data collection procedures and analytic plan are easily replicated, leading to increased trustworthiness of results.

However, there were also several limitations. First, due to its single-center, retrospective design, the results of this study may not be generalizable to other patient populations. Also, we defined reminder system use as positive if the attending anesthesiologist received at least one push notification on the day of the surgery. We were unable to directly link reminders to individual surgical cases, and so our classification of cases likely included some false positives (cases in which the attending received an alert on the same day as, but not directly linked to, a specific case). Nor were we able to confirm acknowledgement of the reminder by the clinician. As a suggestion for future quality improvement initiatives, a more precise definition of system usage and reminder receipt could eliminate such bias. Similarly, the removal of cases with missing data is another potential source of bias. However, we feel that due to the relatively small number of cases with missing covariate data (< 1%), any bias that may be introduced is negligible.

In conclusion, in this single-center retrospective study, we found that an electronic reminder system was associated with timelier PACU order placement, which in turn may lead to improved patient outcomes.