Background

Computerized Physician Order Entry (CPOE) is the process of entering medication orders and other physician's instructions electronically using a computer-based system to ensure standardized, legible and complete orders [1]. Its implementation has been recommended to improve patient safety and outcomes [2, 3] primarily by reducing medication errors [4] that usually arise from faulty prescriptions [5]. However, CPOE itself can lead to new types of errors [6–8], which may be detrimental to patients. This may be more pronounced in critically ill patients, a more vulnerable population due to the severity of their illness, the complexity of interventions done and the diversity of equipments used in the intensive care unit (ICU). The evidence for the CPOE effectiveness in improving the outcomes of critically ill patients is scarce and remains controversial. Two before-after studies in pediatric ICUs showed no significant change in mortality after CPOE implementation [9, 10]. Del Beccaro et al observed a statistically insignificant decrease in mortality from 4.2% to 3.46% in the 13-month pre- and post-CPOE implementation [9]. Keene et al found no change in mortality post-CPOE after adjusting for all covariates (OR, 0.71; 95% CI, 0.32-1.57)[10]. On the other hand, Han and colleagues observed an increase in mortality from 2.8% to 6.6% during the 5-month period after the initiation of a CPOE system at a pediatric hospital with a tertiary care ICU [11]. Literature lacks information on the impact of CPOE implementation on the outcomes of adult critically ill patients. The objective of our study, conducted as a quality improvement project, was to evaluate the outcomes of critically ill patients after CPOE implementation in an adult ICU.

Methods

Setting

The study was conducted at a 21-bed medical surgical closed ICU in a 900-bed teaching tertiary care hospital, King Abdulaziz Medical City-Riyadh, in Saudi Arabia. It was staffed by board certified intensivists 24 hours per days, 7 days a week as described in a study from our ICU elsewhere [12] with rotating residents from different specialties. Nursing staff were multinational with patient-to-nurse ratio of 1:1. The hospital was accredited by Joint Commission International.

Study Design

The study was conducted as an observational before-after study. Data were extracted from a prospectively collected electronic ICU database. We included all consecutive patients admitted to the adult ICU (age ≥ 12 years) in the 24 months pre- and 12 months post-CPOE implementation. We excluded patients with brain death admitted as potential organ donors. For patients who had multiple admissions to the ICU within the same hospitalization, only the first admission was considered in the analysis. The institutional review board of King Abdulaziz Medical City approved the study and due to its observational nature, informed consent was waived.

CPOE Implementation

Before CPOE implementation, physicians wrote medication orders on paper; nurses then faxed the orders to the ICU satellite pharmacy; the pharmacist would screen the order, make necessary adjustments after consulting with the prescribing physician if needed, transcribe the order and dispense the medication. The adult ICU was the first area in the hospital to implement CPOE. Other hospital areas continued to use the traditional paper-based order writing during the study period. The used system was a product from Misys®, which was later acquired by QuadraMed, and allowed medication and nonmedication orders as well as access to the patient's chart, laboratory and radiological data. It included a basic real-time clinical decision support system so that important interactions of prescribed medications, dose checking and allergy notification appeared at the time of prescription as pop-ups. In the year preceding CPOE implementation, a multidisciplinary team prepared electronic order sets and protocols. The team used the existing paper-based order sets including ICU admission orders, traumatic brain injury protocol and antimicrobial therapy order set and converted them with some modifications into CPOE order sets. Some of these order sets, such as the traumatic brain injury protocol, implemented since 2001, had been studied in our ICU and been shown to improve outcomes [13]. This was followed by training of ICU physicians, pharmacists and nurses on the use of CPOE functions. Following the "Go Live" on December 4, 2006, physicians directly entered all medications and most nonmedication orders, such as specific nursing instructions, requisition of laboratory tests and radiological examinations and consultations of physiotherapy and nutrition services, from any computer terminal inside or outside the unit. Pop-up alerts appeared on the screen and required verification before order acceptance. Pharmacists screened electronic prescriptions and advised physicians in case of errors. Nurses checked their bedside computer terminals for new and scheduled orders. Some orders, such as diet and blood transfusions, continued on papers. Technical support and guidance were provided 24 hours a day, 7 days a week for 2 weeks from the hospital's information technology specialists, then as needed. To address the observed difficulties and limitations and enhance the CPOE system, orders and order sets were revised upon agreement of physicians, nurses, clinical pharmacists and medical informatics specialists. Such revisions included transforming complex intravenous orders that required more than 3 steps into single-step orders, optimizing doses and frequency of intravenous antibiotics and simplifying radiological and laboratory orders.

Measurements and outcomes

The following data were extracted from the ICU database: age, gender, Acute Physiology and Chronic Health Evaluation (APACHE) II score [14], source of admission (ward, emergency department, operating room and other hospital), main reason for ICU admission (as per APACHE II definitions), cardiac arrest and sepsis as a reason for admission, the presence of chronic health illness (as defined by APACHE II system), admission Glasgow Coma Scale, admission creatinine and International Normalized Ratio (INR), requirement for vasopressors within the first 24 hours after admission and requirement for mechanical ventilation during ICU stay. The primary outcome was ICU mortality. Secondary outcomes were: duration of mechanical ventilation, ICU and hospital length of stay and hospital mortality. Comparisons were made for different time periods: one, three and 24 months before implementation compared with one, three and 12 months post-implementation. The reason for selecting these periods was to evaluate CPOE effect in the immediate and more extended period of implementation and also to control possible seasonal variation of different outcomes. In addition, we evaluated the impact of CPOE in selected groups of patients, whose outcome was thought to be particularly affected by CPOE. These included: age 12-17 years since Han et al reported an increased mortality in the pediatric patients post-CPOE implementation [11], diagnosis of sepsis because outcomes of these patients can be affected by timely interventions such as timely fluid and antimicrobial therapy [15] and traumatic brain injury as such patients may benefit from protocolized management. Additionally, because patients with higher severity of illness were thought to be more vulnerable, we evaluated the effect of CPOE in patients with high APACHE II score,≥ the median score which was 23.

Statistical analysis

SAS software (version 9.0; SAS Institute, Cary, NC) was used to analyze data. Continuous variables were reported as means with standard deviation and categorical variables as absolute and relative frequencies. The Chi-squared test was used to assess the differences between categorical variables and the student t-test was used for the analysis of differences in the means of continuous variables. To examine the association between CPOE implementation and various outcomes, we used stepwise multivariate logistic regression analysis. Independent variables entered in the model were important factors that clinically affect outcomes and baselines characteristics that were significantly different in the 24 month pre- and 12 months post-CPOE in addition to CPOE implementation status. The following variables were entered in the model: age, gender, APACHE II, source of admission, non-operative trauma and post-operative trauma as main reason for admission, cardiac arrest as a reason for admission, chronic health illness (liver, cardiovascular and renal), creatinine, INR, vasopressors use, and mechanical ventilation. Results were reported as adjusted odds ratios (aOR) for categorical outcomes (ICU and hospital mortality) and beta (β) coefficient for continuous outcomes (duration of mechanical ventilation, length of stay in the ICU and the hospital) with 95% confidence intervals (CI). This analysis was performed for the whole cohort and for the selected subgroups of patients. At α = 0.05, our study had an 80% power to detect a 4.8% decrease or a 5.2% increase in mortality after CPOE implementation using the observed mortality (23.4%) in pre-CPOE implementation period and the number of patients in the pre-CPOE (n = 1638) and post-CPOE (n = 898) periods. A P-value < 0.05 was considered statistically significant.

Results

General characteristics of all patients

In the 24 months pre-CPOE implementation (December 4, 2004 to December 3, 2006), 1638 patients were admitted to the ICU (13104 patient-days) compared to 898 patients (7274 patient-days) in the 12 months post-CPOE implementation (December 4, 2006 to December 3, 2007). Table 1 describes the characteristics of patients admitted to the ICU in the following periods: 24 vs. 12 months, 12 vs. 12 months, 3 vs. 3 months and 1 vs. 1 month pre- and post-CPOE implementation. There were no significant differences in age, admission APACHE II score, admission Glasgow Coma Scale, creatinine and INR in these pre- and post-implementation periods. Compared to the 24 months pre-CPOE implementation, the percentage of female patients and postoperative trauma admissions were higher in the following 12 months (39% vs. 35%, p = 0.04). Also there were significant imbalances between the two groups in these variables: source of admission before ICU, chronic health illnesses and cardiac arrest as a reason for ICU admission. In addition, vasopressors (50% vs. 36%, p < 0.0001) and mechanical ventilation (77% vs. 73%, p = 0.004) were used more commonly in the 24 months pre-CPOE compared to 12 month post-CPOE implementation.

Table 1 Comparison of baseline characteristics of admissions to the intensive care unit pre- and post-computerized physician order entry (CPOE) implementation in four different periods: 24 months pre- vs. 12 months post-CPOE, 12 months pre- vs. 12 months post-CPOE, three months pre- vs. three months post-CPOE vs. one month pre- and one month post-CPOE.
Full size table

Outcomes of patients pre- and post-CPOE implementation

Table 2 compares the outcomes of all patients in the cohort in relationship to CPOE implementation. The implementation of CPOE was not associated with a statistically significant change in ICU mortality when comparing the 24 months pre-CPOE to the 12 months post-CPOE (aOR 0.98, 95% CI 0.7-1.3), the 12 months pre-CPOE to the 12 months post-CPOE (aOR 0.96, 95% CI 0.7-1.3), the 3 months pre-CPOE to the 3 months post-CPOE (aOR 0.83, 95% CI 0.5-1.5), the 1 month pre-CPOE to the 1 month post-CPOE implementation (aOR 1.17, 95% CI 0.4-3.2). Similarly, CPOE implementation was not associated with significant differences in hospital mortality, duration of mechanical ventilation and of length of stay in the ICU and hospital in any of the four periods.

Table 2 Outcomes of patients in different pre- and post-computerized physician order entry (CPOE) periods with adjusted risk calculated using multivariate logistic regression analysis.
Full size table

Outcomes of patient subgroups according to CPOE status

Table 3 describes the outcomes of four subgroups of patients in the 24 months pre- and 12 months post-CPOE implementation. There were no significant differences in ICU mortality of patients 12 to 17 years old, those with sepsis as an admission diagnosis, those with traumatic brain injury and those with APACHE II score ≥ 23.0. Also there were no significant differences in hospital mortality, the duration of mechanical ventilation and of stay in the ICU and hospital within the same periods pre- and post-CPOE in the 4 subgroups.

Table 3 Outcomes of four patient subgroups in the 24 months pre- and 12 months post-CPOE implementation with adjusted risk calculated using multivariate logistic regression analysis.
Full size table

Discussion

Our study demonstrates that CPOE implementation was not associated with any significant improvement in mortality or any change in other outcomes within 12 months of implementation in critically ill patients older than 12 years.

CPOE has been advocated as a tool to enhance patient care and safety and improve resource utilization by ensuring standardized, legible, and complete orders thus eliminating prescription errors, providing advisories orders for drugs that can potentially harm patients and offering physicians evidence-based clinical decision support. Several hospital wide studies showed benefits attributed to CPOE [16–18]. A systematic review that evaluated the effect of CPOE on patient safety found that 23 of 25 selected studies showed 13-99% relative risk reduction in medication errors, 6 of 9 selected studies showed 35-98% relative risk reduction in potential adverse drug events and 4 of 7 selected studies showed 30-84% relative risk reduction in adverse drug events [4]. Other studies showed that the impact of CPOE implementation might extend to mortality benefit. A cross-sectional study of urban hospitals in Texas found that automated order entry was associated with significant reduction in the adjusted odds of death for myocardial infarction and coronary artery bypass graft procedures [19]. Longhurst and colleagues found a decrease in the mean monthly adjusted mortality rate by 20% (95% CI 0.8%-40%, p = 0.03) at a tertiary care children hospital post-CPOE implementation [20]. Moreover, other studies showed a CPOE-associated improvement in hospital resource utilization, resulting in decreased hospital lengths of stay and costs [21, 22].

On the other hand, other hospital-wide reports raised concerns about the effectiveness of CPOE in averting medication errors and promoting patient safety. In fact, some studies demonstrated that CPOE can induce errors. Koppel et al identified 22 situations in which the CPOE system facilitated medication errors [7]. These situations fell into two categories: information errors generated by fragmentation of data and flaws in human-machine interface [7]. The causes of CPOE pitfalls include poor quality of CPOE system, inadequate technical support, healthcare provider resistance to change, insufficient preparation and insufficient user training [23].

Data on CPOE implementation in ICU setting are scarce. On one hand, ICUs environment is well controlled and monitored and has higher staffing level than other areas of the hospital. On the other hand, critically ill patients are a vulnerable population and the healthcare providers working in the ICU have many responsibilities and frequently perform multiple and complex tasks and so are prone to errors. One study looked at the frequency of distraction or interruptions that occur during computerized order entry in one medical ICU and found that one distraction or interruption occurred approximately every 5 minutes and preceded 2 order entry errors [24]. Reports on the impact of CPOE on the outcome of ICU patients have revealed conflicting results. Some studies showed benefit or no harm from CPOE use in the ICU. A prospective trial that compared paper-based order entry in 14 ICU beds (80 patient-days) to CPOE implemented in 8 ICU beds (80 patient-days) found that CPOE was associated with significant lower incidence of medication prescription errors and adverse drug events [25]. Another study showed that CPOE in the ICU was associated with significant reduction in the time between ordering and reporting stat tests [26]. Development of evidence-based decision algorithm for red cell transfusion facilitated by CPOE led to significant reduction in inappropriate transfusions and decrease in blood transfusion from 1.08 ± 2.3 units 0.86 ± 2.3 units (p < 0.001)[27]. In a before-after study at a 20-bed tertiary pediatric ICU, Del Beccaro, et al observed a statistically insignificant decrease in mortality from 4.2% to 3.46% in the 13-month pre- and post-CPOE implementation respectively [9]. Keene et al also found no increased mortality post-CPOE initiation even post-adjusting for all covariates (OR, 0.71; 95% CI, 0.32-1.57) [9, 10]. On the other hand, the report of Han et al [11] in 2005 was alarming. The authors reviewed the mortality data at a pediatric hospital in the 13 months before (n = 1394, 56.7% were admitted to ICU) and 5 months after (n = 548, 56.9% were admitted to ICU) implementation of a different CPOE system (Millenium Powerchart software system, Cerner Corporation, Kansas City, MO) and found a significant increase in mortality post-CPOE implementation from 2.8% to 6.6% (OR: 3.71; 95% CI: 2.13-6.46)[11]. Our study demonstrated no change in the mortality of critically ill patients > 12 years of age post-CPOE implementation suggesting safety of its implementation in our setting.

The observed differences in the outcomes among studies are probably related to inherent methodological differences, patient population as well as differences in the CPOE systems and their implementation process. Additionally, the observed impact of CPOE is influenced by the comparison group or period; as such no benefit will be observed if the CPOE is compared to a very well organized and robust paper-based order system, while the same CPOE will reveal major improvement if implemented where the paper-based system was not well established. This may explain the lack of benefit in our ICU, since the implementation converted pre-existing well established paper-based order sets into CPOE order sets. In particular, our evidence-based traumatic brain injury order set has been shown to improve outcome; hence the lack of benefit of CPOE in this population. Factors related to CPOE system and implementation also greatly influence its impact. CPOE systems vary considerably in their ease to use, ability to customize and the presence and level of clinical decision support. The success of implementation depends on multiple factors including organizational readiness for change, engagement of end users, the level of technical support and the project management. Hence, commitment of hospital management and healthcare providers to CPOE is crucial [28, 29]. Sittig et al's commentary [30] on the study by Han et al [11] illustrates many of the above points. They thought that the increased mortality was related to multiple factors that accompanied CPOE implementation and significantly affected hospital workflow. These factors included quick hospital-wide implementation of the system over a 6-day period, pharmacy centralization for all medications including vasoactive agents which may have affected the care of ICU patients, delay in patient registration causing delays in management of seriously ill patients, increased physician workload leading to less focus on patient stabilization and management at presentation, wireless networking which led to delays in order entry during peak operational periods, order entering taking long time (1-2 minutes for a single order), decreased collaborative patient care due to the diminution of face-to face communication and interaction between nurses and physicians, ICU nurses spending significant amount of time at a computer terminal away from the bedside, and not using order sets [30]. Harrison and colleagues evaluated the unintended consequences of health information technology in general including CPOE and demonstrated other important factors such as the elimination of informal interactions and redundant checks that help detect errors, spending more time on order entry rather patient care, overdependence on decision support system for appropriate prescribing leading to management difficulties when CPOE was unavailable or down [31]. Additionally, Karsh et al described eleven fallacies of health information technology which undermine its efficacy and suggested that collaboration between human factors engineers, applied psychologists, medical sociologists, communication scientists, cognitive scientists and interaction designers were needed to overcome these challenges [32]. In our ICU, CPOE implementation was well planned and closely monitored. Nevertheless, we faced multiple challenges which included occasional server slowness, sporadic resistance to CPOE use, alert fatigue largely due to redundant low risk pop-ups and complex orders. That ICU was the only hospital area that implemented CPOE in the study period was another challenge that necessitated additional training efforts for residents who periodically rotated in the ICU. All of these factors may have contributed to our findings of no difference in outcomes, but subsequently led to the formation of the ICU CPOE committee to enhance CPOE and optimize orders and ordering process. From our experience, these CPOE challenges decreased with time, which suggests that CPOE effectiveness in improving outcomes may require time longer than one year to be observed. It is also important to note that some of the challenges observed in other studies were institution-specific. For example, the observed delay in the registration of patients transferred from other hospitals in Han's et al study [11] was not an issue in our hospital as the registration process did not change post-CPOE implementation. Moreover, a more refined clinical decision support system that would advise on medication dose and duration or on optimal laboratory testing rates, that would utilize patient data and provide guideline-based care recommendations, or that would aid in weaning mechanical ventilation [33] might result in different outcomes.

This study must be interpreted in light of its strengths and limitations. The study used a prospectively collected data and was the first to address the relationship between CPOE and patient outcomes in adult ICU setting. It evaluated a commercial CPOE system which may strengthen the generalizability of our findings. Moreover, we described the CPOE experience at a Saudi tertiary care ICU staffed by multi-national healthcare workers with different backgrounds in information technology. Additionally, the study was done in a period during which there was no change in the ICU structure and function. Among the limitations was the potential time bias considering the pre-post design. However, we adjusted for all potential confounders in our multivariate analysis. Because of the study design, we were not able to examine intermediate endpoints such as technical performance, adverse drug events, adherence to guidelines and staff satisfaction. We believe that these endpoints should be targets for future research.

Conclusions

In conclusion, our study found that CPOE implementation in an adult medical surgical ICU was associated with no improvement in ICU and hospital mortality in the immediate period and up to 12 months after implementation. CPOE can be an important tool to improve health care, but its effectiveness in improving the outcomes of critically ill patients is unproven and may require ICU-specific reengineering and customization to achieve the goal of a demonstrable mortality reduction.

Key messages

  • Implementation of Computerized physician order entry (CPOE) was found to be associated with no significant changes in morbidity and mortality of critically ill patients in the immediate period and up to 12 months afterwards.

  • These findings were also seen in four high risk groups of patients: age 12-17 years, traumatic brain injury, diagnosis of sepsis on admission and admission APACHE II ≥ 23.

  • CPOE effectiveness in the ICU may require ICU-specific reengineering and customization to achieve the goal of mortality reduction.