There has been a continuous increase in the use of anesthesia information management systems (AIMS).1-4 These systems accurately and automatically capture physiological data directly from the patient monitoring devices and anesthesia work stations.5,6 The anesthesiologists must enter other clinical data manually during anesthesia; nonetheless, these data have also been shown to have a high degree of accuracy and record completeness.7,8 Recording specific anesthetic drug administration data into the AIMS is qualitatively different from the two previous categories, as mandatory fields cannot be used to enforce compliance due to the highly variable use and dosage of anesthetic drugs. Therefore, the anesthesiologist remains almost entirely in control of the accuracy of electronic drug records. It is often assumed that computerized records are highly accurate; however, an evaluation is required to determine whether this assumption applies to the records of anesthetic drug administration.

The goal of this study was to examine whether the data on drug administration that anesthesiologists entered manually into an AIMS are accurate in terms of drug name, dose administered, and time of administration, and whether the different stages of anesthesia influence data accuracy.

Methods

This was a prospective observational study comparing clinical anesthetic drug administration data entered into an AIMS (Metavision® Suite MVOR™, iMDsoft, Tel Aviv, Israel) with data on the same drug administration recorded by a trained dedicated observer (K.D.). The study included drug administration by anesthesiologists during randomly selected elective surgical procedures. Cardiothoracic and neurosurgical procedures were excluded as there was no physical space for the observer in these operating rooms. Data were collected from May to November 2010 after the AIMS had been in use for more than three years.

Drug data from the two databases (i.e., the observer and the AIMS) were compared in order to determine the accuracy of the clinical data records.

The anesthesiologist entered clinical drug administration data into the AIMS using buttons (Fig. 1) for predefined doses or for free dose entry. The independent observer was trained to identify and record drug administration by directly observing the use of syringes according to their labels, the concentrations recorded on the label, and the volume (mL) given. For each drug administered, the name of the drug, the dose, and the time of administration were recorded in real time using a database application (Microsoft Access 2003, Microsoft, Redmond, WA, USA) on a computer separate from the AIMS (“observer data”). At the beginning of each day, the time on the observer’s computer was synchronized with the time on the AIMS server.

Fig. 1
figure 1

The form for entering drug administration data into the anesthesia information management system contains buttons for various drugs, grouped using tabs. The form includes buttons for different predefined doses or for free dose entry

Full size image

At the end of the data collection period, drug administration data (drug name, dose, and administration time) as well as the start and end times of anesthesia and surgery for each patient were retrieved from the AIMS (“AIMS data”). The names of the anesthesiologists were anonymized.

The study was performed under the auspices of the Hadassah Center for Clinical Quality and Safety as a quality improvement initiative. The Institutional Review Board of the Hadassah Medical Organization approved the study and waived the requirement for informed consent from the anesthesiologists and patients (0271-14-HMO; June 2014). Due to the observational nature of this study, the anesthesiologists were not explicitly informed about the study objectives. If they asked about the goal of the study, they were informed that it was simply related to the use of the AIMS. If they had further questions, the observer was instructed to refer the anesthesiologist to the principal investigator (A.A.). The anesthesiologists were allowed to request that the observer leave the operating room, and this did occur on one occasion. After the study concluded, the study results were presented at a departmental staff meeting where the anesthesiologists were given the opportunity to request that their data be removed from the database; however, no requests were received.

Data analysis

The observer and AIMS data were matched by manually comparing the two data sets according to case, drug name, and time of administration or dose. Drug administration records that could not be matched for name were considered omissions. Among records matched by name, inaccuracies for dose and time (time discrepancy exceeding ± ten minutes) were identified. The main study outcome was the accuracy of the drug administration records in each of the three levels (i.e., drug name, dose, and time of administration).

The effects of several cofactors on the accuracy of drug administration records were measured as secondary outcomes. These included the pharmacological group, anesthesia phase, and data entry delays. For the pharmacological group, drugs were combined into different groups in order to assess whether inaccuracies clustered according to drug type. Record inaccuracy was compared between the three anesthetic phases: (a) induction: from the start of anesthesia care until the start of surgery, (b) maintenance: from start of surgery until 15 min before the end of surgery, and (c) emergence: from 15 min before the end of surgery until the patient was transported to the postanesthesia care unit. For delays in data entry, each drug administration was associated with three time points, including the actual time of drug administration (observer time), the time marked as time of drug administration in the AIMS (anesthesiologist’s time), and the actual time that the data were entered into the AIMS (time stamp). For example, the anesthesiologists injected propofol at 09:00 (observer time), entered the data into the AIMS at 10:00 (time stamp) by positioning the AIMS time curser at 09:10 (anesthesiologist time). The time-lags between observer time and anesthesiologist time were compared for accurate vs inaccurate entry of drug administration data.

Statistical analysis

Continuous data were compared using two-tailed Student’s t test. Proportions were compared using the Chi square test. Statistical calculations were performed with SAS® Version 9.2 (SAS Institute, Cary, NC, USA). A P value of < 0.05 was defined as statistically significant.

Results

Data were collected on 596 separate drug administration occurrences by 22 anesthesiologists (including three consultant-level anesthesiologists and 19 resident trainees) during 57 surgeries. The mean (SD) number of drug administration records per case was 10.5 (6.6) (Fig. 2). One anesthesiologist requested not to be observed. The 596 drug administration records included the following drug categories: 159 (26.7%) narcotics, 110 (18.5%) hypnotics, 79 (13.3%) cardiovascular drugs, 77 (12.9%) muscle relaxants, 57 (9.6%) antibiotics, 42 (7.0%) local anesthetics, 35 (5.9%) analgesics, 21 (3.5%) antiemetics, and 16 (2.7%) neuromuscular reversal drugs. Drug administration was recorded during the induction phase on 331 (55.5%) occasions, during the maintenance phase on 208 (34.9%) occasions, and during emergence on 57 (9.6%) occasions (Table). The mean observation time per case was 121.3 (66.8) min.

Fig. 2
figure 2

Summary of study results

Full size image
Table Rate of omissions (drug administration observed but not reported in the anesthesia information management system) during the different anesthesia phases
Full size table

Anesthesia information management system and observer concordance

In the AIMS and observer databases, 495/596 (83.1%) drug administration records matched for drug name, with a mean per case of 8.8 (4.7) records. Among the remaining 101 unmatched drug records, 90 (15.1% of all drug records) were noted by the observer but not entered into AIMS by the anesthesiologist (i.e. “omissions”), which represents a median [interquartile range] of 2 [1-4] records made during 32/57 (56.1%) of all surgeries. The median number of omissions per anesthesiologist was 3.5 [1.5-6.5]. Only 5/22 (22.7%) anesthesiologists had no omission inaccuracies, representing 114/585 (19.5%) of observed drug administrations (median, 10 [7.5-44.5]).

There were no drugs charted incorrectly by name. There were no corresponding data in the observer database for 11 AIMS records made during nine surgeries (1.8% of all drug records).

The largest group of drug records omitted from the AIMS was cardiovascular drugs, representing 34/90 (37.8%) of all omitted drugs and 34/76 (44.7%) of all observed cardiovascular drug administrations. The next largest group of drug records omitted was hypnotics, representing 24/90 (26.7%) of all omitted drugs and 24/107 (22.4%) of all observed hypnotic administrations. The 34 omitted cardiovascular drugs included 18 ephedrine (52.9%) and 16 phenylephrine (47.1%) omissions. These represented 45.3% of all observed cardiovascular drug administrations. There were no differences between the mean doses of phenylephrine omitted and those entered into AIMS [141.9 (86.0) µg vs 146.0 (53.1) µg, respectively; P = 0.844], while the doses of ephedrine omitted from the AIMS were larger than those entered into the AIMS [9.7 (2.7) mg vs 6.7 (2.9) mg, respectively; P = 0.02]. The mean doses of the hypnotics omitted from the AIMS were smaller than those entered into AIMS [propofol 33.9 (25.3) mg vs 135 (72) mg, respectively, P < 0.001; and etomidate 3.3 (2.3) mg vs 15 (1.4) mg, respectively; P < 0.001]. Administration of narcotics, antibiotics, and analgesics was rarely omitted from the AIMS (Table).

Dose accuracies

Among the 495 AIMS and observer drug administration records matched for drug name, doses matched on 458 (92.5%) occasions. Among the remaining 37 records, the dose entered into AIMS was larger than the observed dose on 15/37 (40.5%) occasions, while the converse was found on 22/37 (59.5%) occasions. Dose inaccuracies were most frequent among muscle relaxants (10/76, 13.2%).

Timing accuracies

Among the 495 AIMS and observer records matched for drug name, the time of drug administration as entered into AIMS (anesthesiologist time) matched the observed drug administration time (observer time) within ± ten minutes in 476 (96.2%) records. The mean absolute time difference between the observer and anesthesiologist records was 2.7 (3.6) min. On 15/19 (78.9%) occasions, the administration time entered into AIMS (anesthesiologist time) was later than the actual administration time (observer time), while on 4/19 (21.1%) occasions, the anesthesiologist time preceded the observer time.

Dose and time accuracies

For 439 (88.7%) of the 495 AIMS and observer records matched for drug name, both time of drug administration and dose were accurate. There were no AIMS records with both dose and time inaccuracies.

Anesthesia phases

Drug record omissions occurred principally during maintenance: 50/90 (55.6%), and less frequently during induction: 30/90 (33.3%) and emergence: 10/90 (11.1%). During the maintenance phase, 50/203 (24.6%) drug records were omitted from the AIMS, while 30/325 (9.2%) records were omitted during induction and 10/57 (17.5%) records were omitted during the emergence phase (P < 0.001).

Dose and time inaccuracies occurred mainly during induction. Dose inaccuracies accounted for 27/325 (8.3%) records during induction compared with 10/203 (4.9%) records during maintenance and 0/57 (0%) records during emergence (P = 0.07). Time inaccuracies accounted for 15/325 (4.6%) records during induction compared with 3/203 (1.5%) records during maintenance and 1/57 (1.7%) record during emergence (P = 0.11).

Delayed data entry

The mean time lag between actual drug administration (observer time) and AIMS data entry (time stamp) was 19.1 (17.1) min during induction, 3.3 (7.5) min during maintenance, and 3.4 (3.7) min during emergence (induction vs maintenance P < 0.001; induction vs emergence P < 0.001; maintenance vs emergence P = 0.9). The time lag was shorter for drug administration with accurate records than for those with inaccurate dose records [11.9 (15.1) min vs 22.6 (21.9) min, respectively; P < 0.001] and those with inaccurate time records [12.3 (15.9) min vs 23.4 (13.9) min, respectively; P = 0.003).

Discussion

The accuracy of drug administration records entered manually into an AIMS ranges from 96.2% for the correct time, to 92.5% for the correct dose, to 84.6% for an existing electronic record.

Drug omission inaccuracies occurred, principally with cardiovascular drugs as well as with small doses of hypnotics during the maintenance phase. Dose and time inaccuracies occurred during induction and were associated with delayed data entry. Manually entered data are subject to potential error with most anesthesiologists making at least one charting error of some type. Conceivably, this can have medical, management, legal, and research implications.

The majority of omissions were related to vasoactive drugs or to small doses of hypnotic agents. These omissions were not found during induction but occurred mainly during the anesthesia maintenance phase. The occurrence of these omissions during maintenance may suggest that they were not related to the anesthesiologist’s activity level but rather to the drug doses being perceived as non-significant or, alternatively, as “self-incriminating”. During induction, some degree of hemodynamic instability is accepted, but during maintenance, hemodynamic instability might be thought to reflect poor anesthetic technique. In contrast, dose and time inaccuracies occurred principally for drugs administered during induction and were associated with a charting delay, possibly reflecting the limitation of retrospective data recording from periods where anesthesia activity is higher. It is unlikely that the inaccuracies resulted from difficulty in the use of the computerized system since satisfaction with the specific AIMS system in use is typically very high.7 Furthermore, the mean time difference between the computerized record and the observer data was small [2.7 (3.6) min], suggesting that data were entered relatively rapidly. Lastly, omissions were most frequent during maintenance when anesthesiologist activity is usually lowest and time to enter computerized data should not have been a limitation.

Accurate charting in anesthesia consists of three main elements: physiological data, drug data, and procedural data. Each has importance for the medical record per se and for postoperative patient care. For example, difficult intubation must be documented to assist in the safety of future anesthesias. Records of intraoperative drug and fluid administration are important for postoperative care.

Manual paper anesthesia records suffer from inaccuracies in recording vital signs,5 vague timing of drug administration, and illegibility. In addition, manual charting increases workload,9 and paper charts are easily lost. Studies formally validating the accuracy of drug administration data on paper charts are lacking. While studies comparing computerized systems and paper charts have been performed,10,11 these studies were not direct comparisons of the accuracy of the two charting techniques, but rather, they examined whether computerized monitoring of drug delivery (e.g., bar code readers) improved chart accuracy. These studies required written consent, which meant that the anesthesiologists knew they were being observed. In addition, they described accuracy only by reconciling drugs missing from the drug tray at the end of the case with those recorded. Furthermore, the direct comparison between computer and paper chart records11 examined accuracy in terms of many non-drug-related fields (which have been shown to be highly accurate as a result of the use of context-sensitive mandatory fields)7 and compared only total number of drugs administered. In contrast, our study was performed without the direct knowledge of the anesthesiologist and examined the accuracy of every drug administered at three levels: name, time, and dose.

It is important to establish the accuracy of the clinical computerized data as due to their computer-based recording and presentation, as they may be falsely perceived as perfect. As stated, the use of context-sensitive mandatory data fields in AIMS can ensure accurate manual entry of procedural and clinical data (with demonstrated rates of accuracy and chart completion approaching 100%).7 Nevertheless, this technique cannot be applied to drug administration as it varies substantially between patients, anesthesiologists and anesthesia techniques, making automatic program-based quality control almost impossible. Thus, accurate drug administration charting seems to be largely dependent on the anesthesiologist’s motivation. Defining acceptable charting accuracy represents a value judgment. While 100% accuracy should be the objective; realistically, this is unlikely to be achieved. The 15% of omitted drug records reported here might be considered as excessive, but we lack a basis for comparison with either other studies or paper charts.

Limitations

Despite the fact that the anesthesiologists were not explicitly informed about the objectives of the study, a Hawthorn effect12 leading to increased accuracy cannot be excluded. In cases where residents are supervised by attendings, administration of drugs is usually performed by the attending during induction and entered into AIMS retrospectively by the resident. This may explain charting inaccuracies during the induction phase. The sample size of 57 cases with 596 drug records was limited by logistical constraints and the necessity of limiting a Hawthorn effect. The data presented relate to the specific case types included, i.e., a wide range of elective surgical cases excluding neurosurgery and cardiac surgery. It is possible that the incidence of charting errors may be different in lengthy and/or complex cases.

Conclusions

The range of accuracy varies when anesthesiologists manually enter drug administration data into an AIMS. Charting omissions represent the largest cause of inaccuracy, caused principally by record omissions for vasopressors and small doses of hypnotic drugs. Accurate charting is important, and while computerized anesthesia data may be perceived to have a high degree of accuracy, drug administration records are not perfect, and anesthesiologists remain responsible for their accuracy.