Canadian Journal of Anesthesia/Journal canadien d'anesthésie

, Volume 57, Issue 10, pp 898–902

Hospital administrative database underestimates delirium rate after cardiac surgery

  • Rita Katznelson
  • George Djaiani
  • Gordon Tait
  • Marcin Wasowicz
  • Ainsley M. Sutherland
  • Rima Styra
  • Corina Lee
  • W. Scott Beattie
Reports of Original Investigations

DOI: 10.1007/s12630-010-9355-8

Cite this article as:
Katznelson, R., Djaiani, G., Tait, G. et al. Can J Anesth/J Can Anesth (2010) 57: 898. doi:10.1007/s12630-010-9355-8

Abstract

Purpose

Administrative electronic databases are highly specific for postoperative complications, but they lack sensitivity. The objective of this study was to determine the incidence of delirium after cardiac surgery using a targeted prospectively collected dataset and to compare the findings with the incidence of delirium in the same cohort of patients identified in a hospital administrative database.

Methods

Following Research Ethics Board approval, we compared delirium rates in a prospectively collected data research database with delirium rates in the same cohort of patients in an administrative hospital database where delirium was identified from codes entered by coding and abstracting staff. Every 12 hr postoperatively, delirium was assessed with a Confusion Assessment Method in the Intensive Care Unit. The administrative database contained the International Classification of Diseases version 10 (ICD-10) codes for patient diagnoses. The ICD-10 codes were extracted from the administrative database for each patient in the research database and were checked for the presence of the ICD-10 code for delirium.

Results

Data from a cohort of 1,528 patients were analyzed. Postoperative delirium was identified in 182 (11.9%) patients (95% confidence interval [CI], 10.3-13.5%) in the research dataset and 46 (3%) patients (95% CI, 2.2-3.8%) in the administrative dataset (P < 0.001). Thirteen (0.85%) patients who were coded for delirium in the administrative database were not identified in the research dataset. The median onset of postoperative delirium in these patients was significantly delayed (4 [3-9] days) compared with patients identified by both datasets (2 [1-9] days) and compared with patients from the research database only (1 [1-14] days) (P = 0.007).

Conclusion

Postoperative delirium rates after cardiac surgery are underestimated by the hospital administrative database.

La base de données administrative hospitalière sous-estime l’incidence de delirium après une chirurgie cardiaque

Résumé

Objectif

Les bases de données électroniques administratives sont très spécifiques quant aux complications postopératoires, mais elles manquent de sensibilité. L’objectif de cette étude était de déterminer l’incidence de delirium après chirurgie cardiaque à l’aide d’un ensemble de données ciblées et colligées de façon prospective ainsi que de comparer ces résultats avec l’incidence de delirium dans la même cohorte de patients identifiés dans une base de données administrative hospitalière.

Méthode

Après avoir obtenu le consentement du Comité d’éthique de la recherche, nous avons comparé les taux de delirium dans une base de données de recherche colligée de façon prospective aux taux de delirium de la même cohorte de patients dans la base de données administrative de l’hôpital, dans laquelle le delirium a été identifié à partir de codes saisis par le personnel de saisie et d’analyse des données. Le delirium a été évalué à l’aide d’une méthode d’évaluation de la confusion dans l’unité des soins intensifs toutes les 12 heures en postopératoire. La base de données administrative comprenait les codes de la Classification internationale des maladies, 10e révision (CIM-10) pour les diagnostics des patients. Les codes de la CIM-10 ont été récupérés de la base de données administrative pour chaque patient de la base de données de recherche et nous avons vérifié si le code de la CIM-10 pour le delirium était inclus.

Résultats

Les données d’une cohorte de 1528 patients ont été analysées. Le delirium postopératoire a été identifié chez 182 (11,9 %) patients (intervalle de confiance [IC] 95 %, 10,3-13,5 %) dans l’ensemble de données de recherche et 46 (3 %) patients (IC 95 %, 2,2-3,8 %) dans l’ensemble de données administratives (P < 0,001). Treize (0,85 %) patients codés pour le delirium dans la base de données administrative n’ont pas été identifiés dans l’ensemble de données de recherche. Chez ces patients, le delirium postopératoire apparaissait significativement plus tard (médiane : 4 [3-9] jours) que chez les patients identifiés dans les deux ensembles de données (2 [1-9] jours) et que chez les patients inclus seulement dans la base de données de recherche (1 [1-14] jours) (P = 0,007).

Conclusion

La base de données administrative de l’hôpital sous-estime l’incidence de delirium postopératoire après une chirurgie cardiaque.

Delirium is a psychiatric disorder presenting as an acute deterioration of brain function leading to a fluctuating level of consciousness and an inability to maintain attention.1 It is a common complication after cardiac surgery with a prevalence ranging from 3-47% of patients.2-5 This large variation in the reported incidence of postoperative delirium depends on the study design, the patient population studied, and the sensitivity and specificity of the methods used for the clinical diagnosis of delirium.

Postoperative delirium presents a unique and complex clinical condition. The diagnosis of delirium is based solely on the clinical assessment of patients and is often missed during routine clinical practice. This situation may potentially lead to lack of proper documentation and a false representation of the true incidence of postoperative delirium. It has previously been reported that retrospective medical record review has been inaccurate in establishing the diagnosis of delirium in medical, cardiac, and orthopedic patients.6-9 Importantly, medical record reviews are the basis for data extraction and coding for administrative hospital databases. Although the administrative electronic databases are highly specific for postoperative complications, they are known to be relatively insensitive;10,11 however, the magnitude of this discrepancy has not been clearly addressed.

The objective of this study was to determine the incidence of postoperative delirium in a cardiac surgical patient population using a targeted prospectively collected database and to compare this result with the incidence identified in the hospital administrative database. We hypothesized that administrative databases significantly underestimate the real delirium rates when compared with prospectively collected data based on validated screening tools.

Methods

Following Research Ethics Board approval, we compared delirium rates in the same patient cohort identified in two different databases: 1) prospectively collected data in a research database, and 2) retrospectively entered data in an administrative hospital database using ICD-10 coding for the diagnosis.

Assessment of delirium

In the research database, delirium was assessed every 12 hr postoperatively during the patients stay in the intensive care unit using the Confusion Assessment Method in the Intensive Care Unit (CAM-ICU). The CAM-ICU allows monitoring for symptoms of delirium in patients with ventilated lungs and extubated trachea.12 It is based on the Diagnostic Statistical Manual of Psychiatric Disorders criteria and includes a four-step algorithm assessing 1) an acute onset of changes or fluctuations in mental status, 2) inattention, 3) disorganized thinking, and 4) an altered level of consciousness. The patient is determined to be delirious (CAM positive) if he/she manifests both features 1 and 2 and either feature 3 or 4.

The nursing staff performed the CAM-ICU measurements, and nurse-educators trained all nurses in the use of the CAM-ICU method in patients with ventilated or non-ventilated lungs. This training consisted of an introductory lecture and a series of in-service training sessions. The CAM-ICU was not performed after patients were discharged from the ICU.

The research database was created to gather information about postoperative delirium in the cardiac surgical patient population of a large academic hospital. The data entered included the patients’ demographic characteristics, type of surgery, the number of CAM-ICU assessments performed, and the onset and duration of delirium. One of the study investigators collected the CAM-ICU data sheets every week and entered results into the research database.

The administrative database contained the International Classification of Diseases version 10 (ICD-10) codes for patient diagnoses. These codes were entered by professional coders and abstractors who review information in the patient chart following discharge. The information reviewed includes discharge summaries, consultation reports, clinical notes, and interventional reports. The diagnostic code for delirium was based on documentation by a physician. For each patient in the research database, the ICD-10 codes were extracted from the administrative database and checked for the presence of the code for delirium.

We evaluated the charts of the patients who were defined as having delirium in the administrative hospital database but did not have diagnosis of delirium in the research database.

Statistical analysis

The delirium rates were compared between the two groups with the Chi square test for differences in probabilities in a 2 x 2 contingency table. Confidence intervals (CIs) for proportions were calculated at 95%. A probability value < 0.05 was considered significant. Inter-observer agreement between the two delirium scoring methods was determined using Cohen’s kappa. The sensitivity and specificity of detecting delirium cases retrospectively using the administrative database were calculated using the prospective data collection as the gold standard for case finding. Statistical analysis was conducted with the use of SPSS computer software.

Results

Data from a cohort of 1,528 patients were analyzed. The demographic data, surgical characteristics, and postoperative outcomes of this cohort of patients are shown in the Table 1.
Table 1

Demographic data and surgical characteristics

 

Number of patients (n = 1,528)

Preoperative

Age, (yr)

63 ± 13

Male sex

1096 (71)

Peripheral vascular disease

224 (15)

Congestive heart failure

233 (15)

Diabetes Mellitus

256 (17)

Intraoperative

Urgent surgery

491 (32)

Combined CABG/valve surgery

417 (27)

CABG with CPB

793 (52)

CABG without CPB

77 (5)

Heart transplantation

9 (0.6)

Aortic surgery

16 (1)

Other cardiac procedures with CPB

215 (14)

Postoperative

Renal failure requiring dialysis

79 (5)

Myocardial Infarction

17 (1)

Stroke

15 (1)

Length of hospital stay (days)

7.9 [1, 183]

30-day mortality

19 (1)

Data presented as mean ± SD, number of patients (%), or median [range]. CABG = coronary artery bypass graft surgery; CPB = cardiopulmonary bypass

Postoperative delirium was identified in 182 (11.9%) patients (95% CI, 10.3-13.5%) in the prospectively studied research cohort, while only 46 (3%) patients (95% CI, 2.2-3.8%) were coded for diagnosis of delirium in the administrative dataset (P < 0.001). The total number of CAM-ICU assessments performed is reflected in Fig. 1.
https://static-content.springer.com/image/art%3A10.1007%2Fs12630-010-9355-8/MediaObjects/12630_2010_9355_Fig1_HTML.gif
Fig. 1

Number of Confusion Assessment Method in the Intensive Care Unit (CAM-ICU) assessments

A total of 33 (2.2%) patients with delirium were identified in both datasets. The onset and duration of delirium in these patients were similar (Fig. 2). Interestingly, there were 13 (0.85%) patients that were coded for delirium in the administrative database but not identified in the research dataset. The median onset of delirium in these patients was significantly delayed (4 [3-9] days) when compared with patients identified in both datasets (Fig. 3).
https://static-content.springer.com/image/art%3A10.1007%2Fs12630-010-9355-8/MediaObjects/12630_2010_9355_Fig2_HTML.gif
Fig. 2

The onset and duration of delirium in the research and administrative databases. Columns 1 and 2 represent the onset of delirium; columns 3 and 4 represent the duration of delirium in both the research and administrative databases

https://static-content.springer.com/image/art%3A10.1007%2Fs12630-010-9355-8/MediaObjects/12630_2010_9355_Fig3_HTML.gif
Fig. 3

The onset of delirium in the research and administrative databases. Shown are the onset of delirium in patients identified by both the research and administrative databases; the research database only; and the administrative database only; P = 0.007

The inter-observer agreement between the two methods was κ = 0.25. The sensitivity of the retrospective administrative database review to detect delirium was 18.1% (13.2-24.5%). The specificity of the retrospective administrative database review to detect delirium was 99% (98.4-99.5%). Positive predictive value of the administrative data was 71.7% (56.3-83.5%). Negative predictive value was 90.0% (88.3-91.4%).

In the research database, the diagnosis of delirium was made in 105 (58%) patients on the first postoperative day, in 62 (34%) patients from postoperative day two to six, and in the remaining 15 (8%) patients on the seventh postoperative day or later. The duration of delirium in these patients was one day in 111 (61%) patients, two days in 29 (16%) patients, three days in 18 (10%) patients, four days in seven (4%) patients, five days in two (1%) patients, six days in nine (5%) patients, and more than seven days in six (3%) patients.

Discussion

In this observational study we demonstrated that the incidence of postoperative delirium documented in the hospital administrative database was significantly lower when compared with the same cohort of patients in the prospectively collected research database.

The current study confirmed the previous concerns about the reliability of using administrative databases as a source for reporting delirium rates in different patient populations.6-9,13 Using an administrative database as the only source of identification of postoperative delirium could considerably underestimate the real incidence rates14 and lead to conclusions that might conflict with the studies based on validated screening tests for postoperative delirium.15,16

It is likely that the discrepancy in delirium rates between our two databases is based on the practice and methodology of establishing the diagnosis of delirium. It could be a reflection of inadequate data abstraction, poor communication among the care team, and/or incomplete charting by physicians.

Delirium is a clinical diagnosis that cannot be confirmed by laboratory results or neuro-imaging tests. It is based on the clinical evaluation of a patient aimed at identifying the essential clinical features of delirium. The clinical picture of delirium varies and includes hyperactive, hypoactive, and mixed types with symptoms that can differ significantly. Delirium has a fluctuating course; therefore, it can be missed easily during routine clinical examination. Furthermore, the routine clinical practice in the early postoperative period after cardiac surgery includes the administration of sedatives and narcotic analgesics that may mask symptoms of delirium. The fluctuating level of consciousness, the inability to maintain attention, and disorganized thinking (the very symptoms that are used to define delirium) could be interpreted as the side effects of general anesthesia, sedation, and postoperative analgesia. Consequently, the diagnosis of delirium might not be documented clearly in the clinical and consultation notes, and the diagnosis would be omitted in the discharge summaries that are used for diagnostic coding in the administrative database. Even when the diagnosis of delirium is identified in the consultation reports and clinical notes, it would commonly lack the indicators of onset or course of delirium.

Timing of the onset of delirium with respect to the postoperative course is an important factor in understanding the mechanisms of postoperative delirium. Risk factors that are associated with delirium in the immediate postoperative period are likely different from the risk factors triggering delirium days or weeks after surgery. This information is particularly important in evaluating potentially modifiable risk factors of delirium and developing preventive treatment strategies to reduce postoperative delirium. In the current study, the prospectively collected research dataset contained the delirium rates that were established using the CAM-ICU assessments performed every 12 hr after surgery. The CAM-ICU is a validated screening instrument for delirium based on the Diagnostic Statistical Manual of Psychiatric Disorders with high specificity and sensitivity and low inter-rater variability.12 In our research database, the incidence of delirium after cardiac surgery was similar to previous reports in comparable patient populations.5,17

The current study showed that the onset and duration of delirium were similar in patients who were identified by both datasets. However, a small number of patients who were identified only by the administrative database had delayed onset of delirium with a median of four days after surgery. This finding is consistent with the current clinical practice of following the postoperative course of patients undergoing cardiac surgery. The duration of ICU stay after low-risk cardiac surgery rarely exceeds a 24-hr period. Patients are transferred to the surgical floors until discharge from the hospital. The CAM-ICU assessments were limited to the postoperative period in ICU and were not administered on surgical floor. This was the main limitation of the research database in determining the variability in the postoperative course of delirium. Even though the administrative database identified only 13 (0.85%) patients developing late (> four days) delirium, it demonstrated the importance of following patients beyond the immediate postoperative period.

In conclusion, this report found that delirium rates after cardiac surgery were 11.9% in patients identified by the prospective research compared with 3% in the same cohort of patients identified by the administrative hospital databases. This finding reinforced concerns of using an administrative database as a reliable source for identifying delirium rates after cardiac surgery. Caution should be exercised in interpreting results reported from administrative databases, particularly if different treatment and management strategies are employed in an attempt to reduce postoperative delirium. Recording the results of delirium screening tests as separate entities and facilitating advancements in the electronic dataset collection would improve detection and reporting of delirium in the administrative databases. Furthermore, documenting the onset time and duration of delirium would further improve the credibility of administrative databases and make them more consistent sources of information.

Competing interests

None declared.

Copyright information

© Canadian Anesthesiologists' Society 2010

Authors and Affiliations

  • Rita Katznelson
    • 1
  • George Djaiani
    • 2
  • Gordon Tait
    • 2
  • Marcin Wasowicz
    • 2
  • Ainsley M. Sutherland
    • 2
  • Rima Styra
    • 3
  • Corina Lee
    • 2
  • W. Scott Beattie
    • 2
  1. 1.Department of AnesthesiologyToronto General HospitalTorontoCanada
  2. 2.Department of AnesthesiologyToronto General HospitalTorontoCanada
  3. 3.Department of PsychiatryToronto General HospitalTorontoCanada