On the utility of diagnostic instruments for pediatric delirium in critical illness: an evaluation of the Pediatric Anesthesia Emergence Delirium Scale, the Delirium Rating Scale 88, and the Delirium Rating Scale-Revised R-98
- First Online:
- Cite this article as:
- Janssen, N.J.J.F., Tan, E.Y.L., Staal, M. et al. Intensive Care Med (2011) 37: 1331. doi:10.1007/s00134-011-2244-y
Delirium is a poor-prognosis neuropsychiatric disorder. Pediatric delirium (PD) remains understudied, particularly at pediatric intensive care units (PICU). Although the Pediatric Anesthesia Emergence Delirium (PAED) scale, the Delirium Rating Scale (DRS-88), and the Delirium Rating Scale-Revised (DRS-R-98) are available, none have been validated for use in PICU settings. The aim of the present study was to investigate the use of the DRS/PAED instruments as diagnostic tools for PD in the PICU.
A prospective panel study was conducted, under circumstances of routine clinical care, investigating the diagnostic properties of the PAED, DRS-88, and DRS-R-98 in PICU patients at a tertiary university medical center. A total of 182 non-electively admitted, critically ill pediatric patients, aged 1–17 years, were included between November 2006 and February 2010. Sensitivity, specificity, and receiver operating characteristic (ROC) curves were calculated. Three psychometric properties were analyzed: (1) internal consistency (2) proportion of items not rateable, and (3) discriminative ability.
The PAED could be completed in 144 (93.5%) patients, much more frequently than either the DRS-88 (66.9%) or the DRS-R-98 (46.8%). Compared with the clinical gold standard diagnosis of delirium, the PAED had a sensitivity of 91% and a specificity of 98% (AUC 0.99). The optimal PAED cutoff score as a screening instrument in this PICU setting was 8. Cronbach’s alpha was 0.89; discriminative ability was high.
The PAED is a valid instrument for PD in critically ill children, given its reliance on routinely rateable observational signs and symptoms.
KeywordsDeliriumPediatric intensive care unitsCritical illnessInstrumentation
Delirium is a neuropsychiatric disorder secondary to a general medical condition and represents a serious complication of the underlying disease or its treatment . Three subtypes of delirium can be distinguished: (1) hypoactive delirium, which is characterized by apathy, decreased responsiveness, and withdrawal; (2) hyperactive delirium, which is characterized by restlessness, emotional lability, and agitation; (3) mixed delirium, which is a combination of the two other subtypes . The condition is frequently seen in critically ill adult and geriatric patients and is associated with poor prognosis, reflected by worse functional and cognitive outcome, prolonged hospital stay, and a higher mortality rate after discharge from hospital [1, 3–5]. Although the clinical picture of delirium in adults is well known, comparatively scarce attention has been paid to its counterpart in children—pediatric delirium (PD)—even though data suggest that PD is a similarly serious condition [2, 6–8].
It is important to correctly diagnose delirium in children, in order to make early intervention possible. Adult delirium is often missed by clinicians, because it commonly manifests as the hypoactive subtype [9–11]; anecdotal evidence suggests that hypoactive forms, and misclassification thereof, may be even more prevalent in the pediatric population. As a consequence, there is an urgent need for a valid instrument which can be used by staff at pediatric intensive care units (PICU) [2, 12] to assess PD. The Pediatric Confusion Assessment Method for the Intensive Care Unit (pCAM-ICU), the Pediatric Anesthesia Emergence Delirium (PAED) scale, the Delirium Rating Scale (DRS-88), and the Delirium Rating Scale-Revised (DRS-R-98) are instruments that have been developed to diagnose delirium in clinical practice. The pCAM-ICU has recently been validated as a highly reliable instrument in pediatric critically ill children (8–15 years) .
The PAED, which only includes items covering behavioral symptoms, has been validated in children in the age range of 18 months–6 years in a post-anesthesia setting . However, in that study, the diagnosis of delirium was only made if the anesthetist decided that treatment of the condition was necessary. Therefore, delirium could have been underdiagnosed. The DRS-88 and the DRS-R-98 focus more on cognitive functioning  and have only been validated in adults, and not in ICU settings . Although Turkel and colleagues  reported that the DRS-88 could be applied to diagnose delirium in children (6 months–19 years), their study was carried out in retrospect and no control group was available. Thus, to our knowledge, only the pCAM-ICU has been tested in PICU settings in children, whereas the DRS/PAED instruments have not been tested in a PICU setting.
Therefore, the main objective of the present prospective panel study was to investigate, under circumstances of routine clinical care, the use of the DRS/PAED instruments as diagnostic tools for PD in a PICU setting, taking into account both item and sum score levels of the instruments. It was hypothesized that (1) the PAED would be more suitable than the DRS in this setting, as it contains only behavioral items, which are easier to assess in young critically ill children and, therefore, (2) a significant negative impact of younger age was expected with regard to the proportion of rateable items for both versions of the DRS (as it relies on cognitive items that cannot be assessed in the youngest or most ill children), but not for the PAED.
Setting and patients
Data pertained to a cohort of critically ill (defined as a threatening failure of the brain, heart, or lungs) pediatric patients, admitted from November 2006 until February 2010. The setting was a tertiary eight-bed PICU at Maastricht University Medical Centre (MUMC+) in the Netherlands. All non-electively admitted patients to the PICU between the ages of 1 and 17 years, ventilated and non-ventilated, were included. The sampling frame also included patients who had been admitted to the PICU after an elective surgical procedure and who were still at the PICU after 48 h. Patients were excluded if they were (1) non-Dutch speakers and (2) not in need of intensive care. As the information necessary to rate the DRS/PAED instruments is collected as part of routine clinical care at the PICU, under Dutch law the study did not fall under the remit of the medical ethics committee, provided data anonymity is preserved at all times.
The Pediatric Anesthesia Emergence Delirium (PAED) scale
The PAED was developed by Sikich and Lerman  in order to trace emergence delirium (ED). ED is defined as a disorder occurring during awakening from anesthesia in the immediate postoperative period. It is characterized by confusion, hallucinations, delusions and can be accompanied by moaning, restlessness, and agitation. The scale consists of five items which are rated on a 5-point scale ranging from ‘extremely’ to ‘not at all.’ A score of 10 or higher indicates diagnosis of delirium . For the purpose of the present analyses, a maximum of two missing items (2/5 = 40%) was allowed a priori; a weighted sum score was computed in the case of one or two missing items.
The Delirium Rating Scale
The DRS-88 was developed by Trzepacz and colleagues . It consists of ten items encompassing a range of areas which can be affected by delirium such as hallucinations, psychomotor behavior, cognitive status, sleep-wake cycle disturbance, and lability of mood. A score of 10 or higher indicates a diagnosis of delirium [18, 19]. For the purpose of the present analyses, a maximum of four missing items (4/10 = 40%) was allowed a priori; a weighted sum score was computed in the case of four or fewer missing items.
The Delirium Rating Scale-Revised (DRS-R-98)
The DRS-R-98 is a revised form of the original DRS and consists of 16 items, of which the last three are optional and scored only when there is suspicion of delirium (score of 15.25 or higher on the first 13 items); the three optional items serve to differentiate between delirium and other disorders. For the purpose of the present analyses, a maximum of five missing items was allowed a priori if only the first 13 items were scored (5/13 = 39%) and a weighted sum score was computed in the case of five or fewer missing items. If all 16 items were scored, a maximum of 6 missing items was allowed (6/16 = 38%) and a weighted sum score was computed in cases of 6 or fewer missing items .
Clinical/expert judgment of the gold standard
The PICU pediatric neuropsychiatrist (JS) routinely examined all children using the criteria of the revised fourth edition of the DSM-IV . He reviewed and discussed all available information and opinions (from parents, nurses, and the medical charts) with the other members of the PICU team before making a final diagnosis [7, 12]. He also made further categorizations into the hyperactive, hypoactive, and mixed states of PD.
Patients at our PICU are routinely assessed for the presence of delirium under circumstances of quality improvement in clinical care since 2005, using a schedule of twice-daily examinations. The PAED, DRS-88, and DRS-R-98 are completed during the first assessment, which is carried out by the intern over the course of their attachment to the unit, having received on-site clinical training by the pediatric neuropsychiatrist of the unit. The second assessment was carried out—within 3 h—by the pediatric neuropsychiatrist, who was unaware of the screening results. PAED, DRS-88, and DRS-R-98 were completed once a day. Once PD was diagnosed, appropriate management was initiated in collaboration with the PICU team.
Data were analyzed using STATA, version 11.0 . For each patient, a single assessment day during hospitalization was selected. For patients diagnosed with PD, PAED, DRS-88, and DRS-R-98 assessments on the first day of the PD gold standard diagnosis were selected; for patients without PD, PAED, DRS-88, and DRS-R-98 assessments were randomly selected in such a way that groups were frequency-matched on number of admission days until assessment. Exclusion criteria for analysis were unconsciousness or deep sedation according to the clinical judgment of the responsible intensivist treating the patient.
A patient was regarded non-rateable on a scale if more than the maximum allowed number of items was missing, precluding diagnosis. Non-rateability of items was examined as a function of age using one-way analysis of variance (ANOVA, one-tailed).
Three psychometric properties of the PAED, DRS-88, and DRS-R-98 (with exclusion of the three diagnostic items of the last scale, because scoring of these items is optional) were evaluated: (1) internal consistency, expressed as Cronbach’s alpha; (2) number and proportion of items not rateable; (3) discriminative ability, expressed as the area under the receiver operating characteristic (ROC) curve, was calculated per instrument item and for instrument sum scores, with 95% confidence intervals (95% CI). ROC curves allow for exploration of the relationship between the sensitivity and specificity of a continuous predictor (in this case the scores for each item on a scale) and a dichotomous outcome (in this case PD) . Instruments with the largest area under the curve (AUC) are most accurate in distinguishing between patients with and without delirium.
Given the fact that PD is a serious condition requiring immediate intervention, sensitivity was given more weight than specificity in determining the optimal cutoff scores for PD. Cutoff scores for optimal sensitivity can be inferred from the curve. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the diagnosis of delirium were calculated, as well as the likelihood ratio for a positive test result (the ratio between true and false positive rate).
Population characteristics (N = 154)
Patients with delirium (N = 26)
Patients without delirium (N = 128)
Primary PICU indication
Sum score analysis
Of 154 patients, the PAED could be rated, applying the a priori criteria for rateability specified above, in 144 (93.5%), the DRS-88 in 103 (66.9%), and the DRS-R-98 in 73 (46.8%). For the DRS-88, the mean age of rateable (8.0 years) and non-rateable patients (4.3 years) differed significantly from each other (F (1,152) = 20.39; SD = 5.05; p < 0.001). This also held for the DRS-R-98 (rateable patients 9.6 years, non-rateable patients 4.3 years, F (1,152) = 57.68; SD = 5.05; p < 0.001). For the PAED, the difference between rateable and non-rateable groups was in the same direction albeit smaller and not significant (6.9 and 4.9 years respectively; F (1,152) = 1.45; SD = 5.05; p > 0.05). Owing to non-rateability, the PAED missed 3, the DRS-88 12, and the DRS-R-98 22 of the 26 diagnoses of delirium made by the gold standard.
Patients diagnosed with delirium
Pediatric neuropsychiatrist diagnosis
PAED outcome, delirium
DRS-88 outcome, delirium
DRS-R-98 outcome, delirium
Item characteristics for PAED (N = 154)
Goal directedness of movements
Awareness of surroundings
Item characteristics for DRS-88 (N = 154)
Cognitive status during formal testing
Lability of mood
Temporal onset of symptoms
Variability of symptoms
Sleep-wake cycle disturbance
Item characteristics for DRS-R-98 (N = 154)
Perceptual disturbances and hallucinations
Thought process abnormalities
Lability of affect
Sleep-wake cycle disturbance
To our knowledge, this is the first prospective study evaluating the utility of the DRS and the PAED as assessment tools for PD in a PICU setting. The PAED was rateable most often (93.5%) and proved to be excellent in discriminating between patients with and without PD (AUC = 0.99). This is in line with the findings of two recent studies which both found an AUC of 0.98 for the PAED in detecting ED in a post-anesthesia setting [22, 23]. It seems, therefore, that the PAED can be used in both settings. Both versions of the DRS could not be rated as frequently as the PAED, as these scales contain numerous items that are difficult to assess in young children, especially regarding cognition. As the largest part of the PICU population consists of children under the age of 3, this is a severe drawback . In addition, these tools require substantially more time to score. Taken together, in this study, the DRS scales were found to be far less suitable for routine PD assessment at a PICU. However, the PAED is not flawless: there were two false negatives (1 hypoactive and 1 hyperactive) that were probably due to the fluctuating nature and/or the subtle presentation of delirium and two false positives (1 hyperactive and 1 mixed). In these cases, it subsequently appeared that the presentations (emotional and/or behavioral disturbances) were better explained by other somatic complications.
Taken together, the strengths of this study are (1) it was executed during daily regular clinical care, and (2) it also included the (near) majority of the PICU children, namely those in the age range greater than 1 and less than 5 years. This broad inclusion could explain the incidence of PD (16.9%) in our study compared to the 13% found by Smith and colleagues  who only included children above the age of 5 years. However, because the pCAM-ICU is relatively new we did not have sufficient data points to evaluate its use.
Some limitations are also apparent. Firstly, data for 40 of 182 patients (22%) were assessed in retrospect, because patients were not observed during weekends or holidays. In these cases, we had to rely on collateral information from parents, nurses, intensivists, and child neurologists to reconstruct ratings. However, there were no significant differences between the results with or without retrospective data (data not shown). Second, given the fluctuating nature of delirium, it could be that some cases were missed. Thirdly, only one observer rated the PAED each day. Because the PAED contains an element of subjectivity, depending on what a given observer defines as ‘a little bit’ or ‘extreme’ in rating a given symptom, inflated random error may have been the result. More objective anchors should be developed for the PAED to overcome this problem. Unfortunately, reliability analysis was not possible given that the PICU staff over the study period was not yet participating as rater of the different instruments—this is the topic of a current study. Fourth, this is one of the first psychometric studies conducted in a PICU setting  where a higher proportion of missing data can be expected because assessments are complicated by factors such as severity of critical illness, young age, mechanical ventilation, sedation, and limited amount of time available for assessments. Because listwise deletion would lead to too much data loss we used a conservative strategy and deleted subjects for which less than 60% of the items were scored for a specific scale. Missing data were corrected using a weighted sum procedure. Fifth, the findings suggest that the incidence of delirium is much lower than in adults, especially with respect to the hypoactive subtype. The difficulties with the assessment of delirium in young and/or non-verbal patients could be an explanation for the relatively low rate of PD; in addition, the exclusive reliance of the PAED on subtle behavioral alterations may explain the relatively low rate of hypoactive delirium. Finally, replications of the current findings are necessary as it remains to be seen whether results can be generalized to other PICU settings. A sufficient level of involvement of child psychiatrists and their trainees is therefore recommended.
Given (1) the potentially dangerous nature of delirium, (2) the high workload at the PICU, (3) a shortage of pediatric neuropsychiatrists, and (4) the fact that 75% of the children at the PICU are below the age of 3 years, an instrument is needed that is easy to use at the bedside, not time-consuming, and suitable for young children as well. The PAED seems to fulfill these requirements. It is therefore concluded that the PAED should be used for PD screening at the PICU, preferably multiple times per day. Whenever a score of 8 or higher (or other scores, depending on the patient mix of the PICU in question) is obtained for a patient, a pediatric neuropsychiatrist should be consulted to assess whether the child is delirious or not. In this way, the likelihood that PD will be missed may be reduced. Future studies could be directed towards the development of an improved screening instrument for PD, based on the PAED.
The authors want to thank Machteld Telman, MSc, for revising the manuscript, Philippe Delespaul, PhD, for statistical advice, Sabeth Walpot, medical secretary, for the layout and Boris Kingma, MSc, for technical support. All authors have no financial interests to disclose.
Conflict of interest
None to declare.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.