Introduction

Many critically ill children admitted to a specialized intensive care unit (ICU) receive benzodiazepines and/or opioids to reduce pain and anxiety. Long-term exposure to these medications may result in physical dependency. These patients are at risk for withdrawal symptoms after abrupt discontinuation or too rapid tapering-off of these medications [1].

Signs and symptoms of withdrawal can be categorized into three main groups: overstimulation of the central nervous system (CNS), gastrointestinal dysfunction, and autonomic dysregulation [1–3].

A reliable, validated and clinical useful assessment tool is indispensable for monitoring withdrawal syndrome. Two such tools are available, the Sedation Withdrawal Score (SWS) and the Opioid Benzodiazepine Withdrawal Scale (OBWS) [4, 5]. Yet each has its limitations. The SWS includes 12 symptoms. While all these symptoms seem clinically relevant for assessing withdrawal, data on validity and reliability are lacking. The OBWS included 16 symptoms and signs of withdrawal and was evaluated in a small sample of PICU children (n = 15) comparing OBWS scores and nurses’ clinical judgment. Sensitivity of the OBWS at scores 8 (cut-off) or higher was 50% and the specificity was 87%, which implies moderate validity of the scale [5]. Symptoms of the OBWS were indirectly adapted from the Neonatal Abstinence Score [6].

Recently, Franck and colleagues tested the OBWS in 83 pediatric ICU patients and provided a preliminary validity and reliability evaluation of an 11-item reduced version which they named the Withdrawal Assessment Tool-1 (WAT-1) [7]. WAT-1 observation includes four steps. The first is retrieval of information on loose stools, vomiting and temperature of the previous 12 h. The second is a 2-min pre-stimulus observation on state, tremor, sweating, uncoordinated movement and yawning/sneezing. The third is a 1-min stimulus observation on startle to touch and muscle tone. The fourth is maximally a 5-min post-stimulus observation to determine time to gain calm state. In all, the WAT-1 includes four CNS symptoms, two autonomic dysregulation symptoms, two symptoms of gastrointestinal dysfunction and three items derived from the State Behavioral Scale [8]. WAT-1 scores of 3 or higher (on a scale of 0–12) revealed the best sensitivity and specificity (87 and 88%, respectively) in relation to NRS-withdrawal ratings.

Within our line of research we studied the (co-)occurrences of withdrawal symptoms in critically ill children with the ultimate objective of identifying which signs and symptoms are essential for developing a valid and reliable assessment tool.

Materials and methods

Methods

Several steps of scale development were needed to construct the Sophia Observation withdrawal Symptoms-scale (SOS). The institutional review board approved the study; because of the strictly observational and non-invasive nature of this study, they waived the need for informed consent.

The first step was a review of the literature on withdrawal symptoms in critically ill children admitted to a PICU based on an extensive evaluation of the literature as recently published by our group [3]. Based on this review, we selected 24 signs and symptoms for inclusion in a preliminary scale, which we called the Sophia Benzodiazepine and Opioid Withdrawal Checklist (SBOWC). Twelve signs and symptoms concerned the central nervous system, four the gastrointestinal tract, and eight the autonomic nervous system (see Table 1).

Table 1 Percentages of occurrences of withdrawal symptoms for the weaning group (WEAN) and the unsuccessful weaning group (WEAN−)
Full size table

The second step was a prospective observational study in critically ill children admitted to the ICU of the Erasmus MC—Sophia Children’s Hospital between September 2005 and February 2006. Eligible for inclusion were children aged ≤16 years who had received midazolam (benzodiazepine), morphine or fentanyl (opioids) by continuous infusion for at least 5 days.

The SBOWC was assessed within 24 h after tapering-off or cessation of midazolam and/or opioids (morphine or fentanyl). All nurses had received verbal and written instruction on the application of the SBOWC. An instruction manual explaining the signs and symptoms to be observed was available at each patient’s bedside. Nurses’ interobserver reliability (Cohen’s kappa) for the individual items of the SBOWC ranged from 0.59 to 1.0. The intraclass correlation coefficient was 0.85 (95% CI 0.69–0.94). The face validity was supported by the judgement of ten experienced pediatric intensive care physicians. Furthermore, the construct validity of the SBOWC was promising as shown by a Spearman rank correlation coefficient (r s) of 0.51 between total doses of midazolam and maximum SBOWC sum score in 76 children (95% CI 0.32–0.66, p < 0.001). In addition the r s between total doses of opioids and the maximum SBOWC sum score was 0.39 (95% CI 0.17–0.57, p < 0.01, n = 71). The correlation between duration of medication and maximum SBOWC sum score was 0.52 (95% CI 0.34–0.67, p < 0.001, n = 76) [9].

The third step was twofold. In step 3a the underlying structure was identified by multidimensional scaling. Step 3b was to obtain expert opinion on the relevance of each of the SBOWC symptoms. To this end we mailed a questionnaire to all pediatric intensive care physicians and clinical fellows in pediatric intensive care of all eight PICUs in the Netherlands and to critical care nurses of our own PICU. In the first main question they were asked to rate the relevance of each withdrawal symptom in critically ill children using a four-point scale ranging from ‘definitively yes’ to ‘definitively no’. In the second question, they were asked to state which signs and symptoms were definitely necessary for an assessment tool. Furthermore, they were asked to provide demographic aspects: discipline, age and years of work experience.

Step 4 selected the relevant items; signs and symptoms which had insufficient discriminative value were eliminated.

Study population

Children aged ≤16 years admitted to the ICU of our level three children’s hospital between September 2005 and February 2006 were eligible for this study if they received midazolam (a benzodiazepine), morphine or fentanyl (opioids) by continuous infusion for at least 5 days. Exclusion criteria were: status epilepticus treated with midazolam, use of neuromuscular blocking agents (NMBA), and severely disturbed behavior pattern on account of underlying neurological disease.

The observations were stratified into two groups. First, a ‘weaning group’ (WEAN), 932 observations in 76 children obtained within 24 h after decrease and/or discontinuation of midazolam and/or opioids. Those children were thought to be at risk for developing withdrawal symptoms. Second, an ‘unsuccessful weaning’ group (WEAN−), 93 observations in 27 children obtained before increasing midazolam and/or opioids during the weaning process in order to counteract possible withdrawal related signs and symptoms. We expected that the WEAN− group would show more withdrawal related signs and symptoms as compared to the WEAN group.

Statistical analyses

As the frequency distribution of the symptoms of SBOWC was highly skewed, the response categories were dichotomized. Of clinical interest is determination of the interrelationship of the signs and symptoms. Conventionally, a correlation technique is used as measure of association. In this study it would not be of interest to estimate associations between symptoms, however, but rather to estimate the co-occurrences of these symptoms. To that end, Jaccard’s measure of similarity tailored to estimate these co-occurrences was applied.

Expert opinions

Each symptom of the checklist was recoded. A symptom was coded relevant if the expert scored the first question as ‘that’s definitely so’ or ‘that’s true’, or the second question with ‘definitely necessary’. A symptom was deemed relevant if 50% or more of the respondents agreed on its relevance.

Multidimensional scaling

The goal of multidimensional scaling (MDS) analysis technique in this study was to detect meaningful dimensions of observed similarities and dissimilarities (distances) between the withdrawal symptoms. The (dis)similarities between the withdrawal symptoms identified by MDS were based on the co-occurrences of these signs and symptoms. Then, MDS attempted to arrange ‘objects’ (withdrawal symptoms) in a space of a given number of dimensions that does not entail substantial loss of information. Nevertheless, in general, the more dimensions we used in order to reproduce the matrix of dissimilarities, the better the fit of the reproduced matrix to the observed matrix. We considered the Jaccard measure as a similarity measure of co-occurrences. The performances of the models were represented by the normalized raw stress, a measure of model adequacy. This coefficient varies from 0 to 1 and should be <0.05 as a good fit. The Tucker’s φ coefficient of congruence is a measure of correspondence between the distances of the empirical data and the distance derived from the model. Ideally, this coefficient should be >0.95. Identification of the clinical-empirical structure was by means of the computer algorithm PROXSCAL (Proximity Scaling).

The symptoms hypertension and high pitched crying were excluded from analysis. First, it was not always possible to measure arterial blood pressure, e.g., in the absence of an indwelling arterial catheter, in 386/932 (41%) observations. Second, high pitched crying could be observed only in the absence of an endotracheal tube, which was not always the case (629/932, 67%). Furthermore, as the SBOWC asked for observation of either feeding retention or poor feeding, these items were taken together in the MDS procedure.

Final item selection was based on the experts’ opinions and the results of MDS. Inclusion in the SOS required signs and symptoms to have a substantial score (z score) ≥0.30 on at least one dimension of the MDS solution. As a further requirement, a symptom was deemed relevant if 50% or more of the respondents agreed on its relevance.

The robustness of the MDS solution was tested with tenfold cross-validation and presented by the mean (SD) value.

Interobserver reliability was tested for dichotomous items by Cohen’s kappa and the intraclass correlation coefficient (ICC) for continuous data [10]. A Cohen’s kappa below 0.65 was considered unsatisfactory [11]. Statistical analyses were performed with the Statistical Package for Social Sciences (SPSS), version 14.0.

Results

Characteristics study sample

Seventy-nine children (57% boys) met the inclusion criteria. Their median age was 3.4 months (range 0 days to 15.5 years). All 79 received midazolam at a median dose of 176 mcg/(kg h). Seventy-three children (92%) also received opioids at a median dose of 14 mcg/(kg h). Midazolam was administered for a median of 10 days (3–108 days); opioids were administered for a median of 8 days (1–41 days). The median (range) total dose of midazolam was 33 mg/kg (2–595 mg/kg), and that of opioids 4 mg/kg (0–682 mg/kg).

For 76 children, 932 observations were obtained within 24 h after decreasing and/or discontinuation of midazolam and/or opioids (WEAN group). Three patients were discharged before midazolam and/or opioids were decreased or discontinued. Ninety-four ‘unsuccessful weaning’ observations were obtained for 27 children in the ‘unsuccessful weaning’ group (WEAN−). The latter observations were obtained before midazolam and/or opioids were increased during the weaning process in order to counteract possible withdrawal related signs and symptoms.

Interobserver reliability

Twenty-three observations were scored simultaneously by the attending nurse and the principal investigator (EI). The intraclass correlation coefficient was 0.97 (95% CI 0.92–0.98). The interobserver reliability (Cohen’s kappa) for the individual items of the SOS ranged from 0.73 to 1.0.

Bivariate analysis of co-occurrences of withdrawal symptoms

Table 1 presents occurrences of signs and symptoms for the two conditions, i.e., observations performed within 24 h after weaning (WEAN, n = 932) of opioids/benzodiazepine and observations performed before decreasing medication (WEAN−, n = 94). In addition, Table 2 shows the co-occurrences of signs and symptoms for the WEAN group (lower triangle) and WEAN− group (upper triangle).

Table 2 Co-occurrences of withdrawal symptoms (%) in the weaning (WEAN) and unsuccessful weaning (WEAN−) groups
Full size table

For the WEAN group, the symptoms most frequently observed were: tachypnea (29.6%), agitation (21.1%), diarrhea (17.8%), fever (17.6%) anxiety (14.8%), sleep pattern (14.6%) and hypertension (14.6%). For the WEAN− group, agitation (46.2%), motor disturbance (37.6%), tachypnea (31.2%), increased muscle tension (28.0%) and feeding retention (25.5%) were observed most frequently. The signs and symptoms tremors, hallucination, seizure, pupil dilatation, sneezing and yawning demonstrated very low prevalences (from 0.3 to 1.9).

With regard to co-occurrences (Table 2) the highest percentage was seen for agitation co-occurring with motor disturbance, both in the WEAN (9.5%) and WEAN− (25.8%) groups. Groups substantially differed in frequencies of the following co-occurrences: agitation/anxiety, agitation/muscle tone, agitation/motor disturbance, agitation/grimacing, agitation/sleeping, agitation/sweating, muscle tone/motor disturbance. Co-occurrence rates with signs and symptoms such as seizures, pupil dilatation, hallucinations, yawning and sneezing were very low.

Experts’ opinions

In total, 85 experts—22 physicians and 63 nurses—completed the questionnaire. Most of them (84.7%) were female. The median work experience was 8 years for physicians (IQR 4–11.3) and nurses (IQR 4.5–15) alike.

The following symptoms were considered most relevant: agitation, anxiety, inconsolable crying, increased muscle tension, tremors, tachycardia and sweating (84.7–95.3%) (see Table 3). For five signs and symptoms fewer than 50% of the experts agreed on relevance: seizures, feeding retention, yawning, sneezing and mottling.

Table 3 Construction of the SOS based on two methods (MDS and expert opinion)
Full size table

Multidimensional analysis of co-occurrences

The PROXSCAL-procedure with random starts was chosen as the initial configuration. Twenty-one signs and symptoms were entered and the solution of the MDS procedure turned out to be three-dimensional. The normalized raw stress was 0.0498, indicating a good fit. The Tucker’s φ coefficient of congruence equaled 0.97. Decompositions of normalized Raw Stress ranged from 0.02 to 0.06. The tenfold cross-validation of the MDS solution identified a mean normalized raw stress of 0.0501 (SD = 0.002) and a mean Tucker’s φ coefficient of congruence of 0.97 (SD = 0.0009). The conclusion is justified that the solution identified is robust.

The signs and symptoms seizures, pupil dilatation, sneezing and yawning had a z score below 0.30 on either of the three dimensions (see Table 4).

Table 4 Multidimensional scaling, dimensional quantifications (z scores)
Full size table

Finally, based on the MDS results as well as the experts’ opinions, 15 signs and symptoms (Table 3) were included in the SOS (see also Supplementary materials).

Discussion

To our knowledge, this is the first study to identify the underlying empirical structure of co-occurrences of withdrawal symptoms that experts (physicians and nurses) considered to be of relevance in critically ill children. The co-occurrences of these signs and symptoms could be adequately represented in a three-dimensional solution. Nevertheless, the heterogeneity suggested that the signs and symptoms did not constitute homogeneous clusters within the three-dimensional solution. In all probability, this should be attributed to the low levels of occurrences of withdrawal symptoms, and, consequently, to the low levels of co-occurrences. This finding suggests that withdrawal symptoms vary between individuals.

As withdrawal symptoms of children in the PICU are usually treated immediately, extreme reactions are seen only briefly. While ethically correct, because we put the patients’ comfort first, this approach nevertheless makes it difficult to determine co-occurrences of withdrawal symptoms.

Although the underlying empirical structure of the signs and symptoms was unraveled nicely, the question arises whether the composed SOS based on MDS and the experts’ opinions adequately covers all phenomena of the withdrawal syndrome. Still, the signs and symptoms included in the SOS have been extensively described in the literature [3–5, 12]. Also, the tenfold cross validation on children showed that the empirical solution was robust. Therefore, we conclude that the SOS truly covers the withdrawal symptoms.

Several items were excluded from MDS analysis. We do not preclude that assessment of arterial blood pressure (ABP) might be of additional value, particularly in children being ventilated for a long time. This item was excluded because not all children had an indwelling arterial catheter. High pitched crying can be assessed only when children are not ventilated. This item was excluded, therefore, even though experts identified it as relevant.

The item feeding, which consisted of subitems feeding retention and poor feeding, was eliminated at a later stage. Feeding retention was rated in children fed by feeding tube, poor feeding in orally fed children. Feeding retention may be the result of various causes and therefore this subitem is not sensitive enough. Poor feeding is reported in the literature as a withdrawal symptom. Poor feeding was judged relevant by 55% of the experts, but feeding retention was judged irrelevant. Both subitems were combined in the MDS analysis and it was impossible to distinguish which feeding item had a substantial z score. Also, taking into consideration that only very few children (7/76, 9%) of the target group were orally fed for 1–2 days during ICU stay, we decided to eliminate the combined item.

The OBWS, SWS and WAT-1 overlap with our SOS scale, except for the signs and symptoms hyperactive Moro-reflex, high pitched crying, sneezing, yawning, frequent suction required, seizures and pupil dilation [4, 5, 7]. These signs and symptoms were not included in the SOS. Given the inherent age restriction for the Moro-reflex this item cannot be included in a scale for all ages. We included three other signs and symptoms—anxiety, grimacing and tachycardia—which the three other scales do not contain. On the basis of the literature, however, these signs and symptoms were identified as withdrawal symptoms [3, 13].

We believe the SOS is more sensitive for assessing signs and symptoms of benzodiazepine withdrawal than is the WAT-1. Franck and colleagues report that validity analysis suggested that the WAT-1 is better at detecting symptoms of opioid rather than benzodiazepine withdrawal [7]. For a PICU withdrawal scale this might be problematic given the fact that benzodiazepine withdrawal symptoms are frequent. The literature shows a broad spectrum of benzodiazepine withdrawal symptoms [3]. Symptoms such as agitation, anxiety, tremors, insomnia, fever, sweating, and tachycardia have been described for both benzodiazepine and opioid withdrawal. Symptoms such as movement disorder, grimacing, inconsolable crying and hallucinations have only been observed as benzodiazepine withdrawal. The latter three symptoms were included in the SOS and not in the WAT-1.

Franck and colleagues instead incorporated three items of the State Behavioral Scale [7]. A methodological underpinning of this approach is lacking, however.

The scales are nevertheless comparable with respect to demographic characteristics of test study population and psychometric properties. Validation is a never-ending process and it would be worthwhile to compare both assessment tools in a multicentre study with emphasis on sensitivity to change.

Both scales could be easily integrated in daily practice as a standard of care. However, the time to assess withdrawal is 2 min for the SOS, in contrast with the WAT-1 which requires approximately 7 min observation. For daily practice, the clinical utility of an assessment tool increases if the observation time is brief.

A particular strength of the SOS is that it incorporates the opinions of healthcare workers, so as to make it clinically relevant. Also, it does not ask the raters to assess severity of the symptoms, unlike the Sedation Withdrawal Score. Therefore, we feel that it is not only easier to use, but also more reliable, in that the reliability of an assessment tool will increase as symptoms are unambiguous [14].

In future research, we will explore the sensitivity to change of the SOS in critically ill children. Furthermore, cut-off scores, sensitivity and specificity must be defined for treatment purposes. In conclusion, the SOS is feasible for assessing benzodiazepine and/or opioids withdrawal symptoms in critically ill children in an ICU environment.