To construct a reliable and clinically practical instrument for monitoring opioids and benzodiazepine withdrawal symptoms in pediatric ICU patients.
Intensive care unit in an academic children’s hospital.
Patients and participants
79 patients up to age 16 years on intravenous midazolam and/or opioids for ≥5 days. An expert panel of 85 physicians and nurses rated clinical relevance of withdrawal symptoms.
During drug weaning repeated observations were performed with a checklist of 24 withdrawal symptoms described in the literature.
Measurements and results
For 76 children, 932 observations were obtained within 24 h after decrease and/or discontinuation of midazolam or opioids. Most frequent symptoms were tachypnea, agitation, motor disturbance, diarrhea, fever, anxiety, sleep disturbance and hypertension (14.6–29.6%). Multidimensional scaling (MDS) was performed to detect the underlying empirical structure of co-occurrences of symptoms. An expert panel judged clinical relevance of each withdrawal symptom on a four-point scale ranging from ‘definitively so’ to ‘definitively not’. Agitation, anxiety, inconsolable crying, increased muscle tension, tremors, tachycardia and sweating were considered relevant by 85–95% of the experts. On the basis of the MDS results and the experts’ opinions, 15 symptoms were included in the final instrument.
We are the first to develop an assessment tool for withdrawal symptoms in pediatric ICU patients on the basis of the underlying empirical structure of co-occurrences of withdrawal symptoms that experts considered relevant. Future studies need to define cut-off points and clarify psychometric issues.
Many critically ill children admitted to a specialized intensive care unit (ICU) receive benzodiazepines and/or opioids to reduce pain and anxiety. Long-term exposure to these medications may result in physical dependency. These patients are at risk for withdrawal symptoms after abrupt discontinuation or too rapid tapering-off of these medications .
A reliable, validated and clinical useful assessment tool is indispensable for monitoring withdrawal syndrome. Two such tools are available, the Sedation Withdrawal Score (SWS) and the Opioid Benzodiazepine Withdrawal Scale (OBWS) [4, 5]. Yet each has its limitations. The SWS includes 12 symptoms. While all these symptoms seem clinically relevant for assessing withdrawal, data on validity and reliability are lacking. The OBWS included 16 symptoms and signs of withdrawal and was evaluated in a small sample of PICU children (n = 15) comparing OBWS scores and nurses’ clinical judgment. Sensitivity of the OBWS at scores 8 (cut-off) or higher was 50% and the specificity was 87%, which implies moderate validity of the scale . Symptoms of the OBWS were indirectly adapted from the Neonatal Abstinence Score .
Recently, Franck and colleagues tested the OBWS in 83 pediatric ICU patients and provided a preliminary validity and reliability evaluation of an 11-item reduced version which they named the Withdrawal Assessment Tool-1 (WAT-1) . WAT-1 observation includes four steps. The first is retrieval of information on loose stools, vomiting and temperature of the previous 12 h. The second is a 2-min pre-stimulus observation on state, tremor, sweating, uncoordinated movement and yawning/sneezing. The third is a 1-min stimulus observation on startle to touch and muscle tone. The fourth is maximally a 5-min post-stimulus observation to determine time to gain calm state. In all, the WAT-1 includes four CNS symptoms, two autonomic dysregulation symptoms, two symptoms of gastrointestinal dysfunction and three items derived from the State Behavioral Scale . WAT-1 scores of 3 or higher (on a scale of 0–12) revealed the best sensitivity and specificity (87 and 88%, respectively) in relation to NRS-withdrawal ratings.
Within our line of research we studied the (co-)occurrences of withdrawal symptoms in critically ill children with the ultimate objective of identifying which signs and symptoms are essential for developing a valid and reliable assessment tool.
Materials and methods
Several steps of scale development were needed to construct the Sophia Observation withdrawal Symptoms-scale (SOS). The institutional review board approved the study; because of the strictly observational and non-invasive nature of this study, they waived the need for informed consent.
The first step was a review of the literature on withdrawal symptoms in critically ill children admitted to a PICU based on an extensive evaluation of the literature as recently published by our group . Based on this review, we selected 24 signs and symptoms for inclusion in a preliminary scale, which we called the Sophia Benzodiazepine and Opioid Withdrawal Checklist (SBOWC). Twelve signs and symptoms concerned the central nervous system, four the gastrointestinal tract, and eight the autonomic nervous system (see Table 1).
The second step was a prospective observational study in critically ill children admitted to the ICU of the Erasmus MC—Sophia Children’s Hospital between September 2005 and February 2006. Eligible for inclusion were children aged ≤16 years who had received midazolam (benzodiazepine), morphine or fentanyl (opioids) by continuous infusion for at least 5 days.
The SBOWC was assessed within 24 h after tapering-off or cessation of midazolam and/or opioids (morphine or fentanyl). All nurses had received verbal and written instruction on the application of the SBOWC. An instruction manual explaining the signs and symptoms to be observed was available at each patient’s bedside. Nurses’ interobserver reliability (Cohen’s kappa) for the individual items of the SBOWC ranged from 0.59 to 1.0. The intraclass correlation coefficient was 0.85 (95% CI 0.69–0.94). The face validity was supported by the judgement of ten experienced pediatric intensive care physicians. Furthermore, the construct validity of the SBOWC was promising as shown by a Spearman rank correlation coefficient (r s) of 0.51 between total doses of midazolam and maximum SBOWC sum score in 76 children (95% CI 0.32–0.66, p < 0.001). In addition the r s between total doses of opioids and the maximum SBOWC sum score was 0.39 (95% CI 0.17–0.57, p < 0.01, n = 71). The correlation between duration of medication and maximum SBOWC sum score was 0.52 (95% CI 0.34–0.67, p < 0.001, n = 76) .
The third step was twofold. In step 3a the underlying structure was identified by multidimensional scaling. Step 3b was to obtain expert opinion on the relevance of each of the SBOWC symptoms. To this end we mailed a questionnaire to all pediatric intensive care physicians and clinical fellows in pediatric intensive care of all eight PICUs in the Netherlands and to critical care nurses of our own PICU. In the first main question they were asked to rate the relevance of each withdrawal symptom in critically ill children using a four-point scale ranging from ‘definitively yes’ to ‘definitively no’. In the second question, they were asked to state which signs and symptoms were definitely necessary for an assessment tool. Furthermore, they were asked to provide demographic aspects: discipline, age and years of work experience.
Step 4 selected the relevant items; signs and symptoms which had insufficient discriminative value were eliminated.
Children aged ≤16 years admitted to the ICU of our level three children’s hospital between September 2005 and February 2006 were eligible for this study if they received midazolam (a benzodiazepine), morphine or fentanyl (opioids) by continuous infusion for at least 5 days. Exclusion criteria were: status epilepticus treated with midazolam, use of neuromuscular blocking agents (NMBA), and severely disturbed behavior pattern on account of underlying neurological disease.
The observations were stratified into two groups. First, a ‘weaning group’ (WEAN), 932 observations in 76 children obtained within 24 h after decrease and/or discontinuation of midazolam and/or opioids. Those children were thought to be at risk for developing withdrawal symptoms. Second, an ‘unsuccessful weaning’ group (WEAN−), 93 observations in 27 children obtained before increasing midazolam and/or opioids during the weaning process in order to counteract possible withdrawal related signs and symptoms. We expected that the WEAN− group would show more withdrawal related signs and symptoms as compared to the WEAN group.
As the frequency distribution of the symptoms of SBOWC was highly skewed, the response categories were dichotomized. Of clinical interest is determination of the interrelationship of the signs and symptoms. Conventionally, a correlation technique is used as measure of association. In this study it would not be of interest to estimate associations between symptoms, however, but rather to estimate the co-occurrences of these symptoms. To that end, Jaccard’s measure of similarity tailored to estimate these co-occurrences was applied.
Each symptom of the checklist was recoded. A symptom was coded relevant if the expert scored the first question as ‘that’s definitely so’ or ‘that’s true’, or the second question with ‘definitely necessary’. A symptom was deemed relevant if 50% or more of the respondents agreed on its relevance.
The goal of multidimensional scaling (MDS) analysis technique in this study was to detect meaningful dimensions of observed similarities and dissimilarities (distances) between the withdrawal symptoms. The (dis)similarities between the withdrawal symptoms identified by MDS were based on the co-occurrences of these signs and symptoms. Then, MDS attempted to arrange ‘objects’ (withdrawal symptoms) in a space of a given number of dimensions that does not entail substantial loss of information. Nevertheless, in general, the more dimensions we used in order to reproduce the matrix of dissimilarities, the better the fit of the reproduced matrix to the observed matrix. We considered the Jaccard measure as a similarity measure of co-occurrences. The performances of the models were represented by the normalized raw stress, a measure of model adequacy. This coefficient varies from 0 to 1 and should be <0.05 as a good fit. The Tucker’s φ coefficient of congruence is a measure of correspondence between the distances of the empirical data and the distance derived from the model. Ideally, this coefficient should be >0.95. Identification of the clinical-empirical structure was by means of the computer algorithm PROXSCAL (Proximity Scaling).
The symptoms hypertension and high pitched crying were excluded from analysis. First, it was not always possible to measure arterial blood pressure, e.g., in the absence of an indwelling arterial catheter, in 386/932 (41%) observations. Second, high pitched crying could be observed only in the absence of an endotracheal tube, which was not always the case (629/932, 67%). Furthermore, as the SBOWC asked for observation of either feeding retention or poor feeding, these items were taken together in the MDS procedure.
Final item selection was based on the experts’ opinions and the results of MDS. Inclusion in the SOS required signs and symptoms to have a substantial score (z score) ≥0.30 on at least one dimension of the MDS solution. As a further requirement, a symptom was deemed relevant if 50% or more of the respondents agreed on its relevance.
The robustness of the MDS solution was tested with tenfold cross-validation and presented by the mean (SD) value.
Interobserver reliability was tested for dichotomous items by Cohen’s kappa and the intraclass correlation coefficient (ICC) for continuous data . A Cohen’s kappa below 0.65 was considered unsatisfactory . Statistical analyses were performed with the Statistical Package for Social Sciences (SPSS), version 14.0.
Characteristics study sample
Seventy-nine children (57% boys) met the inclusion criteria. Their median age was 3.4 months (range 0 days to 15.5 years). All 79 received midazolam at a median dose of 176 mcg/(kg h). Seventy-three children (92%) also received opioids at a median dose of 14 mcg/(kg h). Midazolam was administered for a median of 10 days (3–108 days); opioids were administered for a median of 8 days (1–41 days). The median (range) total dose of midazolam was 33 mg/kg (2–595 mg/kg), and that of opioids 4 mg/kg (0–682 mg/kg).
For 76 children, 932 observations were obtained within 24 h after decreasing and/or discontinuation of midazolam and/or opioids (WEAN group). Three patients were discharged before midazolam and/or opioids were decreased or discontinued. Ninety-four ‘unsuccessful weaning’ observations were obtained for 27 children in the ‘unsuccessful weaning’ group (WEAN−). The latter observations were obtained before midazolam and/or opioids were increased during the weaning process in order to counteract possible withdrawal related signs and symptoms.
Twenty-three observations were scored simultaneously by the attending nurse and the principal investigator (EI). The intraclass correlation coefficient was 0.97 (95% CI 0.92–0.98). The interobserver reliability (Cohen’s kappa) for the individual items of the SOS ranged from 0.73 to 1.0.
Bivariate analysis of co-occurrences of withdrawal symptoms
Table 1 presents occurrences of signs and symptoms for the two conditions, i.e., observations performed within 24 h after weaning (WEAN, n = 932) of opioids/benzodiazepine and observations performed before decreasing medication (WEAN−, n = 94). In addition, Table 2 shows the co-occurrences of signs and symptoms for the WEAN group (lower triangle) and WEAN− group (upper triangle).
For the WEAN group, the symptoms most frequently observed were: tachypnea (29.6%), agitation (21.1%), diarrhea (17.8%), fever (17.6%) anxiety (14.8%), sleep pattern (14.6%) and hypertension (14.6%). For the WEAN− group, agitation (46.2%), motor disturbance (37.6%), tachypnea (31.2%), increased muscle tension (28.0%) and feeding retention (25.5%) were observed most frequently. The signs and symptoms tremors, hallucination, seizure, pupil dilatation, sneezing and yawning demonstrated very low prevalences (from 0.3 to 1.9).
With regard to co-occurrences (Table 2) the highest percentage was seen for agitation co-occurring with motor disturbance, both in the WEAN (9.5%) and WEAN− (25.8%) groups. Groups substantially differed in frequencies of the following co-occurrences: agitation/anxiety, agitation/muscle tone, agitation/motor disturbance, agitation/grimacing, agitation/sleeping, agitation/sweating, muscle tone/motor disturbance. Co-occurrence rates with signs and symptoms such as seizures, pupil dilatation, hallucinations, yawning and sneezing were very low.
In total, 85 experts—22 physicians and 63 nurses—completed the questionnaire. Most of them (84.7%) were female. The median work experience was 8 years for physicians (IQR 4–11.3) and nurses (IQR 4.5–15) alike.
The following symptoms were considered most relevant: agitation, anxiety, inconsolable crying, increased muscle tension, tremors, tachycardia and sweating (84.7–95.3%) (see Table 3). For five signs and symptoms fewer than 50% of the experts agreed on relevance: seizures, feeding retention, yawning, sneezing and mottling.
Multidimensional analysis of co-occurrences
The PROXSCAL-procedure with random starts was chosen as the initial configuration. Twenty-one signs and symptoms were entered and the solution of the MDS procedure turned out to be three-dimensional. The normalized raw stress was 0.0498, indicating a good fit. The Tucker’s φ coefficient of congruence equaled 0.97. Decompositions of normalized Raw Stress ranged from 0.02 to 0.06. The tenfold cross-validation of the MDS solution identified a mean normalized raw stress of 0.0501 (SD = 0.002) and a mean Tucker’s φ coefficient of congruence of 0.97 (SD = 0.0009). The conclusion is justified that the solution identified is robust.
The signs and symptoms seizures, pupil dilatation, sneezing and yawning had a z score below 0.30 on either of the three dimensions (see Table 4).
Finally, based on the MDS results as well as the experts’ opinions, 15 signs and symptoms (Table 3) were included in the SOS (see also Supplementary materials).
To our knowledge, this is the first study to identify the underlying empirical structure of co-occurrences of withdrawal symptoms that experts (physicians and nurses) considered to be of relevance in critically ill children. The co-occurrences of these signs and symptoms could be adequately represented in a three-dimensional solution. Nevertheless, the heterogeneity suggested that the signs and symptoms did not constitute homogeneous clusters within the three-dimensional solution. In all probability, this should be attributed to the low levels of occurrences of withdrawal symptoms, and, consequently, to the low levels of co-occurrences. This finding suggests that withdrawal symptoms vary between individuals.
As withdrawal symptoms of children in the PICU are usually treated immediately, extreme reactions are seen only briefly. While ethically correct, because we put the patients’ comfort first, this approach nevertheless makes it difficult to determine co-occurrences of withdrawal symptoms.
Although the underlying empirical structure of the signs and symptoms was unraveled nicely, the question arises whether the composed SOS based on MDS and the experts’ opinions adequately covers all phenomena of the withdrawal syndrome. Still, the signs and symptoms included in the SOS have been extensively described in the literature [3–5, 12]. Also, the tenfold cross validation on children showed that the empirical solution was robust. Therefore, we conclude that the SOS truly covers the withdrawal symptoms.
Several items were excluded from MDS analysis. We do not preclude that assessment of arterial blood pressure (ABP) might be of additional value, particularly in children being ventilated for a long time. This item was excluded because not all children had an indwelling arterial catheter. High pitched crying can be assessed only when children are not ventilated. This item was excluded, therefore, even though experts identified it as relevant.
The item feeding, which consisted of subitems feeding retention and poor feeding, was eliminated at a later stage. Feeding retention was rated in children fed by feeding tube, poor feeding in orally fed children. Feeding retention may be the result of various causes and therefore this subitem is not sensitive enough. Poor feeding is reported in the literature as a withdrawal symptom. Poor feeding was judged relevant by 55% of the experts, but feeding retention was judged irrelevant. Both subitems were combined in the MDS analysis and it was impossible to distinguish which feeding item had a substantial z score. Also, taking into consideration that only very few children (7/76, 9%) of the target group were orally fed for 1–2 days during ICU stay, we decided to eliminate the combined item.
The OBWS, SWS and WAT-1 overlap with our SOS scale, except for the signs and symptoms hyperactive Moro-reflex, high pitched crying, sneezing, yawning, frequent suction required, seizures and pupil dilation [4, 5, 7]. These signs and symptoms were not included in the SOS. Given the inherent age restriction for the Moro-reflex this item cannot be included in a scale for all ages. We included three other signs and symptoms—anxiety, grimacing and tachycardia—which the three other scales do not contain. On the basis of the literature, however, these signs and symptoms were identified as withdrawal symptoms [3, 13].
We believe the SOS is more sensitive for assessing signs and symptoms of benzodiazepine withdrawal than is the WAT-1. Franck and colleagues report that validity analysis suggested that the WAT-1 is better at detecting symptoms of opioid rather than benzodiazepine withdrawal . For a PICU withdrawal scale this might be problematic given the fact that benzodiazepine withdrawal symptoms are frequent. The literature shows a broad spectrum of benzodiazepine withdrawal symptoms . Symptoms such as agitation, anxiety, tremors, insomnia, fever, sweating, and tachycardia have been described for both benzodiazepine and opioid withdrawal. Symptoms such as movement disorder, grimacing, inconsolable crying and hallucinations have only been observed as benzodiazepine withdrawal. The latter three symptoms were included in the SOS and not in the WAT-1.
Franck and colleagues instead incorporated three items of the State Behavioral Scale . A methodological underpinning of this approach is lacking, however.
The scales are nevertheless comparable with respect to demographic characteristics of test study population and psychometric properties. Validation is a never-ending process and it would be worthwhile to compare both assessment tools in a multicentre study with emphasis on sensitivity to change.
Both scales could be easily integrated in daily practice as a standard of care. However, the time to assess withdrawal is 2 min for the SOS, in contrast with the WAT-1 which requires approximately 7 min observation. For daily practice, the clinical utility of an assessment tool increases if the observation time is brief.
A particular strength of the SOS is that it incorporates the opinions of healthcare workers, so as to make it clinically relevant. Also, it does not ask the raters to assess severity of the symptoms, unlike the Sedation Withdrawal Score. Therefore, we feel that it is not only easier to use, but also more reliable, in that the reliability of an assessment tool will increase as symptoms are unambiguous .
In future research, we will explore the sensitivity to change of the SOS in critically ill children. Furthermore, cut-off scores, sensitivity and specificity must be defined for treatment purposes. In conclusion, the SOS is feasible for assessing benzodiazepine and/or opioids withdrawal symptoms in critically ill children in an ICU environment.
Anand KJ, Arnold JH (1994) Opioid tolerance and dependence in infants and children. Crit Care Med 22:334–342
Tobias JD (2000) Tolerance, withdrawal, and physical dependency after long-term sedation and analgesia of children in the pediatric intensive care unit. Crit Care Med 28:2122–2132
Ista E, van Dijk M, Gamel C, Tibboel D, de Hoog M (2007) Withdrawal symptoms in children after long-term administration of sedatives and/or analgesics: a literature review. “Assessment remains troublesome”. Intensive Care Med 33:1396–1406
Cunliffe M, McArthur L, Dooley F (2004) Managing sedation withdrawal in children who undergo prolonged PICU admission after discharge to the ward. Paediatr Anaesth 14:293–298
Franck LS, Naughton I, Winter I (2004) Opioid and benzodiazepine withdrawal symptoms in paediatric intensive care patients. Intensive Crit Care Nurs 20:344–351
Finnegan LP, Kron RE, Connaughton JF, Emich JP (1975) Assessment and treatment of abstinence in the infant of the drug-dependent mother. Int J Clin Pharmacol Biopharm 12:19–32
Franck LS, Harris SK, Soetenga DJ, Amling JK, Curley MA (2008) The Withdrawal Assessment Tool-1 (WAT-1): an assessment instrument for monitoring opioid and benzodiazepine withdrawal symptoms in pediatric patients. Pediatr Crit Care Med 9:573–580
Curley MA, Harris SK, Fraser KA, Johnson RA, Arnold JH (2006) State Behavioral Scale: a sedation assessment instrument for infants and young children supported on mechanical ventilation. Pediatr Crit Care Med 7:107–114
Ista E, van Dijk M, Gamel C, Tibboel D, de Hoog M (2008) Withdrawal symptoms in critically ill children after long-term administration of sedatives and/or analgesics: a first evaluation. Crit Care Med 36:2427–2432
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428
Fleiss JL (1981) Statistical methods for rates and proportions. Wiley, New York
Siddappa R, Fletcher JE, Heard AMB, Kielma D, Cimino M, Heard CMB (2003) Methadone dosage for prevention of opioid withdrawal in children. Pediatr Anaesth 13:805–810
Birchley G (2009) Opioid and benzodiazepine withdrawal syndromes in the paediatric intensive care unit: a review of recent literature. Nurs Crit Care 14:26–37
Streiner DL, Norman GR (2003) Health Measurement Scales. A practical guide to their development and use. Oxford University Press, Oxford
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Ista, E., van Dijk, M., de Hoog, M. et al. Construction of the Sophia Observation withdrawal Symptoms-scale (SOS) for critically ill children. Intensive Care Med 35, 1075–1081 (2009). https://doi.org/10.1007/s00134-009-1487-3