Development and Psychometric Evaluation of the Treatment-Emergent Activation and Suicidality Assessment Profile
- First Online:
- Cite this article as:
- Reid, J.M., Storch, E.A., Murphy, T.K. et al. Child Youth Care Forum (2010) 39: 113. doi:10.1007/s10566-010-9095-5
- 69 Views
Although effective in treating a range of childhood psychiatric conditions, selective serotonin reuptake inhibitors (SSRI) have been implicated in the induction of an “activation syndrome” (characterized by symptoms of irritability, restlessness, emotional labiality, etc.) that may represent an intermediary state change that fosters suicidality. SSRI-induced activation syndrome is well-accepted by many clinicians and thought to be relatively common, particularly in children and teens. However, gaps exist in empirical data on phenomenology and tools for early detection. With this in mind, we report on a recently funded National Institutes of Health grant to develop a measure of behavioral activation to be completed in a clinical setting. We discuss the development of this measure—the Treatment-Emergent Activation and Suicidality Assessment Profile (TE-ASAP)—as well as psychometric results from a sample of youth with internalizing disorders who were at varying stages of SSRI treatment. Overall, psychometric data were quite promising, with the TE-ASAP demonstrating excellent reliability (i.e., internal consistency, inter-rater, short-term test–retest stability) and strong validity properties. Through further evaluation of the TE-ASAP in the context of a controlled multimodal trial in youth with obsessive–compulsive disorder, we hope to augment understanding of activation syndrome and, in turn, mitigate risks through early detection of this potentially lifethreatening adverse effect.
KeywordsSelective serotonin reuptake inhibitorsActivation syndromeBehavioral activationObsessive–compulsive syndrome
Selective serotonin reuptake inhibitors (SSRI) have demonstrated efficacy in treating a variety of childhood and adult psychiatric conditions, including depressive disorders (March et al. 2004; MacGillivray et al. 2003); obsessive–compulsive disorder (OCD; POTS 2004; Bandelow 2008); and other anxiety disorders (Walkup et al. 2008; Bandelow 2008). Although the side effect profile of SSRIs is favorable relative to that of older antidepressants (e.g., tricyclics or MAOIs; Murphy et al. 2008), there has been recent controversy about the role of SSRIs in the induction of suicidality in children and adolescents (Goodman et al. 2007). This issue has created (for clinicians, patients, and parents alike) a dilemma regarding the relative benefits and risks of SSRI use—as well as practical inquiry into the efficacious measurement of “activation syndrome” symptoms. In response, we were recently awarded a grant from the National Institutes of Mental Health to develop a measure of behavioral activation and to examine the phenomenology and timing of activation in the context of a multimodal clinical trial for pediatric OCD. The present paper discusses the initial phase of that grant, in which the primary goal was to develop and psychometrically examine a measure of “activation syndrome” symptoms, the Treatment-Emergent Activation and Suicidality Assessment Profile (TE-ASAP). What follows first is a brief history detailing the association between SSRIs and suicidality/behavioral activation.
History of Association Between SSRIs and Suicidality
Evidence of a possible link between SSRI therapy and the onset of suicidal behaviors among pediatric patients initially emerged in the early 1990s (Teicher et al. 1990; Teicher et al. 1993). The first report, by Teicher et al. (1990), discussed six depressed youth who developed suicidal ideation following initiation of fluoxetine treatment. Considerable clinical and empirical attention was subsequently given to the possibility of a relationship between SSRI use and increased suicidal behaviors (Beasley et al. 1991; Letizia et al. 1996). As a result of SSRI-related suicidaility concerns, all antidepressants excluding fluoxetine were banned from pediatric use in the United Kingdom (Department of Health, United Kingdom 2003).
In response to these data and corresponding clinical accounts, a joint meeting of the Psychopharmacologic Drug and Pediatric Advisory Committees to the FDA was held in 2004 to determine the strength of relationship between SSRI initiation/dosage increase and suicidality. This meeting focused on a synthesis of available data (e.g., rigorous clinical trials, case reports), although emphasis was placed on double-blind, placebo-controlled studies submitted to the FDA. A total of 24 acute trials (up to 12-week-long) taking place between 1983 and 2004, and involving approximately 4400 pediatric participants, were analyzed. Most were specific to major depression, but there were several trials regarding OCD and other anxiety disorders. A range of antidepressants was studied, including SSRIs (i.e., citalopram, fluoxetine, fluvoxamine, paroxetine and sertraline); the serotonin-norepinephrine reuptake inhibitor (SNRI), velafaxine; and ‘multiple action agents’ such as mirtazapine and nefazodone. Although there were no completed suicides in those trials that were reviewed, patients treated with active antidepressants (4%) manifested significantly more suicidal behaviors than did placebo-treated patients (2%). Using risk difference analyses, the FDA found that approximately 2–3 of 100 pediatric patients treated with an antidepressant should expect an increase in suicidality (suicidal thoughts and behaviors) above and beyond that found with placebo treatment. Then, in response to these data and the committee’s vote, the FDA issued a “black box warning” in October 2004 to ensure provider-patient/parent discussion; attention to non-medication alternatives; and the monitoring of SSRI-related behavioral changes. While Hammad (2004) reported that neither primary diagnosis nor chemical class of antidepressant moderated findings (Hammad 2004), more recent analyses have suggested otherwise: there has been mounting evidence that those with a major depressive disorder (versus OCD; Bridge et al. 2007); those taking venlafaxine and paroxetine (versus another medication; Bridge et al. 2007); and adolescents (versus adults; Barbui et al. 2009) are at relatively greater risk.
In addition to its desired effects (e.g., enhanced dissemination of information on potential risks), implementation of the “black box warning” was related to a paradoxical increase in youth suicide (Gibbons et al. 2007)—presumably mediated by the attenuation of antidepressant prescriptions for youth. Between the years of 2003 and 2005 (i.e., following the “black box warning”), the increase in youth suicide marked “the largest year-to-year change in suicide rates in this population since the Centers for Disease Control and Prevention began systematically collecting suicide data in 1979” (Gibbons et al. 2007, p. 1356); for instance, the Netherlands experienced an increase of 49% between 2003 and 2005, while the United States had an increase of 14% between 2003 and 2004 (Gibbons et al. 2007). Inversely, SSRI prescriptions for youth had decreased by approximately 22% across the Netherlands and United States during this time (Gibbons et al. 2007).
Phenomenology of Activation Syndrome
The main shortcoming of antidepressant trials was that most lacked an empirically-determined measure of suicide, parasuicidal behavior, or (most notably) non-suicidal behavioral activation. Current literature estimates that activation symptoms occur in 12.3–13% of cases (March et al. 1998; POTS 2004; Riddle et al. 2001). Our clinical experience suggests an even higher rate of activation in the general clinic population—at least 25% (in marked contrast to the 4% occurrence in individuals treated with placebo; POTS 2004). Symptoms of behavioral activation by antidepressants can vary greatly, manifesting as any combination of the following: irritability, agitation, somatic symptoms of anxiety, panic attacks, restlessness, hostility, aggressiveness, insomnia, disinhibition, emotional labiality, impulsivity, social withdrawal, restlessness, hypomania/mania, paranoia or other psychotic symptoms, or other unusual changes in behavior or mood. However, there are limited methods currently available to assess behavioral activation related to SSRI use in youth. Given this, we submitted—and were successfully funded—a grant application to study the nature and timing of antidepressant-induced “activation syndrome” in the context of a clinical trial on pediatric patients diagnosed with OCD. A major goal of this study, and the purpose of this paper, was to develop a measure of activation syndrome that could be completed in a clinical setting. What follows is a discussion on the development of TE-ASAP’s initial version, followed by its psychometric evaluation in a sample of youth with varied internalizing disorders who were at varying stages of SSRI treatment.
The initial phase of the study focused on refining the TE-ASAP for the assessment of activation syndrome. To facilitate this goal, we (1) drew on expert opinion to refine the measure and (2) administered it to a cohort of pediatric patients who were undergoing open-label SSRI treatment for anxiety and depressive disorders. TE-ASAP ratings depended on clinician judgment of the relative accuracy of parent and child responses (obtained together) as well as the pair’s overall accuracy—such that the recording of discrepancies might favor one respondent’s input; or the clinician’s rating might diverge from the pair’s collective report. The latter phase of this grant—which is not reported in detail in this paper—involves an ongoing randomized clinical trial on the phenomenology of behavioral activation in a randomized controlled trial of sertraline in youth with OCD.
Phase 1 participants included 30 children (19 male) who exhibited one of the following psychiatric disorders: OCD; major depression; generalized anxiety disorder; social phobia; or separation anxiety disorder. The age range of participants was 7–17 years (M = 12.1, SD = 2.9), and the vast majority were Caucasian. Diagnoses were made based on administration of the K-SADS-PL (Schedule for Affective Disorders and Schizophrenia for School-Age Children—Present and Lifetime Version) and all other available information collected during a clinical interview. Youth were in varying stages of SSRI treatment at the time of participation. All patients met the following criteria to be eligible for involvement in this phase of the protocol: (a) diagnosed with a Major Depressive Disorder, OCD, or non-OCD anxiety disorder derived from the KSADS-PL; (b) currently taking an SSRI; and (c) between the ages of 7 and 17 years, inclusive. As well, at least one parent per child needed to be available to participate in assessments. On the other hand, children were excluded if they met criteria for psychosis or pervasive developmental disorder. With regard to medication, 10 of the 30 youth initiated treatment within 4 weeks of assessment; 10 were on sustained treatment for 7–12 weeks by baseline; and 10 had taken an SSRI for 13 or more weeks.
Aberrant Behavior Checklist (ABC)
The ABC (Aman and Singh 1986) is a behavioral rating scale of treatment effects. The measure comprises five subscales: (1) Irritability, Agitation, Crying; (2) Lethargy, Social Withdrawal; (3) Stereotypic Behavior; (4) Hyperactivity, Noncompliance; and (5) Inappropriate Speech. The ABC was developed for patients with developmental disabilities but, given the content of the “Irritability” subscale, we wanted to determine whether it had utility for detecting “activation syndrome.”
Child Behavior Checklist (CBCL)
The CBCL (Achenbach and Rescorla 2001) is a psychometrically sound, 118-item scale that assesses specific child behaviors from the parent’s perspective. The CBCL provides a total behavior problem score, several subscale scores, and scores on Internalizing and Externalizing dimensions of dysfunction.
Children’s Depression Rating Scale-Revised (CDRS-R)
The CDRS-R (Poznanski and Mokros 1996) is a semi-structured interview with the child and an adult caregiver, intended to diagnose depression and level of severity. The interviewer rates 17 symptom areas including those that comprise DSM-IV criteria for a diagnosis of depression. The scale is sensitive to symptom change and treatment response.
Swanson, Nolan, and Pelham: IV Parent Scale (SNAP-IV)
The parent-rated SNAP-IV (Swanson 1992) provides a dimensional scaling of the Diagnostic and Statistical Manual of Mental Disorders-IV items for inattention; impulsivity; hyperactivity; and oppositionality, as well as other features without import in the present study. The scale has good predictive validity and excellent reliability, with an alpha of .94 (Bussing et al. 2008).
Suicidal Ideation Questionnaire: Junior (SIQ-JR)
The SIQ-JR (Reynolds 1988) is a self-report measure of a wide range of thoughts related to death and dying; passive and active suicidal ideation; and suicidal intent in children and adolescents. Items are rated on a 7-point Likert-type scale, ranging from 0 (“I never had this thought”) to 6 (“This thought was in my mind almost every day”). The scale has adequate test–retest reliability and construct validity (Keane et al. 1996) as well as good internal consistency (alpha = .94; Reynolds 1988).
Young Mania Rating Scale: Parent Version (YMRS-P)
Barnes Akathisia Scale
The Barnes Akathisia Scale (Barnes 1989) is a commonly-used measure of akathisia in individuals who are receiving antipsychotic medications. The scale rates both observed movements and the subjective experience of restlessness. The scale has good interrater reliability (Barnes 1989) and face validity (Cunningham-Owens 1999).
Participants were recruited through the normal patient flow into a large OCD specialty clinic. Initial screening took place over the telephone, whereby families were assessed for the appropriateness of participation in this component of the study. Assuming successful screening, the family was scheduled to give in-person consent and assent (subsequent to detailed explanation, by study personnel, of the study and its procedures). Once consent and assent were obtained, the participant was assessed by a rater trained in the administration of the K-SADS-PL and CY-BOCS (Children’s Yale-Brown Obsessive Compulsive Scale); and the participant and parent were, together, interviewed by a board-certified child psychiatrist (TKM) in order to inform TE-ASAP ratings. In 28 of 30 TE-ASAP interviews, inter-rater reliability was determined through comparison with secondary coding (by a psychologist or additional psychiatrist observing administration). For eight families, the TE-ASAP was readministered 7 days after their original interviews. The interviewer remained constant. Following any TE-ASAP interviews, and in accordance with standard practices, the primary rating psychiatrist recorded whether she judged any adverse events to be “Definitely Related;” “Probably Related;” “Possibly Related;” or “Not Related” to study medication.
Measure Selection, Development, and Refinement
Based upon review of the extant literature, and discussion among manuscript authors as well as at the FDA hearings in 2004, the constructs initially selected as most congruent with activation syndrome included “irritability;” “disinhibition/impulsivity;” and “akathisia.” Additional domains of interest in the present study were somatic anxiety; hyperkinesis; and mania. Accordingly, the key measures chosen for the present study were the SNAP-IV (targeting impulsivity/hyperactivity); CBCL (targeting aggression, impulsivity, and both externalizing and internalizing symptoms); SIQ-JR (informing on suicidality); ABC (addressing irritability); YMRS-P (targeting symptoms of mania); and Barnes (targeting akathisia). As noted above, all of the preceding measures have established psychometric properties for the pediatric population.
TE-ASAP Measure Development and Refinement
Initial Scale Development. Items were generated predominantly from both the extant literature and the clinical experiences of Drs. Goodman and Murphy with SSRIs in pediatric populations. The initial version of the TE-ASAP—from which the present data were collected—consists of the following subscales1: (a) Irritability (12 items), (b) Akathisia, Hyperkinesis, and Somatic Anxiety (10 items), (c) Disinhibition and Impulsivity (7 items), (d) Mania (10 items), and (e) Self-injury, Suicidality, and Harm to Others (8 items). For each item, parents were asked to separately rate frequency and intensity on a 4-point Likert-style scale, such that 0 = none; 1 = mild; 2 = moderate; and 3 = severe. Thus, three overall scores are derived: an overall frequency score; an overall intensity score; and a total score (comprised of both frequency and intensity ratings).
Informal Feedback and Discussion. Following initial development, the TE-ASAP was circulated among those psychiatrists and psychologists participating in the grant application. Revisions were made to reflect comments on (a) content validity; (b) comprehensiveness; (c) overlap with other constructs, and (d) readability and clarity.
Delphi Procedure. Further content validation of the TE-ASAP was conducted via the Delphi technique (Adler and Ziglio 1996), which uses a highly structured and focused questionnaire approach to establish a consensus opinion among experts. A primary advantage of the Delphi technique over committee action is its allowance for relatively anonymous feedback from experts. In the present study, target group opinion about the TE-ASAP was obtained—and then summarized to inform a follow-up questionnaire. Results from this questionnaire, in turn, informed a third (i.e., the final) questionnaire. Expert discussion occurred with the preceding questionnaires—however, it was not until the third that consensus (regarding those constructs to include in the measure) was achieved.
Alpha coefficients for all TE-ASAP scores were adequate or better (α ≥ .71), with the exception of the TE-ASAP Frequency and Intensity scores for the subscale addressing Self-Injury, Suicidality and Harm to Others (α = .64 and .57, respectively). Internal consistency for the TEASAP Frequency, Intensity, and Total, were high (α = .95, .94, and .97, respectively).
Correlations Among TE-ASAP Total and Subscale Scores
Correlations among subscales of the TE-ASAP
Inter-rater reliability for the TE-ASAP scores was high across subscales. Agreement for the TE-ASAP Total score, Frequency, and Intensity were all high (r = .99, .98, and .99, respectively; p < .01). Convergence was exhibited for other subscales as well, with correlations ranging between .90 and .99 (p < .01).
Reliability Across Assessments
Test-retest reliability of TE-ASAP scores
Convergent and Divergent Validity
Given the high association between TE-ASAP Frequency and Intensity scores, and in the interest of avoiding redundancy, we only report correlations with the Total score. The TE-ASAP Total score was strongly correlated with the following constructs: CBCL Aggression and Delinquent subscales (rs = .80 and .66, respectively; p < .01); SIQ-JR (r = .64, p < .01); and the ABC Irritability subscale (r = .85, p < .001). Correlations of moderate strength were shown between the TE-ASAP Total score and SNAP-IV Inattentive, Impulsive, and Oppositional subscales (rs = .53, .51, and .56, respectively; p < .01); CDRS (r = .57, p < .01); the ABC Stereotypy subscale, (r = .56, p < .05); YMRS-P (r = .55, p < .01); and the CBCL Attention subscale (r = .47, p < .01). Nonsignificant correlations of weak-to-moderate strength were shown with the CBCL Anxiety/Depression and Somatic subscales (rs = .35 and .24, ns) as well as with ABC Lethargy; Hyperactivity; and Inappropriate Speech subscales (rs = .20, .46, .42, respectfully, ns).
TE-ASAP subscale scores (i.e., for Irritability, Mania, etc.) correlated significantly with corresponding constructs assessed by other study measures: The TE-ASAP Irritability subscale correlated moderately-to-strongly with CBCL Attention (r = .39, p = .05); Delinquency (r = .52, p < .05); and Aggression subscales (r = .76, p < .01), as well as with the ABC Irritability factor (r = .79, p < .01). The TE-ASAP Akathisia, Hyperkinesis and Somatic Anxiety subscale correlated weakly with the Barnes Akathisia scale (r = .27, p > .05), and with the CBCL Attention (r = .27, p > .05), Somatic (r = .31, p > .05), and Anxiety/Depression (r = .25, p > .05) subscales. The TE-ASAP Disinhibition and Impulsivity subscale score correlated strongly with CBCL Attention (r = .55, p < .01), Delinquency (r = .82, p < .01), and Aggression (r = .70, p < .001) subscale scores; the SNAP-IV Inattentive (r = .66, p < .01), Impulsive (r = .60, p < .01), and Oppositional (r = .56, p < .01) subscale scores; and the ABC Hyperactivity subscale scores (r = .64, p < .01). Scores on the TE-ASAP Mania subscale correlated strongly with YMRS-P Total (r = .53, p < .01), and weakly with both CDRS Total score (r = .23, p > .05) and CBCL Anxiety/Depression subscale (r = .09, p > .05). The TE-ASAP Self-Injury, Suicidality, and Harm to Others subscale score correlated strongly with the SIQ-JR (r = .72, p < .01) and CDRS (r = .59, p < .01) Total scores, but weakly with the YMRS-P Total (r = .21, p > .05) and CBCL Anxiety/Depression subscale (r = −.03, p > .05) scores.
We report on the development of the TE-ASAP, a measure to assess common symptoms of behavioral activation secondary to SSRI use in children and adolescents. The initial part of this study focused on measurement development, whereas the second phase is an on-going clinical trial of cognitive-behavioral therapy (CBT) with or without sertraline in a large sample of youth with OCD. Overall, findings suggest that the TE-ASAP is psychometrically sound. Reliability estimates were generally strong, with high internal consistency for all subscales; excellent inter-rater reliability; and strong short-term test–retest reliability.
Evidence of convergent validity was found vis-à-vis strong correlations with measures of similar constructs. For example, TE-ASAP Irritability subscale scores correlated moderately-to-strongly with CBCL Delinquency and CBCL Aggression subscale scores as well as with the ABC Irritability subscale. And the TE-ASAP Total score correlated moderately-to-strongly across study measures, suggesting modest convergence with examined constructs. Divergent validity was supported with generally weak correlations between, on the one hand, TE-ASAP subscale scores and, on the other, theoretically divergent constructs (e.g., weak relation between (1) the TEASAP subscale addressing Self-Injury, Suicidality, and Harm to Others and (2) YMRS-P subscale scores).
Although initial psychometric findings for the TE-ASAP were promising, our experiences suggested a number of revisions were necessary. In particular, the clinician-rated version of the measure was somewhat unwieldy to administer, and several items were redundant. Accordingly, a number of items have since been eliminated or reworded, and the measure has been converted to a parent-report format. In pilot testing of the parent-report version, we found that (relative to the clinician-rated version) information was obtained in a more rapid manner; and quality of the information provided appeared to be comparable. As well, given the strong relation found between TE-ASAP Frequency and Intensity scores on the clinician-rated version, directions for reporting were both clarified and parsed for the parent-report version. In the new version, the parent is asked to indicate overall symptom severity based upon the collective impact of: (1) the frequency of each behavior, relative to same aged peers; (2) the intensity of each behavior, relative to same aged peers; and (3) the extent to which each behavior interferes (if at all) with the child’s functioning in family/home, school, and social realms. In addition, to inform the potential for a symptom-medication link, the parent indicates his/her impression of relatedness (dichotomous: yes/no), and whether each symptom is better; worse; unchanged; or new.
We have recently progressed to the second study phase, through which children and adolescents with OCD are randomized in double-blind fashion to (1) sertraline at standard dosing (RegSert); (2) sertraline slow titration (SloSert); or (3) pill placebo (Pla) for 18 weeks. All youth receive cognitive-behavioral therapy starting at the fourth week of the study. The primary aim is to confirm the occurrence and timing of activation syndrome during SSRI treatment. There are several advantages to conducting a study of SSRI-induced activation syndrome in participants with OCD versus depression. First, a diagnosis of pediatric OCD can be determined with relatively greater certainty than can a diagnosis of depression in youth. Second, potential confounds of affective instability and suicidality (at baseline) are present to a lesser extent in OCD than in depression. Third, recruitment of youth with OCD (versus those with depression) into a pediatric pharmacological trial may be aided by the relatively superior benefit-to-risk ratio of SSRIs in this population, and the inclusion of cognitive-behavioral therapy for all participants.
Both the efficacy of cognitive-behavioral therapy and the presence of activation symptoms will be assessed through a comprehensive multi-modal battery. As noted, all three groups will receive cognitive-behavioral therapy for obsessive–compulsive symptoms, beginning after the fourth week of the study (to avoid confounding the impact of exposures on presentation of activation). We predict that measures of activation syndrome will be elevated during the initial 2 weeks in the RegSert versus PLA arm. As well, we expect less frequent development, and lesser severity, of activation symptoms in the SloSert versus RegSert arm. A secondary aim of Phase 2 will be to examine the differential outcome of sequential multimodal treatment for pediatric OCD (i.e., sertraline + cognitive-behavioral therapy) versus placebo + cognitive-behavioral therapy. This secondary aim is in response to the Pediatric OCD Treatment Study II (POTS II; Freeman et al. 2009), a multisite examination of augmentation strategies for partial response history to pharmacotherapy with serotonin reuptake inhibitors (SRI; Freeman et al. 2009). More specifically, POTS II examined the relative efficacies of (1) “gold standard” cognitive-behavioral therapy, to be provided by highly trained psychologists and in conjunction with continued medication treatment; (2) a truncated course of cognitive-behavioral therapy, to be provided by the prescribing psychiatrist and in conjunction with continued medication treatment; and (3) continued medication treatment alone (i.e., without cognitive-behavioral therapy). To date, approximately 10 participants have been enrolled and are at various points in study procedures. We anticipate enrolling up to 90 participants across our two sites (University of South Florida and University of Florida).
It should be noted that the labels (e.g., Irritability) for the different dimensions of the scale are preliminary and intended only to help organize content during the construction of the instrument. These subscale names may be altered once we are satisfied that the domain range and factor structure has been established.