A Psychometric Analysis of the Revised Child Anxiety and Depression Scales—Parent Version in a School Sample

Abstract

The Revised Child Anxiety and Depression Scale—Parent Version (RCADS-P) is a parent-report questionnaire of youth anxiety and depression with scales corresponding to the DSM diagnoses of separation anxiety disorder, social phobia, generalized anxiety disorder, panic disorder, obsessive-compulsive disorder, and major depressive disorder. The RCADS-P was recently developed and has previously demonstrated strong psychometric properties in a clinic-referred sample (Ebesutani et al., Journal of Abnormal Child Psychology 38, 249–260, 2010b). The present study examined the psychometric properties of the RCADS-P in a school-based population. As completed by parents of 967 children and adolescents, the RCADS-P demonstrated high internal consistency, test-retest reliability, and good convergent/divergent validity, supporting the RCADS-P as a measure of internalizing problems specific to depression and five anxiety disorders in school samples. Normative data are also reported to allow for the derivation of T-scores to enhance clinicians’ ability to make classification decisions using RCADS-P subscale scores.

Anxiety and depressive disorders are among the most common psychiatric conditions experienced by youth (Lewinsohn et al. 1993). Current or short-term prevalence rates are approximately 2–4% and 5–6%, respectively, in recent reviews (Costello et al. 2004; Costello et al. 2003), while lifetime prevalence rates range from 6–15% and 15–20%, respectively, in epidemiological studies (Silverman and Ginsburg 1998). Given the significant functional impairment (Birmaher et al. 1996) and increased risk of continued psychopathology in adulthood (Pine et al. 1998) associated with anxiety and depression in youth, the accurate assessment of such difficulties is imperative in aiding both clinical and research efforts.

Traditionally, self-report measures have been the dominant method for assessing internalizing disorders in youth (March and Albano 1996; Southam-Gerow and Chorpita 2007) as they provide an efficient and cost-effective means of gathering information. Although several measures have been developed to assess anxiety and depression [e.g., the Revised Children’s Manifest Anxiety Scale (RCMAS) and the Children’s Depression Inventory (CDI), respectively], the Revised Child Anxiety and Depression Scale (RCADS; Chorpita et al. 2000) maps onto current DSM nosology and indexes the main features of five prominent DSM anxiety disorders [separation anxiety disorder (SAD), social phobia (SOC), generalized anxiety disorder (GAD), obsessive-compulsive disorder (OCD), panic disorders (PD)] as well as major depressive disorder (MDD). This is an advantage over measures developed prior to more contemporary diagnostic classification systems (e.g., DSM-IV; APA 2000), as for example, Chorpita et al. (2005) found that the RCADS targets anxiety and depression more specifically than the RCMAS and CDI, respectively.

Despite youth self-report instruments demonstrating to be useful in measuring anxiety and depression, limitations of such child and adolescent self-reports have been noted (e.g., Kendall and Flannery-Schroeder 1998), speaking to the importance of incorporating parent reports in the assessment of youth internalizing problems (e.g., Jensen et al. 1999; Klein et al. 2005). Researchers have already begun to administer the RCADS to parents (coined the RCADS-P¹) to assist in the measurement of anxiety and depression among youth (e.g., Costa et al. 2009; Watts and Weems 2006; Weems and Costa 2005). Other similar parent-report measures of youth anxiety also exist, including the Multidimensional Anxiety Scale for Children parent version (MASC-P; Baldwin and Dadds 2007), and the Screen for Child Anxiety Related Emotional Disorders-Revised parent version (SCARED-R-P; Muris et al. 2004), both which target DSM related anxiety problems and demonstrate the importance of gathering parent-reported data when assessing anxiety in youth.

Some advantages of the RCADS-P relative to the MASC-P and the SCARED-R-P, however, include the RCADS-P’s ability to concurrently assess both anxiety and depression—a useful feature given the high comorbidity between these disorders in youth (Brady and Kendall 1992). In contrast to the other measures discussed, the RCADS-P is also available for free,² thus supporting the feasibility of its use in a wider variety of settings. The RCADS-P also recently evidenced particularly strong psychometric properties in a clinical sample of youth (N = 490) diagnosed with structured diagnostic interviews (Ebesutani et al. 2010b), including the ability for the RCADS-P anxiety subscales to discriminate between anxiety disorders—an advantage over the SCARED-R-P, which has so far failed to make such discriminations (Muris et al. 2004). Accurate classification percentages based on receiver operating characteristic analyses were also reported, ranging from 71.3% to 85.4%. Given these strengths, the RCADS-P (whether utilized alone or in combination with the RCADS) has the potential to be a comprehensive, efficient, and economical tool in the assessment of youth internalizing problems.

Although the psychometric properties of the RCADS-P have been examined in a clinic-referred sample, the RCADS-P’s psychometric properties remains largely unexamined in non-clinic-referred (community and school) settings. Although some psychometric data based on non-clinic-referred (community) youth have been reported (see studies noted above; Costa et al. 2009; Watts and Weems 2006; Weems et al. 2005), these reports were limited to reliability statistics on the RCADS-P Anxiety Total score only, leaving the remaining six subscales uninvestigated in community and school settings. Additional psychometric studies using non-clinic-referred populations are needed, as evidence suggests that youths from clinic-referred settings are not fully representative of all youths with mental disorders (Goodman et al. 1997).

The present study thus sought to thoroughly examine the psychometric properties of the RCADS-P and all of its subscales in a more representative, community sample of school-based children and adolescents. An additional aim of the present study was to provide normative data, which have not yet been reported for the RCADS-P, to allow for the derivation of T-scores to increase the clinical utility and interpretability of the RCADS-P scale scores. Based on the results from the recent psychometric study on the RCADS-P based on a clinical sample (Ebesutani et al. 2010b), we hypothesized that similar results would be obtained in the present study based on a school-based sample of youth. Specifically, we hypothesized that (a) the RCADS-P 6-factor model would be supported via confirmatory factor analysis (CFA) and would evidence better model fit over a 5-factor model (treating GAD and MDD as a single “distress” factor; Lahey et al. 2008; Watson 2005), (b) the 6-factor structure of the RCADS-P would evidence factor invariance across boys and girls as well as across younger and older youth, (c) reliability of the RCADS-P subscales would be supported via adequate internal consistency and test-rest reliability estimates, and (c) the RCADS-P depression and anxiety subscales would evidence significant correspondence with criterion measures for MDD and corresponding anxiety problems.

Method

Participants

We distributed consents (N = 7,370) to youths and their parents in public and private schools across the Hawaii from 2007 to 2009 to seek their participation in a large school-based study. Among the consents distributed, 26.6% (n = 1,961) of youth and their parents consented to participate, and 65.7% (n = 1,288) of these consenting youths completed forms in school, including the RCADS. Of the participating youths (n = 1,288), 75.8% of their parents (n = 976) completed and returned parent-report forms, including the RCADS-P and the Child Behavior Checklist (CBCL; Achenbach and Rescorla 2001).

Criteria for inclusion in the present study included youth being between 6 and 18 years old (due to the established age range for the RCADS and CBCL forms) and having an available RCADS-P form completed by a parent. Youths’ ages ranged from 8 to 18 years (M = 13.3, SD = 2.9), and thus no youths were excluded due to age restrictions. Additional inclusion criteria required that all forms utilized in the analyses (i.e., RCADS-P, RCADS, CBCL) have 90% or more completed data. As such, nine (0.9%) participants were excluded due to having more than 10% (5 to 24) RCADS-P items missing, leaving a total RCADS-P sample of 967. Among the 747 corresponding RCADS child forms available in the present study, seven RCADS forms (0.9%) were excluded from analyses due to having 5 to 11 items missing, leaving 740 RCADS child forms available for analyses. Among the 743 corresponding CBCL parent forms available, 20 CBCL forms (3%) were also excluded from analyses due to having 12 to 63 items missing, leaving 723 CBCL forms available for analyses.

Youth and primary caregiver demographic information appears in Table 1. All children and parents were fluent in English. Although we did not collect diagnostic data for these school-based youths, we examined the number of youths who scored in clinically-elevated ranges (T-score >65) on the six CBCL DSM-oriented scales to provide estimates of the number of youths in this sample with clinically elevated DSM-oriented problems. In this school-based sample, the following proportions of the 723 youth with available data showed elevations (T > 65) on the CBCL DSM-oriented scales: Affective (13.1%), Somatic (12.4%), Attention Deficit/Hyperactivity (10.2%), Conduct (8.9%), Oppositional (8.7%) and Anxiety Problems (7.1%).

Table 1 Youth and caregiver demographic information

Measures

Child Behavior Checklist for Ages 6–18 (CBCL/6–18; Achenbach and Rescorla 2001)

The CBCL/6–18 measures emotional and behavioral problems in youth. The 120 items on the CBCL are rated by parents as Not True, Somewhat/Sometimes True, or Very True/Often True by youths’ parents. Items are summed to yield (a) Competence and Adaptive scale scores, (b) Syndrome scale scores, (c) DSM-oriented scale scores, and (d) Total Problems scale scores (including Internalizing, Externalizing and Total scale scores). Validity and reliability of the Syndrome and DSM-oriented scales have been documented (Achenbach et al. 2003; Ebesutani et al. 2010a; Nakamura et al. 2009), and extensive normative data are available for children ranging from 6 to 18 (Achenbach and Rescorla 2001). Because Achenbach and Rescorla’s ASEBA (2001) manual recommends using raw scores in order to account for the full range of variation, we conducted all analyses using raw CBCL scale scores.

Revised Child Anxiety and Depression Scales, Child and Parent Versions (RCADS/RCADS-P; Chorpita et al. 2000; Ebesutani et al. 2010b)

The RCADS and RCADS-P are each comprised of 47 items and are designed to assess for the same DSM depression and anxiety disorders in youth. The RCADS and RCADS-P are composed of six subscales: GAD, SAD, OCD, SOC, PD, and MDD. The RCADS and RCADS-P also yield Anxiety Total Scores (sum of all five anxiety scales) and Total (Anxiety and Depression) Scores (sum of all six subscales). The RCADS and RCADS-P ask youths and their parents to rate items according to how often each applies to the youth. Responses range from 0–3, corresponding to “never,” “sometimes,” “often,” and “always.”

The RCADS has been shown to have good internal consistency, convergent and discriminant validity, and a factor structure corresponding to DSM problems in both community and clinical youth samples in the US (Chorpita et al. 2000, 2005) and Australia (de Ross et al. 2002). The RCADS-P also demonstrated good internal consistency, test-retest reliability, convergent and discriminant validity, and a factor structure supporting its six-factor structure in a clinic-referred sample of 490 children and adolescents (Ebesutani et al. 2010b).

Procedure

The current study was part of a larger school-based study of negative emotions in youth, which received Institutional Review Board approval. Parents provided consent through signing and returning take-home forms to their children’s schools. The youths also provided assent in a group format at school prior to data collection. Assistance was provided if children had difficulty reading and/or filling out questionnaires. After the youths completed their questionnaires, they were asked to take corresponding parent forms home (which included the RCADS-P and CBCL), and parents were asked to complete and return the assessment forms to the University via self-addressed, stamped envelopes. Each child received a $5 gift certificate for participation.

A subset of parents (n = 94) participated in a retest of the RCADS-P over an average of 2.1 weeks (SD = 4.35; range = 9 to 46 days) to provide test-retest reliability estimates. This subset of participants consisted of the parents of 37 (39%) boys and 57 (61%) girls who returned the RCADS-P retest packet to the University. The mean age of these youths was 13.5 years (SD = 2.6, range = 8.2–17.9). RCADS-P retest packets were sent to youths’ parents until a minimum of 30 RCADS-P retest forms were obtained for each of the four analysis subgroups (i.e., boys, girls, grades 3–8, grades 9–12). Retest packets were distributed and collected until the end of the study in 2009. Although we achieved our goal of at least 30 RCADS-P retest forms per group (see Table 4), return rates were low.

Data Preparation

Missing data levels were low across the RCADS, RCADS-P and CBCL forms.³ To deal with missing data, however, we imputed missing values using the Missing Value Analysis (MVA) module of SPSS 15.0. The SPSS MVA module examines missing data patterns and imputes missing values through a maximum likelihood method based on expectation-maximization algorithms (Little and Rubin 1987). We calculated each subscale only if it had fewer than 20% of its scale items missing. We used 20% instead of 10% as the cut-off for inclusion to allow scales with low item counts (e.g., CBCL DSM-oriented Oppositional Problems scale; RCADS-P SAD scale) to have one item missing and still be calculated (cf. Ebesutani et al. 2010b; Nakamura et al. 2009). Only a few participants’ subscales were excluded from analyses due to having more than 20% items missing on a given subscale.⁴

Data Analytic Approach

The data analytic approach of the current school-based study was based largely on the data analytic approach of the recent study examining the psychometric properties of the RCADS-P in a clinical sample (Ebesutani et al. 2010b). Following examination of model fit and reliability, we performed validity tests to examine the degree to which the RCADS-P Total Score, Anxiety Total Score, and individual subscales could serve as screens for anxiety and depression, in general, as well as of the specifically targeted DSM disorders.

Model Fit

We used confirmatory factor analysis (CFA) using LISREL 8.8 to assess how well the 47 RCADS-P items fit the hypothesized six-factor structure. We examined the factor structure of the RCADS-P via CFA (as opposed to via exploratory factor analysis) as previous research found support for the hypothesized six-factor structure with the RCADS and RCADS-P (Chorpita et al. 2005; Ebesutani et al. 2010b). We also examined alternative factor structures to the hypothesized six-factor structure, including a five-factor structure (combining GAD and MDD into a single “distress” factor). We examined a five-factor structure as recent research suggested that GAD and MDD may constitute a single “distress” factor (e.g., Lahey et al. 2008; Watson 2005). We also examined whether the hypothesized six-factor structure fit better than a two-factor model of “anxiety” and “depression” (collapsing the five anxiety scales into a single anxiety factor), and a single “internalizing problems” factor.

We evaluated the fit of these factor structures via the Comparative Fit Index (CFI; Bentler 1990), Root Mean Square Error of Approximation (RMSEA; Steiger 1990) and Akaike Information Criterion (AIC; Akaike 1987). CFI values of 0.90 and above conventionally represent good model fit, RMSEA values of 0.08 or lower indicate adequate fit, and RMSEA values of 0.05 or lower indicate excellent fit. We also compared model fit between competing models (e.g., 6-factor model vs. 5-factor model) using the χ ² difference test (Bentler and Bonett 1980).

Scale Reliability and Validity

We examined the reliability of the RCADS-P scales through calculating cronbach alpha coefficients for each scale, item-total correlations with the scale for each item, and test-retest reliability coefficients for each scale.

We then examined the validity of the RCADS-P by asking sequential questions with increasing specificity pertaining to the potential utilization of the RCADS-P scales. These questions were: (a) Does the RCADS-P Total Score specifically target anxiety and depression (apart from other internalizing problems)? (b) Do the RCADS-P Total Anxiety scale and MDD scale specifically target anxiety and depression, respectively? (c) Do the individual RCADS-P depression and anxiety subscales specifically measure MDD and the targeted DSM anxiety disorders?

For the sake of interpreting results, it is important to note that the divergent validity criteria utilized in our analyses were often not orthogonal to the constructs targeted by the RCADS-P scales. For instance, although we used the CBCL DSM-oriented anxiety problems scale as a divergent validity criterion when evaluating the RCADS-P MDD as a measure of depression, anxiety and depression scales are known to be correlated with each other (Brady and Kendall 1992). As a result, we did not expect divergent validity coefficients (e.g., correlation of RCADS-P MDD scale with the CBCL DSM-oriented anxiety problems scale) to be zero. Instead, in these cases, we expected significant and positive correlations to emerge between the RCADS-P scale and the divergent criterion measure—however, we then used Fisher’s z-tests to examine whether that correlation was significantly smaller than the correlation between the RCADS-P scale and its convergent criterion measure. Lastly, due to the number of analyses conducted, we set the significance level to p < 0.001 to reduce type-1 error rates.

Results

Factorial Validity

Fit statistics from the CFA conducted on the full sample⁵ appear in Table 2 and represent adequate model fit for the six-factor model. All factor loadings were statistically significant and ranged from 0.44 to 0.60 (Separation Anxiety factor), 0.49 to 0.71 (Social Anxiety factor), 0.49 to 0.66 (Obsessive-compulsive factor), 0.49 to 0.61 (Panic factor), 0.43 to 0.85 (Generalized Anxiety factor), and 0.48 to 0.59 (Depression factor). The only exception was a Panic factor item (“When my child has a problem, he/she gets a funny feeling in his/her stomach”), which loaded weakly on the Panic factor (0.29). However, given that this item loaded significantly on the Panic-factor (0.48) in a recent psychometric study based on a clinic-referred sample of youth (Ebesutani et al. 2010b), we retained this item and tested the performance of this panic scale (with this item included) in subsequent factorial and validity analyses.

Table 2 Fit statistics for the confirmatory factor analytic models

We next tested the six-factor solution against alternative models, including a single-factor (general negative affectivity) model, and a two-factor (anxiety and depression) model. In addition, given that researchers have recently suggested that MDD and GAD items may cluster together to constitute a single “distress” factor (Lahey et al. 2008; Watson 2005), we tested the original six-factor model against a five-factor (anxiety and “distress”) model, collapsing MDD and GAD into a single “distress” factor. We presented the fit statistics for all three competing models in Table 2. All competing models evidenced significantly degraded model fit compared to the original six-factor model, providing support to the six-factor structure of the RCADS-P: χ ²_diff (15) = 2,329.82, p < 0.001, (1 “distress” factor versus 6 factors); χ ²_diff (14) = 1,849.5, p < 0.001, (2 factor “anxiety and depression” model versus 6 factors); χ ²_diff (5) = 664.1, p < 0.001 (5 factor “MDD/GAD and anxiety” model versus 6 factor model).

We also examined multi-sample CFA solutions to assess the degree to which the hypothesized 6-factor DSM-oriented model was invariant across younger (ages 8–11; n = 360) and older (ages 12–18; n = 607) youths, as well as across boys (n = 436) and girls (n = 531), with respect to factor form and other related model parameters (i.e., factor loadings, factor correlations, error variance). Results of the multi-sample solutions evidenced support for “equal form” of the 6-factor model across younger and older youth (RMSEA = 0.07; CFI = 0.93), as well as across boys and girls (RMSEA = 0.07; CFI = 0.93). The factor-correlation parameter was the only additional parameter that evidenced invariance across both younger and older youth, and across boys and girls. Specifically, allowing correlations among factors to be freely estimated across groups did not significantly improve fit compared with specifying all factor correlation pairs to be equal across younger and older groups $\left[ {\chi_{\rm{freely estimated model}}^2\left( {{2}0{38}} \right) = {6442}.{11};\chi_{\rm{constrained model}}^2\left( {{2}0{59}} \right) = {6476}.{84};\chi_{\rm{difference}}^2\left( {{21}} \right) = {34}.{72},p > 0.0{1}} \right]$, and across boys and girls groups $\left[ {\chi_{\rm{freely estimated model}}^2\left( {{2}0{38}} \right) = {6396}.{78};\chi_{\rm{constrained model}}^2\left( {{2}0{59}} \right) = {6428}.{99};\chi_{\rm{difference}}^2\left( {{21}} \right) = {32}.{21},p > 0.0{1}} \right]$, indicating that the correlations among factors are generally equal across these groups.

Reliability and Validity

Internal Consistency

Cronbach alpha coefficients, alpha-if-item-deleted values and item-total correlations for all RCADS-P scales and items appear in Table 3. All RCADS-P scales evidenced acceptable internal consistency, ranging from 0.68 (SAD scale) to 0.84 (SOC scale). Cronbach alpha coefficients for the RCADS-P Anxiety Total and Total Score were also high (α _{Anxiety Total} = 0.91; α _{Total Score} = 0.93). Although there were a few items that evidenced somewhat low (<0.32) item-total correlation values (i.e., RCADS-P items 3, 18, 33), the associated alpha values if the items were deleted were not substantially greater than the original alpha values that included the items in the scale. We thus retained all original items in the scale for subsequent analyses.

Table 3 Cronbach alpha, cronbach alpha if item deleted, and item-total correlations (N = 967)

Retest Reliability

The 2 week test-retest reliability coefficients for all RCADS-P scales appear in Table 4 and demonstrate favorable reliability for all scales based on the total sample as well as based on the sex and grade-level subgroups.

Table 4 Retest reliability correlation coefficients for the RCADS-P scales by sex and grade-level split

Anxiety and Depression

As predicted, the RCADS-P Total Score correlated significantly and positively with other internalizing-related scales, such as the CBCL Withdrawn/Depressed scale (r = 0.56), Social Problems scale (r = 0.59), and Somatic Complaints scales, (r = 0.57). However, as predicted, z-tests revealed that the correlation between the RCADS-P Total Score and the CBCL Anxious/Depressed scale (r = 0.70)—the convergent validity criterion, given its close association with anxiety and depression—was significantly greater than the largest correlation between the RCADS-P Total Score and the aforementioned divergent validity criteria (i.e., the CBCL Social Problems scale, r = 0.59), z = 5.55, p < 0.001. An identical pattern of results was also evidenced when these analyses were conducted on the Boys-only, Girls-only, Grades 3–8 and Grades 9–12 subsamples, supporting the validity of the RCADS-P Total Score as a measure of anxiety and depression.

Specific Anxiety Total and Depression Scales

We then calculated zero-order bivariate correlations of the RCADS-P Total Anxiety scale and MDD scale with convergent and divergent validity criteria (i.e., the CBCL DSM-oriented Anxiety Problems and Affective Problems scale) to examine whether the RCADS-P Total Anxiety scale and MDD scale specifically target anxiety and depression, respectively. Given that these convergent and divergent validity criteria were not orthogonal (i.e., anxiety and depression are known to be highly correlated with each other; Brady and Kendall 1992) and that the CBCL DSM-oriented Anxiety Problems and Affective Problems scale correlated at r = 0.59 in the present sample, we did not expect these divergent validity coefficients to be zero. Rather, as predicted, we found that the RCADS-P MDD scale correlated significantly and positively with the CBCL DSM-oriented Anxiety Problems scale (r = 0.50), though significantly more with the CBCL DSM-oriented Affective Problems scale (r = 0.72), as evidenced by a significant z-test, z (723) = 8.90, p < 0.001. Similarly, as predicted, RCADS-P Anxiety Total scale correlated significantly and positively with the CBCL DSM-oriented Affective Problems scale (r = 0.55), though significantly more with the CBCL DSM-oriented Anxiety Problems scale (r = 0.62), as evidenced by a significant z-test, z (723) = 2.78, p = 0.005.

We also examined whether youths with scores in the clinically elevated range on the CBCL DSM-oriented Affective and Anxiety Problems subscales scored significantly higher on the RCADS-P MDD scale and Total Anxiety scale, respectively, than youths who scored in the non-clinical ranges. As predicted, the 95 youths scoring in the clinical range on the CBCL DSM-oriented Affective Problems scale scored significantly higher on the RCADS-P MDD scale (M = 8.85, SD = 3.64) than the 872 youths scoring below the clinical range on the CBCL DSM-oriented Affective Problems scale (M = 3.39, SD = 2.88), p < 0.001. Similarly, as predicted, the 51 youths scoring in the clinical range on the CBCL DSM-oriented Anxiety Problems scale scored significantly higher on the RCADS-P anxiety total scale (M = 34.90, SD = 13.91) than the 916 youths scoring below the clinical range on the CBCL DSM-oriented Anxiety Problems scale (M = 17.25, SD = 10.20), p < 0.001. Together, these results suggest that the RCADS-P Total Anxiety scale and MDD scale specifically target anxiety and depression, respectively.

Specific Anxiety Disorders and MDD

We then computed zero-order bivariate correlations between the RCADS-P subscales and corresponding subscales of the RCADS (child version) to examine whether the RCADS-P subscales specifically target MDD and the specific DSM anxiety disorders (GAD, OCD, SOC, PD and SAD). Although we predicted that all correlations with these convergent validity criteria (i.e., RCADS subscales) would be positive and significant, we expected that these correlations would only be moderate in size. This expectation was based on the cross-informant nature of these analyses (i.e., parent vs. child reports), and the robust finding in meta-analyses that children and parents evidence only low to moderate agreement on emotional and behavioral problems (e.g., correlation coefficients ranging from 0.25 to 0.29; Achenbach et al. 1987; Meyer et al. 2001). The correlation between corresponding RCADS-P and RCADS subscales based on the total sample as well as on the Boys-only, Girls-only, Grades 3–8 and Grades 9–12 subsamples appear in Table 5. As predicted, all correlations based on the full sample were significant and in the moderate range. A few non-significant correlations did emerge, primarily among the Boys-only subsample.

Table 5 Parent-child agreement for the corresponding RCADS and RCADS-P scales

Normative Data

To enhance the utility of the RCADS-P for the purposes of referencing particular youth scores to a community sample, we presented normative data (including the means and standard deviations of the RCADS-P scales calculated by sex and grade) in Table 6. To examine for any differences in RCADS-P scale scores across sex and grade level, we conducted a 2 × 5 (sex by grade levels) analysis of variance (ANOVA) for each RCADS-P subscale. Results revealed no significant interaction or main effect for sex or grade-level across all subscales, with two exceptions. Specifically, a significant main effect for grade-level emerged for the SAD scale (F = 43.87, p < 0.001), whereby younger youths scored higher on separation anxiety than older youths. A significant main effect for grade-level also emerged for the MDD scale (F = 4.56, p = 0.001), whereby older youths scored higher on depression than younger youths. These findings are consistent with the notions that separation anxiety disorder is more common among preadolescent children (Last et al. 1992) and that rates of depression increase from childhood to adolescence (Fleming et al. 1989).

Table 6 Ranges, means, and standard deviations for the RCADS-P subscales by sex and grade

Discussion

The RCADS-P demonstrated favorable psychometric properties in the present sample of school-based children and adolescents. Specifically, CFA results supported the six-factor RCADS-P model, which fit equally well across boys and girls as well as across younger and older youth. Consistent with findings from a recent RCADS-P psychometric study in a clinical sample (Ebesutani et al. 2010b), the present CFA results did not support combining the MDD and GAD scales into a single “distress” factor. These results, however, are not consistent with recent findings that support collapsing MDD and GAD into a single construct (e.g., Lahey et al. 2008; Watson 2005). Higa-McMillan et al. (2008), for instance, found that GAD in children appears to have a stronger relationship to depression than to social phobia. More research is thus needed to determine whether MDD and GAD indeed constitute the same “distress” construct in youth, or whether, for example, these discrepant findings are due in part to differences between youth and parent reporting styles and/or due to differences in measurement strategies. The present findings, for instance, were based on parent reports on the RCADS-P (a self-report measure), while Higa-McMillan et al. (2008) findings were based on child reports on the ADIS-IV-C (a clinician guided, semi-structured interview). Future studies controlling for these differences may help clarify this issue related to the GAD and MDD distinction.

Although the previous RCADS-P psychometric study (Ebesutani et al. 2010b) did not identify any significant problems with scale items, the present investigation evidenced a relatively low factor loading of one Panic item (“When my child has a problem, he/she gets a funny feeling in his/her stomach”). Two other items on the SAD scale also evidence somewhat low item-total correlation coefficients [“My child has trouble going to school in the mornings because of feeling nervous or afraid”; “My child is afraid of being in crowded places (like shopping centers, the movies, buses, busy playgrounds)”]. Although inclusion of these items did not substantively degrade the psychometric performance of these scales in the present validity tests, future studies should give particular attention to these items and continue to examine how they may affect measurement precision.

Regarding the reliability of the RCADS-P scales over time, all scales evidenced satisfactory test-retest correlation coefficients. However, the Panic subscale was associated with the lowest test-retest reliability coefficients. This may be due to the episodic and transient nature of panic symptoms (e.g., rapid heart beating, shortness of breath), which may fluctuate and/or change over short periods of time. At the same time, the return rate of retest packets from parents was particularly low, which may have compromised the representativeness of the current sample. Consequently, while the RCADS-P scales appear to provide reliable estimates of MDD and the targeted anxiety disorders, additional research may focus on replicating these finding as well as addressing appropriate assessment intervals for panic symptoms.

The RCADS-P also demonstrated high convergent and divergent validity across all scales. The RCADS (child-report) was utilized as the convergent validity criterion when examining the RCADS-P subscales as both measures target the same DSM-oriented MDD and anxiety problems. However, as child and parent reports are known to only moderately correlate with each other, inclusion of another parent-based measure that targets the same DSM related problem areas would have allowed for a better convergent validity test. Although we did utilized the parent-report CBCL DSM-oriented Affective Problems scale to evaluate the performance of the RCADS-P MDD subscale (given that this parent-reported scale was developed to target MDD and dysthymic disorder; Achenbach and Rescorla 2001), the CBCL’s DSM-oriented Anxiety Problems subscale does not target anxiety disorders at the same level of specificity as the RCADS-P (i.e., the CBCL’s DSM-oriented Anxiety Problems subscale was designed to target the cluster of GAD, SAD and specific phobia; Achenbach and Rescorla 2001). Future studies should thus consider examining the degree of convergence between the RCADS-P subscales and other parent-report measures that target comparable DSM-oriented depression and anxiety subscales.

Although the results of the present study support the psychometric properties of the RCADS-P and the utility of this measure as a useful screen for identifying children and adolescents with depressive and anxiety problems in school settings, there were particular limitations as well as areas for future research and development worth noting. Although the present study was based on a large, ethnically diverse sample, including youths from under-researched populations (e.g., Pacific Islanders), this sample was based solely on a Hawaii youth population and did not include large numbers of specific minority populations that are more represented in several continental US regions (i.e., African American, Hispanic youth). This may pose a limitation to the generalizability of the present findings. Further, there was also a low response rate of parents consenting to participate in the current study, which also contributed to low return rates of our test-retest sample. Although low parent form return rates are typical for parent-based research in school settings (cf. Higa et al. 2006), we compared the demographic data of the youths and families of our sample to the most recent demographic data provided by the U.S. Census Bureau for the Honolulu County - the county of the schools surveyed in the present study—in order to assess how well our sample is representative of students and families of the general Honolulu County population. Based on the most recent and available U.S. Census Bureau data (U.S. Census Bureau 2010), the median household income for the Honolulu County in 2008 was $70,010, and the three most represented ethnicities in 2009 in this county were Asian American (43.9%), White (26.6%) and Multiethnic (16.2%). The median income from the Honolulu County ($70,010) fell in the median income range of our sample ($60,000–$89,000); further, Asian American, White and Multiethnic were also the three most represented ethnicities in our sample. It is notable, however, that there was a smaller percentage of White youths in our sample (4.4%) compared to the percentage of “White persons” reported living in the Honolulu County in 2009 (26.6%). Although our sample nonetheless appears somewhat representative of our targeted population, additional research appears needed with larger and more inclusive samples of youths and families from an increased variety of regions, ethnicities and backgrounds to better understand the generalizability of the present findings. Notably, results of the present study did not identify differences in RCADS-P depression scores between boys and girls, despite girls typically evidencing more depressive symptoms than boys (e.g., Glambos et al. 2004). The degree to which this finding and others inconsistent with previous research is due to characteristics specific to this sample deserves future attention. Relatedly, it is also possible that parent reports of their children’s internalizing experiences are limited in certain ways (given that parents do not have direct insight into their children’s internal states), and this should also be considered when further examining the RCADS-P. Exemplifying this point, Weems et al. (2005) recently found that child report—but not parent report—on the RCADS was significantly related to fear induced physiological response. The degree to which parent reports on the RCADS-P are (or are not) related to such internal states of children warrants future attention.

Interestingly, parent-child agreement conducted on Boys-only and Girls-only subsamples revealed that agreement fell in the expected (moderate) range for the majority of analyses. However, a closer investigation of the data provides additional insight into the complex nature of parent-child (dis)agreement research with regard to age, gender, and type of symptoms. For example, consistent with previous findings on parent-child agreement specific to anxiety subtypes, such as on the SCARED anxiety questionnaire (Birmaher et al. 1997) and on the RCADS-P in a clinical sample (Ebesutani et al. 2010b), parent-child agreement was greatest for SAD. On the other hand, parent-child agreement was smallest for GAD (based on the total sample), demonstrating that the nature of parent-child agreement differs by anxiety type.

With regard to youth sex, parent-child agreement based on the Boys-only subsample fell in the very low range (near zero correlation) for the GAD, panic and social anxiety subscales. De Los Reyes and Kazdin (2005) recently highlighted the complexity of discrepancies between child and parent reports. For example, these authors suggest that informant characteristics, such as youth sex, may affect parent-child agreement, but that the evidence regarding this relationship has been mixed. Some studies found youth sex to be significantly related to parent-child agreement (Grills and Ollendick 2003; Verhulst and van der Ende 1992), whereas other studies have not supported this finding (Choudhury et al. 2003; Christensen et al. 1992). The present results add to this debate by supporting the notion that parent-child agreement may vary somewhat as a function of youth sex (i.e., greater cross-informant agreement among girls). Future research should incorporate these reporter qualities when further evaluating the RCADS-P. Specifically, a related implication worthy of further exploration is that the integration of child and parent reports on the RCADS may thus not be able to rely on a simple additive approach.

Despite the noted limitations and areas for future research, the present study broadened the psychometric support for the RCADS-P to a wider population of school-based youth, and also provided normative data to allow for identifying youths who are clinically-elevated in the targeted areas of depression and anxiety problems. The current findings also provided insight into a variety of theoretical implications regarding the assessment of psychopathology (i.e., models of psychopathology, sampling characteristics, reported agreement). Continued research will benefit from examinations of both the RCADS and RCADS-P in a greater diversity of contexts (e.g., other communities, other languages), investigation of possible adaptations for purposes of briefer assessment models (e.g., Chorpita et al. 2010), and evaluation of the suitability of the measurement structure in light of the ongoing evolution of the clinical and psychiatric nosology (e.g., DSM-V).