Erratum to: J Psychopathol Behav Assess

DOI 10.1007/s10862-010-9208-3

It is proposed within prominent models of social phobia that cognitions play a major role in the persistence of the disorder (Clark and Wells 1995; Rapee and Heimberg 1997). In particular, Clark and Wells (1995) posit that at the core of social phobia, there are three types of persistent maladaptive self-beliefs that lead an individual with the disorder to perceive danger in social and performance situations, and subsequently experience anxiety. Given the centrality of these beliefs in the Clark and Wells (1995) model, it is surprising that there is no validated measure of these beliefs in the social anxiety literature.

According to the Clark and Wells (1995) model, the three maladaptive self-belief types that are held by individuals with social phobia are: (a) excessively high standards for social performance (e.g., “I must be able to convey a favourable impression to everyone”), (b) conditional beliefs concerning social evaluation (e.g., “If people see I’m anxious, they’ll think that I’m weak”), and (c) unconditional beliefs about the self (e.g., “People think badly of me”). In addition to these beliefs, Clark and Wells (1995) proposed that individuals with social phobia engage in several maladaptive cognitive and behavioural processes (i.e., self-focused attention, within-situation safety behaviours, anxiety-induced performance deficits, anticipatory processing, post-event rumination) that ultimately prevent the disconfirmation of the maladaptive self-beliefs. As a consequence, the beliefs persist, and individuals with social phobia continue to experience anxiety in social-evaluative situations.

To assess cognitions related to social anxiety, previous studies have utilized various methods including checklists, thought listing, and questionnaires (see Heimberg 1994, for a review). In particular, questionnaires have been developed to examine cognitions related to appraisals of performance or interpretations of social-evaluative situations (see Clark and McManus 2002; Hirsch and Clark 2004, for reviews). For example, several questionnaires assess cognitions related to speech performance. Hofmann and DiBartolo (2000) developed the Self-Statements During Public Speaking Scale (e.g., “What I say will probably sound stupid”) using undergraduate samples and a socially phobic clinical sample. In one of the undergraduate samples, negative self-statements were found to be positively correlated with social anxiety measures (rs ranged from .37 to .49) while positive statements were negatively correlated with social anxiety measures (rs ranged from −.29 to −.34). Along similar lines, Rapee and Lim (1992) developed the Performance Questionnaire, and used it in their study to allow participants (i.e., individuals with social phobia and non-clinical controls) to appraise their own public speaking performance (e.g., “had a clear voice”, “appeared nervous”). Using the Performance Questionnaire, Abbott and Rapee (2004) found that individuals with social phobia maintained their negative appraisals of their speech task performance over one week whereas non-anxious individuals showed an increase in positivity about their performance.

Complementing studies that have examined cognitions related to public speaking, others have investigated cognitions associated with social interactions using a questionnaire approach. For example, Alden and Wallace (1995) had individuals with social phobia engage in a dyadic social interaction and then asked observers as well as the social phobic individual to rate their performance (e.g., “To what extent did you display facial signs of anxiety, such as blushing?”). Compared to observers, individuals with social phobia endorsed more negative appraisals when asked to judge their performance in the social interaction. In addition, Glass et al. (1982) employed undergraduate samples to develop the Social Interaction Self-Statement Test (SISST), which assesses thoughts related to a social interaction (e.g., “If I blow this conversation, I’ll really lose my confidence”). In a subsequent study, Dodge et al. (1988) reported that the SISST negative statements subscale was positively associated with measures of social anxiety (rs = .60 and .62), while the SISST positive statements subscale was negatively associated with measures of social anxiety (rs = −.57 and −.58). More recently, Beazley et al. (2001) used the SISST (and a modified SISST) to examine the thoughts of individuals with social phobia across three different social-evaluative situations (i.e., impromptu speech, conversation with a same-sex confederate, conversation with a different-sex confederate). They found that individuals with social phobia reported more negative thoughts and fewer positive thoughts during the speech task compared to the two conversation conditions.

While the questionnaires described have attempted to measure transitory, state-like thoughts related to social anxiety, other questionnaires have been developed to assess more stable, trait-like beliefs. Using a clinical sample (individuals with social phobia, individuals with other anxiety disorders), Turner et al. (2003) developed the Social Thoughts and Beliefs Scale (STABS) to assess core beliefs that are characteristic of social phobia (e.g., “Other people are more socially capable than I am”). Interestingly, Turner et al. (2003) found that the STABS is composed of two factors; the first factor is “Social Comparison” and contains beliefs that other people are more socially competent, and the second factor is “Social Ineptness” and contains beliefs related to behaving awkwardly or appearing anxious in social situations. In an undergraduate sample, Fergus et al. (2009) confirmed the Social Comparison and Social Ineptness factors of the STABS, and both factors (along with the total STABS score) were positively associated with measures of social anxiety (rs ranged from .63 to .80). Besides the STABS, Rodebaugh (2009) has recently developed the Core Extrusion Schema (CES) measure that assesses the tendency for high socially anxious individuals to conceal aspects of the self due to fear associated with perceived negative evaluation. The CES measure includes four subscales (Rejection of True Self, Hidden Self, Avoid Mistakes, Present Rejection), and each demonstrated positive correlations with measures of social anxiety (rs ranged from .28 to .57) in an undergraduate sample (Rodebaugh 2009). In particular, the Rejection of True Self subscale (e.g., “Most people don’t really like the kind of person I really am”) taps beliefs about being rejected by others if one was to reveal their true self, and there are some items of this scale that have similar content to items of the STABS.

It thus appears that several types of cognitions related to social anxiety have been assessed in studies within the social anxiety literature. Notably, no measure has explicitly been developed to assess all three of the core maladaptive self-belief types as defined by Clark and Wells (1995). Recently, however, Wong and Moulds (2009) presented the Self-Beliefs Related to Social Anxiety (SBSA) Scale. The SBSA is a new measure that is the first to index the three maladaptive self-belief types proposed in Clark and Wells’ (1995) model. Although it is a new measure, the SBSA does have some similarities with other previous measures (e.g., the STABS) in that it measures stable, trait-like beliefs. Wong and Moulds (2009) originally developed the SBSA to investigate the relationship between post-event rumination and the maladaptive self-beliefs. They found that high socially anxious individuals who participated in a social-evaluative situation and were then instructed to engage in post-event rumination demonstrated a persistence of unconditional beliefs (but not high standard or conditional beliefs) relative to those who distracted following the social-evaluative situation. More recently, Wong and Moulds (2010a) demonstrated that prior to a social-evaluative event, high socially anxious individuals who engaged in anticipatory ruminative processing reported greater endorsement of high standard and conditional beliefs (but not unconditional beliefs) compared to those who distracted. Taking the Wong and Moulds (2009, 2010a) studies together, the differential effect of ruminative processing (both anticipatory and post-event) on the maladaptive self-belief types provide some evidence that the beliefs as measured by the SBSA are distinct constructs. However, Wong and Moulds (2009) only reported preliminary psychometric properties of the SBSA. Specifically, the SBSA and its subscales had excellent internal consistency (Cronbach’s α = .94 for the SBSA; Cronbach’s α = .85 for 4-item high standard beliefs subscale; Cronbach’s α = .91 for 7-item conditional beliefs subscale; Cronbach’s α = .82 for 4-item unconditional beliefs subscale), and satisfactory item-total correlations (.72 to .88 for the high standard beliefs subscale, .73 to .89 for the conditional beliefs subscale, and .79 to .85 for the unconditional beliefs subscale). Wong and Moulds (2009) also reported that the high standard and conditional beliefs had stronger positive associations with fear of negative evaluation than did unconditional beliefs. In addition, conditional and unconditional beliefs had stronger positive associations with depression than high standard beliefs.

Although the SBSA is a measure with potential utility to test the Clark and Wells (1995) model, only the preliminary psychometric properties of this measure outlined above have been reported to date (see Wong and Moulds 2009, 2010a). We thus aimed to elucidate more detailed psychometric properties of the SBSA (factor structure, internal consistency, test-retest reliability, convergent and divergent validity). There were two aspects of our investigation that were of particular interest. First, we sought to examine whether the SBSA had a three-factor structure, consistent with the Clark and Wells (1995) model. Given a priori theory indicating a three-factor model, confirmatory factor analysis (CFA) would be used (Fabrigar et al. 1999). Related to this, we sought to investigate more parsimonious conceptualizations of the maladaptive self-belief types by testing whether the beliefs in fact tap two separate but related constructs (i.e., two-factor model) or whether they tap a single unitary construct (i.e., one-factor model). We also examined the possibility of a more complex four-factor model. Although it is important to test these alternative models in order to provide a more stringent test of the Clark and Wells (1995) model (see Wegener and Fabrigar 2000), there are no other theories on the maladaptive self-beliefs that exist in the current literature to provide more details about these alternative models (e.g., how items would load in the two- and four-factor models). To explore these models further then, before our planned CFA, we planned to use exploratory factor analysis (EFA; Fabrigar et al. 1999). The second issue of particular interest related to validity. Specifically, we sought to examine the validity of the SBSA by investigating its relationships with measures of social anxiety, general anxiety, and depression. In addition, given that previous theory (Clark and Wells 1995) and empirical research (e.g., Wong and Moulds 2009, 2010a) have suggested that the specific maladaptive self-belief types are distinct constructs, we sought to examine the SBSA subscales and whether they would be differentially related to other measures of interest (e.g., a measure of social anxiety). In general, given that this area of research is in its infancy, we investigated the psychometric properties of the SBSA using an analogue research strategy. As social anxiety is considered to be continuously distributed in the general population (Stopa and Clark 2001), an examination of the SBSA employing individuals drawn from a non-clinical population would allow us to identify and measure the beliefs that are likely to also occur in social phobia. Previous research has shown that studies employing clinical samples (i.e., individuals with social phobia) and studies using non-clinical samples (i.e., high socially anxious individuals) yield highly similar findings (see Stopa and Clark 2001).

Our hypotheses were as follows. First, based on the Clark and Wells (1995) model, we predicted that factor analysis of the SBSA (both EFA and CFA) would indicate that a correlated three-factor model would provide the best fit to the data (i.e., items 1, 7, 8, 11 loading on the high standard beliefs factor, items 2, 5, 6, 9, 12, 13, 15 loading on the conditional beliefs factor, items 3, 4, 10, 14 loading on the unconditional beliefs factor) compared to competing four-, two-, and one-factor models. Second, we predicted that all SBSA items would similarly tap maladaptive self-beliefs related to social anxiety (i.e., adequate internal consistency). Related to this, we also predicted that each SBSA subscale would have adequate internal consistency. Third, given the enduring nature of the maladaptive self-beliefs (Clark and Wells 1995), we predicted that the beliefs would be stable over time (i.e., adequate test-retest reliability). Fourth, we predicted that the SBSA would be positively associated with measures of social anxiety (i.e., convergent validity), and that the magnitude of this association would be greater than the association between: (a) the SBSA and measures of general anxiety, and (b) the SBSA and measures of depression. We predicted that there would be a similar pattern of results for the SBSA subscales; specifically, that there would be positive associations between the SBSA subscales and measures of social anxiety, with these associations being stronger than associations between the SBSA subscales and measures of general anxiety, as well as measures of depression. Finally, based on Wong and Moulds (2009), we predicted that: (a) the high standard and conditional beliefs would have stronger positive associations with social anxiety than the unconditional beliefs, and (b) the conditional and unconditional beliefs would have stronger positive associations with depression than the high standard beliefs.

Method

Participants

The full sample of participants were 600 undergraduate psychology students (369 females; mean age = 20.52, SD = 4.10) at The University of New South Wales who participated for course credit. SBSA data collected from the full sample was used for the analysis of factor structure and internal consistency as part of the first phase of the study. Past the midway point of participant recruitment, the second phase of the study was commenced and it was originally planned that the last 275 participants of the 600 participants would also be given a second administration of the SBSA and other questionnaires so that test-retest reliability and validity could be examined. However, practical limitations of testing using an undergraduate sample (e.g., availability of students, attrition) decreased the subsample size. Thus, we had a subsample of 235 undergraduate psychology students (148 females; mean age = 20.45, SD = 3.40) to analyze the test-retest reliability of the SBSA. Importantly, there was no difference in the SBSA total score or the belief subscales scores (all ps > .05) between those participants who were included in the test-retest reliability analysis and those participants who were excluded from this analysis (n = 40) because they did not complete the second SBSA (85% retention). Hence, the exclusion of participants is unlikely to have biased the final subsample. We also had a subsample of 223 undergraduate psychology students (136 females; mean age = 20.64, SD = 3.50) to examine the validity of the SBSA and its subscales. Again, there was no difference in the SBSA total score or the belief subscales scores (all ps > .05) between those participants who were included in the validity analysis and those participants who were excluded from this analysis (n = 52) because they did not complete social anxiety, general anxiety, and depression measures (81% retention). The exclusion of participants is unlikely to have biased the final subsample.

Materials

Self-Beliefs Related to Social Anxiety Scale (SBSA; Wong and Moulds 2009). The SBSA is a 15-item self-report questionnaire that assesses the strength of beliefs about the self in a social context. Consistent with Clark and Wells (1995), it includes items that tap three types of beliefs: (a) 4 items that tap excessively high standards for social performance (items 1, 7, 8, 11), (b) 7 items that tap conditional beliefs concerning social evaluation (items 2, 5, 6, 9, 12, 13, 15), and (c) 4 items that tap unconditional beliefs about the self (items 3, 4, 10, 14; see Figure 1). Participants were asked to rate the extent to which they agree with each belief at the moment when they were administered the questionnaire on an 11-point Likert scale (0 = do not agree at all, 10 = strongly agree). Importantly, participants were not asked to think about a social-evaluative situation or relate the items to a social interaction at the time when they rated the items. SBSA subscale total scores were obtained by summing the item scores within each subscale, and the SBSA total score was generated by summing the subscale total scores. As described in Wong and Moulds (2009), the 15 SBSA items were selected from an initial pool of 49 items that were developed based on the clinical case series presented in Clark and Wells (1995). More specifically, the 15 items were selected because they had the strongest positive associations with fear of negative evaluation after controlling for depression (Wong and Moulds 2009).

Fig. 1
figure 1

Correlated three-factor solution for the SBSA. Note. HS = High Standard beliefs; UNCOND = Unconditional beliefs; COND = Conditional beliefs

Full size image

Fear of Negative Evaluation Scale (FNE; Watson and Friend 1969). The FNE is a 30-item self-report measure of fear of negative evaluation, a construct that is closely related to social anxiety (Weeks et al. 2005). Participants selected a true or false response for each item (e.g., “I am afraid that people will find fault with me”). The FNE has good internal consistency (Cronbach’s α = .90), and is widely used to determine the level of social anxiety of individuals from non-clinical populations (Stopa and Clark 2001).

Social Phobia Scale (SPS; Mattick and Clarke 1998). The SPS is a 20-item questionnaire that measures fear associated with evaluation by other people during routine activities (e.g., eating). Participants rated the items (e.g., “I can feel conspicuous standing in a queue”) on a 5-point Likert scale (0 = Not at all true of me to 4 = Extremely true of me). The SPS has good internal consistency (Cronbach’s α = .89) and validity (Mattick and Clarke 1998).

Social Interaction Anxiety Scale (SIAS; Mattick and Clarke 1998).The SIAS is a 19-item questionnaire that assesses fears associated with social interactions. Participants rated the items (e.g., “I become tense if I have to talk about myself or my feelings”) on a 4-point Likert scale (0 = Not at all true of me to 4 = Extremely true of me). The SIAS has good internal consistency (Cronbach’s α = .93) and validity (Mattick and Clarke 1998), although more recent studies on the SIAS suggest that removing the reverse-scored items strengthens the validity of the scale (e.g., Rodebaugh et al. 2007). We decided to report the full SIAS because analyses revealed that the full SIAS and the SIAS with reverse-scored items removed were strongly correlated (r = .99, p < .01). Additionally, the correlations between the full SIAS and the FNE (r = .56, p < .01), SPS (r = .73, p < .01), and the DASS depression subscale (r = .25, p < .01), were very similar (if not identical) to the correlations between the SIAS with the reverse-scored items removed and the FNE (r = .56, p < .01), SPS (r = .74, p < .01), and the DASS depression subscale (r = .26, p < .01).

Depression Anxiety Stress Scales (DASS; Lovibond and Lovibond 1995). The DASS (short version) is a 21-item self-report questionnaire that assesses the severity of depression, stress, and anxiety over the past week. Each of the subscales have good psychometric properties (7-item depression subscale: Cronbach’s α = .92, 7-item anxiety subscale: Cronbach’s α = .83, 7-item stress subscale: Cronbach’s α = .89). Each subscale total score was doubled to obtain the full DASS score equivalent (Lovibond and Lovibond 1995). Only the depression and anxiety subscales were used in analyses.

Beck Anxiety Inventory (BAI; Beck et al. 1988). The BAI is a 21-item self-report inventory that measures the severity of anxiety. The BAI has good psychometric properties (Cronbach’s α = .85) and validity (Beck et al. 1988).

Beck Depression Inventory—Second Edition (BDI-II; Beck et al. 1996). The BDI-II is a 21-item self-report inventory that assesses the symptoms of depression. It has good psychometric properties (Cronbach’s α = .90) and validity (Dozois et al. 1998).

Procedure

After participants provided informed consent, they completed the SBSA as part of a set of measures that assessed mood and cognitions (i.e., FNE, DASS) before being entered into an experimental protocol. The measures were administered in a randomized order. Participants in the subsample used to examine validity were given additional measures (i.e., SPS, SIAS, BAI, BDI-II) that were administered in a randomized order after completion of the experimental protocol. Participants in the subsample used to examine test-retest reliability were administered the SBSA a second time within one to four weeks of the first administration (M = 9.15 days, SD = 6.12).

Results

Factor Structure

To achieve our goal of performing both EFA and CFA, we randomly split our sample (N = 600) into two subsamples (each with n = 300; Fabrigar et al. 1999). The first subsample was used to conduct the EFA, and the second subsample was used to conduct the CFA. We employed LISREL 8.80 (Jöreskog and Sörbom 2009) to conduct our analyses.

Exploratory Factor Analysis Using the First Subsample

EFA was first conducted without placing any constraints on the number of factors in order to determine the factor structure and pattern of item loadings from the SBSA data in an unrestricted manner (Brown 2006; Fabrigar et al. 1999; Tinsley and Tinsley 1987). Depending on the results of the first EFA, we then planned to conduct an EFA with specification of two-factors, followed by an EFA with specification of four factors, in order to determine the pattern of item loadings for these two alternative models. The single-factor model was not subjected to an EFA because the pattern of item loadings was already known (i.e., items could only load on one factor).

In the first subsample, the SBSA items exhibited minimal skew (range = −.28 to 1.50, M = .83) and kurtosis (range = −1.01 to 1.89, M = .12). Hence, it was appropriate to conduct the EFAs with maximum likelihood extraction. Rotation of the solution was permitted to assist in the interpretation of factors and promax rotation was specifically employed so that if multiple factors emerged, they would be allowed to correlate. A factor loading cut-off of .4 was used.

The unconstrained EFA revealed an easily interpretable correlated two-factor model with all conditional and unconditional belief items loading on Factor 1 (loadings ranged from .44 to .77), and all high standard belief items loading on Factor 2 (loadings ranged from .54 to .99).

We then proceeded to EFA but with specification of the number of factors. Since the unconstrained EFA already revealed a two-factor model, we then conducted an EFA with the specification of four factors. The analysis showed three of the four factors had only two items loading on them. This suggested that the factors were unstable, would be difficult to interpret, and unlikely to be theoretically meaningful (Costello and Osborne 2005; Tinsley and Tinsley 1987).

Based on the EFA results and our research aims, we decided that the unstable four-factor model would not be tested any further. However, we decided to proceed with CFA and examine the three-factor model (derived from Clark and Wells 1995), the two-factor model (derived from the EFA), and the one-factor model in the second subsample.

Confirmatory Factor Analysis Using the Second Subsample

In the second subsample, although the SBSA items exhibited minimal skew (range = −.21 to 1.60, M = .89) and kurtosis (range = −1.05 to 2.15, M = .24) across items (average skew and kurtosis close to zero), certain items exhibited elevated skewness and kurtosis (e.g., Item 2, skewness = 1.60 and kurtosis = 2.15). To guard against such departures from distributional normality, we conducted the CFA with robust maximum likelihood extraction (Fabrigar et al. 1999).

In order to select the optimal model from the CFA, we used the Satorra-Bentler Scaled Chi-Square (SBS χ 2) and a range of other fit indices. In relation to the SBS χ 2, Fabrigar et al. (1999) have noted that for fit statistics based on the χ 2 distribution, when N is large, trivial discrepancies between the model and the data are likely to lead to the rejection of the model even though it may have a reasonable number of factors. Furthermore, Fabrigar et al. (1999) have noted that fit statistics based on the χ 2 distribution test the null hypothesis that the model fits the data perfectly, which is an unrealistic standard for models to reach. Hence, instead of rejecting models based on significant χ 2s, we followed Jöreskog and Sörbom’s (1982) recommendation to evaluate models using smaller χ 2 values to indicate better model fit. In addition to the SBS χ 2, the following fit indices were used (Brown 2006; Fabrigar et al. 1999; Gerbing and Anderson 1993): the comparative fit index (CFI; this value should be .90 or higher for a good fit; the higher this value, the better the fit), the non-normed fit index (NNFI; this value should be 0.90 or higher for a good fit; the higher this value, the better the fit), the root mean square error of approximation (RMSEA; this value should be .08 or lower; the lower the value, the better the fit), the standard root mean square residual (SRMR; this value should be .08 or lower; the lower the value, the better the fit), and the Akaike information criterion (AIC; the smaller the value, the better the fit and the more likely the model is to cross-validate). We also conducted scaled difference in χ 2s tests (SDCS; Satorra and Bentler 1994) in order to test the difference in fit between the three-, two-, and one-factor models.

The three-factor solution overall provided satisfactory fit (SBS χ 2 (87) = 270.53, p < .01, CFI = .95, NNFI = .93, RMSEA = .08, SRMR = .06, AIC = 336.53). The competing and simpler two-factor model (SBS χ 2 (89) = 310.49, p < .01, CFI = .94, NNFI = .93, RMSEA = .09, SRMR = .07, AIC = 372.49) that was derived from the previous EFA did not provide a better fit compared to the three-factor solution. Indeed, the scaled difference in χ 2s test indicated that the correlated three-factor model was a significantly better fit to the data compared to the correlated two-factor model (SDCS χ 2 (2) = 39.33, p < .01). Similarly, the one-factor model with the 15 SBSA items serving as indicators for one latent factor was tested (SBS χ 2 (90) = 454.20, p < .01, CFI = .91, NNFI = .90, RMSEA = .12, SRMR = .09, AIC = 514.20) and did not provide a better fit compared to the three-factor model according to fit indices. Moreover, the scaled difference in χ 2s test indicated that the correlated three-factor model was a significantly better fit to the data compared to the one-factor model (SDCS χ 2 (3) = 150.87, p < .01). The results of the CFA thus indicated that the correlated three-factor model provided the best fit to the data relative to the other competing models. Although this is consistent with the Clark and Wells (1995) model, the CFA results contrast with the previous unconstrained EFA that supported a correlated two-factor model. Nonetheless, the CFA provided a direct comparison of the correlated two-factor model (derived from the EFA) and the correlated three-factor model (derived from Clark and Wells’ theory), and showed the superiority of the three-factor model. In addition to this, the three-factor model has the advantage of being grounded in theory (Clark and Wells 1995), while the two-factor model is predominantly data-driven (i.e., because it was derived from the EFA). Hence, it appears overall, the correlated three-factor model is the best solution. Figure 1 shows the standardized factor loadings for the correlated three-factor model. All factor loadings were significant (ps < .01).

Interestingly, we also tested a correlated three-factor model but with correlated error variances included (e.g., correlated error variances between the conditional beliefs, and correlated error variances between the unconditional beliefs). Correlated error variances represent non-random measurement error (i.e., method effects), and are justified on the basis that conditional beliefs have similar wording, and unconditional beliefs also have similar wording (see Brown 2006). This model (SBS χ 2 (62) = 206.70, p < .01, CFI = .98, NNFI = .97, RMSEA = .09, SRMR = .06, AIC = 322.70) was generally better fitting than the correlated three-factor model without correlated error variances. However, it seems that ultimately a three-factor solution provides the best fit to the data.

Internal Consistency

The SBSA overall had high internal consistency (Cronbach’s α = .92, item-total correlations ranged from .58 to .79). The SBSA also had good internal consistency within each of the belief subscales (4-item high standard beliefs subscale, Cronbach’s α = .83, item-total correlations ranged from .64 to .76; 7-item conditional beliefs subscale, Cronbach’s α = .89, item-total correlations ranged from .69 to .79; 4-item unconditional beliefs subscale, Cronbach’s α = .80, item-total correlations ranged from .58 to .74). Within each of the subscales, Cronbach’s α decreased if any of the items were deleted.

Test-Retest Reliability

The SBSA had good test-retest reliability (r = .82, p < .01). The SBSA subscales also had good test-retest reliability (high standard beliefs: r = .73, p < .01, conditional beliefs: r = .78, p < .01, unconditional beliefs: r = .72, p < .01).

Convergent and Divergent Validity

Table 1 shows the means and standard deviations for the SBSA, anxiety, and mood measures, as well as the correlations between these measures. We were mainly interested in the SBSA, its subscales, and their relationship with social anxiety, general anxiety, and depression. Hence, in order to simplify the analyses, and obtain a reliable estimate of participants’ level of social anxiety, general anxiety, and depression, we created composite scores (see Zeller and Carmines 1980). For example, a social anxiety composite measure was formed by: (a) converting FNE, SPS, and SIAS raw scores to z-scores, and (b) averaging the z-scores for these three social anxiety measures for each participant to form the social anxiety composite measure. This process was repeated using the BAI and DASS Anxiety scores to form the general anxiety composite measure, and the BDI-II and DASS Depression scores to form the depression composite measure. This method of dealing with multiple measures has been used in previous social anxiety research (see Clark et al. 2003, 2006). Each of the composite measures had good internal consistency (social anxiety, Cronbach’s α = .95; general anxiety, Cronbach’s α = .88; depression, Cronbach’s α = .93). Following Cohen and Cohen (1983), we conducted multiple regression analyses to examine the relationships between constructs of interest. We also conducted tests of significance for differences between β coefficients (Cohen and Cohen 1983).

Table 1 Means, standard deviations, and correlations for the measures of social anxiety, depression, anxiety, stress, and maladaptive self-beliefs (N = 223)
Full size table

To examine the SBSA and its unique relationships with social anxiety, general anxiety, and depression, we conducted a regression analysis with the SBSA total as the dependent variable, and the social anxiety composite, general anxiety composite, and depression composite entered simultaneously as the independent variables. As evidence of good convergent validity, the SBSA was found to be strongly (and uniquely) associated with the social anxiety composite, β = .61, t(218) = 11.77, p < .01, weakly (and uniquely) associated with the depression composite, β = .25, t(218) = 3.68, p < .01, and not related to the general anxiety composite, β = .00, t(218) = .04, p = .97. Indeed, social anxiety was a significantly stronger predictor of SBSA scores than depression, t(219) = 4.20, p < .01, and general anxiety, t(219) = 3.72, p < .01. Additionally, the difference between depression and general anxiety as predictors of SBSA scores only approached significance, t(219) = 1.91, p = .06. The three predictors accounted for 54.3% of the variance in SBSA scores.

We then examined the unique relationships between each of the SBSA subscales and the composite measures. A regression analysis was conducted with the high standard beliefs as the dependent variable and the composite measures (social anxiety, depression, and general anxiety) entered simultaneously as the independent variables. Only the social anxiety composite was significantly associated with the high standard beliefs, β = .54, t(218) = 8.42, p < .01. Nonetheless, the three predictors accounted for 30.7% of the variance in the high standard beliefs subscale scores. The analysis was repeated but with the conditional beliefs as the dependent variable. Only the social anxiety composite, β = .55, t(218) = 10.01, p < .01, and the depression composite, β = .19, t(218) = 2.63, p = .01, emerged as significant predictors, and the difference between the strength of these predictors was significant, t(219) = 3.96, p < .01. The three predictors accounted for 48.1% of the variance in the conditional beliefs subscale scores. The analysis was repeated with the unconditional beliefs as the dependent variable. Again, only the social anxiety composite, β = .42, t(218) = 6.86, p < .01, and the depression composite, β = .35, t(218) = 4.37, p < .01, emerged as significant predictors, but the difference between the strength of these predictors was not significant, t(219) = .73, p = .47. The three predictors accounted for 35.6% of the variance in the unconditional beliefs subscale scores.

These results indicate that the social anxiety composite is uniquely related to the high standard, conditional, and unconditional beliefs. In order to see which belief type was most strongly and uniquely related to social anxiety, a regression analysis was conducted in which the social anxiety composite was the dependent variable and the belief types were entered simultaneously as the predictors. As expected, each of the belief types were significantly related to the social anxiety composite. The conditional beliefs was the strongest predictor, β = .38, t(218) = 5.18, p < .01, followed by the high standard beliefs, β = .26, t(218) = 4.29, p < .01, and the unconditional beliefs, β = .19, t(218) = 3.04, p < .01. However, there were no significant differences between the strength of these predictors (all ps > .05). The beliefs explained 49.1% of the variance in social anxiety composite scores.

The results also indicated that the depression composite is related to the conditional and unconditional beliefs. In order to see which belief type was most strongly related to depression, a regression analysis was conducted in which the depression composite was the dependent variable and the two belief types were entered simultaneously as the predictors. As expected, each of the belief types were significantly related to the depression composite. The unconditional beliefs emerged as the stronger predictor, β = .29, t(218) = 3.86, p < .01, over the conditional beliefs, β = .26, t(218) = 3.40, p < .01. However, there was no significant difference between the strength of these predictors, t(219) = .25, p = .80. The beliefs accounted for 25.1% of the variance in depression composite scores.

Discussion

In the social anxiety literature, there is currently no validated measure of the three maladaptive self-belief types as proposed by Clark and Wells (1995). Accordingly, Wong and Moulds (2009) developed a measure of the maladaptive self-belief types—the SBSA. However, to date they have presented only preliminary psychometric data for this measure. Hence, the current study sought to elucidate more detailed psychometric properties of the SBSA (factor structure, internal consistency, test-retest reliability, convergent and divergent validity), and in doing so, validate the measure for use in future research and clinical practice. Examination of the factor structure of the SBSA with an unconstrained EFA revealed evidence to support a correlated two-factor model, against our prediction. However, further testing with CFA indicated that the correlated three-factor model was the best fitting model relative to the competing two- and one-factor models (a four-factor model was also examined but was not considered viable because certain factors were unstable). Theoretical considerations (i.e., Clark and Wells 1995), and the fact that the CFA directly compared the two- and three-factor models and showed that the latter was a better fitting model, led us to accept the correlated three-factor model as the best solution, consistent with prediction. The SBSA and its subscales also demonstrated good internal consistency and test-retest reliability, as predicted. Additionally, as predicted, the SBSA had a unique positive association with social anxiety, and the magnitude of this association was significantly stronger than the association between the SBSA and depression, and the association between the SBSA and general anxiety. This suggests that the SBSA has good validity. In terms of the SBSA subscales, social anxiety emerged as the strongest predictor of the high standard and conditional beliefs (over depression and general anxiety), as predicted. However, in contrast to prediction, the strength of social anxiety as a predictor for the unconditional beliefs was similar to that of depression (and both social anxiety and depression were better predictors than general anxiety). Further examination of the SBSA subscales revealed, against prediction, that the high standard, conditional, and unconditional beliefs did not significantly differ in their ability to predict social anxiety. Consistent with prediction, the conditional and unconditional beliefs were both significantly stronger predictors of depression than the high standard beliefs. Overall, the SBSA showed good psychometric properties.

It is interesting to note the contrasting results between the unconstrained EFA and the CFA. In the end, we decided to accept the correlated three-factor model as the best solution over the other models based on the CFA. However, we highlight that our study is the first to examine the factor structure of the SBSA, and as such, our results should be interpreted cautiously until they are replicated (e.g., future research may provide evidence for the correlated three-factor model, or alternatively, evidence for the correlated two-factor model). It should also be noted that the EFA and CFA in this study were conducted on separate samples, and that sample characteristics may have played a part in producing the contrasting results (Brown 2006). Nonetheless, given that the CFA showed the superiority of the correlated three-factor model over the other competing models, it appears there is evidence to support the Clark and Wells (1995) model and the categorisation of the maladaptive self-beliefs into three types (high standard, conditional, and unconditional beliefs). The current finding that the three-factor model is the best fitting model also justifies the use of the SBSA in future studies in which researchers aim to investigate the maladaptive self-beliefs related to social anxiety and the different belief types. In particular, an important direction for future research will be to examine the specific maladaptive self-beliefs and how they interact with the other components of the Clark and Wells (1995) model (see Wong and Moulds 2009, 2010a, b, 2011). This is consistent with the need to investigate the relationships (especially the causal relationships) between components of Clark and Wells’ (1995) model more generally (Clark and McManus 2002; Hirsch and Clark 2004; Hirsch et al. 2006; Rachman et al. 2000).

In addition to the results of the factor analysis, we showed that the SBSA and its subscales have adequate internal consistency and test-retest reliability, supporting the use of the SBSA. The good internal consistency of each SBSA subscale suggests the items within each subscale similarly tap a specific maladaptive self-belief type. Furthermore, the good internal consistency of the SBSA overall indicates that all of the items similarly tap maladaptive self-beliefs more generally. Besides this, we showed that the SBSA and its subscales have good test-retest reliability. According to Clark and Wells (1995), the maladaptive self-beliefs are persistent and lie dormant until they are activated by perceived social-evaluative threat. Notably, the participants in our study completed the SBSA for both administrations in situations without a social-evaluative threat. Hence, the test-retest reliability analysis essentially demonstrated that the strength of the unactivated beliefs as measured by the SBSA is stable over time, consistent with the enduring nature of the beliefs as suggested by Clark and Wells (1995). Seemingly in contrast to our finding, Wong and Moulds (2009, 2010a) have demonstrated that the maladaptive self-beliefs can change over time. However, unlike our test-retest reliability analysis, the Wong and Moulds (2009, 2010a) studies involved the presence of a perceived social-evaluative threat (i.e., an impromptu speech task). In this context, following Clark and Wells (1995), an impending perceived social-evaluative threat can activate the maladaptive self-beliefs, thus leading to an observed increase in the strength of specific beliefs on the SBSA (Wong and Moulds 2010a). Conversely, a perceived social-evaluative threat that has passed in time can deactivate the maladaptive self-beliefs, thus leading to an observed decrease in the strength of specific beliefs on the SBSA (Wong and Moulds 2009). Taking all of these findings together, it appears the SBSA is suitable for measuring stable unactivated maladaptive self-beliefs over time, and is also able to capture changes in the beliefs when they are influenced by perceived social-evaluative threat.

The SBSA also demonstrated good convergent and divergent validity, lending further support for its use in future studies. Notably, our analysis of validity was strengthened by the fact that we used multiple measures to tap social anxiety, depression, and general anxiety in order to obtain more reliable indices of these constructs. Furthermore, our analysis was strengthened by simultaneously entering the social anxiety, depression, and general anxiety composites into the regression model as predictors and obtaining the unique relationship each construct had with the SBSA. With this in mind, we found that the SBSA had a strong positive association with social anxiety, a weak positive association with depression, and was not related to general anxiety. This suggests that the SBSA taps beliefs that are more characteristic of social anxiety, rather than depression or general anxiety. It is interesting to note that although we were expecting general anxiety to be associated with the SBSA (to a lesser extent than social anxiety), general anxiety in fact did not uniquely account for any variance in SBSA scores. This suggests that the variance in SBSA scores due to any anxiety-related constructs had already been preferentially accounted for by social anxiety, again highlighting the good convergent validity of the SBSA.

Interestingly, when the SBSA subscales were examined, it appears that the conditional beliefs had unique positive associations with both social anxiety and depression, with social anxiety having a significantly stronger association with this belief type than depression. While the conditional beliefs (e.g., “if people see I’m anxious, they will think I’m weak”) have a self-evaluative component (i.e., people think that I’m weak) that is somewhat depressive in nature, it may be that the social element and uncertainty that is inherent in conditional beliefs (i.e., whether or not people will see I’m anxious) predominates, thus explaining why these beliefs are more related to social anxiety than depression. Additionally, our analysis revealed that the unconditional beliefs had unique positive associations with both social anxiety and depression, and that these associations were similar in magnitude. Unconditional beliefs (e.g., “people think I’m boring”) indicate to an individual that other people consistently think negatively of them and that such negative evaluation is likely to apply across situations. The social nature of this (perceived) consequence may explain why these beliefs are related to social anxiety. Furthermore, the absolute and global nature of the negative evaluation of the self contained in the content of unconditional beliefs is characteristic of depressive thinking (Dent and Teasdale 1988; Kovacs and Beck 1978), and this may explain why the unconditional beliefs are also related to depression. Interestingly, these findings regarding the conditional and unconditional beliefs are consistent with the suggestion that socially anxious individuals and depressed individuals share common negative cognitions about the self, which may in part explain comorbidity between social phobia and depression (see Dozois and Frewen 2006). We also found that the high standard beliefs were uniquely associated with social anxiety. This is unsurprising, given that high standard beliefs (e.g., “I must be able to convey a favourable impression”) are likely to lead an individual to attempt to achieve these high standards and in doing so prevent negative evaluation, the core fear related to social anxiety (Clark and Wells 1995; Rapee and Heimberg 1997). Hence, high standard beliefs may ultimately function to prevent negative evaluation, and this may explain why these beliefs are related to social anxiety. Notably, high standard beliefs were not related to depression. Unlike the conditional and unconditional beliefs, high standard beliefs do not contain negative self-evaluative content, and this is a possible explanation for why these beliefs were not associated with depression. It should also be noted that with all three maladaptive self-belief types, none were significantly associated with general anxiety. Given the nature of our analyses (i.e., social anxiety, depression, and general anxiety variables were entered simultaneously as predictors into the regression models), it is likely that the belief types were not associated with general anxiety because (similar to the full SBSA scale) any variance in SBSA subscale scores due to anxiety-related constructs had already been preferentially accounted for by social anxiety. This suggests that the belief types are more closely related to social anxiety than general anxiety, supporting the validity of the SBSA subscales.

Our findings have implications for clinical practice. Given the good psychometric properties of the SBSA and its strong positive association with social anxiety, the SBSA can be used as part of a screening battery with the expectation that individuals with higher levels of social anxiety would score more highly on the SBSA. More specifically, the SBSA may inform treatment planning by indicating specific self-belief types that may be held strongly by an individual seeking assistance with social anxiety. Furthermore, given that the SBSA is sensitive to changes in belief strength over time (Wong and Moulds 2009), it can be administered at pre- and post-treatment to ascertain changes in the strength of maladaptive self-beliefs. Finally, since the SBSA is theoretically linked with Clark and Wells’ (1995) model, it would be an ideal instrument with which to assess the impact of treatments for social phobia that have been developed from this conceptual account (e.g., Clark et al. 2003, 2006; Stangier et al. 2003).

We note some limitations. First, we used a non-clinical sample and did not include a sample of individuals with a diagnosis of social phobia. Future studies should replicate our findings using a clinical sample. Related to this, future studies will need to test whether the SBSA can identify individuals diagnosed with social phobia and discriminate them from individuals with other anxiety disorders and non-anxious controls. Second, our sample was an undergraduate sample and hence was restricted in terms of the age range of participants. Future studies may examine the development of the maladaptive self-beliefs in a younger sample and test predictions made in the Clark and Wells (1995) model (e.g., high standard and conditional beliefs are precursors to late and sudden onset social phobia, while for early onset social phobia, unconditional beliefs develop first prior to high standard and conditional beliefs developing). Third, the SBSA was administered to participants in this study without the presence of a social-evaluative threat. Clark and Wells’ (1995) model suggests that a social-evaluative situation can activate and strengthen the maladaptive self-beliefs. Hence, future studies may test whether exposing participants to a social-evaluative threat before administering the SBSA will change the factor structure of the SBSA. Fourth, our analysis of validity involved the concurrent examination of constructs. Future studies may utilize other methods to examine validity (e.g., predictive validity), as well as examine other constructs (e.g., how the SBSA relates to depressive cognitions). Fifth, we employed only self-report measures in our analysis of validity. Future studies could usefully include behavioural tasks in order to further examine the validity of the SBSA.

Conclusions

Notwithstanding the limitations of our study, we found contrasting evidence regarding the factor structure of the SBSA, but concluded that overall, the optimal factor structure for the SBSA consists of three correlated factors that reflect the high standard, conditional, and unconditional belief types. The SBSA and its subscales demonstrated good internal consistency and test-retest reliability. Overall, the SBSA also demonstrated good validity. The SBSA had a strong significant positive association with social anxiety, a weak but significant positive association with depression, and no significant association with general anxiety. For the subscales, social anxiety emerged as the strongest and only significant predictor for the high standard beliefs. Social anxiety also emerged as the strongest significant predictor for the conditional beliefs, with depression as a secondary but significant predictor. Social anxiety and depression were similarly strong as the only significant predictors for the unconditional beliefs. Given the good psychometric properties of the SBSA, and its theoretical link with Clark and Wells’ (1995) model, the SBSA can be considered a valuable tool for both research and clinical purposes.