Cognitive behavior therapy (CBT) is an effective treatment for a range of clinical presentations using different delivery methods (e.g. group, individual, face-to-face, and Internet-delivered; Andrews et al., 2018; Hofmann et al., 2012). Despite strong efficacy, not all individuals who start treatment engage, complete it, or recover (Andersson et al., 2019; Springer et al., 2018). Identifying, prior to standard CBT protocols, those who might be at risk of poorer engagement and/or outcomes is a crucial research endeavor that can inform alternative clinical protocols and service delivery.

Multiple factors may contribute to lack of response or engagement with CBT. While a comprehensive review of pre-treatment predictors of CBT engagement and outcome is beyond the scope of this paper some key findings warrant mention. Pre-treatment symptom severity is generally considered one of the most consistent predictors of CBT treatment response (Hilbert et al., 2020). There is also some evidence that comorbid conditions can negatively impact response to CBT for anxiety and depressive disorders (e.g., see Goddard et al., 2015; Hansen et al., 2007). Patients’ perceptions of the credibility of therapy and their expected benefits from treatment may also be associated with outcomes (Cohen et al., 2015). There is also evidence that unmodifiable factors like age, gender, and educational status may be associated with poorer engagement and response (Andersson et al., 2019; Karyotaki et al., 2018).

Efforts to identify other unique and potentially modifiable pre-treatment predictors of outcome are needed. Recent research has begun to explore whether the extent to which individuals’ mindsets (e.g., emotional and mental attitudes) and the principles of CBT (hereafter referred to as CBT-mindedness) are consistent impact treatment response (McLellan et al., 2016, 2019). CBT-mindedness can be measured using a brief self-report scale [e.g., the CBT-Suitability Scale (CBT-SUITS)], which assesses attitudes and beliefs that align with CBT but that does not confound symptoms. This measure includes three sub-scales: (1) the “CBT-Rationale sub-scale” that assesses beliefs about how thoughts determine emotions and behaviors; (2) the “Insight sub-scale” that assesses awareness of, and ability to express thoughts and feelings; and (3) the “Behavior sub-scale” that assesses learning from actions. Previous research has shown that CBT-mindedness is generally distinct from depression and anxiety symptoms in undergraduate, community, and face-to-face treatment seeking samples (McLellan et al., 2016), and predicts treatment response amongst those receiving group CBT for social anxiety disorder (McLellan et al., 2019). CBT-SUITS scores were not correlated with age and did not differ by gender or education in undergraduate and treatment seeking samples. However, higher scores were found amongst females and those who attained higher education levels in a general community sample (McLellan et al., 2016). While the factor structure of the measure has been assessed via exploratory or confirmatory factor analysis in the above samples, the internal consistency of the subscales is less than ideal (at times as low as α = 0.62) and therefore warrants further investigation. The existing psychometric studies have been in fairly small samples (n’s = 235, 261, 397) that are predominantly female (76–79%). Critically, while participants in the ‘treatment seeking sample’ were recruited to increase the chances that they were actively seeking help (Study 3, McLellan et al., 2016), 58% of these participants had not received treatment. Furthermore, the previous clinical sample was restricted to those with social anxiety seeking face-to-face CBT (McLellan et al., 2019).

While these initial psychometric and treatment data are promising, further research in larger and more diverse clinical populations (beyond adults with social anxiety) is required to extend the validity of the measure. Furthermore, identifying predictors of treatment engagement and/or response is particularly important for those seeking treatment via alternative delivery methods (e.g., the Internet). While treatment outcomes tend not to differ between face-to-face and therapist guided internet-delivered CBT (iCBT), for many disorders (Carlbring et al., 2018), meta-analytic data have shown dropout rates are higher in iCBT compared to face-to-face CBT for depression (van Ballegooijen et al., 2014). Furthermore, the standardized nature of iCBT programs coupled with the reduced role of the therapist to tailor treatment based on client views or beliefs means that variability in individuals CBT-mindedness at baseline may have a greater effect on engagement and/or treatment response in iCBT than in face-to-face therapy. If CBT-mindedness predicts engagement and/or outcomes providing preparatory modules or modified treatment protocols that address this, or suggesting alternative treatments, means clients may experience greater benefits. However, the relationship between CBT-mindedness and treatment outcome may be more complex. While baseline CBT-mindedness may be predictive of treatment outcomes, it is also possible that constructs like CBT-mindedness may change (increase) during treatment, or may change among those who benefit from treatment. It is therefore important to first investigate whether CBT-mindedness can be modified (as a result of treatment). It then needs to be seen whether treatment related improvements in CBT-mindedness are predictive of treatment response (both treatment completion and change in symptom severity).

As a result, this study investigated the factor structure, divergent, and predictive validity of CBT-mindedness amongst a large sample completing internet-based CBT (iCBT) for their anxiety and/or depression. Based on the existing literature, we hypothesized that:

  1. 1.

    The three-factor structure of the CBT-SUITS will be confirmed (e.g., CBT Rationale, Insight and Behavior);

  2. 2.

    CBT-SUITS total and sub-scale scores would:

    1. a.

      Be weakly associated with depression and anxiety symptom severity and psychological distress, and unrelated to age or sex; and

    2. b.

      Have equivalent relationships with different types of symptoms (in this case, anxiety and/or depression).

  3. 3.

    Pre-treatment CBT-SUITS total and sub-scale scores would predict treatment:

    1. a.

      Completion; and

    2. b.

      Outcomes controlling for pre-treatment symptom severity, psychological distress and demographics.

  4. 4.

    iCBT would increase CBT-mindedness.

  5. 5.

    Increased CBT-mindedness would predict post-treatment symptom severity and distress controlling for baseline symptom severity and demographics.

Methods

Participants

This study included 1132 consecutive users of the THIS WAY UP online clinic, a joint initiative of St Vincent’s Hospital (Sydney, Australia) and the University of New South Wales. THIS WAY UP provides education and psychological treatment courses for adults experiencing symptoms of anxiety and depression (see thiswayup.org.au). All service users registered for iCBT for their anxiety and/or depression between 26th January 2017 and 1st January 2018 and completed the CBT-SUITS as part of their standardized intake assessment. Service users were mostly female (61.6%) and in their early forties (M = 41.62, SD = 14.86, Ranging from 17 to 85 years, IQR = 23). Rurality was classified based on users’ postcodes and the Australian Statistical Geography Standards (Australian Bureau of Statistics, 2013). Of the 882 users who provided their postcode, 29.4% (n = 259) were living in regional or remote Australia. The majority of users accessed treatment for their co-morbid anxiety and depression (54.8%: e.g., the ‘Mixed Depression and Anxiety’ iCBT course), 31.0% sought treatment for their worry (e.g., the Generalized Anxiety Disorder iCBT course), and 14.2% sought help for their depression (e.g., the Depression iCBT course).

Materials

Cognitive Behavior Therapy Suitability Scale (CBT-SUITS)

The CBT-SUITS is a 13-item measure of CBT-mindedness (McLellan et al., 2016). Service users reported the extent to which they agreed with each item from ‘strongly disagree’ to ‘strongly agree’ on a 5-point scale. Evidence of internal consistency (total α = 0.76–0.81), convergent and divergent validity (e.g., associations with anxiety and depression symptom severity and treatment credibility) and predictive validity has previously been reported (McLellan et al., 2016, 2019).

Generalized Anxiety Disorder 7-item (GAD-7)

The GAD-7 is a 7-item self-report measure that assesses GAD symptoms over the past 2 weeks (Spitzer et al., 2006). Service users reported how frequently they had experienced each item as either “not at all”, “on several days”, “on more than half the days” or “on nearly every day”. Total scores ≥ 10 indicate a probable GAD diagnosis (Spitzer et al., 2006). Studies support a one-dimensional structure and provide evidence of internal consistency (α = 0.92), temporal stability (r = 0.83), convergent/divergent validity (e.g., correlations with the measures of anxiety, depression, self-esteem and life satisfaction), criterion validity (e.g., sensitivity/specificity with respect to diagnosis via structured interview), treatment sensitivity, and factorial invariance across age and sex (Löwe et al., 2008; Spitzer et al., 2006).

Patient Health Questionnaire-9 (PHQ-9)

The PHQ-9 is a 9-item self-report measure of depression symptom severity as experienced in the past 2 weeks. Items are rated in the same way as the GAD-7 and a total score ≥ 10 indicates a probable Major Depressive Disorder (MDD) diagnosis (Wittkampf et al., 2007). Internal consistency (α = 0.86) and temporal stability (r = 0.84 over 48 h) are sound (Kroenke et al., 2001). Factor analyses support a one or two-factor structure, and evidence of construct validity is extensive, including convergent/divergent validity (e.g., correlations with measures of depression, health, disability and substance use), criterion validity (e.g., sensitivity/specificity with respect to diagnosis via structured interview), and sensitivity to change (Beard et al., 2016; Dum et al., 2008; Hepner et al., 2009; Kroenke et al., 2001, 2010).

Kessler Psychological Distress Scale (K-10)

The K-10 is a 10-item measure of psychological distress experienced in the past 2 weeks (Kessler et al., 2002). Service users reported how frequently they had experienced each item as either “none of the time”, “a little of the time”, “some of the time”, “most of the time” or “all of the time”. Evidence of factor structure, internal consistency (α = 0.93), test re-test reliability (r = 0.80 over 1–2 weeks), convergent and discriminant validity, and treatment sensitivity has been provided (Furukawa et al., 2003; Kessler et al., 2002; Merson et al., 2021; Sunderland, Hobbs, et al., 2012; Sunderland, Mahoney, et al., 2012; Sunderland, Wong, et al., 2012). The psychometric properties of the instrument are invariant across the adult lifespan (Sunderland et al., 2012; Sunderland, Mahoney, et al., 2012; Sunderland, Wong, et al., 2012). Nationally representative data demonstrate that total scores of 20 (or more) are indicative of the presence of mental disorder(s) (Andrews & Slade, 2001).

Procedure

Service users could choose to enrol in either a self- or clinician-guided version of the respective program, with 63.7% choosing the former option. Supervising clinicians were mostly general practitioners (44.8%), psychologists (26.8%), and medical specialists (17.3%) and retained clinical responsibility for their respective patients across assessment and treatment. As is routine in this clinical service, the prescribing professionals were advised that their patients may not benefit from iCBT if they were experiencing very severe depression, active suicidal ideation, drug or alcohol dependence, schizophrenia, bipolar disorder or were taking atypical anti-psychotics or benzodiazepines. This advice is given because the efficacy of these iCBT courses in people with these characteristics is unknown as they were excluded from the clinical trials that evaluated the courses. However, adhering to these recommendations was at the discretion of the service user and clinician and were not exclusion criteria.

This study was conducted as part of the routine Quality Assurance activities of THIS WAY UP clinic. All service users provided electronic informed consent that their pooled de-identified data would be collected, analysed and published for quality assurance and research purposes by agreeing to the Terms of Use and Privacy Policy of the service (St Vincent’s Hospital Human Research Ethics Committee, 2020/ETH03027). After providing their informed consent and prior to and after treatment, participants completed the CBT-SUITS, K10, GAD-7 and PHQ-9.

Analytic Strategy

Factor Structure (Hypothesis 1)

Exploratory factor analyses were conducted using an oblimin rotation method in the MPlus v5.12 software package including 1 to 5 factors (Muthén and Muthén, 1998–2009). Solutions with an eigenvalue > 1 that had at least three items with loadings ≥ 0.40 were retained for confirmatory factor analysis (CFA, Tabachnick & Fidell, 2007). Consistent with previous CBT-SUITS and transdiagnostic process research (e.g., Goring & Papageorgiou, 2008; McEvoy et al., 2010; McLellan et al., 2016), items were considered to load on more than one factor if the difference between the factor loadings was < 0.10. CFA models were estimated using maximum likelihood methods with robust standard errors (Muthén & Muthén, 1998–2009). The relative fit of the structural models was assessed using the AIC and BIC fit indices (Akaike, 1974; Schwartz, 1978) where lower values indicated improved model fit. Although smaller BIC values indicate better model fit, the magnitude of the BIC difference between two models can also be used to calculate a Bayes factor (eBIC model i–BIC model j), which indexes the posterior odds of preferring the model with the smaller BIC value. As a rule of thumb, BIC differences of more than 10 provide very strong support for preferring the model with the smaller BIC value (Raferty, 1995). The MPlus software package calculates BIC values using the Schwartz method. The Schwartz criterion is equal to minus two times BIC values and because of this, the Bayes factors that are presented here are calculated using e.5 (BIC model i–BIC model j) (Muthén & Muthén, 2008).

Divergent Validity (Hypothesis 2)

After identifying the factor structure of the CBT-SUITS, its divergent validity was assessed. First, Pearson correlation coefficients were calculated between CBT-SUITS total and sub-scale scores and pre-treatment GAD-7, PHQ-9 and K10 total scores. Second, Steiger’s z was used to examine differences between the magnitude of overlapping correlation coefficients to assess whether the CBT-SUITS total and subscale scores were differentially associated with anxiety and depression symptom severity (Diedenhofen & Musch, 2015; Steiger, 1980).

Treatment Sensitivity (Hypothesis 3)

To broadly estimate changes in symptom severity and CBT mindedness across iCBT, paired samples t-tests (with Cohen’s d effect sizes computed using pre-treatment SD) examined differences in GAD-7, PHQ-9, K10, and CBT-SUITS mean scores from pre to post-iCBT (i.e., combining outcomes from the three iCBT courses). Effect sizes of ≤ 0.49, 0.50–0.79, and ≥ 0.80 were considered to be small, moderate and large, respectively.

Predictive Validity (Hypothesis 4)

A series of hierarchical linear and/or logistic regressions were conducted to estimate the predictive utility of the CBT-SUITS in determining treatment completion and outcomes.

First, we examined whether pre-treatment CBT-SUITS total score (and subscale scores) predicted treatment completion after controlling for baseline symptom severity, and demographic and treatment variables. Treatment completion was calculated as the total number of lessons completed (e.g., 0–6 lessons). The following variables were entered into Step 1 of each regression: age, gender, course (i.e., Mixed Depression and Anxiety iCBT vs. GAD iCBT vs. Depression iCBT course), user type (i.e., self-guided vs. clinician-guided) and baseline symptom severity (pre-treatment K10, GAD-7, and PHQ-9 total score). Pre-treatment CBT-SUITS total score (or subscale scores) was then entered at Step 2.

Second, we examined whether pre-treatment CBT-SUITS total score predicted post-treatment symptom severity controlling for baseline symptom severity, and demographic and treatment variables. Three regressions were run with the criterion variables of post-treatment K10, GAD-7, and PHQ-9 total score. The following variables were entered into Step 1 of each regression: age, gender, course (i.e., Mixed Depression and Anxiety iCBT vs GAD iCBT vs Depression iCBT course), user type (i.e., self-guided vs. clinician-guided), and baseline symptom severity (e.g., pre-treatment K10 score when post-treatment K10 score was the criterion variable etc.). Pre-treatment CBT-SUITS total score (or subscale scores) was then entered at Step 2.

Third, we examined whether changes in CBT-SUITS total score from pre- to post-treatment predicted post-treatment symptom severity over and above baseline symptom severity, and demographic and treatment variables of the regression. The above regressions were re-run with the change in CBT-SUITS total score (post-treatment minus pre-treatment score) entered at Step 2.

Results

Sample Characteristics

Table 1 provides the descriptive statistics and estimates of internal consistency of the study measures. Measures of symptom severity indicated that service users were characterized by high rates of probable disorder. Using established cut-scores for these measures, 65.6% of users met criteria for probable GAD (i.e., GAD-7 total score ≥ 10), and 69.7% for probable major depressive disorder (i.e., PHQ-9 total score ≥ 10) (Spitzer et al., 2006; Wittkampf et al., 2007). 89.7% of users reported more intense psychological distress than 95% of community dwelling adults (i.e., K10 total scores ≥ 20) (Andrews & Slade, 2001).

Table 1 Descriptive statistics, internal consistency, and bivariate correlations between measures of symptom severity, CBT mindedness, age, and treatment completion
Full size table

Factor Analysis

A 3-factor solution was favored according to a number of metrics including that there were 3 eigenvalues > 1 (4.32, 2.09, 1.65 and 0.92), the ratio between the successive eigenvalues, and the number of items with dominant factor loadings ≥ 0.40. Two of the 13 original CBT-SUITS items cross-loaded; item 6 on the Behavior and Insight subscales, and item 10 on the CBT Rationale and Behavior subscales. The solution was also readily interpretable and consistent with previous analyses of the CBT-SUITS (McLellan et al., 2016). Rotated factor loadings from exploratory factor analyses are shown in Table 2. Using confirmatory methods, a unidimensional baseline model was then estimated (Model 1). Model 2 estimated the 3-factor structure identified in the EFA. Model 3 estimated the same 3-factor structure from Model 2 but removed item 6 and 10. Using the Bayes factor, there was very strong support for Model 3 (see Table 3). Model fit was improved when items 6 and 10 were removed, hence, all analyses reported herein are based on an 11-item scale. Table 1 shows the mean scores, internal consistency, and bivariate correlations between the 11-item CBT-SUITS (i.e., Model 3) total and subscale scores.

Table 2 Exploratory factor analysis of the CBT-SUITS: factor loadings for the three-factor solution
Full size table
Table 3 Confirmatory factor analyses goodness of fit statistics for the three models
Full size table

Divergent Validity

CBT-SUITS total and sub-scale scores did not vary by gender (all ps > 0.05) and had small or no association with age (see Table 1). CBT-SUITS total and subscale scores did not generally differ between clinician- and self-guided service users (all ps > 0.05) with one exception: self-guided users reported slightly higher CBT Rationale sub-scale scores than clinician-guided users, M (SD) = 15.67 (2.36) vs. 15.38 (2.19), t(1130) = 2.08, p = 0.04, d = 0.13. Small or non-significant correlations were found between CBT-SUITS total and sub-scale scores and the GAD-7, PHQ-9, and K10 (see Table 1). CBT-SUITS total and sub-scale scores were more strongly associated with depression symptom severity than with anxiety symptom severity (CBT-SUITS total score z = − 4.42, p < 0.001; CBT Rationale z = − 4.64, p < 0.001; Insight z = − 2.52, p < 0.01; Behavior z = − 2.20, p < 0.05).

Treatment Sensitivity

Significant improvements in symptom severity, psychological distress, and CBT-mindedness were observed from pre- to post-iCBT in the subsample that completed their iCBT course (see Table 4). Completion of iCBT was associated with large effect size reductions in psychological distress, depression symptom severity, and GAD symptom severity. Small and moderate effect size increases in CBT-mindedness were observed from pre-to-post iCBT.

Table 4 Symptom severity, psychological distress and CBT mindedness at pre- and post-iCBT
Full size table

Predictive Validity

Treatment Completion

Of the 1132 service users who enrolled, 5% did not complete the first lesson, 18.6% completed one lesson, and 14.4%, 11.3%, 9%, and 8.5% completed two, three, four, and five lessons, respectively. Service users typically completed three or four lessons of their online course [M (SD) = 3.59 (2.10)], with a third of users completing all six lessons.

Controlling for age, gender, course (i.e., Mixed Depression and Anxiety iCBT vs GAD iCBT vs Depression iCBT course), user type (i.e., self- vs. clinician-guided), and baseline symptom severity (K10, GAD-7 and PHQ-9 total score) in Step 1, pre-treatment CBT-SUITS total score (entered in Step 2) did not significantly predict the number of iCBT lessons (0–6) completed [Model 1 ∆R2 = 0.04, F(7, 1123) = 6.45, p < 0.001; Model 2 ∆R2 = 0.00, F(1, 1122) = 0.001, p = 0.98]. At both Step 1 and Step 2, age was the only significant predictor of the total number of iCBT lessons completed (Part r = 0.17). The same pattern of results was found when the analyses were repeated for each subscale score.

Treatment Outcomes

Higher baseline CBT-SUITS total scores significantly predicted lower post-treatment K10 scores, but were not significantly associated with post-treatment depression and anxiety symptom severity after controlling for pre-treatment symptom scores, and treatment and demographic variables (see Table 5). In each regression, user age and baseline symptom severity and psychological distress were significantly associated with post-treatment symptom severity such that older service users and those with lower baseline symptom severity and distress were more likely to have lower post-treatment symptom severity and psychological distress. When CBT-SUITS subscale scores were entered (together) in Step 2 (instead of CBT-SUITS total scores), the baseline CBT Insight factor scores significantly predicted post-treatment K10 scores (t = − 2.46, p = 0.01, Part r = − 0.11) over and above other variables. Specifically, a one-point increase in CBT-SUITS is associated with a 0.14-point decrease in K-10 scores, on average. No other baseline CBT-SUITS factor scores were uniquely predictive of post-treatment distress or symptom severity (all ps > 0.05).

Table 5 Predicting post-iCBT symptom severity and distress from pre-treatment CBT mindedness
Full size table

Increases in CBT mindedness from pre- to post-iCBT significantly predicted lower post-treatment psychological distress, GAD symptom severity and depression symptom severity (controlling for baseline symptoms severity, and treatment and demographic variables, see Table 6). Specifically, a one-point increase in CBT-SUITS scores from pre to post treatment is associated with a 0.17-point decrease in K-10 scores, a 0.11-point decrease in GAD-7 scores and a 0.14-point decrease in PHQ-9 scores on average. In each regression, older age and lower baseline symptom severity significantly predicted lower symptom severity and distress following treatment. When the CBT-SUITS subscale scores were entered in Step 2 (instead of CBT-SUITS total scores), only increases in the CBT Behavior factor from pre- to post-iCBT significantly predicted lower post-treatment psychological distress (t = − 3.20, p = 0.001, Part r = − 0.14), GAD symptom severity (t = − 2.37, p = 0.02, Part r = − 0.11), and depression symptom severity (t = − 2.34, p = 0.02, Part r = − 0.10).

Table 6 Predicting post-iCBT symptom severity and distress from changes in CBT mindedness from pre- to post-treatment (n = 363)
Full size table

Discussion

The aim of this study was to investigate the factor structure, divergent and predictive validity of CBT-mindedness in a large sample of adults seeking iCBT for their anxiety and/or depression. Consistent with our first hypothesis, there was strong support for the three-factor structure of the CBT-SUITS, confirming results of previous research (McLellan et al., 2016). However, two of the 13 original items cross-loaded; item 6 on the Behavior and Insight subscales, and item 10 on the CBT Rationale and Behavior subscales. Model fit was improved when these items were removed, and consequently this study found support for the 11-item version of the CBT-SUITS scale.

Regarding hypothesis two, overall, there was good evidence for the divergent validity of the CBT-SUITS scale. As expected, CBT-SUITS scores were unrelated to gender, unrelated or very weakly related with age and pre-treatment symptom severity, and distress. This finding aligns with previous validation work conducted in undergraduate, community, and treatment-seeking samples (McLellan et al., 2016), and replicates this in a larger sample of adults seeking treatment for their anxiety and/or depression and those receiving treatment via the internet. While the relationships between CBT-SUITS and symptoms severity tended to be slightly stronger for depression than anxiety symptoms (baseline CBT-SUITS and PHQ-9 r = − 0.17 vs. baseline CBT-SUITS and GAD-7 r = − 0.07), these differences are unlikely to be clinically meaningful when compared, for example, to the size of the relationship between the symptoms themselves (e.g., the relationship between baseline depression and anxiety symptoms was r = 0.60).

Support was mixed for our fourth hypothesis concerning the predictive validity of the CBT-SUITS. CBT-SUITS total and sub-scale scores did not predict treatment completion or post-treatment depression or anxiety symptoms. Instead, age (for treatment completion and outcome) and baseline symptoms/distress (for outcome) were consistent pre-treatment predictors. However, higher pre-treatment CBT-SUITS scores (total and the Insight subscale) predicted lower psychological distress following treatment. Therefore CBT-SUITS predicted only one of four outcomes. These mixed findings with respect to predicting clinical outcomes do not support our hypothesis but are not unusual. Efforts to identify pre-treatment predictors of treatment response (over and above symptom severity) are plagued by inconsistent findings (Knight et al., 2014; Eskildsen et al., 2010; Hilbert et al., 2020). Further, while CBT-SUITS scores only accounted for 1–2% of the variance in outcomes, this was over and above baseline symptoms, and is a modifiable characteristic, so is still an important target in efforts to improve outcomes.

Of course, efforts to identify predictors that are modifiable, and hence, can be targeted prior to, or during, treatment are crucial for maximizing treatment benefits. This is the first study to investigate if CBT-mindedness increases with CBT as well as examining whether such change predicts outcome. Consistent with hypothesis three, the CBT-SUITS total and sub-scale scores were found to increase across treatment with small to moderate effects. Furthermore, change in CBT-SUITS total and Behavior subscale scores consistently predicted post-treatment distress and depression and anxiety symptom severity. These findings amongst a large group of adults with mixed presentations adds further support to the predictive validity of the CBT-SUITS scale. Although not consistent in all analyses, the results align with existing research that found higher CBT-SUITS scores predicted lower post-treatment symptoms amongst adults receiving group CBT for social anxiety (McLellan et al., 2019).

These findings also extend previous research examining CBT-mindedness as measured by the CBT-SUITS to a large sample of adults undertaking iCBT in the community for anxiety and/or depression. The size of this study and the inclusion of service users seeking treatment for both anxiety and depression lends weight to previous findings that had predominantly used the CBT-SUITS amongst undergraduate or community participants or in small anxiety-focused clinical samples receiving treatment face-to-face. As a result, this study provides evidence that CBT-mindedness may be an important factor for predicting treatment outcomes that spans beyond socially anxious patients examined in prior work, and across treatment delivery methods.

However, the results should be interpreted in light of some limitations. Two items in the CBT-SUITS scale were removed following exploratory and confirmatory factor analyses. This may have been a result of the broader clinical sample utilized in this study compared to previous research. The 11-item structure needs to be replicated in future research. Results also indicated slightly less than ideal levels of internal consistency, especially for subscale scores. While this is likely due to the small number of items that make up each subscale (Tavakol & Dennick, 2011), future research should investigate and attempt to improve their internal consistency. Service users were actively seeking treatment for anxiety and/or depression. While this allows us to generalize the findings associated with CBT-mindedness beyond social anxiety (McLellan et al., 2019) and quasi treatment-seekers (McLellan et al., 2016), service users were demographically homogeneous, mainly comprising females in their early forties who resided in major Australian cities. It is unclear if results would vary with a more heterogeneous sample. Additionally, although the inclusion of treatment for both anxiety and depression is a strength this mean that multiple sub-samples were combined in our analyses. While we controlled for these groupings statistically, future research should investigate whether CBT-mindedness or change therein can predict outcomes across clinical sub-populations. For example, does low CBT-mindedness differentially predict treatment outcomes across self- and clinician guided iCBT? Further, the specificity of the CBT-mindedness is yet to be tested. Future research should assess whether CBT-SUITS scores are especially relevant in predicting outcomes of CBT vs other therapeutic interventions. This will be particularly important for the subscales of the CBT-SUITS that may be conceived as relevant to many therapeutic approaches (i.e. the Insight and Behavior subscales).

Despite these limitations, the results point to some potential implications for clinical practice. It is possible that the experience of CBT itself could promote CBT-mindedness, and increases in CBT-mindedness may promote symptom reductions. The iCBT programs used in this study provide clear psychoeducation and a strong theoretical rationale for skills. While this study is correlational and uncontrolled, so the direction and temporal sequence of the relationship between CBT-mindedness change and symptom change is unclear, these results point to the potential benefit of promoting CBT-mindedness in treatment. This might be achieved in practice by, for example, explicitly encouraging service users to see that changing their behavior or their own actions will change their feelings or problems. Furthermore, given that baseline CBT-mindedness did not predict outcome in this study, it may not be a contra-indication to proceed with CBT for those who do not immediately resonate with a CBT mindset. Ultimately, however, because we did not test the mechanisms by which CBT-mindedness increased, or the temporal sequence of the relationship between change in symptoms and CBT-mindedness, future work involving experimental investigations are needed to examine the possible causal relationships between CBT-mindedness and treatment outcome.

Conclusions

Overall, this study provides additional psychometric detail about the CBT-SUITS. A three-factor structure was confirmed, albeit with a reduced 11-item scale. Scores were generally weakly or unrelated to demographic variables and distress, and anxiety and depression symptom severity. While CBT-mindedness did not predict treatment completion and predicted post-treatment distress (but not depression or anxiety symptoms), this study showed, for the first time, that treatment produced small to moderate effect size increases in CBT-mindedness. Furthermore, change in CBT-mindedness predicted reduced post-treatment distress, and anxiety and depression symptom severity. There is growing evidence that CBT-mindedness may be transdiagnostic and an additional construct to target in an attempt to improve outcomes for more patients receiving CBT (iCBT in this case).