Introduction

The delivery of psychological therapies at scale is an important international issue, with many countries developing programmes for the dissemination of evidence-based practice across common mental health problems. If these are to be successful, systems for training large numbers of therapists to competence (benchmarked to a required standard) must be developed and evaluated. Whilst evaluating the acquisition of therapy competence and patient outcomes during training is important, monitoring the maintenance of competence and patient outcome post-training in the practice setting is essential to realise the public benefit.

Over the last decade, two national training programmes with different training models have substantially increased the availability of evidence-based treatment with promising results. In the UK, the Improving Access to Psychological Therapies (IAPT) initiative, which focuses on evidence-based treatments for depression and anxiety disorders, has reported large pre-post treatment effect sizes for depression and anxiety cases (Clark 2011, 2018a) with large numbers of therapists trained to an agreed level of competence (Branson et al. 2015; Liness et al., under review). In the USA, the Department of Veteran Affairs (VA) focuses on evidence-based treatments for posttraumatic stress disorder and related conditions, with over 10,000 practitioners trained (Rosen et al. 2017) and patient clinical improvement with medium to large effect sizes across disorders (Eftekhari et al. 2013; Karlin et al. 2012). Availability of treatment for common mental health disorders is also being scaled up in low and middle-income countries (Lund et al. 2016; Singla et al. 2017). Promising evidence indicates that non-specialist practitioners can be trained to deliver effective treatment in these settings (Patel et al. 2017; Singla et al. 2017). Whilst a positive picture is emerging for the dissemination of evidence-based therapies, success must be measured in the longer-term by assessing therapists’ clinical competence once they are fully embedded within the clinical services. Two inter-related aspects of therapist skill are involved in assessing the standard of therapy: (1) therapy adherence—whether the therapist implemented the relevant procedures (2) therapy competence—whether procedures were implemented in a competent manner (Blackburn et al. 2001). These are commonly seen as requisite for the effective delivery of therapy in routine practice (Barber et al. 2003; Fairburn and Cooper 2011; Muse and McManus 2013). If training is to demonstrate sustained effects, it is important that therapy adherence and competence, and good clinical outcomes are maintained. This study focuses on therapy competence and clinical outcomes.

Information on the sustained use of evidence-based interventions in the workplace after training has been captured in surveys and questionnaires (Karlin et al. 2012; Liness et al. 2017; Rosen et al. 2016), with trainees self-reporting continued use. However, self-report may be an inaccurate reflection of therapeutic practice (Walfish et al. 2012). Psychological therapists may also drift from evidence-based interventions following qualification (Waller and Turner 2016), thus demonstrating the need for on-going evaluation.

The few studies employing independent observer ratings on clinical assessment to investigate long-term maintenance of trainees’ competence and patient outcomes have generated mixed results. CBT competence and patient outcome improved pre- to 6 months post-training and were maintained at 12-months follow-up for 12 therapists who received brief training in the treatment of depression in the community (Simons et al. 2010). A comparison of three CBT training methods similarly found that adherence and competence—assessed via structured role play—improved pre-to- post-training, and that therapists in a seminar plus supervision and web condition retained gains following training (Sholomskas et al. 2005). Additionally, a study of low-intensity practitioners delivering self-help CBT in the IAPT programme reported significantly better clinical outcome 12 months after training (Branson et al. 2018). However, novice therapists followed-up at 6-, 12-, and 18-months post-training in computer-assisted CBT for anxiety (Brown et al. 2013) demonstrated significant reductions in adherence and a non-significant trend towards reduction in competence over time; booster-training was proposed for novice therapists to limit potential decline in protocol adherence.

The influence of therapist competence on patient outcome presents as a mixed picture (Webb et al. 2010) with some evidence for a relationship to outcome in CBT for depression (Zarafonitis-Müller et al. 2014) and social anxiety disorder (Ginzburg et al. 2012). Higher CBT competence, but not adherence, was associated with better clinical outcomes in novice therapists (Brown et al. 2013). No general relationship was found on UK IAPT-based training courses (Branson et al. 2015; Jolley et al. 2015) apart from a small relationship in depression cases (Liness et al., under review) and some evidence that patients treated by the most competent therapists are more likely to achieve reliable improvement in symptoms (Branson et al. 2015, 2018). Given the mixed findings, small sample sizes, and varied methods, measures, and focus of these studies, further follow-up studies are essential to assess the long-term benefits of large-scale training initiatives, the role of therapist competence, and the transfer of skills from training to the workplace.

Training, Supervision and Service Integration

A work environment and structure that supports training initiatives and encourages therapist development may enhance the transfer of skills learned and maintenance of competence (Beidas and Kendall 2010; Stirman et al. 2016). Within that structure, the importance of on-going consultation and supervision post-training has been highlighted (Edmunds et al. 2013; Herschell et al. 2010; Miller et al. 2004); however, empirical support for the role of supervision in CBT training is limited. A systematic review examining the effects of supervision on CBT therapist competence across five studies reported potential effects on novice therapists’ competence (Alfonsson et al. 2018). Palliative care practitioners randomised to supervision vs no supervision post-brief CBT training differed significantly in skills 6-months post-training, with the no supervision group decreasing and the supervision group increasing (Mannix et al. 2006). Consultation following brief CBT training for adolescent anxiety was deemed essential, as number of consultation hours predicted greater therapist adherence and skill at 3-month follow-up (Beidas et al. 2012).

Informed by emerging evidence, supervision or consultation is a key part of both the IAPT and VA programmes. Weekly ninety-minute group telephone consultation with an expert clinician is built into the VA training model (Karlin et al. 2012). IAPT training incorporates weekly 90-min group supervision for 30 weeks from a CBT therapist accredited by the British Association of Behavioural and Cognitive Psychotherapy (BABCP) as part of the course (Department of Health [DoH] 2018). Both the VA and IAPT supervision models involve case review, feedback on audio-taped sessions and a focus on skill development. IAPT trainees also receive weekly individual supervision from a BABCP-accredited CBT therapist at their work placement site, and evidence 70 h of supervision by the end of training. Supervisors in IAPT services also attend supervision training provided by their trainees’ university to enhance collaboration and to encourage comparable supervision methods during and after training. IAPT graduates who remain in IAPT services continue to receive regular supervision from a BABCP-accredited CBT therapist. Given the specific design of the IAPT initiative, and emerging evidence for the importance of on-going supervision and consultation, exploration of this topic at follow-up is warranted.

IAPT CBT Training

IAPT CBT trainees work in IAPT services whilst being trained in evidence-based CBT for adults with depression and anxiety disorders. They attend university two days per week and work in IAPT clinical-services three days per week, where they maintain a caseload of twelve cases. Training follows a national curriculum (DoH 2011) covering behavioural activation and cognitive therapy models for depression, and evidence-based protocols for anxiety disorders (Roth and Pilling 2008). Trainees are required to treat a range of disorders and are formally assessed on both depression and a range of anxiety cases. Therapists in-training are closely monitored, with regular skills assessments, on-going informal and formal skills feedback, weekly clinical supervision involving live review of therapy clips and whole sessions using the Cognitive Therapy Scale -Revised (CTS-R; Blackburn et al. 2001), and patient outcome measure monitoring. They are encouraged to see training cases, defined as straightforward anxiety or depression presentations with clear main problem descriptors, to enable the trainee to learn National Institute for Health and Care Excellence (NICE) guidance evidence-based protocols for specific difficulties (Roth and Pilling 2008). CBT trainees submit eight cases for formal assessment. IAPT trainees are employed on 1-year training contracts, with an expectation that trainees will continue to work in IAPT services post-training, usually with an increase to full-time work with larger and more diverse caseloads. The intensive training, time, and resources invested in IAPT and other large-scale workforce initiatives, and the lack of studies evaluating the long-term benefits of training, makes follow-up assessment important. This study was set up to investigate therapist competence and patient outcomes during training and post-training when working as qualified CBT therapists.

Aims and Hypotheses

This study aimed to assess if IAPT trainees’ CBT competence gained during training was maintained after training. It also set out to evaluate trainees’ patient outcomes during and after training. It was predicted that

  1. (1)

    the level of CBT therapy competence attained during training would be maintained post-training

  2. (2)

    trainees would achieve good patient outcome in training and at follow-up.

The relationship between therapist competence to patient outcome was also explored, and descriptive information on patient demographics and supervision provision at follow-up was gathered to evaluate continuity of practice in the workplace.

Method

Participants

Therapists were 45 former trainees from seven cohorts (2008–2015) of the IAPT Postgraduate Diploma in CBT at King’s College London. Eligibility criteria were course completion by September 2015, and current employment in a service with ethics approval. Exclusion criteria were primary employment in a non-clinical role at follow-up. Therapist were 73% (n = 33) female and 27% (n = 12) male, and 91% (n = 41) were white and 9% (n = 4) were from a black and/or minority ethnic background. Therapist professions were clinical psychologist (35%, n = 16), psychological wellbeing practitioner (31%, n = 14), mental health nurse (11%, n = 5), counsellor/psychotherapist (9%, n = 4), counselling psychologist (7%, n = 3) and other (7%, n = 3). Median age at training was 33.00 years (IQR = 9.00).

Former trainees were contacted at least 1-year following course completion regarding study participation. Due to timing of ethics approval, trainees clustered into two groups based on time to follow-up: prospective follow-up after 12 months (42%, n = 19) and retrospective follow-up after 12+ months to 6 years (58%, n = 26). Of 119 contacted trainees, 45% (n = 54) agreed to participate. Of those who declined, 43% (n = 28) declined due to workload, 34% (n = 22) due to life-events, and 23% (n = 15) did not respond. Of those who agreed to participate, 83% (n = 45) completed the study. All nine non-completers withdrew due to workload. There were no significant differences in age, gender, profession, pre-training CTS-R scores, mean end-of-training CTS-R scores, or final course result between trainees who completed the study and those who declined to participate or those who withdrew, p > .05.

Measures

Therapist competence was evaluated via assessor ratings of audio-recorded therapy sessions with the 12-item Cognitive Therapy Scale-Revised (CTS-R; Blackburn et al. 2001): score range = 0–72; competence threshold set at ≥ 36. Items are rated from 0 to 6 (0 = non-competent, 6 = expert). Items can be separated into two sub-scales: general therapy skills (Items 1–5) and CBT-specific skills (Items 6–12). The CTS-R has been criticized for inconsistent inter-rater reliability (Gordon 2006; Reichelt et al. 2003; Rozek et al. 2018), a lack of focus on specific mental health problems, and the lack of an empirically validated competence threshold (Fairburn and Cooper 2011; Muse and McManus 2013), although a score of ≥ 36 (item mean ≥ 3) is recommended in the manual. However, the CTS-R is the most widely-used measure of CBT competence on UK training courses and incorporates assessment of adherence as well as competence within item descriptors (James et al. 2001). A more recent study (Kazantzis et al. 2018) reported acceptable internal consistency and inter-rater reliability. In the present study, internal inter-rater reliability was high between pairs of the twenty course markers for a random sample of 25% (n = 31) of the 123 submitted tapes: one-way random single measures ICC (31,30) = 0.85. Lack of resources mean that in most studies course tutors or supervisors are used to rate tapes; however, in the present study a randomly-selected 18% (n = 22) of tapes were second-rated by an external expert who was blind to time (pre-training, post-training, or follow-up). Inter-rater reliability between course and external markers was high: ICC (22, 21) = 0.80.

Clinical outcomes were evaluated using patient self-report questionnaires routinely collected at the start and end of therapy. Depression symptoms for all cases were assessed with the Patient Health Questionnaire (PHQ-9; Kroenke et al. 2001). Anxiety symptoms for all cases were assessed with the Generalized Anxiety Disorder-7 (GAD-7; Spitzer et al. 2006). If provided, anxiety disorder-specific symptoms were assessed using one of the following measures: Social Phobia Inventory (SPIN; Connor et al. 2000), Penn State Worry Questionnaire (PSWQ; Meyer et al. 1990), Obsessive Compulsive Inventory (OCI; Foa et al. 1998), Panic Disorder Severity Scale (PDSS; Shear et al. 1997), Mobility Inventory (MI; Chambless et al. 1985), Impact of Events Scale-Revised (IES-R; Weiss and Marmar 1996), Short Health Anxiety Inventory (sHAI; Salkovskis et al. 2002).

Case complexity was benchmarked against course criteria for complexity, namely primary or comorbid psychosis, personality disorder, autism spectrum disorder, substance dependence, severe and/or treatment-relevant physical health conditions, and severe psychosocial difficulties. Two researchers from the present study rated cases as complex or non-complex based on these criteria extracted from case information. Inter-rater reliability was acceptable, kappa = 0.67, p < .001 (McHugh 2012). All disagreed cases were discussed and an agreed rating was reached.

Design and Procedure

This study employed an observational longitudinal design, assessing trainees’ therapy competence at the start and end of training and at follow-up, and assessing patient outcomes on the course and at follow-up. Therapists and patients provided informed consent for their audio-recordings and clinical data to be used anonymously and confidentially for research purposes.

Therapy Recordings

Trainees submitted therapy audio-recordings of a recent treatment session at the beginning of the course (Baseline), towards the end of training (End-of-Training), and at follow-up (Follow-Up). At all time points, trainees were requested to submit for assessment a representative session rather than their strongest. Only 33 Baseline recordings were available as the baseline evaluation was introduced in 2009; 45 recordings of sessions were available for end-of-training and at follow-up. All course and follow-up therapy recordings were rated by course staff who were accredited CBT therapists and supervisors with formal training and extensive experience rating the CTS-R; assessors were blind to clinical outcome, subject, and session number. Additional CTS-R rater training was not provided for the study as all markers were course staff who had already been trained to use the CTS-R for assessment. All staff who join the course are provided with induction training in using the CTS-R, which includes blind marking multiple recordings with an experienced marker until high inter-rater agreement is achieved. Following the initial training, on-going reliability monitoring of CTS-R ratings comprises second-ratings on all tapes below the competence threshold (CTS-R < 36) and on a random selection of approximately 20% for each tape submission to check rater drift. As reported above, both internal and external interrater reliability was high for randomly-selected tapes in the present study.

Each recording was scored by a randomly selected course marker. Written reviews and ratings of sessions were provided to trainees within a month of submission to encourage incorporating feedback into clinical practice; markers were thus not blinded to the stage of the study.

Clinical Cases

During the course, trainees identified eight cases for on-going supervision and wrote them up as clinical case reports, including outcome measures, after the patients had completed treatment. Correspondingly, participants at follow-up provided clinical outcome data from their eight most recent cases. At follow-up, trainees also provided case information regarding number of treatment sessions and physical and psychosocial comorbidities, and the frequency and content of their current supervision. Data were available for 720 cases in total, with 360 from during the course and 360 at follow-up. Primary diagnoses of the cases on the course were anxiety disorders (73%, n = 264) and depression (27%, n = 96). Primary diagnoses at follow-up were anxiety disorders (58%, n = 209), depression (40%, n = 143), and other (2%, n = 8). Distribution of primary diagnoses differed significantly between course and follow-up cases, exact X2 = 23.64, p < .001. There was a significant difference in the proportion of complex cases on the course (n = 5, 1%) than at follow-up (n = 128, 35%), X2 = 139.53, p < .001. Cases from two therapists were excluded due to missing outcome measure data, thus clinical outcomes analyses were conducted based on 43 therapists’ data. The Benjamini–Hochberg Procedure (Benjamini and Hochberg 1995) was used to correct for multiple comparisons across all hypothesis tests, with false discovery rate Q = 0.05.

Results

Therapist Competence

A repeated-measures analysis of covariance (ANCOVA) found a significant effect of time (Baseline vs End-of-Training vs Follow-Up) on CTS-R scores, controlling for time to follow-up (12 months vs 2–6 years): F(1.83, 57.02) = 11.46, p < .001, partial η2 = 0.27. Post-hoc tests found a significant difference between the estimated marginal means for Baseline (M = 30.45, SD = 4.20) and End-of-Training (M = 40.23, SD = 2.86) tapes, p < .001, but no significant difference between End-of-Training and Follow-Up (M = 40.15, SD = 3.98), p = 0.10.

Having established an overall effect on total scores, corresponding ANCOVAs for the CTS-R subscales (generic and specific) were conducted. There was a significant effect of time on generic CTS-R subscale scores: F(2, 62) = 6.17, p = 0.004, partial η2 = 0.17. Scores differed significantly between Baseline (M = 14.04, SD = 1.82) and End-of-Training (M = 17.10, SD = 1.37) p < .001, but not between End-of-Training and Follow-Up (M = 17.70, SD = 1.84), p = 0.12. There was also a significant effect of time on specific CTS-R subscale scores: F(1.64, 50.98) = 7.42, p = 0.003, partial η2 = 0.19. Scores differed significantly between Baseline (M = 16.41, SD = 2.89) and End-of-Training (M = 22.26, SD = 1.83) p < .001, but not between End-of-Training and Follow-Up (M = 22.45, SD = 2.36), p = 0.68. Figure 1 reports the mean total and subscale CTS-R scores across time; both total and subscale CTS-R scores increased significantly across training and were maintained at follow-up.

Fig. 1
figure 1

Total and subscale CTS-R scores across time

Full size image

Therapist Competence at Follow-Up

At follow-up, 85% (n = 38) of therapists remained competent and 15% (n = 7) were non-competent. At follow-up, no significant (p > .05) differences were found between competent and non-competent therapists using χ2 tests for gender, highest educational qualification, and profession, and Mann–Whitney test for baseline CTS-R total or subscale scores, and end-of-training total or subscale CTS-R scores. Follow-up total and subscale CTS-R scores were significantly greater for competent than non-competent therapists, reported in Table 1.

Table 1 Follow-up CTS-R scores for competent and non-competent therapists
Full size table

Clinical Outcomes

Effect sizes on the course and at follow-up were the mean Cohen’s d for therapists’ eight cases based on national standard deviations by diagnosis (depression, anxiety, or other; NHS Digital 2018). Reliable improvement (RI) rates on the course and at follow-up report the percentage of therapists’ cases that reduced by the clinically-reliable change index for ≥ 1 measure and did not reliably deteriorate on the other (DoH 2014). Appropriate disorder-specific measures were used in place of the GAD-7 for anxiety disorders by diagnosis (DoH 2014). Table 2 reports repeated-measures ANCOVAs assessing change in 43 therapists’ effect sizes and reliable improvement rates over time (Course vs Follow-Up), controlling for time to follow-up. There were no significant changes in clinical outcomes between the course and follow-up.

Table 2 Therapists’ clinical outcomes for course and follow-up
Full size table

Relationship Between CTS-R Scores and Clinical Outcomes

Table 3 reports Spearman correlations between total and subscale CTS-R scores and clinical outcomes for end-of-training and follow-up. There were no significant correlations. There was a positive trend (p = 0.06) between the end-of-training total CTS-R and PHQ-9 effect size. At follow-up, the total CTS-R exhibited positive trends (p = 0.05–0.09) with RI rate and GAD-7 effect size. There were also trends at follow-up (p = 0.07–0.08) between the generic and specific subscales with RI rate, and the specific subscale with GAD-7 effect size.

Table 3 Association between CTS-R scores and clinical outcomes
Full size table

Mann–Whitney tests found no significant differences between competent and non-competent therapists for clinical outcomes, reported in Table 4. There was a trend (p = 0.06) for greater PHQ-9 effect sizes for competent therapists but overall clinical outcomes were stable from training to follow up in both groups.

Table 4 Clinical Outcomes for Competent and Non-Competent Therapists at Follow-Up
Full size table

Clinical Role and Supervision

During training, therapists received weekly 90-min group supervision from a BABCP-accredited CBT therapist for 30 weeks as part of the course. Supervision content included review of live therapy clips every session and discussion of on-going training cases. Trainees also received weekly individual supervision from an accredited CBT therapist at their work placement site, and were required to evidence 70 h of such supervision by the end of training.

At follow-up, 89% (n = 40) of therapists were still employed in IAPT services, and 64% (n = 29) of those were employed within the NHS trust in which they had trained, while 11% (n = 5) were working in CBT specialist services. Supervision was provided by accredited CBT therapists in line with IAPT procedure. At follow-up, 49% (n = 22) of therapists continued to receive weekly and 40% (n = 18) fortnightly clinical supervision, with a median duration of 1.5 h (IQR = 1 h). Competent therapists did not differ significantly from non-competent therapists in the frequency of supervision received: U = 124.50, p = 0.77. Unlike course supervision where audio tape therapy review is an integral part of supervision, only 27% (n = 12) of therapists reported continued use of live supervision methods, i.e. review of actual therapy sessions at follow-up. While 29% (n = 11/38) of the competent therapists at follow-up still reviewed tapes in supervision compared to 14% (n = 1/7) of the non-competent therapists, this difference was not significant: exact X2 = 0.65, p = 0.39.

Therapists reported offering an adequate course of therapy with a mean of 11.82 (SD = 3.20, range = 8–19) treatment sessions on the course, and 12.40 (SD = 5.02, range 7–23) at follow-up.

Discussion

This study aimed to assess if trainee therapists’ gains in competence and clinical outcomes attained during IAPT CBT training were maintained post-qualification in the workplace, and to investigate the relationship between competence and clinical outcomes at both time points. CBT competence was evaluated at the beginning and end of training, and at least 12 months following qualification using the CTS-R. All trainees achieved a recognised minimum level of competence by the end of training, and the majority maintained that level of competence at follow-up. Trainee clinical outcome was also examined, with course and follow-up cases demonstrating significant improvement across all measures despite an increase in complex presentations at follow-up.

CBT Competence

IAPT trainees achieved therapy competence by the end of the course, and the majority maintained competence when working as qualified therapists, notably in line with the expected competence threshold (Blackburn et al. 2001; Branson et al. 2015; Keen and Freeston 2008). While baseline CTS-R scores were lower in the present study (M = 30.45) than in some other CBT training studies (35.72–38.94; see Barnfield et al. 2007; Branson et al. 2015; Keen and Freeston 2008), trainees obtained competence in line with previous literature by the end of training. The CTS-R subscales for generic and CBT-specific items also improved and were maintained at follow-up, adding to the evidence for post-training maintenance of both generic and CBT-specific competence (Sholomskas et al. 2005; Simons et al. 2010). The subset who did not maintain competence post-training was small, which makes conclusions difficult. The seven therapists who were non-competent at follow-up nearly reached CTS-R competence (36) with a median of 33.50, perhaps indicating drift rather than major relapse and that additional input may have enabled them to maintain competence.

Patient Outcome

This study adds to the growing literature that trainees obtain excellent patient outcomes during CBT training (Forand et al. 2011, Öst et al. 2012) and on large workforce training programmes (Branson et al. 2015; Eftekhari et al. 2013) by extending it to post-training. It also supports the maintenance of excellent patient outcomes post-training in the face of increased caseload and complexity. Therapists delivered doses of therapy in accordance with NICE (2011) guidance at both time-points, possibly promoting patient gains. High average number of treatment sessions have been found to improve therapy outcomes (Clark et al. 2018b).

Therapist Competence and Patient Outcome

No significant relationship between therapist competence and patient outcome was found at the end of training or at follow up, although positive trends emerged. Previous training studies reported similar results (Branson et al. 2015; Jolley et al. 2015; Liness et al., under review). There was also no significant difference in patient outcome between competent and non-competent therapists at follow up. The absence of significant findings was likely influenced by the small number of non-competent therapists (n = 7) at follow-up, a somewhat ironic consequence of so many therapists retaining competence. The lack of an association of therapist competence to patient outcome may be due to the restriction of range in therapist competence (Öst et al. 2012). Other factors that may influence the competence-outcome relationship and patient outcome at any given time including therapeutic alliance (Webb et al. 2010) and patient characteristics such as symptom severity and chronicity, motivation and life events (Clark et al. 2018b; Webb et al. 2010), were not accessible in this study. Investigating the effect of patient diagnosis was not feasible in the current study, but diagnosis may moderate the competence-outcome relationship– with stronger evidence for depression (Kazantzis et al. 2018; Webb et al. 2010; Zarafonitis-Müller et al. 2014) and in a single study using a disorder-specific competence measure for social anxiety (Ginzburg et al. 2012). Additionally, the time-point of the recording in the therapeutic process may influence the competence-outcome relationship (Kazantzis et al. 2018), but was not feasible to manipulate in the present study. Emerging positive trends in this study highlight the need for future investigations aimed at investigating the competence- outcome relationship and moderators therein. Future studies would benefit from larger numbers, varied therapist abilities, diverse patient presentations, varied and disorder-specific competence measures, and evaluation of the competence-outcome link at several time-points in the therapeutic process.

Clinical Role and Supervision

The majority of therapists in this study continued to work in IAPT services post-training and a substantial number remained in the same NHS trust. Patient characteristics at follow-up were similar to training cases, with complexity increased slightly as expected. Trainees continuing to work in similar workplace settings and see similar patient presentations may assist the consolidation of therapist competence, and contrasts with previous large-scale training initiatives where skill maintenance difficulties were partially attributed to a lack of continuity from training to the work environment (Brooker et al. 2003). Therapists’ continued use of regular expert supervision may also have supported maintenance of competence (Mannix et al. 2006; Rakovshik et al. 2016). Therapists however reported reduced use of ‘live’ supervision (i.e. review of therapy tapes). This is a key characteristic of training supervision, and its quick reduction post-qualification is noteworthy as therapists have higher and more complex caseloads. Only 1/7 non-competent therapists reported that supervision included feedback on audio recordings of therapy, possibly indicating the usefulness of on-going review and feedback of actual therapy sessions after initial training (Bearman et al. 2017; Weck et al. 2017); however, competent and non-competent therapists did not differ significantly in post-training access to live supervision or in frequency of supervision overall. Small numbers and the lack of a control condition make conclusions difficult. Further investigation is needed on a larger sample post-training.

Limitations

Present findings have the constraints of a naturalistic study with no control condition. Therapists self-selected audiotaped sessions that were submitted for assessment and also patients that they brought to weekly supervision and subsequently included in their case reports. The representativeness of both the audio sessions and patients is unknown as submitted cases only partially reflect overall caseloads. However, trainees were instructed to select representative audio tapes, and patient selection occurred before clinical outcome was known. Clinical outcome at follow-up was requested for the most recent eight cases irrespective of outcome. Several therapists were unable to complete the study due to workload, consequently the sample size was smaller than intended. Delay in ethics approval also affected participation and led to varied timings of follow up submissions for the earlier cohorts. Recruitment, ethics approval and attrition are common difficulties in workplace research (Brosan et al. 2007; Stirman et al. 2015). Further evaluation of training effects and maintenance of competence is crucial for the sustainability of workforce training such as the IAPT programme, and perhaps merits increased resources and funding for services to enable timely research and busy therapists’ participation.

Conclusions and Future Directions

This is the first study to our knowledge that evaluates CBT therapist competence and patient outcomes from the IAPT workforce training programme during training and at follow-up. The maintenance of therapists’ gains post-training is promising for large-scale evidence-based training worldwide. The highly-structured IAPT set-up that encourages training and service collaboration, and emphasizes the importance of regular clinical supervision both during and after training, possibly has a role to play in the sustainability of competence. The generalizability of this structure to less resource-rich environments is unknown and requires further long-term evaluation and access to the necessary resources to increase participation by busy practitioners. The focus on alternative clinical assessment and supervision methods, the training of non-mental health professionals, and the creation of on-line resources are all encouraging developments for embedding evidence-based practices worldwide (Fairburn and Patel 2014; Kobak et al. 2017). Overall, this study demonstrates that a large-scale training programme can equip therapists with sustainable competence to address the challenge of mental health in the population.