Background

Acceptance and commitment therapy

Acceptance and commitment therapy (ACT) has been shown to be efficacious in numerous disorders, such as anxiety and depression (e.g., [1]), psychosis (e.g., [2]), phobias (e.g., [3]), alcohol use, anger, and stress (e.g., [4]), obesity (e.g., [5]), burnout [6], and treatment-resistant patients [7]. ACT aims to increase psychological flexibility, which has been shown to increase mental health in over 250 randomized controlled clinical trials across numerous disorders [8]. Psychological flexibility is defined as someone’s ability to “recognize and adapt to various situational demands; shift mindsets or behavioral repertoires when these strategies compromise personal or social functioning; maintain balance among important life domains; and be aware, open, and committed to behaviors that are congruent with deeply held values” ( [9], p865). Preliminary evidence suggests that symptomatology itself does not explain whether someone is functioning with a high level of well-being [10] and there is also evidence that psychologically flexible responses to changing demands promote advantageous outcomes [11]. This speaks to a tendency in basic behavioral science and research away from disorder-specific characteristics and processes, towards processes that potentially underlie multiple disorders and clinical states [12]. However, despite ACT being a transdiagnostic approach [13], there are few transdiagnostic (i.e., applicable across diagnostic categories as opposed to a single disorder) effectiveness studies (e.g., [14, 15]). Further, despite the studies investigating ACT’s efficacy, important questions remain, for example, specification of the mechanisms of action, stability of change, and how well it can be implemented into routine clinical practice [16]. However, in clinical trials, the mechanisms of change are far from clear, and integrated assessment is as necessary as it is difficult to implement. Overall, this study investigates three key aspects: The effectiveness of ACT, long- term follow-up, and the social context and social processes.

Effectiveness of acceptance and commitment therapy

There is also a need to improve our understanding of the effectiveness of ACT, stressing real-world settings across broad populations with the goal of maximizing external validity [17]. While the importance of Randomized Controlled Trials (RCTs) and efficacy studies is evident, such tightly controlled and randomized studies might neglect crucial factors of what is done in the field [18], and thus less likely reflect conditions under which interventions are used in common clinical practice [19]. Non-randomized trials are often more externally valid than RCTs because they include all patients, are conducted under conditions that more closely match how practitioners will ultimately implement the treatment, and include more context variables (such as patient-practitioner relationship, [20]). Objective and subjective ratings need to be assessed, for instance through paradigms assessing psychological flexibility during a psychological challenge (i.e. a behavioral approach test).

Long-term follow-up

Relatively little is known about the processes that unfold following treatment, for instance, whether or how patients integrate newly learned skills into their daily environment, and how these factors interrelate with well-being of both the patients and their social networks. Processes following treatment have traditionally been examined with a single assessment concentrating on symptomatology, which is why researchers have been encouraged to include longer follow up periods in clinical studies [21]. The period following treatment is risky if patients stop applying what was learned during therapy or are confronted with challenges [21, 22]. This is especially true for treatment-resistant patients (i.e., patients who do not respond to standard, first line treatments), for whom viable treatment options are lacking and even less is known about than about first-line treatments [7]. To determine if ACT can meaningfully impact treatment-resistant patients in routine practice, a study population including treatment-resistant patients across diagnoses treated in real-world settings is essential. Inclusion of non-treatment-resistant patients, however, is important as well to inform about treatment-resistant ones.

Social context and social processes

Factors outside therapy itself, including social processes, are believed to account for approximately 33% of improvement in patients undergoing psychotherapy [23]. Despite the importance of social context, it remains poorly understood how the influence of social surroundings (e.g., supportive vs. counterproductive) longitudinally affects the patient’s well-being and prosociality (i.e., cooperating with others, acting for the well-being of others, and sacrificing for others, [24, 25]) and how this in turn affects the patient’s social context. Tracking the transition into the real world after treatment is therefore crucial. The challenges and risks of relapse during this period may be more extreme for inpatients than outpatients, as their change of environment (i.e., leaving the hospital) is more extreme; however, this has not been tested directly. Furthermore, it is completely unknown how the influence of these social variables is moderated by psychological flexibility. Patients treated in an inpatient setting with more negative relationships in their extended social network have been found to relapse more frequently [26]. This suggests that both close and extended social ties are relevant for outcomes, and that these impact multiple forms of treatments. More research is sorely needed to better understand the mechanisms that influence a patient and their social context, particularly in disorders beyond depression and in non-cross-sectional studies that allow the temporal order of effects to be disentangled [27, 28]. The few existing longitudinal studies have identified a bidirectional link between couple distress and depression [27, 28] and to a lesser extent between couple distress and substance abuse [29, 30] and between couple distress and anxiety [30]. Relatively little emphasis has been placed on outcomes of functioning, well-being, or prosocial behaviors – neither in the treated patient nor the immediate and extended social network. As such, little is known about the interaction between treatment, social context, and these broader outcomes.

Building on these insights, the necessity to understand patients’ behavior in their natural environment becomes clear [31]. Daily life happens in specific environmental contexts, and there is a need to understand these contexts. Implementing Event Sampling Methodology (ESM) allows the examination of patients’ daily life and its stressors, including the assessment of moods, thoughts, symptoms or behaviors, which change over time [31, 32]. Thus, ecologically valid data can be collected in a real-time fashion while capturing dynamic changes of variables [33].

Study aims

The major aim of Choose Change is to longitudinally examine the mechanisms of action involved in an ACT treatment for transdiagnostic patients with varying degrees of treatment experience and treatment success in a controlled effectiveness trial in order to maximize external validity [17]. More specifically, it will be tested how psychological flexibility training (i.e., ACT) influences various outcomes across time and document how the intervention is implemented in patients’ everyday lives following the treatment. Simultaneously, state-of-the-art methods are used to ascertain processes of change and maintain internal validity. Increased use of technological innovations, such as mobile phones, will provide us with more information about the common and specific effects of psychological treatments [34]. This is the first project to test the effects of the social context of treatment settings (in- vs. out-patient) and the transition out of intensive treatment to well-being, functionality, and recidivism following a modern transdiagnostic behavioral treatment promoting psychological flexibility.

Main research areas

We will investigate the immediate and long-term outcomes of treating treatment-resistant patients with psychological flexibility training (i.e., ACT) in two treatment-related social contexts (i.e., inpatient and outpatient), and the influence of other, naturally occurring social contexts (e.g., self-chosen contexts such as home, work etc.) on various treatment outcomes. These research areas include broad outcomes (i.e., indices of well-being, functioning, social interactions, and prosocial behavior) in addition to symptoms and recidivism, thus going beyond a symptom-based focus.

On the basis of these main research questions, we derived the following main hypotheses: First, we expect the ACT intervention to lead to significant and clinically relevant changes during treatment that remain stable or increase at 12-month follow-up. Second, patients treated in the inpatient setting will experience more intense and frequent social stressors in the follow-up period than those treated in the outpatient setting, which will have a negative impact on outcomes. Third, psychological flexibility will moderate the negative impact of social stress on outcomes over time, so that it is buffered in patients with high levels of psychological flexibility.

Method

Design

This is a controlled effectiveness clinical trial. The study contains multiple seven-day-ESM phases, four intensive follow-up assessments at 1, 4, 9, and 12 months following intensive treatment, weekly measures during treatment, and a behavioral approach test, while considering the effects of the social context of treatment settings (in- vs. outpatient), and following a modern transdiagnostic behavioral treatment promoting psychological flexibility. Figure 1 illustrates an overview of the study design.

Fig. 1
figure 1

Illustration of the study design. Note. Smartphone, Heat Induction Test, Assessment is administered at Baseline, Post, and 12-month follow-up. Process assessment is assessed weekly during the intensive treatment phase by the patient, therapist, and nurse (in the case of the inpatient setting). Assessment through questionnaires and interview are administered at 1-, 4-, and 9-month follow up. BL: Baseline; P: Patient; T: Therapist; N: Nurse (only for the inpatient setting). FU-1, FU-4, FU-9, FU-12: Assessments at 1-, 4-, 9-, and 12-month follow up

Full size image

Sample size

Power analyses were conducted with alpha = 0.05, power = 0.8, two-sided test, for within group and between group hypotheses. We assumed a correlation between time points = 0.5 and based on previous work (Gloster et al., 2015), a small to medium effect size was assumed. Required sample sizes were 75 individuals for within group research and 150 patients (75 per group) for between-group hypotheses. Taking missing values into account (ca. 10% dropout), the required sample size in order to have sufficient power for all three hypotheses was thus approximately 170 individuals. We therefore aim to recruit n = 85 patients per treatment modality.

Participant recruitment and selection criteria

Participants are recruited from two specialized clinics practicing ACT (inpatient and outpatient) from ongoing referral and intake procedures. Since ACT is a transdiagnostic therapy which is efficacious for many disorders, and in order to increase generalizability within this effectiveness trial, selection criteria are purposefully kept to a minimum. Inclusion criteria are: Minimum 18 years of age, ability to speak German sufficiently, present for therapy and able to attend sessions, previous treatment (i.e., ≥ 20 sessions of empirically supported psychotherapy and/or minimum dosage and length of an approved drug as recommended by international guidelines), and sign a consent statement will be included the study. In addition, for patients in the out-patient setting, patients will also be included with < 20 sessions therapy experience to enable full modelling of patients from those without treatment experience, to those with treatment experience in the outpatient setting, to those with treatment experience in the inpatient setting. Exclusion criteria include acute suicidal intent, acute substance dependency, (where the primary treatment goal is detoxification), active mania, previous experience with ACT, and inability to read or complete assessments. Otherwise all diagnoses are included (i.e. Affective Disorders, Anxiety Disorders, Somatoform Disorders, Mood Disorders, Anxiety-stress related Disorders, Somatic Disorders, Obsessive-Compulsive Disorder, Impulse Control Disorders, Gambling Disorders, and Personality Disorders). When participants enter the clinic, medication is optimized when necessary, as determined by the attending physician in consideration of patient preference. Medication is then held constant throughout the intensive treatment phase.

Outcome assessments

Assessments in questionnaire and interview format will occur at six primary time points to capture changes that the patients choose in the following domains: Therapy expectancy, psychological flexibility, social interactions, therapy consistent skills, well-being, emotion regulation, symptomatology, stress, and prosocial behavior. ESM will target the same domains. Table 1 shows a brief description of the assessments. Assessments are done by psychology graduates and doctoral students. Assessors are trained to competency and supervised regularly. All utilized assessments were chosen based on covering the respective domain and adequate quality criteria.

Table 1 Description of assessment instruments of the Choose Change study
Full size table

Primary outcome measures will target symptoms and general functioning. These are assessed using questionnaires completed by the patients using the Brief Symptom Checklist (BSCL; to assess symptoms), the World Health Organization Disability Assessment Schedule – 2 (WHODAS 2; to assess functioning), and the Mental Health Continuum – Short Form (MHC-SF; to assess well-being). Primary outcomes are measured at baseline, post, and 12 months following treatment. The MHC-SF is measured at all time points.

Secondary outcomes are included to assess additional domains, such as treatment context, social context, spreading of effects, follow-up, daily context, and behavioral measures (see Table 2 and Table 3). In addition, an experimental paradigm will be used to measure how psychological flexibility impacts patients’ behavioral response to experimentally induced discomfort (i.e., radiant heat pain). This radiant heat pain paradigm is commonly used in research [35, 36] and will be administered at baseline, post, and 12 months after treatment.

Table 2 Overview of domains, constructs, assessments, and respective time points at which each assessment is administered
Full size table

In order to test whether treatment context has an impact on patients’ well-being and functioning (i.e. flourishing) [37], ACT will be administered for both in- and outpatients. For inpatients, a multidisciplinary team is involved, consisting of psychotherapists providing individual and group therapy and the nursing and support staff providing ACT specialized sessions. For outpatients, psychotherapists provide individual ACT sessions. All psychotherapists are certified for this study through attendance of several ACT trainings and provision of a certification role play video rated by a certified ACT trainer. Therapists will make choices based on ACT skills and procedures, which will all be documented following each session. This allows for the documentation of ACT’s clearly defined skills, while enabling a flexible and situationally sensitive application.

Baseline: diagnostics, assessment

During the first week of therapy participants complete informed consent procedures before data collection. All participants complete the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID) [38] to determine diagnostic status. Diagnoses are rated on the Anxiety Disorders Interview Schedule (ADIS) severity rating scale [39]. The diagnosis with the highest severity score determines the primary diagnosis. Following the SCID, participants complete diagnostic-specific and transdiagnostic questionnaires (see Table 2).

Post: assessment

At completion of the intensive therapy phase post assessment is done (i.e. diagnostic-specific and transdiagnostic questionnaires, ESM, and the heat induction test). Except for the SCID and the Credibility and Expectancy Questionnaire (CEQ) procedures and items will be the same as in baseline (see Table 2 and Table 3).

Table 3 Overview of domains and respective time points at which each EMA assessment is administered
Full size table

Follow-up: 1, 4, 9, and 12 months after treatment

In order to assess what was learned during therapy and how psychological flexibility, well-being, and symptoms are associated with each other, patients stay in contact with the research team after completing the intensive inpatient or outpatient treatment phase. Over the following year, four follow-up appointments are conducted at 1, 4, 9, and 12 months after completion of the intensive therapy phase. Follow-up appointments include an online questionnaire and an interview. For the included questionnaires see Table 2.

Assessments measuring mechanisms of action across levels of analysis

Behavioral approach test

A heat induction test (using the TSA-II, [40]) was used in order to test how participants respond to an uncomfortable and ambiguous stimulus not subject to the same demand characteristics as questionnaires. During this test, temperature increases until the patient stops the increase with a click of the mouse. This temperature is then held constant to test tolerance. This test is administered at Baseline and Post (to determine whether the tolerance changes during treatment) and at Follow-Up 12 (to determine whether the tolerance changes after treatment). Before and after participants will complete a pain-related questionnaire.

Weekly psychological flexibility process measure (between baseline and post)

In order to examine whether patients with higher levels of psychological flexibility will profit faster from treatment, weekly psychological flexibility progression will be measured during the intensive treatment phase (between Baseline and Post). For this, the PsyFlex [41, 42] will be used across different information sources. The PsyFlex is a self-developed instrument which aims at measuring state psychological flexibility through capturing all six skills of the ACT Hexaflex [43]. The PsyFlex is filled out by the patients, their therapists, and – in the inpatient setting – their nurse, which results in an evaluation of one person from up to three different perspectives.

Ambulatory monitoring (ESM at baseline, post, and 12 months after treatment)

In order to examine how psychological flexibility and other treatment parameters were applied to stressors in real-time in the participants’ natural environment, how social group interactions affected the individual, and how these aspects changed over time, patients receive a smartphone from the research team and instructions on its usage (i.e. operation of the smartphone, recognition of the signaling tone, answer example questions). Data will be collected at Baseline, Post, and Follow-Up 12. At each time point, patients will carry the smartphone for one week, using a signal-contingent ESM approximately every three hours (e.g., 8 am, 11 am, 2 pm, 5 pm, 8 pm, and 11 pm). Items stem from previous ESM studies [44,45,46,47] and are based on a functional analysis of social interactions [48] due to their individual nature. In addition to the data assessed via questionnaires, GPS data (e.g. time-stamped data regarding location, speed of travel if travelling, and time spent at different locations) of the patients will be collected during the respective ESM week. For assessed domains within the ambulatory assessement, see Table 3.

Close person (baseline, post, and 12 months after treatment)

In order to investigate the impact of the social context, patients are asked to give the research team permission to contact a person of their choice, who is close to them. This close person is then contacted by the research team. After obtaining informed consent from this close person, an online questionnaire is completed at Baseline, Post, and at Follow-Up 12 by the close person, providing information about their relationship and well-being. For the included questionnaires see Table 2.

Therapy integrity

Therapy will consist of an intensive treatment phase lasting approximately twelve treatment sessions. Therapists will be certified for this study. Therapy sessions will be recorded and of those, two will be randomly selected and rated by external and independent ACT experts to assess treatment integrity. Therapy integrity will be further promoted via regular targeted supervision sessions by ACT experts.

Analysis plan

For all analyses we will use multilevel models with time as level 1 and person as level 2 variables. These models are able to deal with the interdependence of observations across time within the same subject. Hypothesis 1 refers to a multilevel model with time as within-subjects variable and no between-subjects variable. Hypothesis 2 is based on hypothesis 1 but will contain in addition one between-subjects factor (group, 2 levels), plus the interaction between time and group, which is our primary focus. Hypothesis 3 refers to a multilevel model with social stress as predictor and psychological flexibility as mediator. Again, we will be interested in the interaction between these two characteristics, which are both time-varying within subjects.

Discussion

The aim of this study is to longitudinally examine the mechanisms of action involved in an Acceptance and Commitment Therapy (ACT) treatment for transdiagnostic patients with varying degrees of treatment experience in a controlled effectiveness trial. This study captures the treatment effectiveness of ACT, in addition to assessing psychological flexibility and patients’ social contexts in multiple ways over multiple time points, up to 12 months after intensive in- or outpatient treatment. A large sample consisting of patients with various diagnoses is recruited, in accordance with ACT being a transdiagnostic behavioral approach. Novel methods of data collection (Event Sampling Methodology, ESM) are combined with traditionally used ones such as questionnaires (implemented at many time points in many ways) and a behavioral approach test (Heat Induction Test). By capturing treatment effectiveness (including follow-up to 12 months), symptoms, emotions, social interactions, stressors, psychological flexibility, spreading of effects to the patient’s social network, well-being, and health-behaviors as well as behavioral variables, and fluctuations of all these, this study will add to the understanding of antecedents, consequences, and inherent processes that contribute to the recovery of psychological disorders and maintenance of gains as well as fluctuations in well-being, functioning and reduced symptomatology. Furthermore, these data allow for a better understanding how time affects the accuracy of participants’ reporting of their own experiences, thereby contributing to a better understanding of how to better probe for such information in the future [49]. Therefore, these data are relevant for both basic clinical theory and clinical care.

The implementation of ESM enables us to capture experiences in participants’ natural environment. ESM is also today’s gold standard and allows us to collect information closer to real life than data collection solely with questionnaires. Since participants’ retrospective recalls are limited to three hours during the ESM week, the participants will have more accurate memories regarding that time window than larger time windows (days, weeks, months, or even years) often enquired about in questionnaires. Combining ESM, behavioral tests, weekly process measures filled out by three sources (patient, therapist, and nurse [in the in-patient setting]), questionnaires, and also information about the participants’ social networks in a large group of in- or outpatients with various diagnoses across different severity levels results in a unique and rich data set.

These data further the understanding of symptoms, general functioning, and well-being by testing across different domains, namely therapy expectancy, psychological flexibility, social interactions, therapy consistent skills, well-being, emotion regulation, symptomatology, stress, and prosocial behavior. They will also provide information about how ACT differentially affects a range of psychological, behavioral, cognitive, and affective outcomes. Different components can be identified and related to symptomatology, maintenance of therapeutic gains, and relapse.

Finally, Choose Change will generate hypotheses that will lead to a number of publications that will focus on how ACT transfers to treatment-resistant patients treated in in- and outpatient treatment settings. Moreover, the mechanisms underlying treatment success and failure remain largely unknown and findings from Choose Change are expected to significantly contribute to our understanding in this area. The results of Choose Change will provide science with important and meaningful insights into the effectiveness of ACT, trans diagnostically, in in- and outpatients, and in a naturalistic setting.