Introduction

This article proposes an initial programme theory to explain the causal mechanisms influencing successful completion of a residential programme for treatment of substance use disorders (SUD). In doing so, we identify the importance of context in activating those mechanisms and outline the usefulness of Realist Evaluation (RE) as a research philosophy for exploring the “black box” of a complex intervention for addictions.

Funders seek evidence-based treatment (Mee-Lee et al. 2010). Although a broad range of interventions for SUD has developed, including pharmacological, psycho-social and spiritual, either alone or in combination, conclusions from systematic reviews of intervention studies have highlighted the gaps in our knowledge. A systematic review of studies published between 1995 and 2012 (Reif et al. 2014) found that although there was moderate evidence to support the effectiveness of residential treatment for addictions, more research was required to identify which clients would respond best to residential treatment and, importantly, “to determine whether treatment services are equally effective for different populations” (p310). There is a distinct absence of evidence to support best-practice interventions for indigenous populations (Taylor et al. 2010). A subsequent systematic review of studies published between 2013 and 2018 found that although treatment retention, completion and continuing care were predictors of recovery, it was not clear what relevant contexts would improve retention or completion (de Andrade et al. 2019). They also observed that there remained a lack of consensus for best practice, which could frustrate funders’ need for evidence-based practice.

The reliance on the gold-standard randomised controlled trial (RCT) to provide evidence of effective interventions may have contributed to the lack of consensus. RCTs may not be appropriate to explain causation for complex interventions because in relying on successionist causation and the analysis of associations they ignore the potential for human agency, the existence of multiple causal pathways and the influence of context on the mechanisms of change and outcomes (Marchal et al. 2013; Van Belle et al. 2017). Addiction has been described as a complex social process. (Maher and Dertadian 2018). Treatment also contains complexity from the number of activities that make up the intervention; the different behaviours and associated motivation required from those delivering and receiving the intervention; the range of outcomes; and any adaptations from the original intervention design (Craig et al. 2008).

Realist evaluation (RE) has been proposed as an effective way of evaluating complex health interventions, through its focus on unlocking the “black box” of an intervention ( Duncan et al. 2018; Pawson and Tilley 1997; Van Belle et al. 2017). Recently there has been a call for realist evaluation to be used to advance understanding of treatment for addictions (Urbanoski 2020). RE aims to make explicit the theory underpinning an intervention (Astbury and Leeuw 2010) by identifying causal mechanisms that explain how outcomes are achieved from interventions (Marchal et al. 2012; Pawson and Tilley 1997). In doing so, RE aims seeks to answer: what works, for whom, in what circumstances and over what timescale (Astbury and Leeuw 2010; Pawson 2013; Pawson and Tilley 1997; Weiss 1997). RE differs from other theory-based evaluation approaches (for example Theory of Change) in its focus on generative causation (Bhaskar 2008) and search for the configurations of context-mechanism-outcome to explain how programmes outcomes are caused (Blamey and Mackenzie 2007).

A realist evaluation assumes the existence of depth ontology, in that the world exists on three levels, the empirical, the actual and the real (Bhaskar 2008; Jagosh 2020; McEvoy and Richards 2006; Schiller 2016), (see Table 1). Epistemologically, we can know about the empirical experiences because they are observable and measurable; the actual comprises events, which may or may not be observable (for example a volcano erupting on an uninhabited island); however, causal mechanisms exist in the realm of the real and can only be hypothesised, though these hypotheses can be tested (Fletcher 2017; Jagosh 2019). Thus, the realist evaluator has to explore each domain and reveal the latent mechanisms that exist in the real and the contexts that activate them (Jagosh 2020; Pawson 2013; Westhorp 2018).

Table 1 Key terms
Full size table

RE requires an iterative process that is guided by theory (Wong 2015). The first stage is the development of an initial programme theory that explicates how the outcomes are caused (Westhorp 2014; Wong 2015). Subsequently, the hypotheses are tested in order to confirm, refine or refute the initial theory (Pawson and Tilley 1997). Despite the importance of the first stage, there have been few reports of the process followed by authors (Fick and Muhajarine 2019; Mukumbang et al. 2018; Smeets et al. 2022; Wong et al. 2013). This study seeks to add to the corpus of knowledge about the development of an initial programme theory.

Aims

This study sought to develop an initial programme theory that would propose possible contexts and mechanisms of change leading to successful completion of a residential treatment programme for substance use disorders.

Setting

The setting for the evaluation is a Salvation Army 26-bed residential centre in New Zealand for adults diagnosed with moderate or severe substance use disorder (DSM-V). In 2020 approximately half of residents identified as New Zealand European and 38.5% as Māori (see glossary of terms). The programme is a complex intervention originally developed from the Community Reinforcement Approach (Meyers and Miller 2001). It is an eclectic programme incorporating motivational interviewing, group work, relapse prevention and community living. Tikanga Māori (cultural values) practices are included and a whānau (family) support programme is also delivered. Core elements of the programme include peer-based practice and support, recovery coaching, attendance at the Salvation Army Recovery Church and self-help addiction services (AA and NA). Residents receive support for mental and physical health from an on-site nurse, a clinical psychologist and a psychiatrist. Post completion, former residents can access support to help with re-integration into their communities and some stay in post-recovery houses provided by the Salvation Army. Successful completion of the programme requires participation in the equivalent of eight weeks of group work and eight one-on-one therapeutic sessions. In the 12 months ending 30 June 2020, 20.28% of clients did not complete treatment.

Method

Design

The building blocks for hypothesis development are the configuration of context (C), mechanism (M) and outcome (O) (Pawson and Tilley 1997). A core principle within realist approaches is that outcomes from a programme occur as a consequence of the influence of context upon the mechanism (Greenhalgh et al. 2017a; Pawson and Tilley 1997), or O = f(M.C) (Pawson 2018). Differences in context account for differences in outcome from the same programme. Document review and interviews are methods commonly used to collect data in realist evaluations (Manzano 2016; Pawson and Tilley 1997). Programme documents can provide information about desired outcomes and potential activities. Interviews offer the opportunity for interviewer and interviewee to engage in a dialogue to explore potential context-mechanism-outcome configurations (CMOC) (Greenhalgh et al. 2017b; Manzano 2016; Pawson 1996). The development of the initial theory is likely to involve programme developers and managers as key informants (KIs) because they have information about how a programme is intended to work and the theory that underpins it (Pawson and Manzano-Santaella 2012; Pawson and Tilley 1997). Consequently, the number of KIs in this stage is often small—Mukumbang and colleagues (2018) interviewed 12 KIs across three programme locations, Fick and Muhajarine (2019) interviewed four KIs and Smeets and colleagues (2020) interviewed seven KIs. Therefore, we collected data from the service’s contractual documents and through semi-structured interviews with four KIs.

Participants

The first author purposively selected most members of the management team and a senior caseworker as key informants, based on their knowledge of the programme, their roles within the organisation and availability for interview (Greenhalgh et al. 2017b; Manzano 2016). One participant had written the programme funding applications. Another had written the programme policies and procedures. A third was an experienced caseworker and the fourth was the clinical lead. Together, they held sufficient knowledge about how the programme operated to facilitate the development of the initial programme theory.

Data collection

Realist interviews are theory-led and purposeful (Manzano 2016; Pawson 1996). They seek to elicit information about the CMOCs that explain how the programme works. Three distinct phases of interview have been identified: theory gleaning, theory refining and theory consolidating (Manzano 2016; Pawson 1996). The theory gleaning phase seeks to identify the context in which the programme operates and how it works. It looks at the whole programme and so interviewees tend to be managers and programme developers. The development of initial theories starts with this phase (Manzano 2016).

The first author conducted five semi-structured interviews with the four key informants at the Bridge offices. Interviews were audio recorded. Key informants signed consent forms prior to the interviews.

An interview guide (Appendix 1) was developed to focus on exploring how the Bridge programme was intended to work and to explore the possible outcomes and the contexts leading to those outcomes (Manzano 2016). Findings from the document review identified desired outcomes and potential activities and these were also used to direct the interviewer’s line of questions. During the interviews the key informants were asked to describe successful and unsuccessful cases and possible triggers for behaviour change. Reasons for the different outcomes were explored to identify contexts and mechanisms.

Ethical approval for this study was provided by the University of Otago Human Ethics Committee (reference H20/172). The authors’ positionality acknowledged that the treatment programme operated in an open system that would influence the beliefs and behaviours of all actors. The lead author briefed all programme staff before data collection began, outlining the research aims and approach. He also outlined his belief that colonisation created the conditions of inequity which led to addiction disproportionately affecting indigenous people and acknowledged his family’s historical role in the process of colonisation. The lead author and two other authors had no prior connection with the Bridge programme before this study. The consultant psychiatrist was aware of the service as part of his clinical role.

Data Analysis

Inductive content analysis (Hsieh and Shannon 2005; Leung and Chung 2019) was used to identify outcomes and activities from the Bridge funding agreements and an organisation service contract proposal. Data from these documents were used to inform the interview guide and the data coding hierarchy.

For interview data, we used both inductive and deductive thematic analysis methods to inform a retroductive theorisation process (Fletcher 2017; Jagosh 2020). The first author transcribed verbatim the interviews onto Microsoft Word and the analysis was completed using NVivo12 to support data management. Two authors analysed interviews 1–3 inductively using thematic analysis (Braun and Clarke 2006) coding for contexts, mechanisms and outcomes. Following comparison and review of their coding, two authors continued inductive analysis of interviews 4 and 5 independently. The first author exported the coding data into Excel for review and collation of themes and creation of draft CMOC.

The first author then reviewed the combined coding deductively and re-categorised the codes using the descriptors programme activity, context, mechanism, behaviour change or outcome following the approach outlined by (Dalkin et al. 2015). The mechanism was used as the starting point for identifying potential CMOC frameworks (Dalkin et al. 2015). Whilst the inductive phase focussed on single words or phrases, in the deductive phase we searched longer passages for explanations for the mechanisms that lead to the events being described in the data.

Subsequently, the first author conducted deductive analysis of the data to identify which critical realist domains were referred to (Ekström 1992; Fletcher 2017; Wiltshire and Ronkainen 2021) and coded data to domains of the empirical (14), actual (48) and the real (3). Using this coding, a map of the key configurations was developed using retroductive inference. Three authors mapped to the initial CMOC configurations and developed revised hypotheses to explain the behaviour change processes (Fig. 1).

Fig. 1
figure 1

Initial programme theory

Full size image

Finally, the map of the CMOC configurations and proposed hypotheses were reviewed individually with three key informants and another clinical lead in the organisation. They supported the map of configurations and hypotheses.

Findings

Key informants (Table 2) described a sequence of outcomes they wanted clients to achieve from the programme. First, to complete the treatment programme; then, to reduce harm and, finally, to achieve abstinence. We identified one intermediate outcome of identity change that contributed to successful residential treatment. The mechanisms leading to these outcomes were a sense of belonging to the recovery community and a sense of hope and purpose. These two mechanisms were activated by organisational, group and individual contexts related to supportive cultures, the management of chaos, and experiencing success.

Table 2 Key informants
Full size table

A summary of the findings from key informant interviews is provided in Fig. 1. It is a conceptual model and shows horizontally the domains of the empirical, the actual and the real. Dotted lines showing links between the domains and hard lines show connections between elements of context within the domain of the actual. Following the retroductive approach, the model starts on the left at the outcomes and moves to the right through the mechanisms and contexts to the notional start, described as “rock bottom” by key informants.

Intermediate Outcome: New Identity

The new identity was centred on the client identifying as a member of the recovery community. It was described as the successful integration of new beliefs about what they could achieve, their role and responsibilities to themselves and others, and also the value they placed on their new relationships.

“They talk about things that they don’t want to lose in life if they were to relapse…They’ve developed and gained some things that they don’t want to give up and have integrated it as part of their new identity” (KI2)

The outcome of a new identity was contrasted with the disconnected identity with which many clients entered the programme. The failure to resolve that disconnection was seen as a key cause for clients not completing the programme.

“Many people…come here…completely disconnected from themselves, from their families. They are angry with the world.” (KI4)

“Their sense of ‘who is my family’, ‘who am I’ is very disconnected and conflicted. And I think the people who often struggle the most to change are those who don’t resolve that conflict around…’where does my loyalty lie?’” (KI4)

Mechanisms of Change

We hypothesised that there were two key mechanisms that generated the antecedent (intermediate) outcomes outlined above:

  1. a.

    Developing a sense of belonging. If clients experience a safe and supportive community [Context] that meets their immediate needs and offers evidence of rewards [Context], then they develop a sense of belonging [Mechanism] that causes them to engage in the programme [Outcome], and to develop a network of support for their life in recovery [Outcome].

  2. b.

    Creating hope, and a sense of purpose. If clients maintain a sense of belonging [Context] and if they can identify with the recovery of peers and staff [Context], then they will develop a sense of hope and purpose for themselves [Mechanism], and they will develop a new identity as a person in recovery [Outcome].

The mechanisms have been described here as inter-related in that the development of belonging to the community is an antecedent for the creation of hope and purpose. Belonging existed in relation to a community of recovery and showed movement away from a relationship with a community of alcohol or drug users. Key informants used the term hope to describe that the client came to believe that an event (recovery) was possible and the term purpose to describe that a client had meaning (goals) in their life:

“There has to be some sort of purpose, to be moving towards something.” (KI2)

Contexts

Contexts related to the mechanisms were stratified at the levels of organisation, group and the individual. Key informants described a complex open system in which the levels of context were inter-related. Managers purposefully created the culture to support change and the opportunities in which clients could experience success.

Development of a Safe and Supportive Community

The safe and supportive community within the programme was characterised by the cultures of the organisation (represented by the norms and rules of the programme) and through the behaviour of the staff and the clients. Organisational norms included the need for the programme to appear meaningful to clients and their support networks. The norm was explained by a key informant describing the circumstances in which a client may not graduate from the programme,

“And so we get there with some people where we just feel like…they haven’t actually contributed in a meaningful way that we think they will have actually benefitted enough to say, yep here’s your certificate of graduation.” (KI2)

The empirical evidence for the staff culture included consistency of enforcement of rules (for example no drugs or alcohol on site or adherence to weekend plans), supporting clients by making their programmes as individualised as possible, and identifying and rewarding good behaviour by clients. Staff culture was seen as contributing directly to success of clients:

“[T]he clients seem to do better when us as a staff are more boundaried and more all on the same page and holding the same boundaries and really holding the line, and the clients seem to respond better to that as opposed to if we seem to be a little bit inconsistent on stuff.” (KI1).

Group Culture

The ideal culture of clients was described in terms of the support they gave each other:

“When [the programme’s] culture is right you’ll see them at Recovery Church when they get their awards, they’ll stand up and clap for them, and cheer each other on and will yell out encouragement”. (KI2).

Here, the key informant used the empirical evidence of vocal support to identify the existence of a supportive group culture.

The key informants described the support networks as comprising contexts of both a small number of key individuals, possibly a group from the same cohort going through the programme, and also wider groups representing the Recovery Church or Alcoholics Anonymous (AA) and Narcotics Anonymous (NA). Further context for the support network to be successful are features of safe housing and a safe community, where safety was described in terms of the absence of alcohol or drug use.

“The way they…talk about it is quite concrete, `If I use again it’s over for me’…`I can’t be around people who use’.” (KI2).

Membership of the support network was described as conditional upon maintaining recovery. Former clients who relapsed would withdraw from the network and re-connect when ready to recommence recovery.

Absence of Organisational Chaos

When effective, the cultures of staff and clients combined to create a context of a calm programme, marked by an absence of chaos. Chaos was evidenced by sudden changes when clients were removed from the programme, or clients exhibited inappropriate behaviour (e.g. non-compliance with programme rules or agreed group ways of working) or mental health problems (e.g. heightened anxiety), or when staff were not settled. At the level of the organisation, a formal welcome process (whakawhanaungatanga) had been introduced to ensure staff and clients were settled and calm during the client induction and had the opportunity to start to develop relationships within the programme. The welcome described was a Māori practice involving the sharing of family history, songs and prayer.

“Sometimes we have really awesome groups and a lot of them will know their pepeha [family history] and…quite often we’ve had…leaders within the client group, and they will really step up and be sharing of themselves and be giving of themselves to purposefully and meaningfully make that connection with people coming in” (KI 5).

The cultural norms and procedures also supported the management of chaos. In groups, the caseworkers would maintain boundaries to ensure clients did not sabotage group work and that they adhered to the programme’s behavioural norms:

“[W]e need to hold the with what our expectations are and that if someone swears at a staff member then there will be consequences”. (KI4)

Management of Personal Chaos and Loss

A key feature of the transition into the programme was the personal chaos related to mental health and loss experienced by clients. Clients entered the programme having completed detoxification. However, the removal of drugs and alcohol and change of environment could highlight mental health problems, leading to voluntary withdrawal from the programme. Key informants described clients facing other challenges associated with loss and grief—the loss of identity, the loss of community, feelings of failure.

“A lot of people who come through the programme have a lot of bravado and a lot of ‘I’m fine, I’m fine’ [Empirical domain]…But actually they’re really not fine under all that. [Actual domain] There’ll be a lot of trauma and a lot of negative critical self-talk…So, for some of the people coming from the community and stopping the drugs and alcohol, ,,,their mental state changes once they are off the drugs and alcohol….And for [clients from prison] which is a low stimulus environment, and then they’ve come here which is a high stimulus environment in comparison…there has been a deterioration in their mental state.” (KI5)

Staff supported clients through periods of personal chaos by imparting skills (for example mindfulness) and building connections with and between clients.

“And I feel like the group dynamic really does hold people.” (KII 5)

Experiencing Success

Staff were purposeful about creating opportunities in which clients could experience success. Although the transition from addiction was characterised by a sense of loss, staff emphasised the importance of creating opportunities for clients to experience success in recovery. The main methods for doing this were individual symbolic rewards (chocolate fish) for providing negative urine screen results and team rewards for tidy houses. Staff were purposeful about the reward process:

“And so that’s a very reinforcing way of where staff would carefully word what it is. We are careful to not just use a…word that isn’t specifically linking what we’re noticing to a behaviour. So, we are trying really to notice and reward and reinforce change that’s positive.” (KII 4).

Relatable Identity Change

An important context for change was that clients could believe that change for them was possible. The programme addressed this by introducing staff with lived experience of addiction as recovery coaches, responsible for working one-on-one with clients and to manage the residences and mandating attendance at AA or NA groups. Recovery coaches were able to tell their stories of recovery and thereby provide clients with hope for change and their futures. The staff believed that clients often came from communities where no one was abstinent and that therefore recovery was perceived by clients to be a unachievable.

“And it’s a bit of a myth sometimes to them that they can even change or that they can have a different life”. (KI2)

The importance of seeing former addicts achieving recovery and the link to hope was also highlighted:

“That’s why the 12-steps and NA and AA are so important because [clients] are seeing people that are similar to them…they’re not miles away in terms of where their journey is…And so that’s that walking, living hope that actually I can make these changes.” (KI2)

Timescale

Key informants were consistent in how they described the timescales involved for the change processes. Clients decided to engage and continue treatment, or not to engage and to leave, during the first two weeks of the programme. For clients who engaged and stayed, further change was noticed towards the end of weeks four or five, when further commitment to the programme was made by clients through expressions of remorse for not having engaged enough, or through seeking extensions. The final two weeks of the programme were also important because they were the period during which the programme was most likely to exit people from the programme for failure to make adequate progress.

Summary

The proposed conceptual model (Fig. 1) shows how the presence of organisational, group and individual contexts work together to generate mechanisms of change (belonging, and also hope and purpose) leading to a pattern of change in identity from disconnection to being a part of the recovery community and subsequently to reduction in harm and abstinence.

Discussion

We used a realist approach to identify potential hypotheses that explain how behaviour change is generated in a residential programme for adults with moderate to severe substance use disorder. We hypothesised that completion of the residential programme depends on the creation of a supportive milieu in which clients experience an absence of chaos, presence of structure and a cycle of reinforcement of success that together trigger mechanisms of belonging, hope and purpose, leading to a change of identity and sense of belonging to a recovery community. Clients who remain disconnected are likely to withdraw from the programme during the first week. Staff assess clients’ behaviour against a normative framework and exit those they do not believe demonstrate sufficient commitment to change.

While other authors have proposed models to explain recovery from addictions (DiClemente 2007; DiClemente et al. 2004; Neff and MacMaster 2005; Prochaska et al. 1992; Simpson 2004; West and Brown 2013) we believe our model is the first to be developed using a realist approach and thus the first to propose mechanisms based on generative causation and to highlight how contexts activates mechanisms and influences outcomes.

Sequential and Developmental Models

Models of behaviour change in addictions frequently cited use sequential or developmental models of change (De Leon 2010; Prochaska et al. 1992; Simpson 2004). These models ascribe client motivation to be core to the change process. The Transtheoretical Model (TTM) of intentional behaviour change proposes that the process of change depends on the client having the motivation, energy and effort to complete a series of tasks that lead to recovery (DiClemente et al. 2004). The Texas Christian University (TCU) model (Simpson 2004) similarly views motivation as a core aspect of the “black box” of treatment success, within a sequential process. The TCU model also identifies relevant success factors comprising self-efficacy, strength of the therapeutic relationship and mastery of self-management. The Therapeutic Community model (De Leon 2010) is under-pinned by sequential structure, with increasing freedoms as a client makes progress, whilst maintaining an environment separate from the conditions of addiction the clients previously lived in, and addressing the mental and physical health needs of clients. Our model does not propose a sequential process of change, though it has a distinct start point (disconnectedness) and intermediate outcome (identity change), and two identifiable points of attrition (in the first two weeks and after four weeks). Within that we have proposed an inter-related model of contexts and mechanisms, emphasising that change appears to be dependent on external context rather than internal motivation.

Hope

Hope, meaning a belief that challenges can be overcome, has been proposed as the foundation of recovery (Substance Abuse and Mental Health Services Administration 2020) and as a key element of recovery in mental health studies (Leamy et al. 2011). Hope Theory (Snyder et al. 2018) explains how motivation can lead to change during treatment. Snyder and colleagues suggested that state hope comprises a pathway to achieving a goal and the belief that one had the capability to use that pathway. This requires an ability to set goals (a purpose) and the skills to manage obstacles. In our proposed programme theory, we highlight the importance of staff with lived experience of addictions as a context to activate the mechanism of hope and purpose. The Bridge programme also provides clients with the knowledge and skills to apply goal setting. Despite the conceptual claims, there have been few published studies of hope in relation to recovery from addictions (Gutierrez et al. 2020; Mathis et al. 2009). A study of homeless veterans resident in, or having completed residential treatment for substance disorder found a modest relationship between state hope and recovery, measured as quality of life and days abstinent (Irving et al. 1998). Improvements in state hope over residential treatment time have also been found (Ekqvist and Kuusisto 2020), though this study did not report on treatment completion or recovery.

Purpose

The search for purpose, or meaning in life, has been characterised as central to coping with distress and a navigating a pathway towards well-being (Frankl 2008) and subjective well-being (Schueller and Seligman, 2010). Authors have described meaning and purpose as central components of spirituality in addiction studies (Cook 2004; Gutierrez 2019), in 12-step programmes and in recovery (Galanter 2007; Laudet et al. 2006). Life meaning has also featured as a key ingredient of human capital in recovery capital models (Hennessy 2017), both separate from spirituality (Cano et al. 2017; Groshkova et al. 2013) and connected to spirituality (Laudet et al. 2006; Sterling et al. 2008). While meaning in life has been found to be one of a group of factors that buffers stress, causation remains unclear (Cano et al. 2017; Laudet et al. 2006). One study of cocaine dependent participants in residential treatment found that purpose in life was not significantly related to treatment completion or length of stay (Martin et al. 2011). Measurement of purpose and meaning within recovery capital has been inconsistent, for example, with meaning measured separately from or as synonymous with spirituality (Burns and Marks 2013; Groshkova et al. 2013; Hennessy 2017; Sterling et al. 2008). Furthermore, there remains a dearth of published studies investigating purpose or meaning in relation to recovery in residential settings (Krentzman 2013). Meaning has also been equated to meaningful activities (Groshkova et al. 2013) or as taking “meaning from life’s events” (p41) (Laudet and White 2008). In our study, purpose has been used as meaning in one’s life. Key informants linked purpose to goal setting in terms of something to move towards. Thus, it appears to be linked inextricably to hope.

Belonging

Sense of Belonging has been defined as feeling an integral part of an environment (Hagerty et al. 1992); in Native North American culture, as connectedness to a community (Hill 2006); and by four dimensions of membership, influence, satisfying needs and emotional connection (McMillan and Chavis 1986). Sense of community has been found to foster hope in recovery among members of sober living homes (Jason et al. 2016; Stevens et al. 2012). The Theory of Human Relatedness (THR) (Hagerty et al. 1993) has been used to explain recovery in an Alcoholics Anonymous setting, where a sense of belonging was proposed as a necessary competence, or process, to promote connectedness to the recovery community (Strobbe et al. 2012). THR posits that humans are predisposed to desire connections to others, and that relatedness can be defined as the level of involvement with others, including environments and organisations. Disruptions to the relatedness leads to a sense of disconnection and can lead to adverse psychological and social effects. Although this theory has not been applied explicitly in a residential setting for addictions treatment, it appears to align with the mechanisms we have termed a sense of belonging and has potential to explain it.

Attrition

Studies of attrition have focussed mainly on individuals’ risk factors (Baker et al. 2020; Deane et al. 2012; Harley et al. 2018), leaving practitioners with generalised advice to focus on those clients at higher risk of attrition, to individualise treatment, and to involve clients in decision making (Brorson et al. 2013). In our preliminary theory we propose a more nuanced view of attrition, that it is the consequence of distinct mechanisms. For those who leave against staff advice, responses from key informants suggested that the mechanism relates to a flight response, triggered by the lack of connectedness to the programme and participants. The mechanism leading to staff to exit clients early appears to be a consequence of the staff seeking to protect the programme values in response to perceived transgressions by clients.

Contexts

Theories of behaviour change often ignore the relevance of context (Blue et al. 2016). Those theories obscure the operation of mechanisms of change because their focus is either too narrow (ignoring the complexity), or too broad (too distant to be applicable). Existing models of behaviour change in addictions substitute treatment factors for contexts. Common factors of effective treatment have been proposed, comprising: support, structure, goal direction, rewards for abstinence, use of abstinence norms, building self-efficacy and coping skills (Moos 2007); engagement and good therapeutic alliance (Mee-Lee et al. 2010; Simpson 2004); community, structure, and meeting clients’ mental and physical health needs (De Leon 2010). However, the authors have treated these factors as variables to be managed, rather than addressing how they might be forces that activate mechanisms (Greenhalgh and Manzano 2021). Whereas our proposed programme theory has highlighted the roles of organisational and programme contexts to activate the mechanisms of change leading to programme outcomes.

Limitations and Future Research

This study aimed to develop an initial theory explaining the change process within the residential programme that could be tested in later studies. In this realist way we have tried to move knowledge forward (Salter and Kothari 2014). Although only four key informants were interviewed over five interviews, they were able to describe the programme in sufficient detail to develop the hypotheses and causal diagram. We will complete a robust process of testing and refining the model through triangulation of views from former clients and other practitioners. Quantitative studies will test the proposed mechanisms of change and the intermediate outcome.

Further research is required to identify the timescales over which the changes occur, and for whom the intervention will work. A particular focus for subsequent research will be the appropriateness of the programme for Māori (members of the indigenous population of New Zealand). They comprise 17% of the New Zealand population (2018 census), yet in 2020 were 38% of the residential programme admissions. The causes and consequences of this over-representation on the recovery process need to be explored with clients. In relation to programme content, the key informants did not differentiate between clients based on ethnicity and the programme features concepts drawn from Māori custom and practice (tikanga) that may also be as appropriate for New Zealand European clients. Again, this supposition needs to be tested with clients. Finally, the findings considered three of the four elements of Context (individual capacities, inter-personal relations and institutional setting) (Pawson et al. 2004). They did not consider wider societal system in which the programme operated, nor the influence of post-treatment support, and these should be tested in future research.

Conclusion

We used a realist evaluation to develop a preliminary programme theory to explain outcomes from residential treatment for SUD. Theories like this are important because practitioners can use them to increase the likelihood of replication of successful treatment. We found that realist evaluation provides researchers with the philosophy and methods with which to explore and start to unravel some of the complexity within residential treatment for addictions. In doing so, we have proposed a move away from researching residential addiction through an individualistic lens that attributes treatment success or failure to the individual and the intervention, towards a theory that highlights the roles of organisational and programme contexts to activate mechanisms of change leading to programme outcomes.