Significance of Leaders for Sustained Use of Evidence-Based Practices: A Qualitative Focus-Group Study with Mental Health Practitioners

The implementation of evidence-based practices in mental health services requires much effort. Pursuing high fidelity, that is the degree to which one follows the intervention as specified, has gained considerable attention as it has been associated with positive outcomes, including improved staff retention (Aarons et al. 2009) and patient outcomes (Bartholomew and Kensler 2010; Hasson-Ohayon et al. 2007). However, even for services with high fidelity score, the implementation has shown to be discontinued (Bond et al. 2014; Salyers et al. 2009). The decisions of practitioners to sustain (continue use after the implementation period) or end the use of a practice after its initial adoption are not well understood (Bond et al. 2014; Greenhalgh et al. 2004). Learning from high-scoring fidelity practitioners may provide us with knowledge concerning the challenges faced by the further use of such implementation in their services.

The intervention that form the basis for this study, the illness management and recovery (IMR), is a standardized psychosocial intervention designed to help people with severe mental illnesses manage their illness and achieve personal recovery goals (Mueser et al. 2006). Five strategies form the basis of the IMR intervention: psychoeducation to improve knowledge of mental illness, relapse prevention, behavioral training to improve medication adherence, coping skills training to reduce the severity and distress of persistent symptoms, and social training to strengthen social support. The intervention consists of a combination of educational, motivational, and cognitive-behavioral techniques. By the use of a workbook with educational handouts, practitioners teach patients weekly for 10–12 months, either individually or in groups. A toolkit has been developed together with fidelity checklists for the practitioners to be guided in the implementation of IMR (SAHMSA 2009a, b).

In this paper, implementation is understood as “a specified set of activities designed to put into practice an activity or program of known dimensions” as suggested by (Fixsen et al. 2005, p. 5). A number of conditions have been singled out as influencing whether the implementation of new interventions such as IMR are successful or not. The framework introduced by Fixsen and colleagues (Fixsen et al. 2013; Fixsen et al. 2009) emphasize three drivers, that is competency drivers, which involve the selection, training and coaching of practitioners, organizational drivers which involve facilitative administrators who prepare and support the use of new interventions, and leadership drivers, which involve adaptive leaders who facilitate and support the implementation. In addition, fidelity assessments are seen as a critical component of implementation as they can monitor continuity of practice (Fixsen et al. 2005).

Practitioners play a central role in the implementation of new practices in terms of their attitudes and behavior (Aarons et al. 2011), and also in terms of their autonomy, i.e., whether practitioners perceive independence in how they perform tasks and operate independently. The insertion of new interventions can lead healthcare practitioners to experience the loss of job autonomy and subsequently to increased staff turnover (Aarons et al. 2012; Rossen et al. 2016). However, the implementation of new practices has also shown to lead to increased job autonomy as it gives practitioners more structure and competence in their work (Aarons et al. 2012). High autonomy is therefore emphasized (Greenhalgh et al. 2004; McGuire et al. 2015). Moreover, a difference has been pointed out between practitioners’ feelings of being autonomous (i.e., self-regulating, oriented toward the interest value of the environment and contextual supports for self-initiation) and autonomy support (i.e., the leaders understanding and acknowledging the perspectives of the practitioners, offering them opportunities to make choices and encouraging self-initiation), both of which have been associated with better job performance (Baard et al. 2004).

The relationship between practitioners and leaders will also affect the implementation of new interventions (Greenhalgh et al. 2004). Similarly to Fixsen et al. (2013), many researchers have argued that leadership is an important determinant for implementation success (Grant et al. 2014; Green et al. 2013; McGuire et al. 2013). Some have highlighted the importance of strong and active leaders who are committed to the entire implementation process for successful intervention (Bond et al. 2014; Rychener et al. 2009; Salyers et al. 2009). A few studies have shown that perceived leader support has been associated with practitioners’ participation in implementation (Sloan and Gruman 1988) and is a significant predictor of a strong climate for implementation (Klein et al. 2001), yet research is needed which explores the specific behavior that leaders may enact in order to facilitate the implementation of interventions (Aarons et al. 2014). A measure was recently developed that differentiated between four types of leader behaviors that are thought to influence implementation; proactive leaders who address implementation challenges, knowledgeable leaders who understand implementation issues, leaders’ support of practitioners’ adoption and use of practices, and perseverant leaders who are consistent, unwavering, and responsive to implementation challenges and issues (Aarons et al. 2014).

Practitioners will also be affected by their patients (Greenhalgh et al. 2004). When they experience an innovation as meaningful, which also appears to be meaningful for the patients, this can have a powerful influence on the decision to adopt the practice (Salyers et al. 2009; Stewart and Chambless 2007).

Similarly to Fixsen et al. (2013), many have highlighted the importance of fidelity and outcome monitoring (Miller et al. 2017). Bond et al. (2009) emphasized a “monitoring philosophy” that is making use of regularly monitoring to guide the further implementation. They suggested that introducing fidelity monitoring to services would provide practitioners with a focus for implementation efforts and a framework for understanding the practice. It would also provide leaders with political documentation and offer validation to teams that achieve high fidelity. While few services embraced the fidelity monitoring philosophy, those that did were more likely to sustain the practice. Thus, the practitioners’ understanding must be worked on for them to experience assessments as a resource for long-term sustainability (Bond et al. 2014; Rychener et al. 2009).

In this qualitative focus-group study, a group of practitioners who implemented IMR was interviewed in which the majority had reached high levels of fidelity, which is the exception rather than the rule (Bartholomew and Kensler 2010; Hasson-Ohayon et al. 2007). A previous study with the same practitioners had also shown that the majority wanted to continue to use IMR after the implementation period (Egeland et al. 2017). The aim was to gain more knowledge concerning the practitioners’ experiences with the implementation of IMR in their services, and how this may influence further use. To our knowledge, no one has previously examined such a select group of high IMR fidelity-scoring practitioners. Qualitative research can contribute to deepening and contextualizing our understanding of the conditions of these practitioners, as well as their decisions on whether or not to continue to use IMR.

This study was part of a project that implemented IMR in nine Norwegian mental health services (A–I) between November 2013 and June 2015. In addition to a large data collection, the qualitative study consists of nine semi-structured focus group interviews with practitioners who had used IMR for 12 months. The study was approved by the regional committees for medical research ethics [REK 2013/2035].

Analysis

All the interviews were audiotaped and transcribed by a professional. As a method for analysis, we have drawn upon the thematic analysis described by Braun and Clarke (2006) and Malterud (2001). This means that when reading through the focus-group interview transcripts, we aimed at an openness and inductive entrance to the data. With regard to interpretation, we leaned towards a realist reading of the data, acknowledging the practitioners’ descriptions, and elaborating and amplifying the meanings that were expressed by the participants. We were concerned with getting closer to the quality and meaning of the practitioners’ experiences as they presented them, seeking to identify patterns and coherences among the participants in the focus group.

The research question that guided the analysis was: “What challenged the practitioners’ further use of IMR?” In the initial reading of the data, the first author and a senior researcher experienced in analyzing qualitative data (MIH) read the transcripts separately. The first author did a close coding of the data, followed by more interpretatively coding of themes that emerged as central in the focus-group interviews, both within and across the interviews. Preliminary themes that emerged were leadership, patient characteristics, and being in time squeeze. In order to make consensus in the analysis, the senior researcher (MIH) independently reviewed the data and noted themes based on the transcripts, followed by in-depth discussions between the two authors. When elaborating themes in the data, some themes were perceived as more prominent of how the practitioners experienced the implementation of IMR in their services and what challenged their further use of the program. For instance, two of the themes that emerged as central in the interviews was the way the practitioners’ notions of whether they would continue to use the program depended on whether they saw it as beneficial for the patients and whether their leaders supported further use in the clinic.

In order to select themes of interest and make consensus about significance and conceptualization of themes, the first author had repeatedly discussions with the senior researcher (MIH) throughout the process of analyses. The results of the analysis was also discussed with a third senior researcher with in-depth knowledge of the program and subject field (KH) that participated in five of the interviews. The conceptualization and understanding of the chosen themes was a thorough process, continually questioning of and discussing the connection between the themes and ensuring that they reflected the data. A visual map and subthemes were developed (Fig. 1), and data extracts were selected that illustrated the themes. Simultaneously, themes were connected to relevant theory and literature.

Open image in new window

Most of the practitioners in the study scored high on fidelity during the study, which indicated that they used IMR according to the manual. However, the practitioners expressed views and concerns that might challenge the sustainability of the program in their services, both in terms of maintaining high fidelity levels and in terms of organizational facilitation. Four themes described these challenges: Prioritization dilemmas despite high autonomy, the practitioners’ experiences of leader engagement, uncertainty about long-term priorities, and that practitioners are guided by the patients’ reception of IMR.

In this current study we examined a group of health practitioners who were implementing IMR, in which all but one of the services had reached high levels of fidelity and almost all practitioners wanted to continue to use the program (Egeland et al. 2017). This was in contrast to earlier studies with mixed fidelity results (Bartholomew and Kensler 2010; Hasson-Ohayon et al. 2007). The findings showed that the practitioners expressed their positive experiences with IMR, and wanted to continue to use it. Nevertheless, they perceived challenges in sustaining the use of IMR in their services. Despite feelings of being highly autonomous, they experienced a high-demanding job schedule. The practitioners perceived that their leaders were supportive, but there were few indications of leaders being particularly engaged in the implementation process. Most of the services lacked goals and visions for implementing IMR, and the practitioners’ further use of IMR was steered by the patients’ reception of it. This indicates that high fidelity is not equivalent to successful implementation, and that other circumstances in the services threaten the sustainability of IMR.

Autonomy has been emphasized when new practice are to be implemented (Greenhalgh et al. 2004; McGuire et al. 2015), as it is presumed to increase practitioner job motivation and prevent job turnover (Aarons et al. 2012). However, despite feelings of high autonomy, many practitioners in the study reported dealing with priority dilemmas towards using IMR. The findings can therefore be understood as not supportive of earlier studies promoting practitioner autonomy as ways of sustaining innovation use. Instead, they may provide evidence that the practitioners need support and guidance, rather than autonomy, for sustaining the use of IMR.

Furthermore, a distinction has been identified between practitioners’ feelings of being autonomous and leaders’ support of practitioner autonomy, with both being associated with better job performance (Baard et al. 2004). However, instead of their perceived autonomy helping them to increase the use of IMR, the lack of leader support for their autonomy caused them to hesitate and therefore acted as a barrier to the long-term sustainable use of IMR. Therefore, the findings indicate that autonomy support from their leaders must be present for the practitioners’ autonomy to sustain the use of IMR.

In spite of similar fidelity scorings, the practitioners in Service C stood out as a good example. This indicates that themes other than fidelity are of importance in sustaining IMR. In addition, they perceived their leader to be engaged, dedicated, and committed to the entire implementation process, which has been seen as an important determinant (Aarons et al. 2012; Bond et al. 2014; Salyers et al. 2009). This also corresponds to good implementation leadership, as described by Aarons et al. (2014).

Even though most of the practitioners in the other services experienced that they were supported by their leaders, the leaders’ active engagement differed greatly. It seemed as if the leaders who did not actively engage in the implementation process left the responsibility for the follow-up to the practitioners. As a result, there was a difference between passively supportive leaders and actively engaged leaders. While the former made it possible for the leaders to disengage themselves from the process, the latter tied the leaders to the process, compelling them to act. Based on the findings of this study, we hypothesize that leadership requires more than merely supportive leaders in order to influence practice’s sustainability.

Findings showing that practitioners were being influenced by their patient’s reception of IMR supports earlier research showing that patients have an important impact on the practitioners’ further use of practices (Greenhalgh et al. 2004; Salyers et al. 2009; Stewart and Chambless 2007). Although the external team gave systematic evaluations of the patients to the practitioners, many practitioners based their decisions regarding IMR on a limited sample and displayed a great deal of confidence in their own judgment. Adaptations to interventions are common and sometimes necessary to sustain them. To figure out which pieces of the practice to keep, which patients it would work for, how to recruit and engage them, and how to fit it into their usual workload. This is not necessarily just signs of poor likelihood of sustainability. However, to obtain wanted effect from the intervention, a planned and considered adaptation is necessary, rather than ad hoc (Aarons et al. 2012). Outcome monitoring was not considered as a tool for the further evaluation of IMR, and therefore these practitioners were at risk of rejecting a practice despite good evidence.

As earlier findings (Bond et al. 2009) indicate, the monitoring philosophy, which is considered an important standard in implementation science, seems to be difficult to implement in everyday practice. Similarly to what Rychener et al. (2009) experienced, the practitioners’ understanding must be worked on for them to experience assessments as a resource for program improvement.

The current study used the framework introduced by Fixsen et al. (2013, 2005). Several of the findings support this framework, such as the importance of leadership and fidelity monitoring. Yet, little is said about practitioner autonomy or whether it is guided by patient reception. Therefore, the framework could benefit by emphasizing the health-practitioners’ role in the implementation process beyond the competency drivers that focus on selection, training, and coaching.

There were some strengths and limitations of the current study. The sample consisted of voluntary practitioners in mental health services who achieved high fidelity scores, which is the exception rather than the rule. The results may not be transferable to the implementation of other practices that are required by service leaders or management. The implementation advisory group carried out the interviews, which may have restricted the respondents’ feedback. However, we believe that we managed to reveal both challenges and facilitators in the implementation of IMR in all of the services. In addition, the study was based on the practitioners’ perceptions of the implementation. We do not know to what degree the leaders were engaged or to what degree the practitioners were autonomous. The project was characterized by bottom-up implementation, with voluntary recruitment. The practitioners managed most of the implementation activities, which may have challenged sustainability.