This study of organizational readiness provides insight into the complexity of engaging in organizational change18 and evidence-based quality improvement. Each intervention site had different strengths and weaknesses, but these would likely not have been revealed without the triangulated ORC and semi-structured interview data.
Implementation strategies at each site, in the site-level controlled trial, were tailored based on the findings from the baseline assessment. Sites A and B were both ready for change, so site A served as the lead implementation site, given its adaptability and emphasis on professional growth and trying new techniques. The site developed testimonials for the other sites on the successes of their implementation strategies. At site B, a strong sense of cohesion was built on by having the opinion leader talk to the staff as a whole about implementation and having frequent meetings with all staff where the project was discussed and problems were addressed.
Sites C and D required more tailoring of the context of the intervention to give it its best chance of acceptance. In site C, training needs were addressed by heightening awareness of gaps in care through use of opinion leaders and educational programs; changing leadership was compensated for by keeping the remaining team the same and by maintaining a consistent message about the mission and goals of the intervention targets; autonomy was encouraged by allowing clinicians to design aspects of their interventions (wherever possible given the evidence base) around the VISN-established care targets.
In site D, adaptability issues were addressed by bringing the opinion leader and key staff on board first and providing direct guidance on the specifics of implementing the intervention; efforts were made to link study goals with administration’s goals in order to address mission issues. Cohesion was enhanced by building teams within the clinic, and as with site C, autonomy was encouraged by allowing clinicians to design the implementation of their interventions.
Overall, intervention sites were moderately ready to change, and may have been identified by VISN leadership because they were perceived to be ready to change (sites A and B) and/or in need of change but also in need of additional support (sites C and D). The latter sites required more flexibility from the coordinating site as to how to implement the intervention, whereas the former sites could implement the interventions closer to the original intent. As noted, half (51%) of the intervention sites’ respondents had worked at the VA for 11 years or more, and the response rate was relatively low among staff (57%). This could suggest that intervention site staff in general were entrenched in their routines and/or perhaps had seen change efforts that had not worked or that had not been sustained and were skeptical about engaging in the project. The low response rate may indicate resistance on the part of the intervention sites to the presence of the project. Despite numerous targeted efforts to increase the response rate, those who consented but refused to complete the instrument were dogmatic in their refusal, usually due to the perception that the instrument would take too much time to complete.
The program needs subscale reflected the most variability in responses, with high standard deviations in three of the four sites. This variability may be due to differences in educational and professional backgrounds such that respondents had differing emphases on the areas in which their clinics needed additional guidance. Training needs, program needs, and change were the domains with generally lower scores, and staff attributes scores remained high. The latter finding could be because these questions focused on perceptions of one’s own professional motivation (growth) and personality style (adaptability), which might be more proximal and stable than most of the other subscales, which call for evaluation of the clinic and therefore might be more distal and subject to fluctuation depending on daily work dynamics.
Next Steps in Organizational Assessment and Implementation
EQUIP-2 falls within a burgeoning number of studies that assess barriers and facilitators to implementation and then focus on refining implementation strategies during intervention.19 Staff expectations, perceptions, and attitudes may encourage or inhibit adoption of evidence-based practices,20 and are thus critical to investigate and address at the outset of a quality improvement project through formative evaluation methods.21 Also critical to investigate are the larger systemic and structural factors that may affect attitudes towards and experiences of change. For example, in a system such as the VA, change from the “top” is expected by staff, and typically the changes are not voluntary or optional.
Key stakeholder perspectives will be gathered mid- and post-intervention, and organizational readiness will be assessed again post-intervention to see if readiness changed over time. This long-term perspective on adoption of innovation has been recommended by others who have noted that change toward evidence-based care in mental health care is a slow and uneven process, warranting a longitudinal perspective.22,23 Issues such as organizational vision and commitment may affect the long-term sustainability of innovations, so ongoing assessment of these issues could be critical.24 With a longitudinal perspective, this study will be able to explore associations between patient outcomes and organizational readiness, as has been demonstrated by others.25 Whitley and colleagues26 suggest that implementation of illness management and recovery requires strong leadership, an organizational culture that embraces innovation, effective training, and committed staff. Lin and colleagues1 suggest that employees’ perceptions of an organization’s orientation, activities, and support of quality management are associated with their perceptions of whether implementation of quality improvement activities will lead to improved patient outcomes.
Limitations
This study has limitations in that the sample sizes are not large and there is variability across sites, so generalizability is limited. Data from administrators was particularly limited, so insight is not provided here as to the ways in which administrators and staff may differ in their perceptions of organizational readiness, as operationalized by the ORC and as explored in the interviews. Additionally, taking the average survey response from a group of employees is a limited way in which to assess organizational readiness, at least in part because employees likely differ in their awareness of and contribution to readiness. This study was not designed to test variable contributions to readiness, but an investigation that tests this idea would be clinically and empirically valuable.
Conclusion
One of the main points of this quality improvement effort was the need to acknowledge from the outset that clinical settings, though they may be similar in many ways, are never completely equivalent despite best efforts to have them equivalent from an empirical standpoint. Accordingly, we designed the study to explore the ways in which the sites were not equivalent, i.e., to assess in some depth each site’s organizational climate, with more emphasis on the intervention sites because we wanted to maximize uptake of the intervention in order to improve quality of care.
Organizational change, though difficult to achieve, can occur. Adoption of evidence-based care in specialty mental health is critical for the improvement of patient outcomes.27 The onus is on implementation researchers to continue to identify factors that facilitate successful adoption of appropriate and effective clinical practices. One step toward the identification of these factors is thorough assessment of organizational readiness for change, so subsequent intervention efforts are more carefully attuned to the strengths and barriers present in each site. Ideally, with locally tailored implementation strategies, adoption of evidence-based practices will increasingly become the norm, and patients seeking services will receive the care that they need in order to have optimal health outcomes.