Background

Chronic obstructive pulmonary disease (COPD) is an irreversible, progressive, and debilitating respiratory illness characterized by airway inflammation and airflow limitation [1, 2]. It is the fourth leading cause of death and disability [3] and the third leading cause of hospitalizations in the US It is estimated that COPD will become the leading global cause of death by 2033 [4, 5]. The US Veteran population is especially vulnerable to COPD as Veterans are four times more likely to be diagnosed with it, more susceptible to its complications, and have a higher COPD mortality rate compared to the civilian population [6].

Although COPD is not fully reversible, it is treatable when evidence-based approaches to its management such as medication optimization, adherence review, inhaler technique, and symptomatic assessment are used [7]. While these best practices are well-established in the literature and clinical guidelines, it remains a challenge to embed these recommendations into routine primary care delivery models [8, 9]. As a result, only one-third of US patients with COPD receive evidence-based treatment [9]. Barriers to implementing and scaling COPD evidence-based best practices include lack of informatics infrastructure, limited staffing and practitioner engagement, and high workload [9, 10].

COPD care bundles, which combine multiple evidence-based clinical interventions for COPD management into one service [11] have demonstrated positive patient outcomes [12,13,14,15,16,17,18,19,20] Yet there is a need to better understand the best approaches to promote scale-up of COPD bundles across multiple settings [15].

This evaluation explores whether a virtual implementation package, COPD Coordinated Access to Reduce Exacerbations (CARE) Academy (Academy), built capacity for implementing a COPD care bundle in the Veterans’ Health Administration (VA). The VA is the largest integrated health care system in the United States with 171 VA Medical Centers (VAMCs) with unique processes, cultures, priorities, and geographic barriers that can make scaling best practices difficult.

Initial design of the Academy began in 2018 with the development of a clinical training program that was refined and tested across two VAMCs [16]. The program was found to have a positive impact on clinician confidence and interprofessional collaboration; however, clinicians reported they needed additional guidance and resources to overcome logistical barriers to implementing COPD CARE [17]. Furthermore, a national implementation team, including experts in pharmacy and COPD management, recognized that different implementation strategies were needed to scale COPD CARE more rapidly.

To address these obstacles and promote effective service reach, the national implementation team developed a more comprehensive implementation package over a 12-month period with support from VA experts in Dissemination & Implementation (D&I) science [21]. The Academy is a 5-week virtual training program led by national COPD CARE experts who serve as external facilitators. It is comprised of five discrete implementation strategies [22] including implementation facilitation (IF) (Fig. 1), which is an interactive approach to addressing implementation challenges through forming supportive relationships [23]. The Academy provides facilitation through cohort-based learning by convening clinicians from multiple VAMCs working together to implement the COPD CARE bundle at their respective VAMCs. The Academy trains clinicians and provides them with resources to serve as internal facilitators and support implementation at their VAMCs. The external facilitators are included to promote discussion and shared problem solving. Virtual discussions, guided implementation resources, informatics support, and clinical training support were integrated within the IF model (Fig. 1).

Fig. 1
figure 1

Core components of the Academy

Full size image

Evaluation conceptual framework

We applied the RE-AIM conceptual framework [24, 25] to measure the impact of the Academy on building capacity for implementing COPD CARE. RE-AIM emphasizes translating evidence-based interventions into practice and has five domains and associated measures for examining an intervention’s reach (R), effectiveness (E), adoption (A), implementation (I), and maintenance (M). RE-AIM was selected as a guiding framework to assess the Academy impact due to its inclusion of implementation outcomes in addition to effectiveness outcomes [24].

The specific aims of this evaluation were to assess the impact of the Academy on clinicians’ capability to implement COPD CARE (effectiveness) and to assess the reach, adoption, implementation, and maintenance of the Academy as it relates to clinicians’ capability for implementation. Future evaluations are planned to examine more distal outcomes, such as the effect of the Academy on the implementation of COPD CARE as indicated by increased use of best practices and improvements in Veteran care.

Methods

Design

This quality improvement evaluation utilized a mixed-methods approach to obtain retrospective feedback from clinicians about their perceptions of the Academy and its impact on their capability to implement COPD CARE. This evaluation was determined not to meet the federal definition of research and qualified for a quality improvement exemption.

Setting and sample

The Academy was implemented in two cohorts in Fall 2020 and Spring 2021. Cohort one involved five Midwestern VAMCs and Cohort two involved eight VAMCs from the West and East coasts and the Southwest. Two approaches were used to identify the VAMCs: [1] the Academy was promoted through a national VA website known for promoting promising practices to VA leaders [26], and [2] program developers strategically engaged with Clinical Pharmacy Executives across the VA to identify VAMCs with a strong interest in the program. VAMCs that agreed to participate were instructed to complete a pre-implementation workbook, which involved identifying implementation team members, including an implementation lead, responsible for guiding the overall process at their site and a clinician lead responsible for conducting the front-line clinician training.

Data sources

Three primary data sources were used for this evaluation: [1] administrative records, [2] a survey, and [3] a semi-structured interview and are described in Table 1.

Table 1 Data source, description, and timeframe
Full size table

Measures

Descriptions and data sources for each RE-AIM domain are detailed in Table 2.

Table 2 Description and data source for RE-AIM domain measures
Full size table

Data analysis

Descriptive statistics were calculated for all quantitative survey items and thematic analysis was used to summarize the qualitative open-ended items. To assess changes in self-reported capability, the non-parametric Wilcoxon signed-rank test was used for the 10 Likert scale items. No adjustments for repeated testing were made and an alpha level of 0.05 was used. IBM SPSS Statistics (Version 28) [27] was used for the statistical analysis. For structured interview data, descriptive statistics were calculated to summarize the eight Likert scale items and frequencies were calculated for the dichotomous yes/no items. An independent evaluator with no direct affiliation to the VA conducted a thematic analysis of the open-ended items separately, and a consensus of final themes was agreed on through group discussion with the project lead. The thematic analysis was conducted using NVivo [28]. Initially, an inductive approach was taken using open coding. Survey and interview findings are presented within the RE-AIM framework to identify indicators of the RE-AIM effectiveness and implementation outcomes for each domain. We used the SQUIRE 2.0 reporting guidelines when writing this paper [29].

Results

The results are presented for each RE-AIM domain.

Reach

Thirty-six clinicians from 13 VAMCs participated in the Academy in Fall 2020 and Spring 2021. Across all 13 sites, 264 front-line clinicians completed the COPD CARE clinician training. This group included 130 pharmacists, 117 nurses, and 17 other front-line clinicians (e.g., respiratory therapists).

Effectiveness

Sixteen clinicians who served as clinician leads and implementation leads from the 13 VAMCs responded to the survey. Thirteen (81%) respondents were Clinical Pharmacist Practitioners and three (19%) reported other professions (i.e., inpatient care med-surg Tele Nurse). Clinicians reported significant increases in their capability to complete implementation efforts after participation in the Academy across ten items representing implementation tasks (p < 0.05) (Table 3).

Table 3 Changes in clinician capability to complete implementation tasks before and after Academy participation
Full size table

Adoption

Over 90% of clinicians responding to the survey reported complete or nearly complete attendance at all five of the Academy weekly discussions. Clinicians from 12 (92%) VAMCs participated in the semi-structured interview; one VAMC declined the request for an interview. Interviewed clinicians reported high utilization of Academy resources, with the workbooks being used by clinicians at all 12 (100%) VAMCs, followed by 11 (92%) VAMCs using the live virtual debrief meetings and the COPD CARE resources available through a shared network drive. Interview responses indicated that clinicians at three-fourths or more of VAMCs reported using the Academy weekly emails (83%), weekly YouTube videos (75%), and monthly post-Academy follow-up meetings (75%).

Implementation

Acceptability

Clinicians’ perceptions of the Academy content and delivery approach suggest they were satisfied with these aspects of the Academy [30]. Interview findings indicate that clinicians at all 12 (100%) sites viewed the Academy content to be complete and covering critical aspects of implementing the COPD CARE service and clinicians at 83% of VAMCs reported the approach to delivering Academy content was effective.

Clinicians reported that they valued the team-based support aspect of the Academy including the opportunities to be part of the virtual discussions and a learning collaborative. Survey results indicated that nearly all clinicians (94%) found that learning from colleagues at other VAMCs during the Academy and attending the weekly live sessions (81%), were some of the most valuable aspects of the Academy.

The interviews corroborated the survey findings. Table 4 presents representative clinician quotes. Related to the Academy clinical training support, clinicians at about 75% of VAMCs viewed the clinical training content as helpful for preparing front-line clinicians to deliver COPD CARE. This was reiterated in the interviews with a clinician sharing that the clinical training content had far-reaching beneficial effects on increasing clinician comfort and motivation to treat COPD (Table 4). Clinicians shared how valuable it was to have the opportunity to problem-solve with clinicians from other VAMCs. However, clinicians also reported the Academy lacked sufficient content in certain areas (e.g., exploring spirometry in greater depth, additional resources to describe the COPD CARE referral process, and additional informatics support) (Table 4).

Table 4 Themes and representative quotes
Full size table

Appropriateness

Clinicians at nearly 70% of VAMCs perceived the Academy to be critically important and clinicians at 75% of VAMCs reported having the necessary tools and resources to implement COPD CARE after Academy participation. Clinicians found the external IF approach to be appropriate. Participants felt supported, motivated, and encouraged by the support from the national facilitator (Table 4). However, some clinicians reported challenges to Academy participation; describing ways in which the Academy may not have been perceived as suitable or practical as an implementation package for those VAMCs. For example, one clinician found that the weekly Academy topics were not always aligned with where their VAMC was in the implementation process (Table 4).

Maintenance

Clinicians from 92% of responding VAMCs reported long-term utilization of Academy resources and clinicians from 75% of VAMCs reported participating in the post-Academy meetings with other participants. The Academy had a lasting effect on sites, which was reflected in VAMC’s integration of the Academy into the site organizational structure. Several clinicians reported that their implementation teams continued to have regular communication after the Academy (Table 4). For many, the Academy served as a lasting resource. Clinicians reported long-term use of the training materials and resources months after the Academy, suggesting its lasting effect and value (Table 4).

Discussion

Guided by the RE-AIM framework we evaluated the Academy’s impact on building capacity for implementing COPD CARE, including clinicians’ perceived capability to implement COPD CARE. The use of IF as the overarching approach paired with additional strategies seemed to demonstrate positive outcomes across all RE-AIM domains. In this evaluation, we found that the fully-virtual, cohort-based IF approach was successful at building capacity for implementing COPD CARE at a large number of VAMCs simultaneously. This approach eliminates geographic and cost barriers to participation, increasing the reach of the Academy.

Interview findings suggested clinicians were committed to Academy participation and had a high degree of resource utilization, indicating successful adoption. A majority of clinicians were satisfied with the content and delivery approach and viewed the Academy as a useful and practical approach to implementing COPD CARE, which are indicative of successful implementation. These positive RE-AIM implementation outcomes likely contributed to the effectiveness of the Academy at increasing clinicians’ perceptions of their capability to implement COPD CARE. As clinicians embraced the Academy and participated in the virtual discussions with other VAMCs, they felt supported and their perceived capability to be successful at implementing COPD CARE increased. The linkage between the Academy’s strategies and increasing perceived capability or self-efficacy is supported by Bandura’s Social Cognitive Theory [31].

Findings from this evaluation add to the literature on the development of internal facilitators to build capacity for implementing best practices [23, 32, 33]. The Academy participants reported receiving the guidance and resources they needed to successfully implement, suggesting that the external facilitators provided the necessary knowledge and guidance for skill development [32]. The Academy supported relationship building and the creation of a supportive environment, which was reflected in clinicians’ reporting that they valued the opportunity to interact and problem solve with the external facilitators as well as with other internal facilitators from other VAMCs who were implementing simultaneously. The Academy also provided training in COPD management and materials and support for clinicians to train their colleagues, which is a necessary support for internal facilitators in healthcare settings [33].

Our findings suggest that the Academy was successful in enhancing clinician’s perceived capability to complete implementation tasks, which likely contributed to minimizing barriers to implementing COPD best practices [9, 10]. To promote practitioner engagement and address limited staffing, the Academy provided training and resources for clinicians to engage with colleagues in other disciplines (e.g., nursing, respiratory therapy) and obtain their buy-in to collaborate on the implementation of COPD CARE. This is reflected in our finding that clinicians’ reported increased capability to gain support from leadership for COPD CARE and to form collaborations with other services after participation in the Academy. The development of such collaborations promotes sharing of staffing responsibilities, addressing barriers related to limited staffing. The Academy also promoted the development of an informatics infrastructure through the provision of COPD CARE referral dashboards and clinical note templates. Clinicians were guided through the process of initially installing informatics tools and were provided training materials to support them in training their clinician colleagues in the use and application of the informatics tools.

Despite the positive outcomes in our evaluation, some clinicians identified challenges to participating in the Academy and highlighted content or resources they perceived to be lacking, such as additional training and informatics support to improve patient referrals. Opportunities exist to explore this barrier and enhance the COPD CARE referral process.

There were some potential limitations to this evaluation. First, clinicians assessed their pre-Academy capabilities after Academy participation, which introduces the potential for bias. This was mitigated in part by discussing their perceptions of the Academy on their capabilities during the interview. Second, some of the interviews involved multiple clinicians from a VAMC, which may have affected their willingness to openly share their experiences with the Academy. To the extent possible, interviewers made efforts to create a safe environment to facilitate honest feedback. Third, some clinicians did not participate in the interviews and important differences in their viewpoints may not have been captured in data collection. While there was variation in interview participants across VAMCs, the evaluation team ensured that at least one clinician who was designated as an implementation lead was involved in each interview. Future evaluations will more carefully consider sampling approaches. A limitation of this evaluation is its sole focus on the front-line clinician perspective and not other stakeholders’ (e.g., leadership, clinic managers) perspectives. Furthermore, VAMCs engaged in this evaluation elected to implement COPD CARE, which may have increased clinician motivation to complete the Academy. Notably, these efforts to disseminate COPD CARE were made during a global pandemic, clinicians were not provided with additional salary support or protected time for their implementation efforts, and national facilitation of the program relied heavily on part-time VA pharmacy interns.

Conclusions

Through this evaluation, we documented the impact the Academy had on enhancements to clinician perceptions of capability to implement successfully. We also identified potential areas of improvement for the Academy as an implementation package to support the scale-up of COPD CARE. These lessons learned are important to inform future Academy improvements as it is rolled out to additional VAMCs. This evaluation adds to the growing evidence base supporting the efforts to scale COPD CARE. It builds on a previous iteration of the implementation package focused solely on clinician training and paves the way for future evaluations to further examine the Academy’s impact and future iterations of post-Academy strategies.