Implementation Evaluation of a Complex Intervention to Improve Timeliness of Care for Veterans with Transient Ischemic Attack

Background The Protocol-guided Rapid Evaluation of Veterans Experiencing New Transient Neurologic Symptoms (PREVENT) program was designed to address systemic barriers to providing timely guideline-concordant care for patients with transient ischemic attack (TIA). Objective We evaluated an implementation bundle used to promote local adaptation and adoption of a multi-component, complex quality improvement (QI) intervention to improve the quality of TIA care Bravata et al. (BMC Neurology 19:294, 2019). Design A stepped-wedge implementation trial with six geographically diverse sites. Participants The six facility QI teams were multi-disciplinary, clinical staff. Interventions PREVENT employed a bundle of key implementation strategies: team activation; external facilitation; and a community of practice. This strategy bundle had direct ties to four constructs from the Consolidated Framework for Implementation Research (CFIR): Champions, Reflecting & Evaluating, Planning, and Goals & Feedback. Main Measures Using a mixed-methods approach guided by the CFIR and data matrix analyses, we evaluated the degree to which implementation success and clinical improvement were associated with implementation strategies. The primary outcomes were the number of completed implementation activities, the level of team organization and > 15 points improvement in the Without Fail Rate (WFR) over 1 year. Key Results Facility QI teams actively engaged in the implementation strategies with high utilization. Facilities with the greatest implementation success were those with central champions whose teams engaged in planning and goal setting, and regularly reflected upon their quality data and evaluated their progress against their QI plan. The strong presence of effective champions acted as a pre-condition for the strong presence of Reflecting & Evaluating, Goals & Feedback, and Planning (rather than the other way around), helping to explain how champions at the +2 level influenced ongoing implementation. Conclusions The CFIR-guided bundle of implementation strategies facilitated the local implementation of the PREVENT QI program and was associated with clinical improvement in the national VA healthcare system. Trial registration: clinicaltrials.gov: NCT02769338 Electronic supplementary material The online version of this article (10.1007/s11606-020-06100-w) contains supplementary material, which is available to authorized users.


BACKGROUND
Approximately 3300 veterans with transient ischemic attack (TIA) are cared for in a United States Department of Veterans Affairs (VA) Emergency Department (ED) or inpatient ward annually. 1 TIA patients are at high risk of recurrent vascular events [2][3][4] ; however, interventions which deliver timely TIA care can reduce that risk by up to 70%. [5][6][7][8] Despite the known benefits of timely TIA care, gaps in care quality exist in both privatesector US hospitals 9 and the VA healthcare system. 10 A formative evaluation of TIA acute care in VA indicated that most facilities lacked a TIA-specific protocol and that clinicians struggled with uncertainty regarding the decisions to admit TIA patients for timely care. 11,12 The objective of the PREVENT trial 13 was to evaluate the effectiveness of an implementation strategy bundle to promote local adaptation and adoption of a multicomponent QI intervention to improve TIA care quality. The bundle was based upon several implementation frameworks and our previous success employing external facilitation with VA clinical teams. 14 First, we used the integrated Promoting Action on Research Implementation in Health Services (iPARIHS) framework 15 to guide external facilitation to assist local champions cultivate clinical teams, and disseminate professional education materials related to acute TIA care, and to facilitate local QI efforts using performance data. Second, based on prior studies which have distinguished CFIR constructs between high and low effective implementation, 16 we operationalized key constructs (Planning; Goals & Feedback; and Reflecting & Evaluating) from the CFIR inner setting and implementation process domains 17 a priori. Participating teams were trained on these concepts and received reinforcement through external facilitation.
The specific aim of this evaluation was to examine the effect of the implementation strategy bundle on implementation success. We hypothesized that clinical teams which engaged in the implementation strategies and locally adapted the PREVENT program components would realize the greatest implementation success.

Setting
VA facilities were rank ordered in terms of the quality of TIA care based on seven guideline-concordant processes of care and invitations to participate were sent to VA facilities with the greatest opportunity for improvement. Recruitment stratified by region continued until six facilities agreed to participate. PREVENT sites were pragmatically allocated to the stepped-wedge trial in three waves based on the ability to schedule baseline and kickoff meetings.

Quality Improvement Intervention
The rationale and methods used for the development of the PREVENT intervention have been described elsewhere. 13 The provider-facing QI intervention was based on a prior systematic assessment of TIA care performance at VA facilities nationwide as well as an evaluation of barriers and facilitators of TIA care performance using four sources of information: baseline quality of care data, 10 staff interviews, 11 existing literature, [18][19][20][21][22] and validated electronic quality measures. 10 The PREVENT QI intervention included five components (see Appendix 1): quality of care reporting system (see Appendix 2), clinical programs, professional education, electronic health record tools, and QI support including a virtual collaborative.

Implementation Strategies
PREVENT employed a bundle of three primary implementation strategies: (1) team activation via audit and feedback, 23,24 reflecting and evaluating, planning, and goal setting 17 ; (2) external facilitation (EF) [23][24][25] ; and (3) building a community of practice (CoP). 26 In addition, PREVENT allowed for local adaptation of its intervention components and the coordinating site provided EF to the site champion and team.
Active implementation of PREVENT at each site began with a full-day kickoff meeting facilitated by the coordinating site, and involved multidisciplinary staff members from the participating site involved in TIA clinical care. The site team used the PREVENT Hub, a Web-based audit and feedback platform (see Appendix 2), to explore their facility-specific quality of care data and identify gaps. Using approaches from systems redesign, 27,28 site team members brainstormed about barriers to providing highest quality of care, identified solutions to address barriers, ranked solutions on an impact-effort matrix, and developed a site-specific action plan that included high-impact/low-effort activities in the short-term plan and high-impact/high-effort activities in the long-term plan. Local QI plans were entered into the PREVENT Hub, and metrics were tracked on the Hub allowing teams to monitor performance over time. Using the Hub to observe other participating sites' QI activities and performance, facility teams could learn which QI activities either did or did not improve metrics at peer sites. In addition, the coordinating team introduced strategies for activating teams: team planning, goal setting and feedback, and reflecting and evaluating using examples and data from past stroke QI teams and presenting a video of a VA stroke QI team in practice.
During the 1-year active implementation period, the site team members joined monthly PREVENT virtual collaborative conferences which served as a forum for sites to share progress on action plans, articulate goals for the next month, and review new evidence or tools. EF was provided by the PREVENT nurse trained in Lean Six Sigma methodology 28 and quality management.

Evaluation Approach
The stepped-wedge 29,30 implementation trial included six participating sites where active implementation was initiated in three waves, with two facilities per wave. The unit of analysis was the VA facility.

Measurement
We employed a mixed-methods design to evaluate this complex implementation intervention with prospective data collection from multiple sources. Qualitative data sources included the following: semi-structured interviews, observations, field notes, and Fast Analysis and Synthesis Template (FAST) facilitation tracking. 31 Interviews were conducted in-person during site visits or by telephone at baseline, and at 6 and 12 months after active implementation. Key stakeholders included staff involved in the delivery of TIA care, their managers, and facility leadership; we also accepted "snowball" referrals from key stakeholders. Upon receipt of verbal consent, interviews were audio-recorded. The audio-recordings were transcribed verbatim. Transcripts were de-identified and imported into Nvivo12 for data coding and analysis. 32 Using a common codebook, two team members independently coded identical transcripts for the presence or absence of CFIR constructs as well as magnitude and valence for four CFIR implementation constructs (i.e., Goals & Feedback, Planning, Reflecting & Evaluating, and Champions). Valence (+ or −) was scored for each construct if it was present and influencing the implementation of PREVENT at that site. 16,17 Magnitude was scored as 2 if it had a strong influence on PREVENT implementation, 1 if it had a weak or moderate effect, and 0 if it had a neutral effect. The evaluation team conducted formal debriefings after each kickoff, site visit, and collaborative call. These observations were recorded and transcribed for analyses.
We also used the FAST template, which is a structured electronic log, as a rapid, systematic method for extracting key concepts across data sources including interviews, collaborative calls, and Hub utilization data. 31 We adapted an external facilitator tracking sheet for prospective collection of the dose and contents of site-specific, external facilitation provided by the evaluation team to participating site teams. 25 Facility Baseline Characteristics. The measure of quality of care was the "without-fail" rate (WFR), defined as the proportion of veterans with TIA who received all of the processes of care for which they were eligible from among seven processes of care ( Table 1). The WFR was calculated at the facility level based on electronic health record data using validated algorithms. 10 In addition to the baseline WFR for each facility, data from the Office of Productivity, Efficiency and Staffing (OPES) were obtained to classify neurology and emergency medicine staffing levels (https://reports.vssc.med.va.gov/ReportServer/Pages/ ReportViewer.aspx?/OPES/WorkforceRpt/WorkforceReport&rs:Command=Render&rc:Parameters=True&Specialty=Inter-nal%20Medicine&FiscalYear=FY%202018). We report the annual TIA volume at each site (seen in the ED and inpatient setting) and the proportion of patients who were admitted.

Implementation Evaluation
Teams reported on implementation progress on a monthly basis during the virtual collaborative calls. Teams were encouraged to adapt PREVENT components best suited for their local context and addressed gaps in care; thus, a fidelity evaluation was not applicable. The first of three primary implementation outcomes included the facility team's number of implementation activities (defined as an intentional activity planned to change practices; aligned with each site's action plans (see Appendix 3) and completed during the 1-year active implementation period). 14 The second outcome included final level of team organization (defined as the degree of team cohesion and Group Organization [GO Score]) 16,33 for improving TIA care at the end of the 12-month active implementation period. The GO Score 16, 33 is a measure of team activation on a 1-10 scale for improving TIA care based on specified practices (see Table 2). The evaluation team independently determined each site's GO Score by discussing evidence from the study data sources during team meetings and then voting independently using a digital, real-time anonymous ballot until 80% agreement was reached. The rationale for using both implementation outcomes was that they measured two distinct but complementary aspects of implementation: number of activities completed is an overall measure of implementation action, whereas the GO Score describes the degree to which the facility team is functioning as a unit to implement facility-wide policies and structures of care.
As an additional clinical measure of implementation outcome, the final column in Table 3 indicates whether or not the facility achieved ≥ 15-point improvement (in absolute terms to reflect planned vs temporal change) in their WFR over their 1year course of active implementation (see Table 1).
Using a mixed-methods approach 34,35 grounded in the CFIR, 16, 17 we conducted a cross-case and data matrix approach 36 to evaluate the degree to which the sites engaged in the bundle of implementation strategies; the association between implementation strategies and implementation and clinical success; and the associated contex-tual factors. Given that PREVENT's implementation strategies and outcomes were tracked and rated by the coordinating team prospectively, no data were missing.   The teams joined monthly PREVENT collaborative conferences which served as a forum for facility team members to share progress on action plans, articulate goals for the next month, and review any new evidence or tools. Collaborative conference attendance includes the total number of calls that team members joined during the 1-year active implementation period plus the first month during the kickoff when the kickoff occurred early in the month for a total of 13 possible calls, as well as the number of unique team members who participated. After active implementation completion, participants were permitted to join the calls but no longer participated in updates. For example, at facility F, the 7 unique team members' (excluding the champion) participation distribution was 36 call attendings over the 22 possible calls. Note, not all 7 team members participated in all 22 calls. Wave 1 sites had the opportunity to attend more collaborative conferences than wave 2 or wave 3 sites † The Web-based PREVENT Hub provided performance data about a broad range of processes of care, healthcare utilization, and other aspects of care; these data were updated monthly for every VA facility. The Hub also displayed facility implementation plans and included a library of resources. Hub utilization is displayed in terms of the number of total visits by any team member from the facility during the 1-year active implementation period; the number of unique individuals is in parentheses ‡ EHR refers to electronic health record **The GO Score refers to the Group Organization Score for improving TIA care quality; it is a measure of team activation and cohesion. The GO Score is measured on a scale of 0-10 based on specific practices in place during a given time period and scored by the evaluation team. A score of 0-3 indicates the absence of a facility-wide approach; 4-5 reflects a developing facility-wide approach; 6-7 denotes basic proficiency with the presence of a comprehensive facility-wide program; and 8-10 indicates the presence of a mature, facility-wide system that can sustain key personnel turnover. Because the GO Score reflects team activation and cohesion, some sites with individuals who are actively engaged in quality improvement but without a robust team approach may have lower GO Scores but still complete a relatively high number of implementation activities ***Site D had a total of 3 local champions over the course of one year of active implementation. The first champion left the organization within the first 6 months of active implementation; the second champion emerged from the team and lead the transition; and the third champion joined the transitional champion to implement the action plans

Baseline Context
The baseline facility context and QI team characteristics of the six participating facilities are provided in Table 1, listed in order of their baseline quality (WFR). The WFR for sites B, C, and D was similar to the fiscal year 2017 national WFR average of 34.3%, whereas site A was substantially below and Sites E and F were considerably higher than the national WFR. Sites E and F also had the highest level of neurology staffing. Emergency medicine staffing was similar across sites. More than 50% of TIA patients were admitted to the hospital, but admission rates were lowest at the two sites (A and B) with the lowest WFRs. The annual TIA patient volume varied from 13 to 46. At baseline, no sites had active teams in place working on TIA care quality, indicating that all of the teams began the active implementation period from a similar starting point. The participating site QI teams were diverse but generally included members from neurology, emergency medicine, nursing, pharmacy, and radiology; some teams also included hospitalists, primary care staff, education staff, telehealth staff, or systems redesign staff.

Implementation Strategies
Over the course of the 1-year active implementation period, we observed an overall high site engagement with each of the implementation strategies. In Table 2, we present the dose of implementation strategies delivered within the overall strategy bundle: EF, community of practice, Hub (audit and feedback), and local adaptation of PREVENT. The total number of completed implementation activities and final GO Score after 12 months of active implementation are also presented in Table 2. Site labels are retained from Table 1 and are listed in order of the three waves.

Audit and Feedback
We observed frequent usage of the Hub.

External Facilitation
Facility QI teams and champions engaged with the EF during active implementation. Education on PREVENT components The GO Score refers to the Group Organization Score for improving TIA care quality; it is a measure of team activation and cohesion. The GO Score is measured on a scale of 0-10 based on specific practices in place during a given time period. A score of 0-3 indicates the absence of a facility-wide approach; 4-5 reflects a developing facility-wide approach; 6-7 denotes basic proficiency with the presence of a comprehensive facility-wide program; and 8-10 indicates the presence of a mature, facility-wide system that can sustain key personnel turnover. Because the GO Score reflects team activation and cohesion, some sites with individuals who are actively engaged in quality improvement but without a robust team approach may have lower GO Scores but still complete a relatively high numbers of implementation activities during the first half of the year, quality monitoring, planning, and networking between and within sites were the most commonly employed EF topics. Overcoming barriers and data reflection and evaluation were also frequent EF tasks.
The EF was really helpful in allowing me to …call and vent, and she was also really very encouraging. …That was an interesting lesson to learn that you might feel like you're unsuccessful because of that one particular metric, …and so I appreciate her lending me her ear... When the [EF] knew that I was encountering a barrier that was related to physicians, without asking, she would immediately provide me the data that I needed to discuss with that provider to make them understand what we were doing and that was really helpful.

Community of Practice
Sites were active participants in the virtual CoP. A site representative was present on all calls. Site champions attended an average of 80.5% of the monthly collaborative conferences (range 66 to 92%) during the active implementation period. All six teams participated in a promotion ceremony which was attended by local VA facility leadership as well as all of the PREVENT sites' team members; peers from other sites acknowledged the implementation successes and lessons learned from the graduating site.

Implementation Outcomes
The implementation outcome data (Tables 2 and 3) indicated that implementation took place at all facilities given that all sites successfully completed at least 15 implementation activities (range 15-39, mean 26.5) as part of their action plans to improve the quality of TIA care over 12 months. Despite heavy clinical demands at all participating sites, none of the teams withdrew from PREVENT. Table 3 provides longitudinal data for implementation outcomes as well as scores on CFIR constructs related to the implementation strategies which PREVENT facilitated among the local clinical champions and QI teams during the year of active implementation. The sites were ranked in terms of the final GO Score. All sites achieved a substantial increase in the GO Score from baseline to the midpoint of the active implementation period (6 months): mean of 1 at baseline (no facility-wide approach) to 5.5 at 6 months (some facility-wide approach activities). All sites began at the same level, 1, because they had no preexisting organization around TIA. The kickoff and the Hub allowed teams to examine their data, identify gaps in care, develop an action plan, and start to come together as a team; these two elements of the PREVENT intervention were the key factors responsible for the observed improvements in team activation across sites during the first 6 months of active implementation.
I came out of it [kickoff] feeling that I knew what the issue is…I know what the goal is, and I have information sources so that I'm able to do it. …. one of the biggest things that I see is that I think that it really helped to come up with like more of a team… I think [the kickoff] played a large role of how we decided we wanted to proceed … I feel like that was the first time we've, we really kind of drilled down on, on a good …starting plan …of an outline of what we wanted to accomplish.
The matrix display in Table 3 also indicates the key role of champions in promoting implementation success, as there was a direct link between the strong presence of effective champions (i.e., "+2" level) and the GO Score at both the 6-and 12month timepoints. At 6 months, the correspondence between a champion score of + 2 and a GO Score of ≥ 6 (cf. sites F, A, and C) was 100% (3/3). None of the other CFIR constructs systematically distinguished the three sites with scores ≥ 6 from the three sites without. Moreover, all three of these sites with GO Scores ≥ 6 at the 6-month timepoint ultimately achieved ≥ 15 point improvements (in absolute terms) in their WFR rates over the 1-year active implementation period, explaining 75%, 3 of the 4 sites, with ≥ 15 point gains.
At 12 months, the correspondence between a champion score of + 2 and a positive gain in the GO Score between the 6-and 12-month timepoints was likewise 100% (cf. sites F, A, B, D). As before, none of the other constructs systematically distinguished the four sites that improved their GO Score between the 6-and 12-month timepoints from those that did not. Furthermore, Table 3 indicates that the presence of an effective champion was a necessary but not sufficient condition for the strong presence of the CFIR constructs of Reflecting & Evaluating, Goals & Feedback, and Planning. For all five of the rows in Table 3, where a + 2 score appeared for any of these three CFIR constructs (cf. site F, 6 and 12 months; site A, 6 and 12 months; site B, 12 months), the correspondence with a champion score of + 2 was 100% (5/5). The reverse was not true, however: the correspondence between sites with + 2 scores for champions with + 2 scores for all 3 CFIR constructs of was only 43% (3/7) (cf. site A, 12 months, Reflecting & Evaluating = + 1; site C, 6 months, Planning = 0; site D, 12 months, all 3 constructs = +1). This indicates that the strong presence of effective champions acted as a pre-condition for the strong presence of Reflecting & Evaluating, Goals & Feedback, and Planning (rather than the other way around), helping to explain how champions at the + 2 level influenced ongoing implementation.
Site E had multiple individuals who engaged in clinical champion activities, and therefore had lower champion construct scores and achieved lower GO Scores. For example, individual team members at site E often engaged in activities that addressed processes within their own service areas, rather than carrying out coordinated, cross-service efforts led by a central champion. The negative valence in site D's clinical champion construct reflected turnover in their local champion early in active implementation which resulted in the lowest total number of activities completed. However, with the replacement of the local champion, addition of new team members, dedicated EF, and CoP and Hub engagement by the new champion and team members, site D's GO Score improved. Although the sites completed a diverse range of implementation activities (Table 2), the most common categories included activities related to professional education (e.g., teaching house staff) and implementation of clinical programs (e.g., prospective patient identification systems). Teams that engaged in planning to the greatest degree were those with the highest number of completed implementation activities (Table 3). Sites varied considerably in terms of the timing of the implementation activities completed over the 1 year of active implementation (Fig. 1, Appendix 3). Half of the teams began strongly with implementation soon after the kickoff whereas other teams engaged in more activities during the latter half of the year.

DISCUSSION
Our data suggested that the presence of an effective champion was key for implementation success . Indeed, when one site lost its champion, implementation progress was halted, and then revived with the replacement of the local champion who was subsequently supported with EF. Effective champions, individuals who "drive through" imple-mentation according to CFIR 16 , appear to play a critical role in engaging in the implementation strategies including the EF, and fostering teams in reflecting and evaluating, goal setting, and planning: activities which were related both to the total number of implementation activities completed and the degree of team cohesion. 37,38 Our results also suggest that an alternative approach to implementation occurred at a site with a distributed champion model-one in which several individuals shared the responsibility for actions usually performed by a champion. In this context, the individuals limit the activities in which they drive through implementation to their specific clinical area versus the overall program. Although this site (E) did not achieve high scores for team functioning, they did complete many implementation activities. A key to the success of this approach is the degree to which the various champions are able to independently complete a given implementation activity with limited guidance by a central champion PREVENT site champions were diverse, including staff from neurology, nursing, pharmacy, and systems redesign. Our study findings indicated that the professional discipline of the champion was less important than the role they played in either performing implementation activities themselves or engaging other team members in the QI process.
The site team members, and especially the champions, regularly contacted the EF who provided information, support, and encouragement across a broad range of topics. Two key EF activities merit further description: conducting chart reviews and facilitating implementation of the patient identification tool. Clinical champions were sometimes dismayed when the monthly performance data indicated that some patients had failed process measures. The EF conducted targeted chart review which identified gaps in care or documentation; this chart review information supported the champions in their efforts to engage in quality improvement. The EF also worked with teams to implement a patient identification tool to identify patients with TIA who were cared for in the ED or inpatient setting. This tool was used at some sites to prospectively ensure that patients received needed elements of care and at other sites to retrospectively identify opportunities for improvement. Given that many of the champions were clinicians without prior QI experience, the EF was able to help connect clinicians with local clinical informatics staff to implement the patient identification tool.
These findings have direct relevance for healthcare systems like the VA where quality improvement resources may need to be targeted at lower performing facilities which may lack existing teams, have small patient volume, and vary considerably in terms of baseline context. The current findings emphasized the importance of EF and indicate that EFs should be flexible given the heterogeneity in site needs. Moreover, the combination of the in-person kickoff meeting and the Hub at the launch of the active implementation period was critical in three domains/areas: the development of site-specific action plans that were based on site-specific performance data; the early formation of team identity; and to the training of champions and QI teams on how to reflect and evaluate on their data. The high degree of Hub usage suggests that healthcare systems implementing QI programs should provide a forum for providing updated site-level quality of care data as well as resources for quality improvement that can be readily shared across sites. Finally, healthcare systems should consider supporting QI teams within CoP that serve as supported arenas for public accountability for making progress on action plans, sharing lessons learned, and providing encouragement.

Contribution to the Literature
To our knowledge, the PREVENT implementation strategy bundle is one of the few implementation interventions to operationalize CFIR implementation process constructs as strategies a priori to train its QI teams and prospectively evaluate its uptake. Indeed, a recent review of CFIR usage in implementation research recommended future research identified the need for research with prospective CFIR use with a priori specification. 34 Moreover, we provided specifications 39 on the set of implementation strategies delivered as a bundle. In addition, we described the level and timing of engagement and implementation activities across QI teams. These findings provide evidence for specific implementation strategies in the setting of a complex clinical problem when no quality of care reporting system exists. 40 Our results are aligned with a recent systematic review of the effect of clinical champions on implementation which concluded champions were necessary but insufficient for implementation. 41 An emergent finding was that modest and strong positive CFIR planning construct was related to implementation success (especially with respect to the number of implementation activities completed).

Limitations
The primary limitation of PREVENT is that implementation occurred only within VA hospitals. Future research should evaluate implementation in non-VA settings. Because several implementation strategies were deployed simultaneously, it was difficult to disentangle the unique effects of each strategy. Although a six-site sample was sufficient to make case comparisons, future studies might include a larger number of facilities to evaluate additional implementation outcomes.
In summary, this study found that engagement in a bundle of CFIR-related implementation strategies facilitated the local adaptation and adoption of the PREVENT TIA QI program and that two alternative approaches to the role of champions were associated with implementation success. Authors' Contributions All authors participated in the revision of the manuscript and read and approved the final manuscript. DMB: obtained funding and was responsible for the design and conduct of the study including intervention development, data collection, data analysis, and interpretation of the results. TD, LJM, EJM, NR: instrumental in the design of the intervention, implementation strategies, data collection, data analysis, and interpretation of the results. TD and DMB drafted and revised the manuscript along with edits from LP, EJM, NR, BH, LM. BH: instrumental in the design of the intervention and data analysis. LP: participated in data analysis, and interpretation of results. AJC, JM, SB, CA: participated in data analysis. JJS, AP, ZY, LDM: participated in study design, data analysis, and interpretation of results. BG and MK: participated in the intervention design and data collection. Data Availability: These data must remain on Department of Veterans Affairs servers; investigators interested in working with these data are encouraged to contact the corresponding author.

COMPLIANCE WITH ETHICAL STANDARDS:
Disclaimer: Support for VA/Centers for Medicare and Medicaid Service (CMS) data is provided by the VA Information Resource Center (SDR 02-237 and 98-004). The funding agencies had no role in the design of the study, data collection, analysis, interpretation, or in the writing of this manuscript.
Ethics Approval and Consent to Participate: PREVENT and the Implementation Evaluation team received human subjects (institutional review board [IRB]) and VA research and development committee approvals. Staff members who participated in interviews provided verbal informed consent. PREVENT was registered with clinicaltrials.gov (NCT02769338) and received human subjects (institutional review board [IRB]) and VA research and development committee approvals. In addition, the implementation evaluation received separate human subjects (IRB) and VA research and development committee approvals (IRB 1602800879). Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. org/licenses/by/4.0/.