Background

Stroke is a chronic disabling condition [1] that is expected to affect an increasing number of individuals due to population growth and aging [2]. Clinical practice guidelines for inpatient stroke rehabilitation settings provide clear treatment recommendations aimed at improving motor function, postural control, and mobility [3,4,5]. Despite evidence that guideline adherence is associated with functional recovery [6] and patient satisfaction [7], studies conducted in Canada and in the UK reveal that guideline application is inconsistent [8,9,10,11].

To narrow these knowledge to practice gaps, the Knowledge to Action (KTA) Process [12] recommends an evaluation of barriers to knowledge use and tailoring of knowledge translation (KT) strategies to address those barriers. Research conducted by our team and others has shown that implementation of a stroke rehabilitation guideline in the inpatient hospital setting presents unique challenges [13, 14]. Owing to a broad research base [15, 16], stroke rehabilitation guidelines consist of an extensive number of treatment recommendations [3]. Although multiple efficacious treatments may be appropriate for a patient, there may be insufficient time to apply all of them thus requiring individual health professionals to prioritize and select [13]. This process is complicated by the recommendation to deliver stroke rehabilitation by a core interdisciplinary team consisting of physical and occupational therapists, nurses, and speech-language pathologists, physiatrists/physicians, social workers, and dietitians [3]. Team functioning and communication may affect how well members prioritize and coordinate implementation of treatment [13]. The complexity of efficacious treatments in terms of the number of steps and technical skill required also varies widely which has implications for education and training [13]. Finally, stroke can lead to heterogeneous profiles of sensorimotor, communication, and cognitive impairment that cause variable levels of disability. The type and magnitude of stroke-related deficits, the incidence of shoulder pain which occurs in almost 30% of patients [17], and patient preferences can further influence health providers’ decisions to apply a recommended treatment [13, 18].

A multi-component intervention is required to overcome the challenges to integrating stroke rehabilitation guidelines targeting improvement in motor function, postural control, and mobility. Facilitation, defined as “the process of enabling (making easier) the implementation of evidence into practice” (p. 579) [19, 20] is a recognized strategy and core component of the PARiHS (Promoting Action on Research Implementation in Health Services) framework [20,21,22] that could potentially enable collaboration within stroke teams to implement a comprehensive stroke rehabilitation guideline. Reviews of facilitation [19, 23] have characterized it as both an individual role incorporating project management and leadership and a process involving individuals and teams. Facilitators may use a range of strategies to assist individuals and teams to apply evidence in practice as facilitation should be tailored to the needs of the local context [19, 22, 23]. For example, a local facilitator may organize training sessions to address a need to build clinicians’ capacity to implement a specific treatment and check-in at regular intervals to help maintain motivation levels. Facilitators may also approach managers to enable the purchase of equipment or the re-organization of therapy space if these are the issues hindering practice change. In the context of inpatient stroke rehabilitation, employing multiple facilitators recruited from core health professional groups involved in interdisciplinary stroke rehabilitation teams was considered a novel and potentially effective strategy for enabling guideline implementation.

In rehabilitation research, guideline provision combined with interactive educational sessions to review best practices has been associated with improved performance of recommended practices compared to mailing of the guideline among rehabilitation providers [24, 25]. Previous trials [24,25,26], however, have targeted adherence to ≤10 recommendations in one professional group and provided limited description of clustering effects and adjustment for covariates. No studies examined use of local facilitators from different professional groups to promote guideline implementation. The Stroke Canada Optimization of Rehabilitation by Evidence Implementation Trial (SCORE-IT) was a cluster randomized trial designed to evaluate the extent to which a multi-modal, facilitated KT approach to implementing a stroke rehabilitation guideline was more likely than passive strategies, such as mailing the guideline and educational materials, to improve patient function in the inpatient rehabilitation hospital setting (Bayley MT, Wood-Dauphinee S, Richards CL, Salbach NM, Desrosiers J, Eng JJ, et al.: Facilitated knowledge translation improves stroke rehabilitation outcomes: The SCORE-IT cluster randomized controlled trial, under review) [27]. Among patients with stroke treated at facilitated KT sites (n = 410), the odds of demonstrating a high level of functional walking capacity, measured using the 6-min walk test, were 1.63 times (95% CI: 1.23–2.17) the odds observed among patients at passive KT sites (n = 367) (Bayley MT, Wood-Dauphinee S, Richards CL, Salbach NM, Desrosiers J, Eng JJ, et al.: Facilitated knowledge translation improves stroke rehabilitation outcomes: The SCORE-IT cluster randomized controlled trial, under review). The facilitated KT intervention was not associated with gross manual dexterity (measured using the Box and Block Test) among patients with stroke (OR: 1.69, 95% CI: 0.72–4.01). A mixed methods process evaluation was completed to help explain the results related to patient outcomes. In the qualitative process evaluation [27], focus groups were conducted with 33 nurses, therapists, and managers from 11 of the 20 study sites in the facilitated and passive KT groups to explore their experiences with SCORE-IT. The qualitative analysis yielded four themes describing factors that facilitated or hindered implementation of the KT interventions and clinical integration of the recommended treatments [27]. Themes included: presence/absence of facilitation, agreement that the intervention was practical, familiarity with the recommended treatments, and environmental factors (e.g., staff turnover, lack of space or equipment) [27]. Facilitating factors, such as the presence of an individual who provided stroke teams with support and motivation throughout the trial, and experience with using some of the treatment interventions, were described by participants in both study groups [27]. A fifth theme, namely improved team communication and interdisciplinary collaboration, was identified as an unexpected positive trial outcome that served to facilitate the clinical application of treatment interventions in both study groups [27]. While results from the qualitative process evaluation have increased our understanding of site, provider and treatment characteristics that may have influenced implementation of the study interventions and recommended treatments, the extent to which each recommended treatment was implemented has not been reported and may help to explain why the facilitated KT intervention was associated with improved functional walking capacity but not gross manual dexterity among patients. Thus, this paper presents the results of a quantitative process evaluation of SCORE-IT designed to evaluate the extent to which stroke teams implemented the recommended treatments targeting upper extremity (UE) and lower extremity (LE) motor function, postural control, and mobility in each intervention group.

Methods

A national, 2-parallel group cluster-randomized, pragmatic trial was conducted from 2007 to 2009. The effectiveness of a facilitated and passive KT intervention for implementing a stroke rehabilitation guideline was evaluated by comparing patient outcomes related to walking capacity and manual dexterity post-intervention. To understand how implementation of guideline recommendations may have influenced study outcomes related to patient function, stroke teams in each intervention group were asked to complete self-report checklists to record their implementation of 18 recommended treatments with each patient seen over a 2-week period pre- and post-intervention. The ethics board at each site and affiliated university approved the study protocol.

Eligibility and recruitment

Sites with designated rehabilitation beds, an interdisciplinary stroke team with at least one nurse, one physical therapist (PT) and one occupational therapist (OT), and regular inpatients post-stroke (i.e., ≥1 person post-stroke on the unit daily), were considered eligible. Sites with these characteristics were targeted as the treatment recommendations were developed for implementation primarily by nurses, PTs, and OTs in an inpatient rehabilitation setting [13]. Recruitment involved study leads sending letters of invitation to site managers/physiatrists and interviewing to screen for eligibility and obtain consent. All therapists and nurses working on the stroke rehabilitation unit were eligible to participate. A research assistant (RA) hired for each site obtained informed consent from rehabilitation providers. Details of patient eligibility and recruitment are described elsewhere (Bayley MT, Wood-Dauphinee S, Richards CL, Salbach NM, Desrosiers J, Eng JJ, et al.: Facilitated knowledge translation improves stroke rehabilitation outcomes: The SCORE-IT cluster randomized controlled trial, under review).

Data collection

Following recruitment, site representatives were asked to complete a site readiness checklist that required them to provide information on the language of documentation (English/French), university affiliation (full/partial or none), rehabilitation unit location (freestanding/integrated with a general hospital), and stroke patient volume (expected number of stroke patients/year). Site RAs abstracted patient sociodemographic and clinical data from health records.

The outcome was change in the percentage of patients for which inpatient stroke teams implemented each recommended treatment pre- to post-intervention. All inpatients with stroke were expected to need the majority of treatments. For select treatments that are applied only when indicated (e.g., to reduce hand edema/shoulder pain), a similar proportion in each group was expected to require each treatment owing to randomization. A self-report checklist was created for therapists to document name, profession, and which of the 18 recommended treatments was implemented for each patient. To mitigate social desirability bias [28], a section was added to the checklist where therapists could indicate that a treatment would have been implemented if time had permitted. A similar checklist was created for nurses to report on implementation of 7 treatments (sit-to-stand training, use of LE external support, walking practice, UE range of motion and/or stretching, interventions to prevent shoulder pain, task-specific training of the UE, and education of patients/caregivers on how to handle the affected UE). Site RAs oriented therapists and nurses, who were blinded to study hypotheses, to the checklists and asked them to complete a checklist at the end of every treatment session with patients post-stroke during a two-week period pre- and post-intervention.

Randomization

A biostatistician, not involved in study recruitment or data collection, used R™ statistical software to stratify hospitals by language of documentation (English/French) and randomly assign them to either the facilitated or passive KT group using a 1:1 allocation ratio. Site staff were informed of their group assignment following completion of pre-intervention data collection on treatment implementation.

Interventions

The SCORE-IT interventions are described in detail elsewhere (Bayley MT, Wood-Dauphinee S, Richards CL, Salbach NM, Desrosiers J, Eng JJ, et al.: Facilitated knowledge translation improves stroke rehabilitation outcomes: The SCORE-IT cluster randomized controlled trial, under review) [27]. Intervention development was guided by the KTA process and by the results of a qualitative study in which implementation of the stroke rehabilitation guideline was piloted at five inpatient rehabilitation hospitals in Canada [13]. Analysis of transcripts from focus groups involving 79 rehabilitation professionals (physical and occupational therapists, nurses, and directors/managers) identified lack of time, staffing issues, training/education, therapy selection and prioritization, equipment availability, and team functioning/communication as key barriers to guideline implementation. In alignment with the KTA process, the facilitated KT intervention was designed to address these barriers. The facilitated KT intervention included funding for two local facilitators, one nurse and one therapist, to provide 4 h per week of protected time to support guideline implementation over a 16-month period. Having a facilitator from both nursing and allied health was expected to facilitate interdisciplinary collaboration and address barriers related to team functioning and communication [13]. Facilitators attended a two-day workshop where they received media releases for promoting the guideline among clinicians, slide presentations of the treatment protocols, and training in how to apply treatments and run small group education/training sessions. This was designed to prepare facilitators to run local small group education/training sessions to address barriers related to inadequate education/training in how to apply the treatments in clinical practice for existing and new staff. Facilitators were also provided with an outline of strategies used to foster guideline implementation in the pilot study [13], a practice-change toolkit [29], and education in change management. They completed activities to compare current with recommended practice, identify barriers to practice change, and develop a plan that incorporated behavior change strategies to address local challenges to guideline implementation. This was expected to prepare facilitators to address other site-specific barriers related to, for example, insufficient equipment and motivation to change practice [13]. Stroke teams were provided with SCORE guideline booklets that included treatment recommendations and evidence-based treatment protocols and pocket reminder cards designed for therapists or nurses to enable quick and easy access to descriptions of protocols. These resources were expected to address barriers related to inadequate knowledge of and time to read recommendations [13]. Teleconferences and a web-based platform were provided for facilitators to communicate and share successful strategies.

Sites in the passive group received SCORE guideline booklets that did not include treatment protocols, and a handbook [30] and educational video on the use of standardized assessment tools post-stroke. Clinicians were invited to join a list serve to ask questions and share their experiences with the trial.

Sample size

Post hoc power calculations were performed. Given 1381 observations available to analyze treatment implementation by nurses and therapists, accounting for clustering of observations within patients (mean patient-level intracluster correlation coefficient (ICC) across treatments of 0.12; mean cluster size of 8 observations per patient) yielded an effective sample size of 751 independent observations (375 per group) [31]. With 375 observations per group (2-sided alpha = 0.05) and a baseline implementation rate of 30%, there was 80% power to detect a between-group difference of 10% in the rate of treatment implementation.

Given 547 observations available to analyze treatment implementation by therapists alone, accounting for clustering of observations within patients (mean patient-level ICC across treatments of 0.09; mean cluster size of four observations per patient) yielded an effective sample size of 431 independent observations (215 per group). With 215 observations per group (2-sided alpha = 0.05) and a baseline implementation rate of 10%, there was 80% power to detect a between-group difference of 10% in the rate of treatment implementation.

Analysis

The unit of analysis was a binary variable that represented whether a patient received a recommended treatment or not during a treatment session. To account for potential clustering effects at the level of the hospital, provider, and patient, a random-effects logistic regression analysis was carried out in SAS v9.3. The analysis included the following steps. First, estimates of the unadjusted rate of treatment implementation within each group pre- and post-intervention, change pre- to post-intervention, and between-group comparison of change were obtained using proc. nlmixed. Next, tests for random variation at the site, provider and patient levels were performed. A significant test result (α = 0.05) in more than 33% of models was the criterion for including a clustering variable in all final models.

A final model was constructed for each treatment with intervention group, evaluation time (pre or post), an interaction term of group by time, clustering variables and covariates entered as independent variables using proc. glimmix. Covariates included site location (freestanding/integrated with a general hospital), and size of stroke service (expected #stroke patients/year), and patient motor function (Functional Independence Measure [32] (FIM) motor subscore) and comorbidity (Charlson score [33]) on admission. A significant interaction term (α = 0.05) was used to indicate whether change in the extent to which patients received a treatment was greater following the facilitated than following the passive KT intervention after adjusting for clustering and covariates.

Results

Figure 1 presents the CONSORT diagram describing the results of recruitment, randomization, and data collection. Of the 20 participating sites, 10 were randomized to the facilitated KT intervention and 10 were randomized to the passive KT intervention. Facilitators from all sites in the facilitated group attended the training workshop (Bayley MT, Wood-Dauphinee S, Richards CL, Salbach NM, Desrosiers J, Eng JJ, et al.: Facilitated knowledge translation improves stroke rehabilitation outcomes: The SCORE-IT cluster randomized controlled trial, under review). Three sites were removed from the analysis because they had no data (n = 2) or only pre-intervention data (n = 1) due to technical issues with the database. Of the three sites removed, two were from the passive group and were non-academic, and located in a general hospital with 86 and 35 expected patients with stroke/year. The third site removed was from the facilitated group; it was a freestanding site, partially-affiliated with a university, with an expected volume of 90 patients with stroke/year. Thus, data from nine and eight sites in the facilitated and passive group, respectively, were analyzed. The CONSORT diagram indicates the number of unique providers and patients involved in this process evaluation, and the number of checklists submitted by providers pre- and post-intervention by study group.

Fig. 1
figure 1

CONSORT flowchart

Full size image

Table 1 describes characteristics of sites that provided process data. Just over half of the sites in the facilitated and passive groups had no academic affiliation and were freestanding rehabilitation hospitals. The expected number of patients with stroke admitted per year for sites in the facilitated and passive groups was, on average, 95 and 105, respectively. Table 2 describes characteristics of the patients for whom treatment implementation checklists were completed by intervention group and sampling time point. The median age of patients ranged from 62 to 73 years (depending on group, timepoint, and received treatments), with the majority being men (52–69%), ischemic stroke (64–76%), with some arm (CMSA median 2–3) and leg impairment (CMSA median 3). Charlson comorbidity score and the proportion of patients with ischemic stroke were significantly higher in the passive group pre-intervention and post-intervention, respectively.

Table 1 Site characteristics
Full size table
Table 2 Patient characteristics on site admission by intervention group and sampling time point
Full size table

Additional file 1: Table S1 describes checklist completion by provider group (see Additional file 1). Nurses contributed the greatest percentage of checklists in the facilitated KT group (50 and 42%, pre- and post-intervention, respectively) and in the passive KT group (39 and 49%, pre- and post-intervention, respectively) for treatments that RNs, OTs, and PTs were asked to apply. Additional file 1: Table S2 provides the ICC value for the effect of clustering on treatment implementation at the site, provider and patient level for each of the 18 treatments. A significant effect of clustering on treatment implementation was observed at the site, provider and patient level in 0, 67, and 39% of models, respectively. The median ICC across treatments for sites, providers and patients was 0.06, 0.21, and 0.08, respectively.

Additional file 1: Table S3 presents mean cluster sizes in terms of the number of providers per site, number of patients per provider, and number of checklist forms per patient pre- and post-intervention by study group (see Additional file 1). Cluster sizes at the site level indicated that the average number of providers contributing data to the analysis across groups and timepoints ranged from 10 to 15 for 7 treatments implemented by RNs, OTs, and PTs and from 5 to 7 for 11 treatments implemented by OTs and PTs. The average number of patients per provider contributing data to the analysis across groups, timepoints, and treatments ranged from 2 to 3. The average number of checklists completed per patient in the analysis across groups and timepoints ranged from 6 to 13 for 7 treatments implemented by RNs, OTs, and PTs and from 3 to 4 for 11 treatments implemented by OTs and PTs.

Outcomes

Table 3 presents unadjusted estimates of the percentage of patients receiving each treatment pre- and post-intervention, the change in the percentage, and the between-group comparison. Seven of the 18 treatments, including training of sit-to-stand, sitting balance, and standing balance, task-specific training (i.e., stairs), walking practice, interventions to prevent shoulder pain, and task-specific training (i.e., self-care tasks), were being implemented at least 15% of the time in both groups at baseline.

Table 3 Unadjusted intervention effect on change in implementation of 18 treatments
Full size table

After adjusting for clustering at patient and provider levels and covariates, the facilitated KT intervention was associated with a significant improvement in the implementation of sit-to-stand training (p = 0.028) and walking practice (p = 0.043), and the passive KT intervention was associated with improved implementation of standing balance training (p = 0.037). Adjustment for the stratification variable did not change the interpretation of the results. Further analysis of which provider groups changed their practice (see Additional file 1: Table S4) revealed that the unadjusted percentage of patients receiving sit-to-stand training was higher post- compared to pre-intervention for nurses (30 vs 10%), and PTs (67 vs 49%) in the facilitated group, and for PTs (58 vs 39%) in the passive group. The unadjusted percentage of patients receiving walking practice was higher post- compared to pre-intervention for nurses (14 vs 6%), OTs (37 vs 21%) and PTs (80 vs 70%) in the facilitated group, and for nurses (14 vs 11%), OTs (25 vs 14%) and PTs (76 vs 68%) in the passive group. The unadjusted percentage of patients receiving standing balance training was higher post- compared to pre-intervention for OTs (36 vs 34%) in the facilitated group, and OTs (38 vs 27%), and PTs (82 vs 45%) in the passive group.

Discussion

This is among the first process evaluations of a guideline implementation trial involving the use of interprofessional local facilitators in rehabilitation. Findings indicate that a facilitated KT intervention, with local nurse and therapist facilitators, tailoring of strategies to address local barriers, and a guideline with treatment protocols, was of limited effectiveness compared to passive guideline dissemination in improving short-term uptake of a comprehensive guideline by inpatient stroke rehabilitation teams. The process evaluation revealed that the facilitated KT intervention was associated with improved application of only two treatments (sit-to-stand training, walking practice), whereas the passive KT intervention was associated with improved application of one treatment (standing balance training). Results from this process evaluation suggest that superior functional walking capacity observed among patients post-stroke following the facilitated compared to the passive KT intervention (Bayley MT, Wood-Dauphinee S, Richards CL, Salbach NM, Desrosiers J, Eng JJ, et al.: Facilitated knowledge translation improves stroke rehabilitation outcomes: The SCORE-IT cluster randomized controlled trial, under review) resulted from an improved application of sit-to-stand and walking training by stroke teams. Process evaluation findings also indicate that the facilitated KT intervention was not associated with improved gross manual dexterity among patients because this intervention was not associated with improved uptake of treatments targeting UE function.

Results from the current evaluation combined with findings from the qualitative process evaluation of SCORE-IT [27] may help to explain why sit-to-stand and walking practice were more likely than other treatments to be adopted in the facilitated KT group. Facilitation, specifically support and motivation that individuals provided to staff at sites in each group, was perceived to promote the implementation of the recommended treatments [27]. It is possible that facilitation of sit-to-stand and walking training was provided more consistently across sites in the facilitated KT group than in the passive KT group. Results of the qualitative analysis also showed that both familiarity and agreement with recommended treatments because they are “clear and practical to follow” [34, 35] likely helped to promote their uptake [27]. Sit-to-stand and walking training were implemented in at least 15% of patients in each group at baseline (unadjusted estimates) which suggests that some providers had the expertise to perform these treatments and considered them relevant. Sit-to-stand and walking training are also simple, task-oriented mobility treatments that are relevant to daily living. Complex treatments that either involve multiple steps (UE constraint-induced therapy) or technology (functional electrical stimulation, treadmill training) were rarely implemented at baseline and demonstrated either no change or reduced application post-intervention despite being supported by Level A evidence (i.e., evidence from at least one randomized controlled trial, meta-analysis, or systematic review). Based on the SCORE-IT qualitative findings, this was likely because health professionals found that these treatments were time-consuming, and required special training or equipment that was not consistently available across sites [27]. Finally, asking nurses to apply sit-to-stand and walking training, in addition to therapists, appeared an effective facilitated KT strategy as the percentage of patients receiving sit-to-stand and walking practice by nurses increased by 20% (vs a decrease of 11% in the control group) and 8% (vs 3% in the control group), respectively. Improved team communication and interprofessional collaboration were noted as an unintended outcome of SCORE-IT [27]. The improved practice of nurses, likely supported by the nurse facilitator in the facilitated KT group, was particularly influential in the current study as nurses provided a large proportion of the treatment data in the multivariable analysis.

Standing balance training, which increased in the passive KT group, is also a simple task-oriented treatment. Because providers receiving facilitated KT were implementing standing balance training at a high rate at baseline (68% for PTs), they may have prioritized improving adoption of other treatments [13]. Results from the qualitative sub-study [27] indicate that a greater degree of facilitation of and/or agreement with the practicality of standing balance training in the passive compared to the facilitated group, may help to explain why this practice improved in the context of passive dissemination of the stroke rehabilitation guideline.

Despite multiple strategies and resources, the facilitated KT intervention was unsuccessful in improving integration of 18 treatments concurrently. The facilitated approach may not have adequately addressed barriers to integrating numerous treatments simultaneously and complex treatments that were unfamiliar to providers. Targeting fewer treatments and providing individual hands-on training and access to an expert may be a more effective approach based on results from previous guideline implementation trials for low back pain [24] and whiplash [26] rehabilitation.

Limitations

The primary limitation of this process evaluation was the use of self-report measures of practice that cannot capture clinical judgment or patient preferences and are vulnerable to over-reporting. This limitation, however, would have affected both groups similarly. It could not be determined if treatments received were appropriate due to the unavailability of clinical data at the time implementation was evaluated. Results provide average rates of implementation after controlling for patient and hospital characteristics to optimize comparability between groups.

Conclusions

A facilitated KT intervention incorporating a guideline with treatment protocols and trained local nurse and therapist facilitators was of limited effectiveness compared to passive guideline dissemination in improving short-term uptake of a comprehensive guideline by inpatient stroke rehabilitation teams. Conducting this process evaluation as part of the trial was valuable as it revealed the nature of the practice change, in terms of the type of health providers involved and the type of and extent to which treatments were implemented, underpinning patient function outcomes observed in the main analysis. The combination of quantitative and qualitative process evaluation findings provided a basis for hypothesis generation related to designing KT interventions to overcome challenges to integrating treatments recommended in stroke rehabilitation guidelines in the context of interdisciplinary team care. Specifically, KT strategies that better address the need for staff training and team functioning to account for treatment complexity and prioritization post-stroke may be needed. Finally, the study design and analytical approach described in the current study, which involved consideration of multi-level clustering effects, and adjustment for site- and patient-level covariates, is innovative and will help to advance the field of implementation science in the context of rehabilitation guideline implementation.