Background

Stroke is one of the leading causes of death and disability worldwide [1]. Despite declines in age-standardised stroke incidence and mortality rates in recent years, the global burden of stroke remains high with over 80 million stroke survivors worldwide [1, 2]. To successfully reduce the negative impacts of stroke, high-quality health and care practices are needed across the entire stroke care pathway. Reorganising stroke services and implementing changes at a system-level are increasingly being recognised as ways of enhancing coordination across the pathway, optimising care processes, and improving outcomes for stroke patients [3,4,5]. Implementing these transformative changes in stroke care is likely to involve a critical mass of stakeholders across different organisations and will require the application of effective quality improvement (QI) methodologies.

Whilst there are many examples of good stroke care practices, these are not always shared between organisations. Quality improvement collaboratives (QICs) offer a unique opportunity for key stakeholders from different organisations to take part in a series of collaborative activities [6]. The QIC approach, first formalised by the Institute for Healthcare Improvement (IH), is a short-term structured programme, usually between 6 and 15 months, designed to support ‘breakthrough’ improvement in a focused topic area [7]. Teams from different organisations are brought together in ‘learning sessions’ to share and learn best practices and QI methods, and ‘take home’ learning to their organisation to test changes locally in ‘action periods’ [7]. Previous systematic reviews have evaluated the impact of QICs, reporting largely positive effects on improvement measures [6, 8]. Attempts to shed light on the potential determinants of QIC success have proposed the influence of external support [9], leadership [9], team functioning [9, 10], and collaborative learning [10, 11]. However, this literature has emphasised the need for further exploration of whether QIC effectiveness is dependent on the focus (e.g. clinical population), and if there are specific contextual factors that support or hinder QIC success [6, 8,9,10]. The importance of involving patients and carers in decisions about improving the care they receive [12], and the consideration of health inequalities when improving health and care services [13], is widely recognised, but to date, no review of QICs has examined the extent to which patients and carers were involved, or health inequalities were considered.

To build on previous QIC reviews, this systematic review assessed the effectiveness of QICs for driving improvements in stroke care and used the Consolidated Framework for Implementation Research (CFIR) [14] to explore the facilitators and barriers to using a QIC to improve care for this clinical population. The review also sought to consider the extent to which QICs in stroke care involved patients and carers and considered health inequalities.

Methods

Searches

This systematic review was registered with PROSPERO (CRD42020193966) and designed in accordance with recognised guidance and reporting standards (see Additional file 1 for the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) checklist [15]). Studies were identified through searching five electronic databases (MEDLINE, CINAHL, EMBASE, PsycINFO, and Cochrane Library) from their inception to 5th June 2020 and were limited to studies published in English. A search strategy using a combination of Medical Subject Headings and keywords related to ‘stroke’ and ‘quality improvement collaborative’ was developed with the assistance of an information specialist (see Additional file 2). Additional studies were identified through screening reference lists of included studies and relevant reviews.

Study selection

Studies of any design referring to a QIC conducted in an adult stroke care setting, which reported primary effect measures and/or perspectives of participating multidisciplinary stroke teams, were included. The QIC approach was defined in line with previous reviews [6, 8, 9], consisting of the following core elements: (1) a specified topic; (2) clinical and QI experts working together; (3) multiple teams from multiple sites participating; (4) a model or framework for improvement with multiple tests of change; and (5) a series of structured collaborative activities in a given timeframe, involving learning sessions and visits from mentors and facilitators. Conference proceedings and reviews were excluded from the review. Two reviewers independently screened the titles and abstracts of all retrieved citations against the eligibility criteria using Rayyan [16]. Full texts of potentially relevant citations were then obtained and independently assessed by two reviewers. Disagreements at any stage were resolved through discussion with a third reviewer, and where necessary the wider review team. Reasons for exclusion at full-text screening were documented.

Data extraction and quality assessment

Data was extracted from the included studies by one reviewer using a pre-piloted form in Microsoft Excel, and checked by a second for completeness and accuracy. Any disagreements were resolved through discussion with a third reviewer. The following data items were extracted from each study: authors, year of publication, country, aim, study design and setting, improvement area, QIC description and components, and any relevant outcomes. The extent to which patients and carers were involved, and health inequalities considered, was also noted. Data relating to the factors influencing stroke care improvement when using a QIC was extracted, in addition to those specifically labelled as facilitators and barriers. The Mixed Methods Appraisal Tool (MMAT), a critical appraisal tool designed for reviews which include quantitative, qualitative and mixed methods studies [17], was used to assess the methodological quality of included studies.

Data synthesis

Detailed summaries of the study characteristics were collated. A vote-counting method based on the direction of effect was used to identify if there was any evidence of an effect in the included studies [18]. This approach was used due to heterogeneity observed in the studies, particularly in the outcomes assessed, and has been previously used in a similar review assessing the effectiveness of QI interventions [19]. For each outcome type (process, patient, and other), studies were categorised into five groups based on the ratio of outcomes demonstrating positive directional change, either from baseline to end of the study or when an intervention group was compared to a control group: (1) all outcomes; (2) more than half of the outcomes; (3) half of the outcomes; (4) less than half of the outcomes; and (5) no outcomes.

Extracted facilitators and barriers were mapped to the Consolidated Framework for Implementation Research (CFIR) [14] by one reviewer and verified by a second. The CFIR is comprised of five key domains (intervention characteristics, outer setting, inner setting, characteristics of individuals, and the implementation process), each containing constructs enabling the exploration of factors that influence implementation success [14]. This framework was selected as it focuses on organisational and contextual factors related to implementation, which was identified as most suitable for the collaborative nature of a QIC. It also served as a structure to explore factors across different study types. Thematic analysis was used to categorise facilitators and barriers for each relevant construct of the CFIR [20]. This stage was divided equally between two reviewers, with uncertainties resolved through discussion.

Patient and public involvement in the review

A member of the public worked with researchers to develop the data extraction form, ensuring that the extent of patient and carer involvement, and whether improvements were patient-centred, were considered when extracting data, and reviewed this paper.

Results

The search strategy retrieved a total of 1179 citations. After the removal of duplicates, 815 citations were screened based on title and abstract, of which 68 records underwent full-text assessment. A total of 20 papers were identified for inclusion in the review, including two additional papers found through citation checking (Fig. 1).

Fig. 1
figure 1

PRISMA flow diagram

Full size image

Study characteristics

Twenty papers describing 12 QICs used in stroke care were included; four randomised controlled trials [21,22,23,24,25,26], four cross-sectional studies [27,28,29,30], three interrupted time series studies [31,32,33,34], four before-and-after studies [35,36,37,38], and two qualitative studies [39, 40]. A summary of the included QICs is presented in Table 1. QICs were conducted in the USA [23, 29, 33,34,35, 38], UK [21, 31], Netherlands [22, 37], Australia [24] and Taiwan [36] between 2005 and 2020. Most QICs [21,22,23,24, 29, 31, 33,34,35,36,37] focused on improving urgent and/or acute stroke care. Key improvement areas included increasing thrombolysis treatment rates [22, 24, 29, 34, 36, 37], accurate and timely stroke screening and documentation [21, 23, 31, 33, 35,36,37], and increasing compliance in the full delivery of care bundles [21, 31]. Nine QICs took place in secondary care settings (e.g. hospitals) [21,22,23,24, 28, 29, 34,35,36], two QICs were based in pre-hospital care (e.g. emergency services) [31, 33], and one QIC was based in a primary care setting (e.g. general practice) [38]. One QIC took place across more than one setting type [28, 37], with stroke services from hospitals, rehabilitation organisations and nursing homes participating. The number of organisations participating in the QICs varied; some had between 10 to 15 sites [22, 23, 31, 34, 35], whilst others had between 20 and 24 sites [21, 24, 28, 33, 36]. Professionals involved in the QICs included QI experts, doctors, managers, nurses, and allied health professionals; some of whom were identified as specialist stroke clinicians and practitioners. There was variability in some QIC components; the number of learning sessions (from two to five), local QI methods used (plan-do-study-act cycles, driver diagrams, process maps), length of the QIC (from 6 to 48 months), and additional activities (teleconferences, workshops, site-based meetings). Most QICs used electronic/web-based data systems to measure performance [21,22,23,24, 33, 35, 37], and four QICs specified the use of a national registry [21, 35, 37, 38].

Table 1 Summary of included QICs
Full size table

Quality assessment

The MMAT revealed that most papers were of medium to high quality [21,22,23,24,25,26,27, 29,30,31,32, 34, 36,37,38,39,40]. Two papers which scored as low quality [28, 35] either confirmed or added to the findings and so were included. Reliability of findings on quality assessment decisions is referred to in Tables 2 and 3.

Table 2 Facilitators identified in the QICs mapped to the CFIR domains and constructs
Full size table
Table 3 Barriers identified in the QICs mapped to the CFIR domains and constructs
Full size table

Effectiveness of QICs in stroke care

Across the included studies, the effectiveness of QICs was categorised into three types of outcomes: process, patient, and other. Of the 14 studies (from ten QICs) with quantitative data, all reported process outcomes (e.g. door-to-needle times, blood glucose testing, discharge prescriptions) [21,22,23,24,25,26, 28, 31,32,33,34,35,36,37], seven studies (from six QICs) reported patient outcomes (e.g. mortality, quality of life, discharge delay) [22, 24, 28, 34, 36,37,38], and seven studies (from six QICs) reported other outcomes (e.g. staff engagement levels, perceptions of interventions, use of QI methods) [24, 25, 27, 29, 30, 39, 40]. All 14 studies reported a positive directional change in 50% to 100% of their process outcomes [21,22,23,24,25,26, 28, 31,32,33,34,35,36,37]; indicating that QICs were associated with improving clinical processes in stroke care. Of the seven studies reporting patient outcomes, three reported a positive directional change in 100% of these outcomes [28, 34, 37], two reported a positive directional change in less than half of their patient outcomes [22, 38], and two reported no change [24, 36]; suggesting that QICs may not be as effective in improving stroke patient outcomes. Of the seven studies reporting other outcomes, five reported no change [24, 27, 29, 39, 40], and two reported a positive change in these outcomes [25, 30]. Subgroup analyses, conducted by publication year, country, study setting, number of improvement areas, duration of QIC, number and length of learning sessions, and quality assessment judgement, identified no clear associations (see Additional file 3).

Facilitators and barriers

Facilitators and barriers to implementing improvements in stroke care when using a QIC are summarised and mapped to the relevant CFIR domains and constructs in Tables 2 and 3, respectively. The following descriptions of the key facilitators and barriers identified are presented in the five CFIR domains.

Intervention characteristics

Six QICs reported factors related to the complexity and adaptability of the QIC intervention. Complex QI processes, or those requiring system re-design and multi-professional coordination, were more challenging, difficult to implement and unlikely to support change in the short-term [28, 34, 35]. Conversely, where indicators for change were kept simple and the stroke team had more control over them, improvement was more likely to be achieved [23, 39]. Identifying a specific geographical unit or designated team with recognised responsibility was viewed as important and may have encouraged a greater response to the QIC [23, 39]. Demonstrating the success of QI processes on delivery of care also highlighted their adaptability; for example, staff reported ‘spill over’ effects for other clinical conditions [31], and staff suggested that the QIC model could be applied to other aspects of stroke care like endovascular therapy [34].

Outer setting

Features of the external environment were identified as influencing improvement across all but one QIC [22]. External factors, such as the presence of national-level policies and incentives during the QIC [23, 26, 29, 38], or delays in securing contractual arrangements [35], influenced the extent to which organisations improved stroke care. Having little to no experience of previous QI initiatives, such as lack of familiarity with national data registries, meant improvement was less likely to happen for some organisations [34, 36]. The reported complexities associated with treating stroke, including challenging clinical presentations [36], being cared for in different areas of the hospital [35], and capturing accurate data on stroke onset [26], were barriers to achieving QI for all patients and all elements of stroke care.

Inter-organisational collaborative action, particularly during learning sessions, facilitated the exchange of ideas, best practices and experiences between organisations that would not normally work together [28, 33, 36, 39]. These exchanges stimulated teams to ‘take home’ learning to their organisation [28]. Relationships between organisations were fostered through the networking and communication opportunities offered by the QIC [28, 29, 33, 39]. It was reported that collaboration led to cooperation between teams, emphasis on the need for QI, and awareness of ‘being part of a chain of care’ [28]; and created ‘a sense of belonging’ and a ‘shared repertoire’ [39]. Though inter-organisational collaborative action was reported to facilitate improvement across some QICs [28, 29, 34, 36], the ‘Stroke 90:10’ QIC found that variability in performance, attendance, enthusiasm and contribution of teams created tension between organisations, which was not conducive to successful collaborative QI [39].

Inner setting

Factors in this domain were the most highly cited across all QICs. Insufficient organisational support (e.g. lack of prioritisation and inadequate allocation of time and resources for stroke QI) was reported as a significant barrier [24, 27, 28, 31, 33, 35, 37, 39, 40]. Structural changes (e.g. staff turnover) were also reported to negatively impact implementation [22, 24, 28, 29, 31, 40], and in one case led to an organisation withdrawing from the QIC [22]. QI was challenging for organisations that had limited access to equipment or patient data to measure performance [28, 35, 40]. Access to useful information delivered during QIC activities, however, empowered teams to develop knowledge of best practice, patient care and QI methods, which in turn facilitated stroke service improvement across some QICs [25, 28, 31, 33, 35, 40].

Leadership was noted to be associated with achieving improvement across some QICs [27,28,29, 31, 33, 35, 39]. Difficulties in obtaining support from leaders or changes in leadership hindered team participation in QI [28, 33, 39]. Some QICs highlighted how additional meetings and regular communication with leaders were successful tools to overcome these barriers and obtain buy-in from leaders to implement stroke care improvements [27,28,29, 31, 35]. Regular communication of QI activities and progress fostered support and recognition, provided intra-organisational networking opportunities and enabled change [28, 29, 33, 35, 39, 40]. Providing feedback to staff also supported improvement [23, 26, 31, 33, 35, 39]. Positive feedback mechanisms included audit and feedback [39], annotated control charts [31], provider prompts [31], and storyboards [35]. Learning sessions and access to experts motivated change by providing opportunities to share and learn best practices and become familiar with QI tools [33, 35, 36, 39]. Engagement with QI processes was influenced by capacity and willingness to learn [29, 30, 39] and tailoring the content and accessibility of learning sessions to suit participants [28,29,30, 40].

Characteristics of individuals

Individual characteristics were reported to influence improvement across six QICs. The perception of and response to QI processes differed depending on profession. Perceptions towards the effectiveness of thrombolysis were thought to have affected implementation for one QIC [24, 30], whilst another struggled to obtain support for QI measures due to a perception amongst emergency department staff that there were no quality issues surrounding stroke care [35]. Engaging staff from the outset may encourage more positive responses from colleagues towards the implementation of QI processes [27, 31]. Staff who perceived changes as a means of improving patient care, or creating a greater sense of purpose, were more likely to adopt them and look out to other organisations as well as their own [31, 39]. Other individual characteristics identified as influencing improvement included length of service [27], motivation [28, 31, 40], problem-solving [40], and enthusiasm [28].

Process

Ten QICs cited facilitators or barriers to QI associated with engaging appropriate individuals and executing the QIC intervention. Achieving improvement was difficult where there was low to moderate engagement in QI processes [24, 31], and where it was perceived that there was insufficient engagement from clinicians [27] or emergency department staff [35]. Engaging with all staff, particularly leaders, involved in delivering stroke care from the inception of the QIC and throughout was thought to facilitate change [27, 28, 31, 35, 39, 40]. Whilst external facilitators were found to empower teams to take ownership of changes in one QIC [40], another reported that sole reliance on local champions to support the change process was not necessarily sufficient and that more collaborative working was needed [24].

Inconsistencies in delivering the QIC intervention, for example implementation delays [31, 35], longer periods between learning sessions [22], and only having two learning sessions [30], negatively impacted motivation and improvement. Conversely, consistency in applying the QIC model with adequate team participation throughout and the use of a structured approach featuring measurable outcomes, supported improvement [25, 28, 29, 35]. Some QICs highlighted that whilst this intensive intervention facilitated initial improvement, when QIC support and resources were withdrawn, continued improvement might not be sustainable [23, 24, 34, 35]. QICs with longer-term data collection found no continued improvement in door-to-needle times [34], and declining thrombolysis rates [24], when the QIC ended.

Patient and carer involvement and health inequalities

Patient and carer involvement rarely featured in the QICs. None undertook qualitative data collection of patient or carer perspectives of QI, or explored whether their experience had changed as a result of the QIC. An English ambulance service QIC concluded that as patients were the care receivers, their experiences should inform QI [27]. All but one QIC [38] were focused on improving clinical quality rather than patient-centred improvement areas, and only half of the QICs measured patient outcomes [22, 24, 34, 36,37,38]. Whilst unwarranted variation between stroke services was a motivation for improvement in two QICs [21, 28], the context of socioeconomic health inequalities associated with stroke was not present in most QICs. One USA QIC factored health insurance and poverty level into their analysis to assess whether QI activities decreased hospitalisations for stroke in all populations [38].

Discussion

This systematic review assessed the effectiveness of QICs in improving stroke care and explored the facilitators and barriers associated with using the QIC approach. It was considered important given the possible benefits from using a QIC in reorganising stroke services and implementing system-level changes in stroke care. In line with previous QIC reviews [6, 8], the present review found that QICs support positive change for some outcome measures, particularly those related to improving clinical processes. Echoing concerns from these reviews [6, 8], evidence of effectiveness was limited due to the low methodological quality of some studies and the heterogeneity of study design, meaning that meta-analysis was not possible. Whilst QICs were associated with improving clinical processes in stroke care and to some extent patient outcomes, effects on staff engagement, perceptions, and uptake of QI methods were limited. The short-term and intensive nature of a QIC may have restricted the extent to which some measures could be affected. Patient-based outcomes or those related to individual behaviour or organisational change may require longer-term monitoring and embedding of QI processes. Few QICs assessed whether improvements continued or were sustained when the QIC ended. In those that had longer-term follow-up, outcomes had remained the same [34], or worsened [24]; suggesting that when QIC support was withdrawn, continued or even sustained, improvement may not be possible. It has been noted that encouraging a project-like approach to QI can be harmful for continuous improvement [41], supporting the idea that when a QIC ends, the gains achieved during the programme may attenuate as teams re-focus efforts on other aspects of care delivery.

Many factors identified by this review as supportive to QI were consistent with findings from other QIC reviews [9, 10], indicating they are not unique to this clinical population. Use of the CFIR domains to map facilitators and barriers has highlighted the importance of the inner and outer setting when using a QIC to improve stroke care. This substantiates results from the wider QI literature [42, 43], indicating that contextual factors within the organisation and external environment influence the extent to which improvement can be achieved. The positive effect of collaborative interaction (e.g. inter- and intra-organisational networking opportunities) identified, is also evident in previous explorations of QICs [10, 11], including in a recent realist review proposing collaborative ‘capacity building’ as a mechanism for change [9]. The present review’s findings, particularly those related to the influence of networking and access to information, corroborate several conclusions reached by Zamboni and colleagues [9]. Importantly, identifying engagement as a key facilitator further supports the present view that engagement plays a vital role in harnessing QI within an organisation [9, 43]. Despite this emphasis, greater efforts to understand how to increase engagement, who to engage with, and at what stages in the process, could better inform how to optimise a QIC in stroke care.

Given the prominence of factors within the inner setting, QIC success may rely on an organisation’s capacity to participate. This may form the basis of key criteria to be met before subscribing to the approach. Addressing barriers associated with a lack of organisational support, consistently identified across the wider QI literature [41,42,43], is likely to support stroke care QI. Alternative QIC formats such as virtual collaborative events may alleviate some barriers associated with QIC participation (e.g. time commitment) [44]. Intervention and individual characteristics specific to stroke were identified as barriers to implementing improvements using a QIC. In addition to patient-level barriers, such as challenging clinical presentations and the accuracy of stroke data, complex changes in stroke which involved different hospital areas and teams were more difficult to achieve with a QIC. The focus for future QICs may therefore be limited to implementing smaller process changes in stroke care and only with certain cohorts of stroke patients. The perceptions of and response to QI, and in some cases the intervention itself (e.g. thrombolysis), differed depending on profession across some QICs. Given that QICs were less likely to be associated with increasing engagement, changing perceptions, or increasing the uptake of QI methods; exploring ways in which to address these aspects of QI in stroke care deserves attention in future studies.

Patients and carers were not involved in the QICs, and the context of health inequalities was rarely considered. Despite the importance of involving care receivers in improving health services [3, 12], evidence of how and in what circumstances to involve them in QI, remains limited. The lack of consideration of health inequalities in the QICs was unsurprising, as those conducted in secondary care settings tend to focus on administering treatments for presenting health conditions rather than on addressing the underlying determinants of health and equitable access to services.

The findings from this review could be used to inform practice and the direction of future research. First, factors found to influence improvement, such as engagement and organisational support, should be considered by those planning future QIC initiatives in stroke care to enhance chances of success. Developing a tool to assess the presence or absence of the factors found in this review could be useful to support a healthcare organisation in the effective implementation of a QIC to improve stroke care. Second, the lack of stroke patient and carer involvement identified in this review suggests that there is a need for future studies to explore the ways in which patients and carers could be involved in a QIC. Utilising qualitative methodology similar to other participatory projects in QI [45, 46], to characterise how patient and carer experience and knowledge can contribute to a QIC may help to evaluate if their involvement could support a more patient-centred approach to implementing improvements in stroke services. As the focus of many QICs was implementing smaller process changes in discrete parts of the stroke care pathway, future research should be conducted to identify how system-level change can be achieved and whether a QIC would support this. Such studies could adopt the conceptual framework for implementing major system change developed by Fulop and colleagues [47], employing a QIC as the implementation approach and evaluate its potential to influence outcomes associated with system-level change. Lastly, there is a need for further exploration of the sustainability of improvements once QIC support is withdrawn, and how to support continued improvement and ongoing inter-organisational networking. Applying theories as identified in a recent systematic review [48], could identify potential avenues for sustainment strategies and advance understanding of how to sustain improvement and networking when a QIC ends.

This systematic review was conducted using standardised methods, a well-established implementation framework to consider facilitators and barriers, and included public involvement. In addition to searching five academic databases, scoping searches of the grey literature were conducted, and no additional records were identified. Though the searches were comprehensive, it is possible that some relevant papers may have been missed by not systematically reviewing those not published in English. QICs included in this review did not report negative changes across outcome measures, indicating a potential publication bias as QICs with negative findings are less likely to be published than those with positive results. In addition, the majority of studies reported process outcomes and very few reported patient outcomes, and therefore whilst QICs appear to be associated with improving clinical processes in stroke, it should not be assumed that these are directly associated with patient improvements and could highlight a potential shortfall of research in this area [49].

Conclusion

QICs are associated with improving clinical processes in stroke care; however, their short-term nature means uncertainty remains as to whether they benefit patient outcomes. Although helpful with improving elements of the stroke care pathway, evidence around using QICs to achieve system-level change is equivocal. Further research is needed to explore the sustainability of improvements when QIC support is withdrawn. QIC implementation can be compromised by both individual and organisational level barriers. It is evident that engagement, communication, and access to best practice examples could be key to enhancing QIC success in improving stroke care. As a result, future efforts to drive stroke care improvement using a QIC should be informed by these facilitators and barriers.