INTRODUCTION

Over two decades ago, the National Academy of Medicine (then Institute of Medicine) published a foundational report focused on “crossing the quality chasm” in healthcare.1 Since then, a robust body of literature and multiple national initiatives have focused on improving the quality of care in routine practice.2,3,4,5 Two relatively isolated fields—implementation science (IS) and quality improvement (QI) science—have developed theories and methods to address, optimize, and evaluate change in clinical practice (see definitions in Figure 1). IS offers strategies to promote uptake of evidence-based practices in order to improve the quality of care as well as to produce generalizable knowledge.6,7,8 QI—or more recently improvement science—originated from industry and aims to increase the quality, value, and safety of healthcare by changing specific processes within a specific healthcare system/setting.7,8,9,10

Figure 1
figure 1

Definitions for implementation science (IS) and quality improvement (QI).

Full size image

A number of recent articles focus on distinctions between IS and QI.7, 8, 11 However, a growing body of research articulates alignment and potential synergies for these fields. Within cancer care delivery research, Koczwara and colleagues suggest that although IS and QI emerged from different scientific orientations and have fought for intellectual differentiation, in reality they present complementary and synergistic approaches to support practice change.10 Within the nursing literature, Granger (2018) identifies growing semantic entanglement between these fields and focuses on articulating their distinctions,11 while simultaneously citing prior work by her team that combines these approaches within the same studies.12, 13 Check and colleagues reviewed 20 highly cited cancer-related IS and QI studies published in the last 5 years and concluded that “studies use different terminology and emphasize different methodological aspects in reporting but share similarities in purpose, scope, and methods, and are at similar levels of scientific development” and were “well-positioned for alignment.”14

Practice-based research networks (PBRNs) provide an opportunity to empirically explore the application of IS, QI, and IS/QI in actual practice. Practice-based research networks (PBRNs) originated in the 1970s as groups of primary care clinicians and academic researchers affiliated to investigate questions of importance to their clinics and patients.15, 16 In 1994, there were 28 active PBRNs in North America;17 as of August 2020, 185 PBRNs were registered with the Agency for Healthcare Research and Quality (AHRQ).18 PBRN structure supports ongoing commitment to network members that transcends a single research project, is focused on linking community-based clinicians with academic investigators, and helps build the capacity of members over time.19 Much of the PBRN literature has highlighted the integration of research and QI as standard practice.2, 19,20,21,22 However, no studies that we are aware of have explored how PBRNs have conducted IS and QI over multiple years of network existence or the contribution of these approaches within the same projects.

Therefore, the objective of this study was to systematically examine IS and QI projects over a 12-year period within one PBRN, the Oregon Rural Practice-based Research Network (ORPRN).23 Our goal was to address three research questions: (1) How are IS, QI, and IS/QI projects similar or different?; (2) How have IS, QI, and IS/QI projects changed over time?; and (3) In blended IS/QI projects, what synergies exist between the two approaches? We hypothesized that the majority of studies included in the review would include elements of IS and QI and would use IS to inform intervention selection and evaluation and QI to support local practice change. We anticipate this analysis will advance key gaps in IS and QI literature (namely the interface between these fields), describe how PBRNs draw from both approaches, and advance development of implementation laboratories.

METHODS

We conducted a longitudinal, comparative case study to explore IS and QI projects within one PBRN. Data abstraction and analysis modeled best practices for systematic reviews.24 The Oregon Health & Science University Institutional Review Board classified this study as non-human subjects research (IRB #20370).

Setting

ORPRN was founded in 2002 with a focus on rural clinics, but expanded in 2010 to engage clinics throughout the state.25 As of October 2020, ORPRN employed 41 staff, including 14 regionally based practice facilitators26, 27 charged to maintain community relationships while supporting network projects. Between 2014 and 2019, ORPRN engaged more than 360 primary care clinics in more than 84 research, technical assistance, and improvement projects (see network map in Appendix 1). Clinics displayed diversity in ownership, clinic size, and electronic health records. Roughly 56% of these clinics were system-owned, 10% were classified as Federally Qualified Health Centers (FQHCs),28 and nearly 50% were located in rural/frontier areas.29, 30

Over time, ORPRN’s projects have evolved with the changing healthcare landscape, including an increase in hospital/health system–owned clinics, use of electronic health records, quality metric performance and value-based care, and integration of clinical team members (e.g., community health workers).31,32,33 Like many PBRNs, ORPRN adapted by engaging new health system partners (e.g., payers, hospital administrators) and embracing the principles of community-based participatory research (CBPR).19, 34, 35 ORPRN also affiliated into a meta-network of PBRNs to enable the conduct pragmatic clinical trials, comparative effectiveness research, and improve access for big data analyses.36,37,38,39

Project Identification and Inclusion

One co-author (CD) identified all projects and studies (henceforth referred to as projects for simplicity) conducted in ORPRN from network inception. This list was finalized by the lead author (MMD) based on a secondary review of ORPRN’s digital project folders and faculty CVs. We excluded projects initiated prior to 2007 because we lacked access to complete records; included projects thus spanned a 12-year period from January 2007 through January 2019.

To classify project type, we used definitions informed by Bhattacharyya and colleagues for IS6 and Batalden & Davidoff for QI9, and accounted for recent articles comparing and contrasting these methods7, 8, 10 (see Figure 1). Two authors (MMD, LM) reviewed the titles and descriptions of eligible projects and made a preliminary classification of each as IS, QI, IS/QI, or other based on the original applications. From the initial list of included projects, three authors (CD, RG, CC) independently reviewed 12–14 projects each, abstracting data and making a secondary classification. We used the following pragmatic definitions: a project was classified as IS if it evaluated efforts to promote the adoption of an evidence-based intervention into practice and produced generalizable knowledge and QI if it evaluated efforts to improve the quality, value, or safety of care in specific settings. Projects including both elements were classified as blended IS/QI. One author (MMD) compared preliminary and secondary classifications. We reconciled discrepancies via group discussion; if consensus was not achieved, the senior author (NE) adjudicated the final decision. We excluded projects classified as other—which included infrastructure development awards, demonstration projects, formative research, and research to generate new evidence—to facilitate direct comparison between IS, QI, and IS/QI projects as specified by our research questions.

Data Collection and Analysis

We developed an abstraction template informed by the Standards for Reporting Implementation Studies (StaRI),40 Statement and Standards for Quality Improvement Reporting (SQUIRE)41, and prior work exploring the interface between IS and QI (see Appendix 2).10, 42 Data abstracted included details on the funding source, start/end years, objective, topic, design, setting, evidence-based practice, theoretical framework, and practice change strategies. In relation to theoretical framework, we included any mention of theories, models, or frameworks used to inform the implementation process, explain outcomes, or to support evaluation.43 To classify practice change strategies, our team developed an a priori list of common implementation strategies used by ORPRN (e.g., practice facilitation, education/training, process improvement) and added strategies inductively during abstraction (e.g., community engagement, technology system changes). Although categories were informed by the work of Powell and colleagues, we used the term practice change strategy to align with prior work and because the strategies observed rarely used individual implementation strategies or with existing strategy clusters.14, 44, 45

We gathered data via two sources: documents and informal interviews with ORPRN staff. Documents (e.g., applications, planning documents, manuscripts, reports) were the primary information source. We also conducted informal interviews by phone, email, or in person with the PI or project manager when necessary to clarify missing details (e.g., funding source, practice change strategies). One author (MMD) reviewed all abstracted information for accuracy and assessed project complexity on four dimensions (stakeholder number, geographic location, and variety of interests as well as project interdependencies) using a scale modified from Vidal and colleagues (see criteria in Appendix 3).46

The study team characterized and compared the 26 included projects in relation to each research question through a series of weekly review meetings. We utilized qualitative synthesis, which included integrated and interpretive methods, to draw conclusions.47 The study team identified and drafted a case example in order to illustrate how IS and QI elements manifest in the same project and to highlight potential synergies.

RESULTS

Included Studies

As detailed in Figure 2, 23% of the projects (23/99) occurred at ORPRN prior to 2007 and were excluded due to incomplete access to project information. After title and preliminary review, 38 projects underwent full review and 26 were included in the final analysis (see Appendixes 4 and 5). Included projects were classified as IS (n=4), QI (n=8), and IS/QI (n=14).

Figure 2
figure 2

Consort diagram of included and excluded studies.

Full size image

Characteristics

As noted in Table 1, projects addressed preventive services (e.g., cancer screening, vaccinations), care delivery models (e.g., patient-centered medical home, integrated care), specific diseases, geriatric topics, and medication management. Projects averaged 3.8 years in length; IS and QI projects were slightly shorter on average (3.3 and 3.0 years respectively) than those coded as IS/QI (4.4 years). Thirteen agencies funded these projects, including federal sources, national foundations, and local sources including state agencies and health plans.

Table 1 Characteristics of IS, QI, and IS/QI Studies Conducted in the Oregon Rural Practice-based Research Network (ORPRN) from January 2007 to 2019, N (%)
Full size table

Objectives, Setting, and Design

All 26 projects used evidence-based interventions (see Table 1). Half focused on implementing programs while the remaining focused on improving clinical care practices (27%) or increasing guideline concordant care (23%). Implementation of evidence-based programs was most common for projects categorized as IS/QI (71%) compared to IS (25%) or QI (25%).

Projects primarily occurred in primary care (50%) or engaged clinics and community settings (46%). The number of sites involved in projects varied widely, from a minimum of a single clinic to as many as 130 (see Appendix 4). Notably, 81% of projects occurred only in ORPRN. The three projects that involved multiple PBRNs were either classified as IS (1/4) or IS/QI (2/14). Half of the projects classified as IS or IS/QI used randomized designs while no QI projects did. IS/QI-classified projects also used staggered implementation designs (29%), pre/post (14%), and technical assistance support (7%).

As portrayed in Figure 3, projects had an upward trend in complexity over time, driven by an increase in the diversity of the geographic location of stakeholders, the number of stakeholders involved, and the variety of stakeholder interests. Increased complexity occurred for all project classifications (see Appendix 6).

Figure 3
figure 3

Study complexity score by year of funding, average, and individual components.

Full size image

Theoretical Basis and Practice Change Strategies

As detailed in Table 1 and Appendix 4, almost half of the projects (12/26) did not specify a theoretical framework. Theories were common in IS (50%) and IS/QI (86%) projects, while no QI projects included them. RE-AIM48 was used in three projects (12%); the following frameworks appeared in two projects each: the Solberg model,49 the Chronic Care model,50 and the Consolidated Framework for Implementation Research (CFIR).51 Four projects integrated two or more frameworks.

On average, 3.5 practice change strategies were used per project with practice facilitation as the most prevalent (81%, see Appendix 5). Other practice change strategies used in more than half of the projects included education/training (73%), process improvement (62%), and performance data in the form of audit and feedback or clinical quality reports (62%). All IS/QI projects used practice facilitation as did the majority classified as QI (75%) while only one IS project did (25%). All IS projects used education/training and the majority used performance data (75%).

Unique and Synergistic Contributions

Review of the projects classified as IS revealed a pattern of “top down” features in that an intervention and/or improvement target was driven at least in part by the research team or funding announcement. IS projects often utilized theoretical frameworks and prioritized the collection and synthesis of findings across participating sites. QI projects displayed “bottom up” features in that local stakeholders were involved in the selection of intervention and/or improvement targets and a focus on enabling change within a specific setting. QI-classified projects also focused on alignment to local context/needs and utilized rapid implementation processes to support change and to build clinic capacity. IS/QI projects displayed both of these elements by encouraging a focus on locally tailored improvement couched within rigorous cross-project evaluations to produce generalizable knowledge.

The case example in Figure 4 of AHRQ’s EvidenceNOW Healthy Hearts Northwest (H2N) initiative demonstrates how elements of IS and QI were often integrated within the same project. H2N was designed to help 250 small- to medium-sized primary care clinics in Oregon, Washington, and Idaho improve care to address cardiovascular risk factors (e.g., aspirin, blood pressure, cholesterol, and smoking) by implementing the latest evidence-based interventions (IS components). H2N was designed to enable an evaluation by the research team and participation in a cross-project evaluation to determine which change strategies were most effective (IS components). However, the case also describes how the practice facilitator focused on prioritizing change within a specific setting, illustrated in the focused work with clinic Z (QI components). Specifically, the practice facilitator did this by taking the overall project guidance for change, assessing the context of the larger health system and needs of the specific clinic, and tailoring their approach over time. For work with clinic Z, the facilitator served as a bridge between the larger health system’s centralized QI team and this individual clinic. Specifically, they helped the clinic review the current evidence; consider innovative changes that had worked in other clinics and to identify improvement priorities; overcome organizational barriers and gain access to data to inform improvement; and set improvement targets and utilize iterative QI cycles to enhance protocols and improve their metrics (QI components). Ultimately, this clinic became the top-performing clinic on the blood pressure metric within the health system.

Figure 4
figure 4

Case example of Healthy Hearts Northwest – a study blending implementation science (IS) and quality improvement (QI).

Full size image

Many of the IS/QI projects displayed similar methods for bringing elements of IS and QI together, as displayed in the case example (Fig. 4). One IS/QI project focused on integrating clinic and community programs to manage obesity (aka CLEMENTE) included QI elements in that the topic was a local priority, the interventions were operationalized based on locally available resources, and PDSA cycles were utilized to improve referral processes over time. In addition, a cross-project evaluation found that linkages were most effective when staff or clinicians bridged between the clinic and community-based settings through employment or volunteer roles with the referral programs. Another project, classified as QI only (CLIPS), was designed as an IS project at the time of funding, but never brought the cross-project evaluation to fruition. Thus, local contexts changed care delivery, but data were never utilized to produce generalizable findings to inform future implementation efforts. While these patterns repeated for many projects, the study team noted multiple challenges distinguishing between IS and QI studies because of similarities in the definition, setting, and methods used for evaluation.

DISCUSSION

We analyzed 26 projects classified as IS, QI, or IS/QI that were conducted in one PBRN (ORPRN) over a 12-year period; over half (53.8%) were classified as IS/QI. Many characteristics of IS, QI, and IS/QI studies were similar. Notably, all studies focused on implementing evidence-based practices, programs, or guidelines. However, half of the IS or IS/QI projects used randomized designs while none classified as QI did. None of the QI used theories to inform implementation or evaluation, compared to 86% of the IS/QI projects and 50% of the IS projects. The number of practice change strategies used were similar across all classification types; however, practice facilitation was most common in QI and IS/QI projects while education and training were used in all IS projects. Regardless of classification type, all projects displayed an upward trend in complexity over time based on an increase in the number of stakeholders involved, variety of interests, and diversity in geographic location.

In our longitudinal comparative case study review, projects that included both IS and QI elements—such as illustrated in the case example in Figure 4—did not only enable local practice change through QI efforts but also provided data to help evaluate implementation and to provide generalizable findings to inform future work. As illustrated in the case example, one can rarely do IS (integrate evidence into practice) without at least some QI (the local, applied, relational approach that turns ideas into actions). In addition, efforts from QI can be lost if they are not rigorously evaluated using IS methods to compare across sites in order to help determine what works, when, and why.

Figure 5 presents the synergistic benefits of IS and QI, and the iterative relationship between them, which emerged from our review of projects conducted within one PBRN. In contrast to prior work suggesting that QI is outside of implementation research,52 our study demonstrates how the two approaches can and should be brought into alignment. QI can be considered the mechanism by which the principles of IS are operationalized in order to support local practice change. IS in turn provides theories to inform implementation, evaluate efforts to produce generalizable knowledge (e.g., monitor issues of intervention fidelity and the role of context, identify determinants that influence the success or failure of efforts), and to disseminate findings to help others seeking to make similar changes. Integrating IS and QI within the same projects present opportunities to enhance the implementation and sustainability of research in practice. Such approaches align with the concepts from participatory implementation science, the learning evaluation approach, and recent work highlighting methods to systematically integrate IS and QI within projects to facilitate research translation.12, 13, 53,54,55,56

Figure 5
figure 5

The implementation science (IS)/quality improvement (QI) action cycle.

Full size image

Implementation strategies—or in our case “practice change strategies” to account for bundling—are an important element of both IS and QI which warrant additional study. Practice facilitation was used in 81% of the included projects. A growing body of evidence supports the effectiveness of practice facilitation in helping clinics implement clinical guidelines or improve care delivery.57, 58 Practice facilitators are trained professionals who use organizational development, project management, QI, and practice improvement methods to build the internal capacity of a clinic while helping supporting practice change initiatives.26, 27, 59 While facilitation is identified as one of 73 implementation strategies by Powell and colleagues,44, 45 PBRNs routinely use practice facilitation as a central and unifying approach to deploy practice change strategies that are tailored to local context and stakeholder needs.27 This may be because facilitators serve as boundary spanners between the type 1 (slow, systematic) thinking used to create evidence-based programs and interventions and the type 2 (automatic) thinking that occurs in practice.60 Richie and colleagues have described facilitation as a “meta-strategy” for implementation in that facilitators assess local contexts, help adapt interventions, and deploy tailored practice change strategies based on local priorities and needs61 and facilitation is the active ingredient of the integrating Promoting Action on Research Implementation in Health Services (iPARIHS) framework.62

There are a few notable limitations of the current study. First, our review was limited to projects conducted over a 12-year period (2007–2019) within one PBRN. ORPRN is an established PBRN with nearly two decades of experience supporting research and QI initiatives, and findings may vary for new or smaller PBRNs. Second, we excluded projects that were not classified as IS, QI, or IS/QI. We speculate that although these projects were not specifically designed to change practice, they likely set an important foundation for future IS and QI efforts. Additionally, “non-implementation studies” may be less demanding on practices and PBRN staff and thus provide opportunities to build the resilience needed to support practice change. Third, our description of theoretical frameworks was inclusive of frameworks used at any point (e.g., from implementation process to evaluation)43 and did not evaluate the impact process or implementation frameworks had on outcomes. Finally, our classification of studies as IS, QI, or IS/QI was difficult given the similarities in these approaches and differences in language and reporting.10, 42 Our team addressed this challenge by returning to the definitions for IS and QI identified in Figure 1, pragmatically defining elements of IS and QI projects, using multiple reviewers and group discussions to reconcile differences in coding, and conducting informal interviews with ORPRN staff in order to clarify gaps and/or locate additional project documents. Despite these limitations, our results provide important insight into how IS and QI elements often appear within the same projects and create potential synergies.

Future research could explore IS and QI characteristics in the full spectrum of projects conducted within PBRNs and to see if similar patterns emerge in other settings conducting pragmatic research such as in learning healthcare systems or other implementation laboratories. Research is also needed to test associations between theoretical frameworks, practice change strategies, and study outcomes to inform best practices and mechanisms of action in IS/QI projects.63

CONCLUSION

Implementation science (IS) and quality improvement (QI) are two approaches designed to improve clinical practice. While frequently viewed as distinct disciplines, our review found that the majority of projects conducted in one PBRN over a 12-year period included elements of both IS and QI. IS and QI provide complementary tools to help clinics integrate evidence-based practice, programs, and guidelines into routine clinical care. IS provides robust research evidence while QI provides a process by which to effectively engage and interact with local contexts. In turn, IS supports rigorous evaluations of these local QI efforts to support identification of what works and why, and to disseminate findings beyond individual settings. Strategic alignment of IS/QI within projects may help bridge the gap between scientific discoveries and routine practice.