Introduction

Learning health systems (LHSs) were first defined by the Institute of Medicine in 2007, as a system where “science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience” [1]. This approach to health system restructuring provides a promising opportunity to enhance value-based healthcare (VBHC). Value-based healthcare (VBHC) places patients at the forefront of health care services, while emphasizing quality of care over number of healthcare interactions [2, 3]. This approach also aims to reduce costs without sacrificing value [2, 4]. LHSs also place patients at the centre of the health system, with continuous learning from patient experience and outcome data cycling back into the system to improve care [5]. The LHS concept therefore aligns with the goals of VBHC [6]. There has been a global shift towards VBHC, and this concept is now recognized as a top health system priority [7, 8]. Despite the opportunity for LHSs to achieve VBHC across health systems, there are gaps in our understanding of how to operationalize LHSs.

Since its inception in 2007, the idea of LHSs has evolved to include several descriptions and features. Lavis et al. (2018) proposed seven characteristics reflective of a LHS: (1) engaged patients; (2) digital capture, linkage and timely sharing of relevant data; (3) timely production of research evidence; (4) appropriate decision supports; (5) aligned governance, financial and delivery arrangements; (6) culture of rapid learning and improvement; and (7) competencies for rapid learning and improvement [9]. In 2019, Menear and colleagues developed a LHS framework comprised of four key elements: (1) core values; (2) pillars and accelerators; (3) processes; and (4) outcomes [6]. The framework presents a structure in which health systems can work towards delivering more VBHC [6]. Another review by Zurynski and colleagues (2020) included over 200 LHSs papers and reported on the LHS terminology, frameworks, barriers and enablers of LHSs across the literature [10]. Studies in this scoping review used varying terms to describe LHSs, including learning health networks, rapid learning systems and learning healthcare systems [10]. Clearly, LHSs are gaining traction as a valuable model for healthcare organizations, and despite the varied terminology, the central focus on rapidly incorporating evidence into practice to enhance VBHC remains consistent.

While there is ample literature describing LHS characteristics, there is little information on how to put this model into practice. Effective implementation of LHSs has the potential to improve patient outcomes, reduce costs and enhance quality care [6]. So, without a proper understanding of LHS implementation, research and health system resources are lost. A narrative review of LHS frameworks by Allen et al., identified a roadmap to assist organizations in creating LHSs [11]. The authors presented a logic model with key inputs, outputs and outcomes, based on the core features of 17 LHS frameworks and models [11]. This study is a valuable resource to help health systems move towards a LHS model, however, this roadmap has not yet been applied to LHS-focused studies. Further, to date, no reviews have explored the types and outcomes of implementation strategies used by existing LHSs. As such, while the idea of LHSs is promising, it is still unclear how LHSs are operationalized across different health care organizations and countries. The scoping review by Zurynski et al. identified several functioning LHSs, but they did not describe how these LHSs were implemented or the outcomes of the implementation process [10]. Evidently, there is a need to unpack and synthesize the implementation process of LHSs across health care settings.

Implementation science is a field of research focused on methods and strategies to facilitate the uptake of evidence-based intervention and policies. Implementation strategies are techniques used to support the effective uptake of an intervention [12]. The Expert Recommendations for Implementing Change (ERIC) taxonomy includes a comprehensive list of 73 implementation strategies that were developed based on a review of the evidence and expert consultation [12]. Implementation scientists often apply these strategies in their research to ensure an intervention is delivered in a systematic, evidence-based way. Further, measures related to the implementation process can provide helpful information about whether an intervention led to meaningful change [13]. Tierney et al. established a list of 10 implementation measures specifically for evidence synthesis studies that provide additional insight as to the value of an intervention or research project [13]. While both the ERIC taxonomy and Tierney’s implementation measures have been applied to previous implementation research, they have not yet been applied to LHS-focused evidence synthesis work. With a clear gap in the evidence related to LHS implementation, there is an opportunity to understand how LHSs have been designed. Therefore, the aim of this study is to systematically synthesize the evidence on the implementation of LHSs across different health care organizations and countries. This aim will be achieved through the following objectives:

  • Describe the LHS characteristics used across studies and health care organizations

  • Identify the number and types of implementation strategies used to transition to a LHS

  • Describe the LHS outcome measures applied across studies

Methods

Study design

A mixed methods systematic review (MMSR) was conducted following the Joanna Briggs Institute (JBI) methodological guidelines for MMSR [14]. A MMSR allows for the comprehensive overview of a broad research question or phenomenon of interest and may include evidence from qualitative, quantitative, and mixed methods study designs.

This review was registered in PROSPERO (CRD42022293348) and the protocol was previously published [15]. The Preferred Reporting in Systematic Reviews and Meta-Analyses (PRISMA) checklist was used to report the findings of this study [16].

Inclusion criteria

Following JBI guidelines for MMSR [14], the PICo (Population; Phenomenon of Interest; Context) framework guided the question development and identification of inclusion criteria. This review aimed to answer the following research question: How do healthcare organizations implement and evaluate the transformation of learning health systems?

Population

Studies were considered for inclusion if they described a LHS, including rapid learning systems, rapid learning healthcare, learning healthcare systems, learning health systems or other similar LHS terms. Due to the inconsistency in reporting on LHSs, studies which described components of a LHS without using LHS terminology were excluded.

Phenomenon of Interest

This review included studies reporting on implementation strategies and/or outcome measures associated with the adaptation of LHSs. Implementation strategies include any procedure, approach, or method to implement, assess or evaluate the uptake of LHSs. The ERIC taxonomy of 73 strategies was used to identify implementation strategies reported across studies [12]. Any reported outcome measures were identified using Tierney et al.’s list of 10 implementation measures [13] to provide further insight into the value of the intervention and implementation process. Studies were further tagged as either provider, patient, population or healthcare cost-related outcomes, to reflect the quadruple aim of enhancing health systems [17, 18].

Context

Studies conducted in any health care setting were included. Health care settings may include hospitals, academic medical centres, primary care clinics, community health centres, practice-based networks or individual departments or clinics that provide health care to patients. Any country and size of healthcare organization were considered for inclusion. Non-healthcare settings, such as academic institutions, government, or non-government organizations where care is not directly provided to patients, were excluded.

Types of studies

Quantitative, qualitative, and mixed methods studies were considered for inclusion in this review. Additionally, descriptive papers reporting on the implementation of LHSs were included if they were peer reviewed. Grey literature sources such as policy reports, case studies or conference proceedings were included if they described the implementation strategies and/or outcome measures of LHSs. Protocol papers were excluded but forward citation searching was conducted to find any published studies stemming from the protocol. Similarly, reviews were excluded but the reference lists of identified reviews were manually searched for relevant papers.

Search strategy

Six databases were searched up to July 2022, for key terms related to LHSs, implementation and health care, using a comprehensive search strategy developed by a research librarian trained in knowledge synthesis. The search strategy was peer reviewed (PRESSed) by an independent research librarian to validate the approach. The databases included CINAHL (EBSCOhost), Medline (Ovid), Embase (Elsevier), Nursing and Allied Health Database (ProQuest), PAIS (ProQuest) and Scopus (Elsevier). Boolean operators and MESH terms were used accordingly for each database. An example search strategy for CINAHL can be found in Additional file 1: Table S1. No restrictions were placed on date of publication or language. A grey literature search was conducted to identify additional relevant articles. This involved searching ProQuest’s Dissertations and Theses Global, a targeted search of the websites of three pre-identified relevant organizations, and a systematic Google search to identify relevant sources. The grey literature search strategy can be found in Additional file 1: Table S2 while a comprehensive description of the search strategy can be found in the published protocol paper [15]. Additional articles were retrieved through backward and forward citation searching of reference lists of included articles.

Study selection

Identified articles were uploaded to the data management software, Covidence (Veritas Health Innovation, Melbourne, Australia), and duplicates were removed electronically. Two reviewers (MS and CJ) independently screened the titles, abstracts and full texts of identified articles based on the predetermined inclusion criteria. Any discrepancies in screening decisions were resolved through discussion by the reviewers, with an independent, third reviewer (CC), helping to reach consensus as needed. Studies were deemed eligible for inclusion if they reported on the implementation of LHSs, as outlined in the predetermined inclusion/exclusion criteria. One reviewer screened the first five pages of the returned grey literature search results. Relevant articles were uploaded to Covidence and followed the same screening approach as the database search results. The screening results, along with reasons for exclusion, were reported in the 2020 PRISMA flow diagram [16].

Data extraction

Following the screening stage, data were extracted from each included study using a pre-determined, data extraction form. The data extraction process was independently pilot tested by three reviewers (CJ, MS and DS) on a sample of included articles (n = 5). Data extractors met to discuss discrepancies in the data extraction process and changes were made to the extraction sheet as needed. Data were then extracted from the included articles by one reviewer (CJ) and verified by a second reviewer (MS) to ensure consistency and reliability of results. Weekly team meetings were held during the data extraction phase to discuss any arising concerns with the included articles.

Extracted data included study characteristics such as country, year of publication, study design, objective, population and description of LHS. Any reported details about implementation were also extracted. Qualitative findings were extracted as themes and sub-themes and included concepts related to implementing LHSs, such as stakeholder experiences in how healthcare organizations shifted to a LHS model. Evaluation measures and outcome data were also extracted from qualitative studies when available. Quantitative data related to the implementation and/or evaluation of LHSs were extracted. Quantitative findings included patient-related outcomes, cost effectiveness, provider outcomes, pre-post data, changes in population health and impact on the health system.

Data synthesis and integration

The LHS details from each study were synthesized using Lavis et al. seven LHS characteristics [9]. For example, studies reporting on patient engagement were tagged as such, and an overview of the most and least common LHS characteristics were reported narratively. Similarly, the implementation strategies described by authors were synthesized and categorized using the ERIC taxonomy of implementation strategies [12]. A descriptive synthesis of the most common strategies was reported. Outcome and evaluation details were also synthesized by mapping the reported outcome measures to Allen et al.’s list of 10 outcomes for LHSs [11], while Tierney’s list of implementation outcome measures was used to further categorize how studies reported implementation outcome measures [13]. These details provide a comprehensive picture of how LHSs have been implemented across various health care settings and whether the implementation of LHSs led to changes in health care costs, patient, provider, or population level outcomes.

A convergent integrated approach was used to synthesize the data in this review. This approach involves extracting quantitative and qualitative data separately, followed by an integrated synthesis of all sources of data [14]. First, the findings from quantitative studies were transformed into ‘qualitized’ data. This involved creating a narrative description for each quantitative study's key findings, by extracting the key findings from each study and then reviewing and refining them by two independent reviewers. A similar approach was used with qualitative study findings from included qualitative and mixed methods papers. The final, agreed upon narrative description of key findings from all studies were then integrated using thematic analysis by categorizing and pooling similar findings together. This involved two reviewers independently coding all findings, and then determining a list of common themes. The themes were then refined and finalized through further discussion among three authors, until consensus was reached. A final list of integrated findings was then reported in tables and text.

Critical appraisal

Included studies were critically appraised by two reviewers (MS and CJ) using the relevant JBI critical appraisal tool, according to study design. The purpose of the tools is to appraise different types of study designs, to provide an objective summary of the design quality. JBI is a reputable organization that specializes in access, appraisal and application of the best available evidence for evidence-based decision making in health and service delivery. The appraisal tool questions can be found in Additional file 1: Table S3. Qualitative studies were appraised using the qualitative checklist and scored out of 10. Cross-sectional studies were scored out of eight, cohort studies were scored out of 11 and quasi-experimental studies were scored out of nine. The reviewers independently appraised each study and then met to discuss final scores. In instances of differently scored items, the reviewers met to discuss their scores until a consensus was reached. A third reviewer was consulted in cases where a decision could not be reached. Final scores were presented as a percentage alongside extracted data in tables, with a detailed overview of study scores presented in a separate table. Grey literature sources and descriptive case studies were not appraised due to a lack of a relevant appraisal tool, and therefore did not receive a critical appraisal score. This was documented in the results tables. In line with mixed methods systematic review methodology [14], no confidence of findings summary table was developed. Due to the heterogeneous nature of mixed methods reviews, it is not recommended to complete this step.

Results

A total of 5171 studies were identified in the database search, of which 3147 were removed as duplicates. Of the remaining 2024 studies, 27 were deemed relevant and one additional study was found through hand searching reference lists [19]. The grey literature search returned over 700 000 resources, and after reviewing the first five pages of each search, 40 were screened in full text and seven were included in this review. Therefore, a total of 35 resources, describing 31 unique LHSs, were included in this review.

The main reasons for exclusion included wrong article type (n = 78), not related to implementation (n = 69), not about LHSs (n = 49), wrong study design (n = 39), duplicate study (n = 9) or not a healthcare setting (n = 1). A complete list of the search process can be found in the PRISMA flow diagram (Fig. 1).

Fig. 1
figure 1

PRISMA flow diagram of included studies. *Consider, if feasible to do so, reporting the number of records identified from each database or register searched (rather than the total number across all databases/registers). **If automation tools were used, indicate how many records were excluded by a human and how many were excluded by automation tools.From:  Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71. For more information, visit: http://www.prisma-statement.org/ 

Full size image

Study characteristics

Of the included studies, the majority were conducted in the United States (US) (n = 21) followed by Canada (n = 4), the United Kingdom (UK) (n = 3), Australia (n = 1), Sweden (n = 1), Netherlands (n = 1), and Europe (n = 1). Four studies reported on LHSs implemented across international borders. The date of publication ranged from 2014 to 2022, with most studies (24/35) published between 2019 and 2022. Of the 35 included studies, five were of qualitative design [20,21,22,23,24], four were cross-sectional studies [25,26,27,28], two were cohort studies [29, 30], one was quasi-experimental [19] and 23 were descriptive case studies [31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52]. All five of the qualitative studies involved semi-structured interviews and reported on stakeholder views related to implementing LHSs. Together, the qualitative studies described 20 themes related to implementing LHSs. The remaining quantitative and descriptive case studies varied in their designs and outcomes of interest. The case studies described the implementation of LHSs, without reporting on a specific methodological approach to data collection and analysis. The type of health setting varied across studies, with the majority of LHSs implemented in hospitals at a multi-institutional level, often for a specific health condition. The study population varied across studies, with 11 reporting on a pediatric population [26, 28, 29, 35,36,37, 39, 41, 44, 46, 52], 16 reporting on health system leaders or employees [20,21,22,23,24,25, 27, 31,32,33,34, 43, 45, 49, 50, 53] and eight related to adult populations with various clinical presentations [19, 30, 38, 40, 42, 47, 48, 51]. Study characteristics can be found in Table 1.

Table 1 Study characteristics and quality score of included papers
Full size table

Learning health system characteristics

Synthesis of LHS characteristics revealed that of the 31 unique LHSs (reported across 35 studies) eight included all seven of Lavis et al. characteristics [35, 36, 41, 42, 44, 45, 47, 53]. Two studies [25, 52] only reported two LHS characteristics, and these included the ‘digital data capture’, ‘timely production of evidence’ and ‘appropriate decision supports’ characteristics. Digital data capture was reported most often, in 31 studies. Examples of this characteristic included when a LHS incorporated an electronic health record (EHR) as part of the system or the use of dashboards and databases to enable data sharing. Patient engagement, aligned governance, and the culture of rapid learning and improvement characteristics were reported least often, in 21 studies each. Table 2 outlines the number and types of LHS characteristics reported across studies. A more detailed description of the LHS constructs identified in each study can be found in Additional file 1: Table S4.

Table 2 Learning health system characteristics based on Lavis et al.’s seven characteristics
Full size table

Implementation strategies

All study authors reported on the implementation of a LHS, including the specific implementation strategies used, and/or implementation outcomes. Of ERIC’s taxonomy of 73 strategies, 33 different implementation strategies were used across studies. The most common implementation strategies were change record systems (n = 20) [22, 26, 28,29,30,31,32,33,34,35,36, 38, 40, 45, 46, 48, 50,51,52,53], conduct local consensus discussions (n = 7) [19, 28, 29, 32, 37, 39, 49], audit and provide feedback (n = 6) [19, 22, 25, 43, 50, 53], build a coalition (n = 5) [20, 26, 41, 44, 46], and develop and organize quality monitoring systems (n = 5) [21, 27, 35, 44, 46]. Fourteen strategies were reported only once across studies. Table 3 provides an overview of the reported implementation strategies with an example from select papers.

Table 3 Overview of identified implementation strategies based on the ERIC taxonomy
Full size table

Study outcomes

Systematic adoption of evidence-based practices (EBP) was the LHS outcome reported most often (n = 17) according to Allen et al.’s list, followed by knowledge to action latency (n = 12) and population health (n = 12). Using Tierney’s list of implementation measures, the majority of studies commented on intervention complexity (n = 16) and adoption (n = 13). Only two studies spoke about implementation cost [28, 29] and no studies discussed fidelity as part of their LHS implementation approach. Based on the quadruple aim (patient, provider, population and health care cost), 23 studies reported on outcomes related to patients [19,20,21, 24, 26, 28,29,30,31,32,33,34,35,36, 38, 39, 44, 46,47,48, 50, 52, 53], eight studies addressed provider outcomes [20, 22,23,24,25, 27, 39, 43], 15 studies were related to population-level outcomes [26, 27, 36,37,38,39,40,41,42, 45, 46, 48, 49, 51, 53] and five studies reported on healthcare costs [28, 29, 31, 34, 35]. Table 4 provides an overview of study outcomes.

Table 4 Overview of outcome data including patient, provider, population and health system cost outcomes and implementation outcomes
Full size table

Integrated findings

The key findings reported across studies were heterogeneous in nature and therefore a meta-analysis was not possible. Rather, the main findings from all studies were integrated and thematically organized. Quantitative outcome data was qualitized and pooled along with the qualitative and descriptive study outcomes to reveal three main integrated findings and six sub-findings. These integrated findings are described in text below and in Table 5.

Table 5 Overview of integrated study findings along with sub-findings and supporting examples from the literature
Full size table

Integrated finding 1: the digital infrastructure of LHSs optimizes health service delivery

Multiple studies reported on how the implementation of a LHS impacted health service delivery, such as by allowing for the rapid inclusion of evidence into practice or by providing an infrastructure to support digital data capture. Three categories were grouped under this main finding. The first describes how LHSs can allow for better integration of data and evidence into clinical practice. Fifteen studies aligned with sub-finding 1a and reported on how the implementation of a LHS can allow for better integration of data and evidence into practice, such as having a platform that highlights chronic pain in patients to inform care [26] or a database with information about patients with multiple sclerosis that clinicians use to inform decisions [30]. Sub-finding 1b includes how LHSs promote the implementation of digital data capture. Sixteen studies described how a LHS can provide an opportunity for digital data capture [19, 21,22,23,24, 31, 32, 39,40,41, 44,45,46,47, 50, 51]. In some cases, the digital infrastructure was not available or valued prior to the implementation of a LHS, but with the health system transformation, systems were able to see how infrastructure could support the collection of digital data. This was often viewed as key to the LHS implementation process. Sub-finding 3c describes that access and availability of data through a digital platform supports rapid changes to practice and policy. Seven studies aligned with this finding and described how the LHS allowed for faster changes to practice and policy [25,26,27,28,29, 33, 38]. In some studies, this was a result of having electronic data more readily available to practitioners, allowing for rapid decisions and changes to patient care.

Integrated finding 2: learning health systems have a positive impact on patient care and health outcomes

The second integrated finding is represented by 17 studies [19, 22, 28, 29, 32, 33, 35,36,37,38,39, 42, 43, 45, 49, 51, 52] which reported positive patient outcomes as a result of implementing a LHS. These studies included varied patient outcomes such as improved diagnosis, screening or referral rates [38], reduced readmission rates and decreased length of stay [29], and improved prescribing practices [19, 39, 51, 53].

Integrated finding 3: learning health systems can influence health care providers and the health system

The third main finding is related to the impact of LHSs on the health system, including the physical environment and people within the system. This finding included three sub-findings. The first sub-finding, 3a, states how implementation of a LHS may help foster a culture of learning and improvement for sustained success. Eight studies related to this finding and described how the LHS changed the culture within their healthcare organization [21, 23, 31, 33, 34, 39, 43, 45]. This included having a stronger culture of continuous learning and improvement, with several studies describing this feature as being crucial to LHS sustainability overtime. Sub-finidng 3b describes how health system leaders identify challenges in implementing LHSs, despite recognizing their value. Four studies aligned with finding 3b and reported the challenges of implementing a LHS [20, 24, 47, 50], such as facing financial barriers to implement and sustain the digital infrastructure [24] or in getting buy-in from key stakeholders [20]. Sub-finding 3c includes how LHSs may result in cost savings for the health system. Five studies talked about the potential cost savings of LHSs, following an economic analysis of their respective LHSs [28, 29, 31, 34, 35].

Methodological quality

Of the 35 included studies, 12 were appraised for methodological quality [19,20,21,22,23,24,25,26,27,28,29,30]. The remaining 23 papers were grey literature or descriptive case studies and were therefore not eligible for appraisal. Studies were appraised using the relevant JBI appraisal tool and a score was assigned based on percentage of criterion met in each study. Quality scores ranged from 25 to 91%, with five studies receiving a score of 75% or higher [20, 21, 26,27,28]. Table 6 provides an overview of study scores for the 12 appraised studies.

Table 6 Overview of methodological quality for appraised studies (n = 12)
Full size table

Discussion

Learning health systems offer a promising approach to advance VBHC; however, it is unclear how LHSs are operationalized across different health care organizations and countries. Our mixed methods review addresses this gap by highlighting and synthesizing the types of implementation strategies most used when transitioning to a LHS and provides some outcome data from functioning LHSs. Researchers and health system leaders may use these findings to support their own LHS implementation and evaluation efforts.

LHS characteristics and implementation strategies

Studies highlighted the importance of digital infrastructure to capture data and integrate it back into the system to inform decision-making. This is a central component of LHSs described across the literature [6, 9, 54], and it is not surprising that this was a key finding in our review. Digital data capture was included as part of the LHS description in 31 studies, while digital infrastructure to support health services delivery was revealed as a key integrated finding across studies. Further, changing record systems was identified as the most common implementation strategy, highlighting that most health systems adapted their digital infrastructure in some way to implement their LHS. To support other LHS features, such as incorporating patient and provider experiences, and having timely production of evidence, it makes sense that establishing the appropriate digital infrastructure is a preliminary step. This aligns with previous LHS research that found a lack of infrastructure, digital registries and electronic systems for capturing patient data were common barriers to developing a LHS [10]. Clearly, establishing infrastructure for digital data capture is central to LHS implementation and therefore, researchers and health systems leaders may want to prioritize this aspect of implementation for advancing LHS transformation.

Only 21 studies included patient engagement as part of their LHS description, while one study included patient, consumer, and family feedback as an implementation strategy [36]. Another two studies reported involving patients or consumers to enhance the uptake and adherence of the LHS intervention [26, 42]. Evidence suggests that patient-centeredness in healthcare leads to improved patient outcomes and quality of care [55]. However, patient engagement has been lacking in LHS literature, with patient-clinician partnerships cited least often in a synthesis of LHS papers [10], and some LHS frameworks excluding this dimension altogether [54, 56]. While it is crucial not only to engage patients in LHS development, but to incorporate patient experience data back into the system, there are challenges in achieving this. Patients want LHSs to be transparent about the use of their data [57] and to have open communication and shared decision-making with their clinical provider within the LHS [58]. Patient-centred care should be central to any healthcare system, including LHSs. However, more research is needed to understand how to best engage patients in the development, implementation, and evaluation of LHSs to ensure health system transformation remains patient-centered.

Along with patient engagement, aligned governance and a culture of rapid learning and improvement were reported the least often (in 21 papers each). Only eight studies [35, 36, 41, 42, 44, 45, 47, 53] touched on all seven LHS characteristics in their study description. While it is possible that authors were not comprehensive in their LHS reporting, the findings from this review clearly indicate that key components of LHSs are not consistently reported across the literature. Aligned governance is a challenging feature to implement in LHSs [10, 59], largely due to ethical issues in policies and regulations [10, 60]. There are also barriers to establishing a culture of learning and improvement in LHSs due to low buy-in from health system stakeholders [35, 61]. This was a key integrated finding of this review, reported in multiple studies [20, 24, 47, 50]. Regardless, this aspect of the LHS was also recognized as important for ensuring it would be sustained overtime [43, 48, 49, 52]. Clearly, all LHS characteristics are important but there are challenges in incorporating each feature, which means some LHSs are missing key components. An evaluation framework or checklist would be useful to address these challenges in LHS implementation and to ensure researchers are meeting the requirements of a fully functioning LHS.

In this review, we used ERIC’s taxonomy of 73 implementation strategies to code how LHSs were implemented in health service organizations. Of the 73, only 33 distinct implementation strategies were employed, with changing record systems cited as the most common. Much of the literature highlights the need for a culture of rapid learning and improvement to facilitate LHS transformation [21, 49, 62]. Despite this critical component, few studies identified in our review employed implementation strategies that would adequately target culture change. While changing record systems are required to support the data linkage component of a LHS, LHSs will never be fully realized unless strategies aim to facilitate a culture of rapid learning and improvement. Future LHS initiatives should consider building on the strategies identified in this review including conducting local consensus discussions and building a coalition. Future LHS research should test additional implementation strategies that have not yet been evaluated for LHS implementation, including strategies that facilitate the development of stakeholder interrelationships and support clinicians to engage in LHS activities [63].

LHS outcomes

Of the 35 studies in this review, 17 reported positive patient outcomes following implementation of their LHS. This finding is promising as one of the main goals of a LHS is to achieve VBHC, including improving patient outcomes and providing better quality care [2]. However, it is unclear what mechanisms directly led to improved patient outcomes and whether certain LHS features are more strongly associated with positive outcomes. Few studies reported on outcomes related to provider, population and health system costs and most studies did not evaluate the impact of LHS implementation. There is a need for further evaluation research to explore the full impact of LHSs, including how well it addresses the quadruple aim. Finally, the majority of LHSs in this review targeted a particular health condition or patient population. These LHSs were often conducted at an individual department level or as a multi-institutional network with several condition-specific departments working together. Few studies reported on a LHS at the organization level, such as across an entire hospital. This aligns with findings from a recent review that found only four of 76 studies described a LHS as an entire hospital system [64]. With the concept of LHSs still new, it makes sense that researchers may want to first establish a LHS for a particular patient population before expanding more broadly across an institution. However, there are examples of larger scale LHSs, such as the Swiss LHS, being implemented nationally across Switzerland [65]. It is important for researchers to learn from these broader health system examples to continue to scale and spread the efforts and impact of the LHS model.

Implications for research, practice and policy

This comprehensive mixed methods systematic review illustrates important implications for research and health system leaders. First, this review highlighted the value of having a robust infrastructure to support digital data capture when implementing a LHS. Health system leaders and researchers should to prioritize this aspect of LHSs for an effective transition. Second, efforts are needed to support the engagement of patients into the development, implementation, and evaluation of LHS. Third, although many strategies are described in the implementation science literature, few were described in the included LHS studies. Future research should test additional strategies that facilitate partnerships development and engaging clinicians in LHS. Lastly, this review provides evidence that LHSs can lead to improved patient outcomes but there is a need for further evaluation studies on the overall effectiveness of LHSs and their impact on patient, provider, population and cost-related outcomes. With most LHSs being implemented at either an individual clinical unit or a multi-institutional network for a particular medical condition, there is a need for future implementation research to explore large scale health system transformation, such as organizational (e.g., entire hospital), provincial or state-wide health system transformation.

Strengths and limitations

A key strength of this review is the comprehensive mixed methods approach. With the literature on LHSs continuing to emerge, it was important to capture the broad range of studies related to implementation. The inclusion of grey literature allowed for additional case study examples of LHSs to be explored and synthesized. Several aspects of implementation and LHS characteristics were used to categorize studies, which may be useful for health systems researchers and administrators to understand how they can apply similar approaches in their respective healthcare settings. A limitation of the current study includes the potential for biases in the critical appraisal process. While the studies were appraised by one reviewer and verified by a second, some of the questions in the JBI appraisal tool require interpretation of the authors, which allows for potential bias. Additionally, some of the studies included in this review did not have an appraisal tool that fit the methodology, and thus were unable to be appraised. The inconsistent and evolving LHS terminology was a challenge in this review, as some studies may have been excluded if they did not explicitly refer to their intervention as a LHS. This was necessary to avoid irrelevant papers but may have unfairly excluded studies that were truly LHSs but used lesser-known terminology. While no restrictions were placed on language or country, the majority of included studies were conducted in high-income, English-speaking countries. There is a need to explore LHSs in the context of low and middle-income countries.

Conclusion

In this mixed methods systematic review, we described the implementation of LHSs in various healthcare settings, including implementation strategies, outcome measures, and components of functioning LHSs. As the field of LHS science and practice continues to advance, research is needed to better understand the impact of LHSs on patient, provider and population outcomes, and health system costs. Health systems researchers should continue to apply the LHS concept in practice, with a stronger focus on evaluating implementation strategies and outcomes.