Background

Over a decade ago, the sustained use of evidence-based practices (EBPs) was identified as a gap in the literature. Evolving debate among experts suggest sustainability should be considered a distinct concept that occurs “(1) after a defined period of time, (2) the program, clinical intervention and/or implementation strategies (hereafter referred to as EBPs) continue to be delivered and/or, (3) individual behavior change (i.e. clinician, patient ) is maintained, (4) the program (EBP) and individual behavior change may evolve or adapt while (5) continuing to produce benefits for individuals/systems [1]”. Despite growing interest, the timing and understanding of how to sustain the use of EBPs remains a relatively unexplored field of research [2, 3] and least understood part of the translation research process [4] that has challenged practitioners and researchers alike. Evidence reveals the integration and sustainability of EBPs in clinical practice is “an iterative, dynamic” [5] and “complex process,” [6] which poses a significant challenge. Emerging discourse indicates efforts to sustain EBPs in healthcare should be guided by conceptual frameworks, models or theories (hereafter collectively referred to as F/M/Ts) [1, 7,8,9,10,11,12] to better understand the factors that impact sustainability as a distinct concept [13, 14], over time, in a range of distinct healthcare settings [3, 10, 11]. Thus, a critical analysis of existing sustainability F/M/Ts relevant to acute care contexts was conducted as a way to understand the meaning of key concepts, factors, and their relationships to ultimately provide direction for practice and research.

Increasing demand on healthcare organizations to improve patient outcomes [10, 15, 16] in an efficient, cost-effective manner [17, 18]) has resulted in the growing expectation that EBPs be informed by research, be effective and sustainable to inform clinical decision making [19, 20]. In response, healthcare organizations have undertaken a number of quality improvement initiatives [10]. Despite efforts, variable rates of sustained use of EBPs exist ranging from none to full adherence [2], not only among various healthcare professionals but also within different settings [1, 9, 10, 17, 20,21,22,23,24]. Researchers argue the decay of sustained EBPs [17, 23, 25, 26], also referred to as the “improvement evaporation effect” [25, 26] can be attributed to the limited use of theoretical F/M/Ts [27, 28]. To overcome these challenges and to advance knowledge, researchers [7, 11,12,13,14] recommend the use of F/M/Ts to examine the factors that impact sustainability as a distinct concept, especially in complex acute care environments [11].

Recent reviews/syntheses reveal a lack of use/empirical testing of existing F/M/Ts [10, 19, 24], highlight several diverse perspectives, applications and constructs deemed useful for sustainability [10], and few F/M/Ts that focus solely on the sustainability of EBPs within acute settings [10, 24, 29]. Specifically, the majority of sustainability F/MTs and approaches are designed for use in non-specified healthcare settings (37% or 23/62) (e.g., healthcare organizations or systems), followed by 31% (19/62) specified for use in public health, 26% (16/62) in community settings, and only 3% (2/62) primarily focused within acute care [10]. To date, a review that examines how to improve the sustainability of EBPs in acute care settings has not been conducted [11]. Given healthcare expenditures are reported to be the largest in hospitals (36.9% in 2018) [30], exclusively identifying relevant concepts and factors related to sustainability in this challenging setting will likely be of considerable benefit to research and practice, potentially improving the quality of care and reducing costs. Clearly, a gap exists regarding which existing sustainability F/M/Ts are applicable and what factors are relevant when trying to sustain the use of EBPs primarily in acute care contexts [10, 13].

The aims of this study were to (i) identify existing healthcare F/M/Ts that explicitly address the process of sustained use of research (EBPs/guidelines/innovations/clinical protocols/programs/interventions) and are recommended for use within acute care contexts or unspecified healthcare organization/setting; (ii) compare F/M/Ts, using a theory analysis approach, to identify key concepts and factors that influence/predict the likelihood of successful sustainability of EBPs; and (iii) provide a list of relevant sustainability F/M/Ts, concepts, and core factors to act as a guide for practice and provide direction for future research within acute care contexts.

Methods

Search strategy and data sources

Two different data sources and related search strategies were used to identify existing healthcare sustainability F/M/Ts. First, a full text review, abstraction and appraisal of all F/M/Ts included in two existing knowledge syntheses of sustainability in healthcare [1, 10] was conducted to determine overall alignment with the aims and eligibility criteria established for this study given their original purposes, scopes and related methodologies (see Table 1). Second, to identify relevant sustainability F/M/Ts published after the two syntheses, a new systematic search of all published articles, dissertations/theses, systematic and scoping reviews, and concept analyses was conducted using the same eligibility criteria guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) reporting standards [31, 32] (see Additional file 1). The selection of healthcare databases, search terms, and strategy was supported by a health science librarian and peer-reviewed by a second using the PRESS EBC Checklist [33]. A search of CINHAL, MEDLINE, Embase, and ProQuest databases was conducted with results limited to citations published between 1 January 2015 and 3 July 2018, based on end dates of the two syntheses. A hand search of references from included citations was undertaken. Details of key terms and search strategies are available in Additional file 2. Ethical approval was not required for this review.

Table 1 Inclusion and exclusion criteria for systematic review (January 1, 2015, to July 3, 2018)
Full size table

Eligibility criteria

Eligibility criteria were designed to examine sustainability as a distinct concept, as per Moore et al.’s [1] definition, and to identify concepts and factors that related solely to the sustained use of EBPs, after the initial rollout, in complex healthcare environments such as acute care [3, 34]. A checklist of inclusion and exclusion criteria was developed to guide selection of citations (see Table 1). During the process, four coauthors (LNP, JS, BD, CB) reviewed a sub-sample of citations [25] to refine and ensure criteria could be consistently applied. To be eligible, citations needed to be published in English; in a peer-reviewed journal; include sustainability or implementation and sustainability F/M/Ts recommended for use in acute care or an unspecified healthcare organization/setting; and represent the most current/refined version.

A citation was excluded if the F/M/T was not recommended for healthcare; was recommended only for use within a specified setting other than acute care (e.g., public health or community); if it contained only an implementation F/M/T; and if it contained an implementation and sustainability F/M/T without an explicit breakdown of related sustainability factors. Notably, this study was not designed to examine the influence of implementation on sustainability.

Data collection process and analysis

A data collection form was piloted by four coauthors (LNP, BD, CB, JS) with 50 randomly selected citations to ensure comprehensiveness prior to screening. The form required minimal modification. To ensure inclusiveness, level 2 full text screening of all citations was conducted in two steps: (i) screening of results from two syntheses was completed by one reviewer (LNP) and reviewed by four coauthors (BD, IG, CB, JS); (ii) screening of systematic review results was completed by two independent reviewers (LNP, IM). Final decisions regarding inclusion were made jointly by LNP and coauthors (BD, IG, CB, JS). Disagreements were resolved through discussion and consensus.

A theory analysis of the identified F/M/Ts was undertaken as a means of understanding their theoretical underpinnings, paying particular attention to key concepts/factors influencing sustained EBP use within acute care [35]. According to Walker and Avant [36], theory analysis involves consideration of seven elements: (i) determining origins, (ii) examining meaning of concepts and their relationships; (iii) analyzing logical adequacy of concepts and relational statements to determine predictive ability to generate hypotheses, (iv) determining usefulness for practice and predicting outcomes, (v) defining generalizability across settings, (vi) defining the degree of parsimony and language clarity, and (vii) determining testability (see Table 2). Modifications to the theory analysis elements/tool included adding a subjective rating scale for both parsimonious (full or partial) and language (clear, somewhat unclear, unclear). Analysis involved entering findings into a master chart to facilitate comparisons. All factors identified in the appraisal were then extracted and collated. Qualitative content analysis was completed by identifying and placing all related and similar factors together (identified as core factors) and then into broad themes, which were inductively identified from F/M/Ts [37, 38]. Factors cited in four or more F/M/Ts within each theme were identified as common factors.

Table 2 Theory analysis elements applied to sustainability frameworks/models/theories
Full size table

Results

Of the 2967 citations identified, 8 met the inclusion criteria (e.g., four from Moore et al. [1], three from Lennox et al. [10], one from the new systematic review) and were eligible for theory analysis (Fig. 1). Rationale for excluded citations is documented in Additional file 3. Most F/M/Ts containing both implementation and sustainability phases did not explicitly provide a detailed breakdown of the sustainability concepts/factors and were excluded. Those that did were recommended for use in community and/or public health settings and were excluded.

Fig. 1
figure 1

PRISMA flow diagram for combined syntheses (Moher et al. 2009)

Full size image

Framework/model/theory characteristics and quality appraisal

Origins

The F/M/Ts were published between 2005 and 2016; the majority (n = 6) published after 2010. The originators are from Europe (n = 4) [25, 39] [40, 41]; North America (n = 3) [13, 34, 42]; and Australia (n = 1) [43] (see Table 3). Various methodological approaches used by originators to develop F/M/Ts included focused systematic or literature reviews (n = 4) [25, 34, 40, 42]; integrative reviews of frameworks or theory [41, 43]; a Bayesian research co-production approach [39]; and a concept analysis [13]. The F/M/Ts were reported to be based on theoretical and empirical work of scholars from different fields of study/disciplines with varying theoretical perspectives on sustainability. Specifically, these included the diffusion of innovations theory [42, 43], organizational change theory [25], organizational and management theory [25, 39, 42], ecological theory [34, 43], total quality improvement theory [25], psychological theory [41], theory of routines [40], and multiple healthcare discipline theories [13]. Six F/M/Ts were designed to be operationalized to guide practice and/or research at an organizational or unit/departmental level. Two were specified for use at the project/initiative level [34, 39]. Three F/M/Ts were explicitly recommended for use in a hospital [13, 40, 41], and the remaining five were recommended for use in any unspecified healthcare organization/setting. Some F/M/Ts were intended for multiple audiences, namely, researchers [13, 25, 34, 39,40,41,42] practitioners [13, 34, 39,40,41,42], policy-makers [34], administrators [13], and funders/grantors [42]. The most common motivation was to add to the body of evidence/knowledge to either guide research or better understand how to successfully sustain effective improvements in practice.

Table 3 Origins of sustainability frameworks/models/theories for acute care settings
Full size table

Meaning of the F/M/T

To examine how originators defined the constructs of sustainability, conceptual definitions for sustainability were mapped to the five constructs of a comprehensive definition recently published by Moore et al. [1] (see Table 4). Notably, one author did not provide nor reference an explicit definition of sustainability [42]. Two definitions included all five established constructs [25, 39], and four definitions included all but one construct [13, 34, 40, 41]. Similar to Moore et al. [1] findings, the most commonly described construct for sustainability was “continued delivery or use,” which was combined equally with the “evolution or adaption” construct cited in seven out of eight definitions.

Table 4 Framework/model/theory sustainability definitions mapped to sustainability constructs by Moore et al. [1]
Full size table

Differing from Moore et al. [1] findings where most publications did not define the timeline for sustainability, the “after a period of time” construct was included in 75% (6 out of 8) of the definitions. However, these time-related references were undefined and unquantified. The “continued benefits” construct occurred in five out of eight definitions signifying the importance of the perceived goal to enhance outcomes (on individual, unit, organization, system level). The “maintain behavior change in individuals” construct reflected how a broad range of EBPs may interact with individuals or teams to maintain behavior change for sustainability. Although this was the least commonly described construct, it occurred in half the definitions: two F/M/Ts recommended for use in acute care [40, 41] and two for use in unspecified settings [25, 39].

Two similar constructs of sustainability currently not included in the Moore et al. [1] definition emerged during the analysis: defining sustainability as a “process,” [13, 25, 34, 39, 40, 43] or as a “stage/phase of ongoing use” post-implementation [41, 42]. These views were supported by several theoretical perspectives given F/M/T origins, revealing a new construct that describes the nature of sustainability to be “ongoing/continuous and process-like.”

Synthesis of factors and themes

Initially, 152 sustainability factors were extracted from the 8 F/M/Ts. Qualitative analysis identified 37 core factors, which grouped into 7 themes: (1) characteristics of the innovation/EBP; (2) adopter/user factors influencing sustained use (3) leadership and management influences/factors; (4) inner context (practice setting/organization) factors where EBPs are delivered; (5) inner processes/infrastructure factors that support the EBPs (e.g., processes, methods, systems, structures, or strategies); (6) outer context or broader system factors; and (7) outcomes descriptions without defined factors. Further synthesis identified 16 common factors (occurring in four or more F/M/Ts), which are highlighted with an asterisk in Table 5.

Table 5 Synthesis of themes and factors found in sustainability frameworks/models/theories for acute care (n = 8)
Full size table

A subgroup analysis comparing the themes and factors among the specified acute care F/M/Ts [13, 40, 41] with those recommended for unspecified healthcare settings [25, 34, 39, 42, 43] was conducted. Results are available in Additional file 4 and Table 5. Notably, originators collectively identified all seven themes within both subgroups. Only three out of 37 core factors were uniquely identified among all F/M/Ts: two core factors were separately identified in two different F/M/Ts within the acute care subgroup (e.g., behavioral change strategies [41], financial funds, and non-financial resources [13]), and one core factor was identified within the unspecified setting subgroup (e.g., barrier identification [43]). Given minimal subgroup differences, all F/M/Ts were included in the theory analysis.

The themes were defined by terms used by originators. The “adopter” theme is defined as a stakeholder, staff, user, adopter, actor, or individual using the innovation/EBP. Of note, the Sustainability of Innovation Theoretical Framework (hereafter Fox SITF) [43] and Sustainability of Healthcare Innovations Framework (hereafter Fleiszer SHIF) [13] focused exclusively on the presence and influence of champions. The “inner context” theme refers to the context, practice setting or organization, while the “inner process” theme includes processes, methods, systems, structures or strategies used within the context. The “innovation” theme, defined as a new process, change, product, practice, or programme in six F/M/Ts, is not evident in two F/M/Ts [40, 41]. Similarly, the “leadership and management” theme refers to leadership style, approach, behaviors, engagement, support, or feedback in six F/M/Ts [13, 25, 34, 39, 41, 42]. The “outer context” theme, referencing conditions, context, systems or environment external to the inner context, is not evident in three F/M/Ts [39,40,41]. The outcome theme is described in four F/M/Ts as “outcomes on a spectrum from high to nil” [13], sustained “teamwork behaviors” [41], “consequences” [25], or “continuation of benefits” [34].

Inclusiveness of themes and factors

Three F/M/Ts [13, 25, 34] contain all seven themes with one F/M/T [42] containing six themes. The inclusiveness of 6–7 themes in 50% (4 out of 8) F/M/Ts highlights the importance of all themes and related factors for the sustainability of EBPs within acute care contexts. The innovation [40], leadership and management [34, 40, 41, 43], outer context [39,40,41], and outcome [34, 40, 41, 43] themes were not evident in all F/M/Ts. The Framework and a Measurement Instrument for Sustainability of Work Practice (hereafter Slaghuis FMIS-WP) [40] contains only three themes and related factors as it represents a portion of a larger conceptualization on sustainability unpublished. The 37 core factors primarily are distributed among 6 themes, given the outcome concept/factors are undefined. All F/M/Ts contain core factors from the adopter, inner context, and process themes. Fifty-seven percent (21 out of 37) of the core factors are contextual contingent including inner context, inner processes, and outer context core factors thus highlighting the influence context may have on the sustainability of EBPs in acute care. One F/M/T contained all 16 common factors [13].

Concept/factors relationships

All originators described the relationship between the factors as non-discrete or dynamic, which may interact either in varied combinations or degrees on different levels. How this occurs, however, was not made explicit by definition/statements. The use of arrows to imply direction or potential influence between concepts/factors was used in seven F/M/Ts. Uniquely, the National Health Service Sustainability Model (hereafter Maher NHS-SM) [39] originators used three overlapping colored circles representing broad concepts to illustrate a level of dynamic interaction among the related factors within the concepts. The use of arrows or circles failed to clarify how the interactions between factors occurred. The Dynamic Sustainability Framework (hereafter Chamber DSF) [34] originators specified a “dynamic relationship” that exists between and among the three concepts (e.g., innovation, practice setting, broader system) and changes over time, but how to interpret this was unclear. Uniquely, the DCOM Framework with Realistic Evaluation (hereafter Frykmann DCOMF) [41] originators used relational statements to identify key influences impacting relationships between factors not evident in other F/M/Ts, namely, four mechanisms of behavior change: direction, competence, opportunity, and motivation. All originators recommended further testing to seek greater clarity about relationships between concepts. Fleiszer SHIF [13] originators suggest their framework is representative of a mid-range theory, and further understanding of the relationship between concepts and factors is essential.

Assumptions

Key assumptions underlying the F/M/Ts include (i) the concept of sustainability is only partially mature [13], dynamic [34, 40], or ambiguous having different meanings in different contexts [25]; (ii) sustainability considers change (either strategic and/or incremental) as a central influence [25, 34, 39,40,41, 43]; (iii) evolving fit and/or adaption of the EBP is expected [13, 25, 34, 39, 40]; and (iv) success overtime is based on whether or not the EBP remains beneficial [13, 25, 34, 39, 41, 42].

Schematics

All originators provided schematic representations illustrating key concepts/factors claiming to be operational and able to guide sustainability efforts and future research. Four F/M/Ts depict unidirectional graphical representations that assume a continuum or processual stance focusing on the EBP and its ongoing implementation process in context influenced by internal or external factors [13, 25, 34, 41] thus implying the goal of maximizing the fit between the EBP and the context. Originators of the remaining schematics provided a simple, high-level representation depicting the interplay among the set of factors [39, 40, 42, 43]. Notably, in all schematics, each factor category was represented as equal relative to one another given their image size. In fact, originators contend the relative significance of the factors cannot be determined a priori, except the Maher NHS-SM [39] where relative weighting within and among the factors is provided based on empirical evidence.

Empirical testability

To date, evidence of further testing of four F/M/Ts has occurred [13, 34, 39, 40]. Notably, the Maher NHS-SM [39] has been empirically tested in the USA, Canada, UK, South Africa [14, 21, 45, 46] and in low- to middle-income countries [47]. All originators recommended practical testing (application and evaluation) in multiple contexts using different methodologies to broaden conceptual understanding and further development/refinement. Specifically, research using a systemic and process-orientated lens to uncover the complexities and dynamics of the concept was recommended [13].

Parsimonious and language clarity

Five F/M/Ts were subjectively rated by coauthors (LNP, BD, IG, CB, JS) as parsimonious, with clear language, terminology, explicit definitions for factors, and without repetitions noted [13, 25, 39, 41, 42]. The remaining three F/M/Ts were rated as partially parsimonious based on the lack of completeness [40] or the use of vague definitions and concept relational statements [34, 43].

Logical adequacy

Originators claimed all F/M/Ts as operational and capable of guiding research and practice to explore factors influencing the sustainability of healthcare EBPs. Originators of four F/M/Ts explicitly provided either testable hypotheses [34, 39, 42] or testable scales for the concepts [40]. The chambers DSF [34] originators proposed seven tenets related to the ongoing improvement of EBPs emphasizing a “central goal of continuously optimizing the fit between the innovation and the dynamic (changing) delivery context to achieve maximum benefit” [34]. The Model for Sustaining Innovations (hereafter Racine MSI) [42] originators provided 12 propositions, which align with three main factor categories (innovation legitimacies, intermediary functions, conditions of local adopters) but assert it does not predict the likelihood of sustainability. Conversely, the Maher NHS-SM [39] originators defined 10 measurable factors, which are weighted within and among each other, providing a testable hypothesis and a prediction of sustainability for the improvement. originators of the four remaining F/M/Ts [13, 25, 41, 43] identified measurable factors/variables to guide research and data collection without explicitly defining the impact of the factors for outcomes concept but rather state it can vary based on the innovation, conditions, and contexts.

Logical fallacies

Minimal inconsistencies related to the content within the “adopters” and “outcomes” themes were noted among all F/M/Ts by coauthors (LNP, BD, IG, CB, JS). Specifically, within three F/M/Ts, the “adopters” theme was not identified as distinct but rather considered part of the inner context [13, 34] or inner processes [41] themes. In the Fleiszer SHIF [13], the deliberate positioning of individual characteristics within the inner context versus distinct, similar to the leadership and management theme, was not explicit. Chambers DSF [34] originators did not identify “staff” separately but rather part of the “practice setting” or inner context. The staff/team member is not explicitly identified as a separate theme by originators of Frykman DCOM [41], yet the entire framework is focused on revealing how behavior change interventions influence the sustainability of staff/teamwork behaviors. The failure to distinguish adopters, either as individual [2, 48] or collective agency [49] influences, as a separate theme by originators is inconsistent with other F/M/Ts noted in recent syntheses [10, 29] and this study. Furthermore, originators of four F/M/Ts, identified “outcomes” as a theme [13, 25, 34, 41] represented by the combined influence of factors from within their frameworks. Outcome factors were undefined in all F/M/Ts.

Usefulness

Originators claimed the F/M/Ts have multidisciplinary relevance and practicality to inform health professionals, administrators, policy-makers, and/or funders to identify inadequacies, refine theory, and ensure the development of the concept. Uniquely, originators of Racine MSI [42] contend their model provides a “blueprint or agenda” [42] with clear practical implications. Maher NHS-SM [39] originators assert their model is intended to provide a platform for quality improvement for all healthcare disciplines. Other originators indicated their F/M/T can be used across multiple healthcare settings [34, 41], for nursing-specific settings [13, 43] or at the micro-level of work practice [40]. Originators of Sustaining Organizational Change Framework (hereafter Buchanan’s SOCF) [25] presented a practical guide outlining a range of potential influences/factors at different levels of analysis.

Tools

Two F/M/Ts provided tools [39, 40]. Maher NHS-SM [39] includes a manual, user guide, diagnostic tools, videos, and an interactive option, all of which can be used to assess and predict the likelihood of the sustainability of change in clinical practice using a systematic approach. The Slaghuis FMIS-WP [40] includes an instrument to measure sustained changed work practices related to improvement processes, which originators have tested [40, 50].

Generalizability

The Slaghuis FMIS-WP [40], Fleiszer SHIF [13], and Frykman DCOMF [41] were all designed to guide practice and research in acute care settings. Specifically, Slaghuis FMIS-WP [40] and Frykman DCOMF [41] were designed for changing complex healthcare environments (hospitals) where high turnover and interdependence between multiple professionals often exists. The Fleiszer SHIF [13] was designed for use in diverse hospital nursing contexts at the unit/organizational level. The Fox SITF [43] was recommended for use in unspecified nursing contexts. The Maher NHS-SM [39] was designed to guide practice and research at the project/initiative level and has been tested in several non-specified healthcare settings [21] including hospitals [51], community settings [14], and globally [47]. The Buchanan SOCF [25], Racine MSI [42], and Chambers DSF [34] were designed for use in non-specified healthcare contexts for a broad range of interventions at a project/initiative level [34] or unit/organizational level [25, 42].

Discussion

This systematic review is the first to include a comprehensive analysis of healthcare sustainability F/M/Ts with a primary focus on identifying key concepts influencing the sustained use of EBPs in acute care contexts. Our search revealed the vast majority of F/M/Ts relating to sustainability were designed specifically for use in community and public health settings, which is congruent with the current literature [3, 10]. Notably, only three F/M/Ts were primarily focused on the sustainability of EBPs within acute care settings [13, 40, 41], and five were recommended for use in non-specified healthcare organizational/settings [25, 34, 39, 42, 43]. Recommended target domains for use across disciplines imply general learning can be gathered to inform sustainability for practice and research using an interdisciplinary approach. Addressing sustainability challenges from a variety of theoretical perspectives and disciplines is equally pivotal to understanding this concept in acute care as reported in other healthcare sectors [3].

The two most commonly described constructs cited in the F/M/Ts for sustainability were

“continued delivery or use” and “evolution or adaption” constructs. The prominence of these constructs emphasizes the continuous use and evolutionary nature of sustained EBPs in context over time and is congruent with Moore et al.’s [1] previous developed definition of sustainability. Furthermore, this analysis provides insight into sustainability as a “process” or “stage/phase” of ongoing/continuous use of EBPs post-implementation. This finding is congruent with researchers who argue sustainability is not an all or nothing “phase or endgame” [34] nor an “outcome” [52] but rather a “process of managing and supporting the evolving EBP” overtime [34]. Some contend it is a “matter of degree of sustained change” [18, 53] to be viewed as a “continuous phase” [54] or a “continuum” [14] or a “process” [10]. The importance of this construct is consistent with a recent review [10], ultimately adding new knowledge to the current definition [1]. The shift in perspective of sustainability as a “process or ongoing/continuous stage/phase” [3, 10], together with the EBPs’ evolutionary nature and dynamic interaction/influence among the factors overtime [3, 34], highlights the complexity of planning and measuring sustainability and the need to consider how strategies for sustainment overtime differ from implementation and/or potentially overlap.

Results provide a resource of eight F/M/Ts and hypothesized factors that can be used by acute care team members planning or currently implementing EBPs with the goal of improving patient outcomes. Our synthesis of the concepts/factors revealed 37 core factors which cluster around 7 themes specifically defined by the F/M/T originators to be relevant to acute care settings. Four F/M/Ts containing all [13, 25, 34] or most [42] of the themes provide a knowledge base for practitioners and researchers to evaluate the sustained use of EBPs within their acute care setting. Four themes align with those deemed useful in any setting by Lennox et al. [10] (e.g., (i) initiative design and delivery = inner processes, (ii) people involved = adopters, (iii) organizational setting = inner context, (iv) external environment = outer context), and three add to the current knowledge, namely, leadership and management, characteristics of the innovation, and outcomes. The equal distribution of core factors among six of the seven themes (excluding outcome) signifies the relative importance of each theme for the sustainability of EBPs in acute care. Notably, several factors support the conceptualization of sustainability as “a dynamic construct that allows for adaptation in response to new or changing populations, evidence, policies, or other contextual influences” [3]. The combined contextual factors (57% or 21/37) influencing sustainability related to acute care contexts include (i) the integration of four layers of context factors influencing the sustained use of complex organizational change practices (e.g., individual, interpersonal relationships, internal context, wider infrastructure system) [41]; (ii) attention to the complexity, multi-layered, ever-changing organizational setting [13, 25, 34]; (iii) the adaptability of the innovation/EBP to context [13, 34, 39], and (iv) the dynamic process of routinization of innovations/EBPs as a source of change [40]. Arguably, contextual factors impacting sustainability within and among departments or sites will likely provide insight into why the sustained use of EBPs may vary within the same acute care setting. In turn, this likely will affect the strategies needed for sustainment.

Differences amongst F/M/Ts lie in the overall structures, the degree of refinement, substantiation to date, and identified gaps. Each F/M/T reflects a different conceptualization of sustainability evident in the varied schematics. The use of vague/minimal terminology defining concepts/factors and their relationships increases the potential for multiple interpretations. Sustainability outcomes were depicted in three F/M/Ts as a range (e.g., decay to sustainability to development [25], a spectrum from high to nil [13] or as an ongoing stage/phase of implementation [41]). The Chambers DSF [34] defined outcomes as the “continuation of intended benefits.” The outcome theme is not explicitly defined in the remaining four F/M/Ts [39, 40, 42, 43]. Consistent with other researchers [3], we recommend future inquiry focus on articulating sustainability outcomes.

Identified gaps among the eight F/M/Ts were revealed by examining their concepts/factors and tools. Variation existed related to the inclusiveness of each factor and labeling of themes. For example, the absence of any type of ‘financial factor’ in the Racine MSI [42] to guide stakeholders offers little insight into how this factor influences sustainability. Additionally, Fleiszer SIHF [13] did not include a separate “adopter theme” but instead recognized the role of leadership and management as distinct. A lack of focus on facilitation as a factor either explicitly or implicitly or its inclusion and the perceived need for it is not evident in most F/M/Ts [25, 34, 39, 41, 42], except the Fleiszer SIHF [13]. Originators of the Chambers DSF [34] were distinct in their acknowledgement of the “dynamic relationship” between three “changing” concepts (innovation, practice setting, broader system), their focus on “benefits beyond helping patients” and the “fit of the innovation” with existing routines/processes. Despite this acknowledgement, the potential risk of overlooking the impact on patient outcomes has been recognized [21]. Slaghuis FMIS-WP [40] originators claim their framework is part of a larger unpublished framework. Lastly, only two of the F/M/Ts included tools to measure changed work practices [39 40]. To date, minimal evidence for instrument reliability and validity is available for these tools.

Strengths

This systematic review is the first to include a comprehensive analysis of healthcare sustainability F/M/Ts for EBPs with a primary focus on acute care context. Seven themes primarily related to acute care were identified, four that align with a current review [10], and three that add to current knowledge (e.g., characteristics of the innovation, leadership and management, and outcomes). By identifying factors and themes/constructs relevant to acute care settings, this work has the potential to aid sustainability for those planning or currently implementing EBPs. The analysis offers insight into sustainability as a “process” or “ongoing stage of implementation” adding to the current definition. For the first time, factors (mechanisms) influencing the sustainability of behavior changes in an acute care setting (see Table 5) are integrated into a synthesis adding to the current knowledge base [41]. Additionally, the modified theory analysis criteria can be used as a tool to guide practitioners, researchers, and students in the appraisal of emerging or existing F/M/Ts, related concepts, and factors.

Limitations

There are limitations to consider when interpreting the results of this review. First, a systematic review was conducted for conceptual F/M/Ts related to the sustainability of healthcare EBPs from January 1, 2015, to July 3, 2018. Frameworks/models/theories prior to these dates were identified from two existing knowledge synthesis, dated 1946 to March 2017 inclusively. Inclusion criteria varied within each synthesis, and therefore, there is a risk some F/M/Ts may have been missed. Second, the new systematic review, designed to identify recently published F/M/Ts included four key databases, known to focus on healthcare and/or implementation science, among the 14 combined databases used within the two syntheses. There could be sustainability F/M/Ts in databases restricted to the social sciences or organizational management literature that may have been missed. However, healthcare was the primary focus. Third, the qualitative analysis of the main themes and related factors was conducted independently by one reviewer, then analyzed/reviewed by coauthors. Analysis using a deductive approach might draw different conclusions. Lastly, interpretations made as part of the theory analysis are based on the reviewers’ subjective appraisal [36]. These items are clearly marked in Table 2.

Conclusion

Sustainability is an emerging field of study. Given the ever-changing nature and complexity of acute healthcare settings and related costs, it is imperative practitioners and researchers consider the use of sustainability F/M/Ts to guide their practice and inquiry to ensure EBPs are sustained effectively, continue to inform clinical decisions and contribute to improved patient outcomes. Principally, selecting one of the eight sustainability F/M/Ts proactively to plan, evaluate and interpret findings is recommended. Then consider the context level for F/M/T use, specify the goals of sustainability, and determine if the concepts and factors listed apply [55]. We also recommend future inquiry adopt the use of mixed methodologies to explore the complex relationship between implementation factors and outcomes (including sustainability), and determine their level of influence using Proctor’s Framework [56]. Additionally, using a theory analysis approach to examine F/M/Ts containing both implementation and sustainability could provide new insight into the relationship of factors over time (e.g., early, mid-process, and long-term) and/or the potential impact of implementation on the sustainability phase.