Background

Implementation science has emerged as a vital, multidisciplinary research field in the wake of the evidence-based movement [1]. Thus far, research in the field has focused mainly on identifying factors affecting the initial uptake and use of evidence-based practices (EBPs). Less attention has been given to the sustainment of implemented practices—i.e., continuous evidence-based practice (EBP) use, as intended, over time in ongoing operations, often involving adaptation to dynamic contexts [2,3,4]. Many EBPs are adopted, only for their benefit to wane [5,6,7]. The declining quality, intensity, and comprehensiveness of EBP use following implementation are well-documented [8,9,10], but how and why EBP use is sustained remains unclear.

There are several reasons for the limited knowledge about EBP sustainment. First, there is a lack of conceptual clarity in the literature [11]: Researchers use a range of terms to describe sustainment, including sustainability, which is a related, but distinct, term that refers to preparedness for sustained use or the characteristics of a new practice which will enhance its sustainment [12, 13]. Other terms that may reflect an ongoing process expected to result in sustainment include, for example, continuation, durability, institutionalization, sustained use, and routinization [8]. Hence, it is possible that the same underlying concept (i.e., sustainment) is described using different terms (synonymy) or the same key terms might be defined in different ways (polysemy). The use of imprecise concepts and terms makes knowledge exchange and learning on this topic difficult. Second, there are methodological challenges to studying sustainment: The post-implementation duration required to achieve sustainment is unclear [8], and the period required for assessing sustainment may exceed grant funding periods. Third, knowledge regarding EBP sustainment may be limited by the lack of an agreed-upon theory, model, or framework (hereafter referred to in combination as TMFs).

Using TMFs to conceptualize sustainment could alleviate issues related to synonymy and polysemy and offer a structure for organizing and comparing findings across study settings. In addition, TMFs could help guide all phases of sustainment research and practice, including assessment of the form and degree of sustainment, identification of determinants of sustainment, selection of strategies for promoting sustainment, and evaluation of sustainment-related outcomes [14]. In particular, theories (in contrast to models and frameworks, which do not specify theoretical relationships in a way that explains how or why EBPs are sustained) are needed to suggest strategies for promoting EBP sustainment [2, 13].

The extent to which TMFs are used to advance understanding of sustainment is unclear. We consolidated knowledge from published reviews of healthcare sustainment studies to identify TMFs with the potential to conceptualize sustainment, evaluate past uses of these TMFs in sustainment studies, and assess the TMFs’ potential contribution to the development of sustainment strategies. A recent systematic review by Penno and colleagues identified and analyzed existing TMFs that focus on the sustainability of EBPs in specific healthcare settings [15]. Penno and colleagues’ review examined the concepts and factors associated with sustainability within the TMFs (i.e., what influences sustainment). Our review complements Penno and colleagues’ review by identifying TMFs that explain the relationships included among constructs to inform sustainment strategies (i.e., why constructs are thought to influence sustainment). We conclude by recommending TMFs that have the greatest potential to advance understanding of sustainment in future research and thereby may contribute to identifying potential strategies for EBP sustainment.

Method

Search strategy

To identify TMFs that have been used in healthcare sustainment research, we drew upon recently published reviews of sustainment studies. To identify reviews, we followed an approach similar to that of Moore et al.’s search for knowledge syntheses of sustainability in healthcare interventions [16]: We used the PubMed search filter for reviews to identify articles with the terms “sustainability,” “sustainment,” “durability,” “institutionalization,” “routinization,” “continuation,” or “sustained” in the title and published in the past 10 years. We used this range of terms to account for the common use of different terms to describe the concept of sustainment [8].

Inclusion criteria

To be included, we required articles to (1) be written in English, (2) be published in the past 10 years, (3) review articles of sustainment studies, and (4) report on the use of TMFs among included studies.

Study selection

Three authors selected records for inclusion in the study. These authors conducted title, abstract, and full-text review, searching for inclusion of sustainment as a key construct of interest. The three authors resolved discrepancies through discussions, and they reached consensus. Two authors then reviewed the full text of the remaining articles, confirming evidence of reviews of sustainment studies with reports of TMFs in each record.

Data abstraction and analysis

TMFs used in sustainment studies

We reported the frequency with which included articles used a TMF for conceptualizing sustainment, identified all TMFs used by the studies included in the selected review articles, and reported the prevalence of the TMFs across studies. Although they reported on the use of TMFs in studies they included, Shigayeva and Coker (2015) [17] did not indicate the number of included studies that used TMFs, so we reviewed all 108 empirical studies included in Shigayeva and Coker (2015) [17]. From those studies, we abstracted information regarding whether a TMF was used and, if so, which TMF. Two authors independently reviewed 20% (n = 22/108 empirical studies included in Shigayeva and Coker (2015) [17]) of the articles to ensure reliability in abstraction; a single investigator abstracted information from the remaining articles. We excluded articles reporting framework development.

Relevance of TMFs for understanding sustainment

To evaluate the relevance of TMFs for understanding sustainment, we evaluated the TMFs using four criteria: degree of familiarity of the TMF to researchers, operationalized in terms of frequency or usage, and three additional criteria from the Theory, Model, and Framework Comparison and Selection Tool (T-CaST) [18], a user-friendly tool designed to help select TMFs for implementation research. T-CaST guides researchers through rating candidate TMFs’ performance with respect to four domains: usability, testability, applicability, and acceptability. Our study objectives were to identify TMFs that could advance understanding of sustainment and contribute to identifying strategies for sustainment. To achieve these objectives, two authors used T-CaST to rate the performance of TMFs identified in the review from their own perspectives with respect to the following criteria:

  1. (1)

    TMF is familiar to key stakeholders (i.e., implementation researchers). We defined familiarity as being used in two or more articles.

  2. (2)

    TMF provides an explanation of how included constructs influence sustainment and/or each other (i.e., TMF hypothesizes relationships among constructs rather than simply listing them).

  3. (3)

    TMF includes meaningful, face-valid explanations of proposed relationships (i.e., hypothesized relationships among constructs are logically consistent and plausible).

  4. (4)

    TMF includes sustainment as an outcome.

Application of the first criterion left 28 TMFs we then evaluated in terms of criteria (2)–(4).

T-CaST rates TMFs 0, 1, or 2 with respect to relevant criteria, where 0 = TMF does not fit criterion, 1 = TMF fits criterion moderately well, and 2 = TMF fits criterion well. Thus, after limiting TMFs to the 28 cited by two or more included articles, a TMF could score from 0 (TMF does not fit any selected criteria well) to 6 (TMF fits criteria well). We excluded nine of the 28 remaining TMFs because they were too broad in scope (e.g., “theories of organizational change and innovation”), too vague to identify (e.g., “intervention theory”), or were not TMFs (e.g., “child survival sustainability assessment”). Two authors, trained implementation scientists, independently coded nine TMFs and resolved discrepancies through discussions until they reached consensus; they then individually coded the remaining 10 TMFs.

Results

Search results

This search yielded 733 publications. Of these, 709 publications were excluded because they were not published in English, were not published in the past 10 years, did not review sustainment studies, and/or did not report on TMF use among included studies. A subsequent review of article titles and abstracts narrowed this down to 24 publications deemed relevant. Upon full-text review, 15 additional articles were excluded because, upon closer inspection, they did not meet inclusion criteria. There remained nine review articles which contained information about 648 empirical studies of sustainment in healthcare settings. This process is summarized in Fig. 1 [8, 17, 19,20,21,22,23,24,25].

Fig. 1
figure 1

PRISMA 2009 flow diagram

Full size image

Description of included reviews

Table 1 displays characteristics of included reviews (systematic, scoping, and literature reviews) spanning multiple fields and disciplines, such as global health [27, 28], chronic disease [19], and communicable disease [29]. Collectively, the reviews covered from 1979 to 2017. Four reviews had indicated the quality appraisal of the articles. Six of the nine reviews were explicitly limited to English language articles published. Four of the nine reviews reported formally assessing the quality of the included articles.

Table 1 Reviews of sustainment studies
Full size table

TMF use

Seven of the nine identified reviews reported the proportion of included studies that explicitly used a TMF [8, 17, 19,20,21,22,23,24]; an estimated 194/478 (41%) of the studies included in these seven reviews used a TMF to study sustainment (see Table 1 for details on estimate).

The remaining two reviews focused exclusively on articles that used TMFs to study sustainment [26, 29]. For example, Lennox et al. [26] reviewed the literature for publications reporting the use of TMFs (i.e., models, checklists, tools, processes, strategies, conceptualizations, and frameworks) and found that 37% (23/62) of the included studies did not report using a theory, although the included studies may have used models and/or frameworks.

Table 2 reports the 76 unique TMFs cited in the 648 studies included across the 9 reviews. The most frequently cited TMFs were diffusion of innovations (n = 16), ecological theories (n = 10), complexity theory (n = 10), and normalization process theory (n = 6). Most of the TMFs that were cited across the 9 reviews (48/76) were only used by one of the 648 studies.

Table 2 TMF cited in included studies
Full size table

Relevance of TMFs used in sustainment studies

The TMFs that we scored received ratings ranging from 2 to 5 (see Table 3). The TMF with the highest score was institutional theory, which scored 5 out of 6. TMFs scoring 4 out of 6 included the model of institutionalization [31], diffusion of innovations theory [32], open systems theories [33], normalization process theory [29], organizational learning theory [34], the health belief model [27], network theory [35], the theory of planned behavior [28], the organizational sustainability framework [36], and the theory of organization routines [37].

Table 3 Relevance of sustainment TMF
Full size table

Table 3 shows our evaluation of the relevance of the TMFs for conceptualizing sustainment. As can be seen from the bottom row in Table 3, compared to other rating criteria, the TMFs performed best on criterion 2, inclusion of meaningful, face-valid explanations of proposed relationships (total T-CaST score across TMFs, 29). Notably, however, the dynamic sustainability framework only scored 1 on this criterion because the framework shifts between empirical statements about how sustainability occurs and normative ones about what change agents should do to sustain EBPs.

The TMFs performed worst with respect to criterion 3 (including sustainment as an outcome). For example, normalization process theory, which received a score of 1 on this criterion, proposes face-valid relationships among constructs, but the proposed antecedents to sustainment were difficult to distinguish from sustainment itself.

The TMFs also showed inconsistencies in their explanations of how constructs influence sustainment and/or each other (criterion 1; total T-CaST score across TMFs, 19). Several TMFs (e.g., open systems theories, organizational learning theory, complexity theory, and ecological theories) lacked discrete constructs that might be operationalized as antecedents to sustainment.

Discussion

In our review of 9 reviews, we found that TMFs are underused in sustainment research. This finding parallels previous reports of the underuse of TMFs in implementation science more broadly [38,39,40]. In most of these reviews, which spanned multiple disciplines and topics, fewer than half of included studies reported using a TMF to conceptualize sustainment. This may reflect studies’ focus on understanding whether an intervention was sustained, rather than determinants of sustainment [26]. As the field shifts from accumulating evidence of poor EBP sustainment to understanding determinants of sustainment, lack of TMF use in sustainment studies represents a missed opportunity to realize the benefits of using TMFs, including their potential to advance a shared understanding of how and why EBPs are sustained [14]. Among those studies that reported using a TMF, there was little convergence on which of the 76 TMFs that we identified in sustainment research to date were used. Indeed, 48 studies which used a TMF did not appear in any of the other studies. This lack of convergence may contribute to a fractured understanding of sustainment across studies, settings, and fields and may retain concerns regarding synonymy and polysemy. Improved reporting and use of TMFs across multiple studies should improve understanding of this critical topic.

Of the 28 TMFs (Table 2; TMFs 1–28) used more than once across the 648 studies, 11 TMFs received a T-CaST score of 4 or 5 out of 6 possible points, suggesting potential relevance to understanding sustainment. Nonetheless, the information underlying these quantitative scores warrants some qualitative exploration. We found that sustainment was seldom the outcome of interest in the TMFs used in sustainment studies. This finding may reflect a shortage of TMFs that specifically target sustainment, lack of researcher’s familiarity with sustainment-focused TMFs, doubts about their utility, or a preference among researchers for broader TMFs. Whatever the cause, the lack of attention to sustainment as the outcome of interest limits the TMFs’ ability to advance our understanding of sustainment. Our understanding of sustainment may be enhanced by converging upon TMFs that include sustainment as the outcome of interest or explicitly acknowledging the limitations for sustainment research of TMFs that include a different outcome. Few of the TMFs explained the mechanisms through which included constructs influence sustainment. Without a strong explanation of the causal pathways that lead to sustainment, the TMFs make limited contributions to identifying and developing potential sustainment strategies. Identifying causal pathways that lead to sustainment implies conceptually sound sustainment strategies. For example, a TMF that suggests that organizations are subject to pressure from norm-setting institutions to sustain a particular practice implies influencing institutional policy as a potentially high-leverage strategy. Further, despite evidence suggesting that some conditions are unique in influencing sustainment, few TMFs distinguished determinants of sustainment from determinants of other, related outcomes (e.g., adoption, implementation) [41].

Although 11 TMFs scored fairly well based on our criteria, each had significant limitations. For instance, normalization process theory (NPT), which describes the social processes leading the routinization of EBPs, explains the relationships among included constructs; however, raters had difficulty disentangling NPT’s concepts of “integration” and “embeddedness” from its core constructs (e.g., coherence, cognitive participation, collective action, reflexive monitoring). Although normalization in NPT is related to (or perhaps even synonymous with) sustainment, articles that presented NPT did not offer a clear conceptual distinction between “integration” “embeddedness” (i.e., sustainment) and implementation. Another class of TMFs, open systems theories, which broadly propose that organizations are strongly influenced by their environments, offered a meaningful, face-valid explanation of sustainment. However, open systems theories did not include discrete constructs, thereby limiting our ability to operationalize or falsify the theory [14]. Further, open systems theories are an umbrella that encompasses several theories, not a singular TMF.

Institutional theory [42,43,44], which had the highest T-CaST score (5/6), proposes that organizations sustain practices in response to three key pressures related to maintaining legitimacy: mimetic (e.g., mimic other organizations’ behavior), coercive (i.e., meet expectations from organizations providing critical resources, including funds and legal permission to operate), and normative (e.g., act in accordance with professional norms). In other words, the institutional theory suggests that organizations may be more likely to sustain practices if they are under mimetic, coercive, or normative pressure to do so. Further, the institutional theory suggests several moderators of the influence of mimetic, coercive, and normative pressure on organizations’ sustainment of practices. For example, the institutional theory suggests that organizations may be more inclined to mimic other organizations (i.e., sustaining or failing to sustain a practice) in the face of uncertainty and that organizations may be more inclined to cede to coercive pressure from organizations on which they are more dependent.

Understanding of sustainment may be enhanced by understanding the role of mimetic, coercive, and normative pressures and potential moderators that institutional theory proposes. It may also be enhanced through testing strategies for sustainment that the institutional theory suggests: Sustainment may be promoted by assessing the pressures to which an organization may be subject with respect to sustaining a given EBP and then leveraging pressures for sustainment and deflecting those for discontinuation of the EBP.

Despite its potential contribution to understanding sustainment, institutional theory offers limited insight into potentially influential factors at levels other than the outer setting, including the inner setting and individual levels. To some extent, neo-institutional theory [45] may address this limitation. For example, neo-institutional theory suggests that individuals make choices because they see no alternative. Psychological theories, such as theories concerning habits [46], may also offer insight into individual-level determinants of sustainment. Another important critique of the institutional theory is that the institutional pressures that it conceptualizes may be sufficient for superficial organizational change, but sustaining practice may require more intrinsic motivations [47, 48]. The limitations of the institutional theory—or any given TMF—suggests that combining multiple relevant TMFs may be an optimal approach to conceptualizing a construct as complex as sustainment.

Notably, the TMFs that appear to be most relevant to sustainment focus on social dynamics among individuals and organizations. Although sustainment is often inextricably linked with economic resources for delivering EBPs, economic theories for explaining sustainment did not emerge in this review [8]. Indeed, stable financial resources are necessary for sustainment, but our findings highlighting the importance of social processes and relationships suggest that economic and rational choice theories are likely insufficient for explaining whether and how EBPs are sustained in such complex care settings.

A limitation of our study is that there may be other TMFs that are relevant for understanding sustainment but have not yet been cited in healthcare sustainment research. For example, structuration theory [49] explains how social systems create and reproduce structures that uphold and discontinue EBPs. The structuration theory has the benefit of a multilevel orientation to understanding EBP sustainment. Similarly, the contingency theory suggests that EBP sustainment depends on conditions that incentivize sustainment in the face of competing courses of action. Future efforts should assess the relevance of other potentially relevant TMFs for conceptualizing sustainment. And, as the evidence base around sustainment grows, and TMFs are more widely used to support it, the field may benefit from identifying examples of sustainment studies that have effectively applied TMFs.

In this study, we treated TMFs as distinct although we recognize that there are areas of overlap of constructs across TMFs. Identifying overlap of constructs across TMFs may be aided by ongoing efforts to assess the validity and reliability of measures of sustainment-related constructs [50]. Further, as a review of reviews, our study relied upon the information that included the review provided by the authors regarding how TMFs were used in included studies. The degree to which TMFs were used can vary from mere citation to rigorous testing and linkage to specific constructs. Although this level of detail was beyond the scope of our study, we acknowledge the significance of this area for future research. Relatedly, despite the potential for reviews of reviews to leverage existing reviews to develop new knowledge, it is possible that our review of reviews [51, 52] was not entirely comprehensive, and we did not independently review methodological quality.

Another limitation of our study is that, with the exception of the studies cited by Shigayeva and Coker (2015) [17], we did not independently evaluate studies included in the remaining eight reviews. Thus, our estimate of the proportion of studies citing TMFs relies on these reviews’ findings, and the number of studies that we report may include duplicates. Consequently, for the proportion of studies citing TMFs, both the denominator and the numerator may be overestimates; however, the proportion that we report is likely valid. Although we documented assessments of quality conducted by authors of included reviews in Table 1, we did not independently evaluate the quality of published reviews. To the extent that extant reviews’ quality is limited, our findings may not accurately reflect the use of TMFs or their relevance for sustainment research. Nevertheless, our work represents an effort to consolidate existing knowledge to answer novel questions as implementation scientists increasingly appreciate the importance of sustainment.

We used T-CaST to rate the performance of TMFs identified in the review from our own perspectives. Future work should evaluate the relevance of sustainment TMFs from the perspectives of users; users’ perspectives of TMFs’ relevance for their work (e.g., whether they use TMFs for data collection and/or analysis) may enhance our understanding of TMFs’ contributions to conceptualizing sustainment.

Conclusion

Leveraging published reviews of sustainment studies, we identified institutional theory as a promising TMF for advancing our understanding of sustainment. Incorporating theories that meet the criteria that we advanced above has the potential to promote shared understanding of EBP sustainment. Shared understanding of EBP sustainment will be enhanced by repeatedly applying a few sustainment theories. To limit the synonymy and polysemy that has fractured our understanding to date, we must also agree upon operational definitions of included constructs. Then, we may be able to compare the performance of selected theories and understand the implications of sustainment research for identifying promising sustainment strategies.