Introduction

The field of implementation science has made significant advancements towards understanding how evidence-based practices, programs, and policies are adopted and implemented in different clinical and community settings [1, 2]. While there is a wide range of evidence-based interventions, it is still the case that less is known about successfully implementing and sustaining these programs in a range of settings [3, 4]. Implementation science has emphasized the development and use of theories, models, and frameworks (TMF) to guide and understand translation of research into practice [5, 6] and has prioritized understanding how health care and public health programs interact with context and both implementers and beneficiaries to influence adoption, engagement, equity [7], implementation, reach, effectiveness, and sustainment [8,9,10,11].

There is increasing interest [12, 13] but still limited understanding of which contextual factors [3, 14,15,16,17] have an impact on the initial uptake; equitable implementation, reach, effectiveness; and sustained use of complex health interventions [18] in a variety of clinical and community settings, which inhibits the translation of research into practice [10]. More specifically, there is a need to document and understand the impact of the dynamic context in which interventions are integrated [19,20,21,22,23]. The Practical, Robust Implementation and Sustainability Model (PRISM) was developed to fill this need using key concepts from research on and models of chronic care, the diffusion of innovations, quality improvement, and measures of population-based effectiveness for translating research into practice [3, 4].

PRISM was developed as a contextually expanded version of the broadly used Reach, Adoption, Effectiveness, Implementation, and Maintenance (RE-AIM) framework, and as a pragmatic and intuitive model to improve translation of research-tested interventions into clinical and community practice and ultimately population health impact [24]. PRISM can be considered a determinant and evaluation framework in the classification suggested by Nilsen [25] and, as illustrated in Fig. 1, considers how perspectives of the program, policy, or intervention design; the external environment; the implementation and sustainability infrastructure; and the characteristics of multiple levels of “recipients” (e.g., implementers, beneficiaries) influence program adoption, implementation, and maintenance. Within the program or intervention design domain and the recipient domain, PRISM incorporates the perspectives of both the patients (recipients or participants) and the organizational members at different levels of influence (e.g., top leadership, mid-level managers, and frontline staff) to help understand what factors within and external to the organization need to be considered and addressed for successful implementation and sustainability of complex interventions [26]. A relatively unique aspect of PRISM, compared to most other models addressing context, is explicit inclusion of the domain of the “implementation and sustainability infrastructure” (e.g., clear roles and responsibilities related to this program; timely data reporting capabilities, strong communication channels). Inclusion of this domain was based on experience in healthcare settings in which those settings that were able to implement and sustain programs most consistently had the type of infrastructure and support resources noted in this domain. The outcome measures hypothesized to be influenced by PRISM contextual factors include RE-AIM outcomes of reach, effectiveness, adoption, implementation, and individual- and organizational-level maintenance (www.re-aim.org) [4, 27].

Fig. 1
figure 1

The Practical, Robust Implementation and Sustainability Model

Since 2008, PRISM has been used across diverse topics, populations, and settings. The purpose of this citation analysis and scoping systematic review was to explore and describe how PRISM has been used, how it has been operationalized, for what populations and settings, and with what outcomes. Specifically, the review aimed to (1) summarize use of PRISM to date; (2) identify and address conceptual and methodological issues to potentially improve research in this area; and (3) provide recommendations for research teams and practitioners about using PRISM to improve translation of evidence-based interventions in a variety of clinical and community settings.

Methods

We conducted a systematic scoping literature review to determine how PRISM has been used. A literature search was conducted using three databases PubMed, Web of Science, and Scopus. The search algorithm included ((“Practical” AND “Robust” AND “Implementation” AND “Sustainability” AND “Model”) OR (“Practical, Robust Implementation and Sustainability Model”)) OR (“Practical Robust Implementation and Sustainability Model”).

The date range was from 2008 (publication date of the original PRISM manuscript) through September 2020. Further, following the approach described by Bergstörm et al. [28], we conducted a reverse citation search of the original PRISM article (i.e., index article) in PubMed reviewing all papers that cited this index paper to identify studies that used PRISM [4]. Only peer-reviewed articles were included in this review. First, duplicates were removed from the results. Full-text articles were obtained for all articles identified in the searches and were independently reviewed by two researchers (BR, JC) to determine eligibility. Articles were excluded if they did not cite or specify PRISM or were not yet published in a peer-reviewed journal. We adapted and refined categories previously developed by Field et al. [29] to classify the extent PRISM was used in the articles (i.e., referenced the model, framing a field or in a review, informed by the model, directed by the model, or integrated the model into the study design) (Table 1). Classification of articles per the Field et al categories [29] was done by two members of the team (BR, JC). Discrepancies in classification were resolved through consensus discussion and the involvement of the other members of the abstraction team (CG, BG). Because one purpose of this review was to explore the degree to which PRISM propositions were supported (i.e., relationships between PRISM contextual factors and RE-AIM outcomes), only studies that met the criteria for ‘integrated use,’ which required explicit linking of study activities to PRISM domains and/or subdomains, of the PRISM framework were included [29]. Following the database searches and initial eligibility screening, the list of included articles was circulated to an expert panel of implementation science researchers via a national listserv to obtain recommendations for additional articles that described an integrated use of PRISM not identified in the original search.

Table 1 Categories and definitions for the use of the Practical, Robust Implementation and Sustainability Model (PRISM) in published peer-reviewed literature, adapted from Field et al. (2014) [29]

All articles that were categorized as integrating PRISM into their study design (i.e., integrated use [29]) were selected for full abstraction. Articles that described the same study were bundled as primary and companion papers and were abstracted together. Data abstraction was conducted by four researchers (BR, BG, CG, and JC). Data abstraction forms were developed based on prior work of the co-authors in reviewing the use of the RE-AIM framework [30, 31]. We expanded these abstractions forms to include PRISM contextual domains of characteristics of the intervention, characteristics of the recipients (including the multi-level organizational setting), implementation and sustainability infrastructure, and external environment. The abstraction form was then iteratively refined as the team tested it on the abstraction of four studies and then programmed into a REDCap database [32, 33]. The rest of the eligible articles were assigned to pairs of researchers to independently abstract. The pairs of coders met to review their abstractions, resolve discrepancies, and reach consensus. Consensus, rotation of abstraction teams, and iterative group discussions were used to ensure the accuracy and consistency of data abstraction.

The database was used to capture information from three key domains: study characteristics (setting, design, population, etc.), PRISM contextual domains, and RE-AIM outcomes [32, 33]. Articles were categorized as addressing planning/development, dissemination, implementation, evaluation, and/or sustainment. To describe the degree (i.e., quality and extent) of operationalization of PRISM domains, we used a scale of 0-5 in which 0 indicated poor operationalization and 5 indicated excellent operationalization. Ratings were separately made for each type of use of PRISM (i.e., planning/development, dissemination, implementation, evaluation, and/or sustainment). Two abstractors per article/study rated the degree of operationalization and used a consensus approach to decide on the final rating. They also documented the rational for their rating in a qualitative manner. If articles included lessons learned about operationalizing PRISM, this information was also systematically abstracted (see Additional file 1). Key themes from lessons learned were synthesized based on the review of this data by the lead author.

Results

Included studies

Of the 237 unique articles identified, 57 were excluded for not meeting the initial inclusion criteria (Fig. 2 and Table 1). A total of 180 articles were identified as using PRISM to some degree (i.e., referenced the model, framing a field or in a review, informed by the model, directed by the model, or integrated the model into the study design). As shown in Fig. 3, the number of publications increased over time as we moved from 2008 (n=3) to 2019 (n=31). Overall increases in the proportion of directed and integrated articles were also observed (see Table 1 for definitions).

Fig. 2
figure 2

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram of systematic review of articles using Practical, Robust Implementation and Sustainability Model (PRISM)

Fig. 3
figure 3

Number of articles using the Practical, Robust Implementation and Sustainability Model (PRISM) across years (2008 through September 2020*)

Of the 180 initially identified articles, 29 (16.1%) included PRISM as a potential means for understanding the current state of research or advancing research in a specific field (i.e., framing), and 14 articles (7.8%) included PRISM in a review of frameworks (i.e., reviews). Eleven articles (6.1%) mentioned that PRISM informed their study without specifying how the model was used (i.e., informed), and another 23 articles (12.8%) provided more details as to how PRISM was used in developing their study but lacked specific details about how the domains were operationalized (i.e., directed). Seventy-two articles (40.0%) referenced PRISM in either the introduction or discussion as an alternative model that was not used in their study methods (i.e., referenced). Finally, 31 articles (17.2%), representing 23 unique studies, were determined to integrate PRISM into their study design and were abstracted (Fig. 2) [29]. One additional article was added to the final abstraction as it was identified as a later published result for an initially included protocol article (the total number of publications in the integrated category was n=32). For the full list of included papers, their categorization by use, and search source, see Additional file 2.

Study characteristics

Of the 23 abstracted studies, 17 were conducted in the USA, and only three were conducted in low-middle income countries. Thirteen studies addressed primary or secondary prevention as the main topic area followed by seven studies related to mental health, four on veterans’ health, three on cancer, two on infectious disease, two on reproductive health (including OBGYN), one on clinical guidelines, and eight were coded as other topics. Eighteen of the studies addressed health equity, as defined as “reducing and ultimately eliminating disparities in health and its determinants that adversely affect excluded or marginalized groups” [34] (Table 2). Most frequent operationalization of addressing health equity was through the focus of the study on underserved populations.

Table 2 Summary of original studies included in a systematic review of the use of the Practical, Robust Implementation and Sustainability Model (PRISM)

The target populations for the studies were mainly focused on the providers or delivery agents (n= 20), but most also included the setting level (n=16) and patient or community (n=15 for both groups). One study targeted payors and another targeted policy makers (Table 3). Most studies that integrated PRISM were conducted in the clinical out-patient setting (n=13) with a minority of studies being conducted in community settings (n=4). Additional settings identified were clinical in-patient (n=3), schools (n=2), or as part of a national health initiative (n=2). Two studies were conducted in other settings (i.e., a care transition context and a regional evaluation). The study design varied greatly across studies with seven being stand-alone protocol papers, five being randomized controlled trials, four quasi-experimental pre/post designs, three case studies, one cohort study, and one narrative piece (Table 3). Of note, nine were classified as having a design different from above such as qualitative implementation study, cross-sectional study, and adaptation study. The type of study varied from nine that were specifically implementation focused; six that were type 2 hybrid effectiveness-implementation; three efficacy studies; and two were effectiveness studies. Five studies were classified as pilot studies and five did not fit into any of the study type categories. The most frequently used methods used were qualitative (n=8), mixed methods (n=7) and multi-method (n=6). Only two studies reported strictly quantitative methods. In terms of socioecologic level, 14 studies used setting, four individual, three delivery agent, and two had multiple levels at which randomization and/or data collection happened.

Table 3 Summary of domain use of Practical, Robust Implementation and Sustainability Model (PRISM) and Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) frameworks (N=23)

PRISM use

PRISM was used primarily for study evaluation (n=19), but also for planning and development (n=10), and implementation (n=10). Obviously, many addressed multiple implementation stages. Four studies addressed sustainment, and only one study addressed dissemination (Additional file 3). The research team identified two studies, Ayele, R. A., et al. [36] and Leonard et al. [48], that used PRISM for planning and development, implementation, and evaluation. Woodbridge, M. W., et al. [61] was found to be an exemplar in evaluation and sustainment. There were no exemplar studies identified for dissemination.

Table 3 provides further details on the extent to which studies included the various context domains of PRISM and Table 4 shows what sub-domains were addressed. Twenty-two studies included perspectives on the Program/Intervention from the organizational lens, with nearly three-quarters addressing barriers for frontline staff. There were 15 studies including Program/Intervention from the patient perspective focusing on patient barriers, being patient-centered, and addressing service and access. Recipient characteristics at the organizational level (n=18) focused on management support and communication as well as shared goals and cooperation. Recipient patient characteristics reported (n=16) were primarily related to demographics. Most studies (n=21) included Implementation and Sustainability Infrastructure and performance data was the most frequent sub-domain reported within this factor. Studies addressing the external environment domain (n=19) were primarily focused on community resources.

Table 4 Inclusion and operationalization of Practical, Robust Implementation and Sustainability Model (PRISM) elements across all articles included in review by dimension and evaluation criteria: 2008–2020

In terms of the operationalization of the PRISM context domains, most domains were operationalized using qualitative methods followed by quantitative, mixed methods, and muti-methods approaches and only one study used a narrative approach. The specific distribution of the methods for each domain is provided in Table 4. Specific techniques used for data collection about each domain were not as consistently reported but a general review of themes identified interviews, focus groups, and surveys as the most common approaches of data collection with a few studies reporting the use of more innovative approaches such as brainwriting or process mapping) (data not shown). When rating the degree of operationalization of the PRISM domains across the various stages of use of PRISM (i.e., planning/development, dissemination, implementation, evaluation, and sustainment) we found that average ratings were similar across the stages and ranged from 2.3 to 3.1 on the scale of 0 to 5 where 0 was poor and 5 excellent. Operationalization of the PRISM domains for supporting sustainment was rated on average the lowest and implementation the highest. Lower ratings were due to a lack of specificity on how the PRISM domains were operationalized for the given stage.

Only eight included studies provided specific lessons learned about the contribution of PRISM to planning, implementation, and/or evaluation (Table 2). The following key themes were identified when reviewing the lessons learned: (1) PRISM was useful in supporting pre-implementation planning and adaptation of interventions to multilevel contexts [42, 45, 48]; (2) PRISM was feasible and useful in supporting the implementation of interventions in multilevel contexts [26, 46]; and (3) PRISM with its multilevel consideration of determinants was helpful in organizing multilevel influencers of implementation [59, 60].

There were six studies that included all six contextual domains. Despite the RE-AIM outcomes being an explicit part of the original PRISM model (Fig. 1 and Table 3), only two studies directly connected individual PRISM domains with RE-AIM outcomes, and another four studies included RE-AIM outcomes without directly connecting them to the PRISM context domains. Even the two papers in which a connection between the context domain and RE-AIM was made, the studies did not attempt to explicitly test relationships between the contextual factors and RE-AIM outcomes. Of the six studies that included RE-AIM, all but one addressed implementation, three addressed reach, three adoption at the staff and/or setting level, one included effectiveness, and one maintenance at the individual and setting level. One included all RE-AIM dimensions except effectiveness and no studies included all five RE-AIM outcomes.

Few studies combined PRISM with another TMF (n=7) and even fewer described the adaptation of PRISM for the study context (n=2). TMFs that were used in combination with PRISM included the expanded framework for reporting adaptations and modifications to evidence-based interventions (FRAME) [65], the learning evaluation [66], the Social Ecological Model [67], and Lean Six Sigma [68]. Adaptations to PRISM focused on the modifications of domains to align with the context of the study. Goplan et al. adapted and tailored PRISM to reflect the context of their 4Rs and 2Ss for strengthening families intervention [45] and Knudsen et al. modified how the implementation and sustainability infrastructure and external environment are structured based on their community and collation partners [47].

Discussion

PRISM was developed to specify multi-level contextual factors related to implementation research outcomes included in the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework [4, 27, 69]. Our review summarized published reports of use of PRISM from 2002 to 2020 and attempted to report on findings related to relationships between PRISM contextual factors and RE-AIM outcomes. Nearly 200 articles referenced PRISM, used it to inform or direct their research, or integrated it within the research design—with the numbers growing over the past decade. The studies reflected a diverse body of literature that applied PRISM across the stages of intervention planning, implementation, and evaluation. While 23 studies (represented by 32 articles) integrated PRISM within their research methodology, very few included PRISM contextual factors and RE-AIM outcomes together. None made an explicit attempt to test the connection between contextual domains and RE-AIM outcomes, making it difficult to determine the relationships between specific contextual constructs and unique implementation outcomes.

We noted that the purpose and use of PRISM varied across included studies and the level of specificity on how PRISM was integrated and measured was often lacking, making it difficult to compare findings across studies. Similarly, it was challenging to rate the degree and quality of use of PRISM as there were few studies using or reporting on the PRISM sub-domains comprehensively. More guidance on how to use PRISM especially as it relates to its sub-domains should be provided for future studies. Furthermore, measures should be linked to specific PRISM domains and sub-domains to support a more comprehensive use of the model.

Studies that have integrated PRISM in their research methodology reflect a body of literature that is relatively early in its development. As a result, few studies reported using PRISM as a framework to describe potential relationships between context and outcomes. Indeed, many of the studies have used PRISM in planning and characterizing dissemination and implementation settings and “recipients” (participants, implementers, and organizations). A large proportion of these studies examined implementation and sustainability infrastructure issues when planning for intervention implementation. Explicitly reporting on this dimension is seen as one of the strengths of PRISM (explicit focus on this construct is relatively unique among context-oriented implementation science frameworks) and conceptually is likely to be strongly related to sustainment, although this remains to be tested. We were surprised to find that 78% of the studies addressed health equity in some way. This focus was primarily operationalized by the primary focus of the studies on underserved populations. We believe that the main reason for this high percentage of health equity focus is due to the focus of the RE-AIM outcomes on representativeness which lends itself to health equity applications [20]. This trend should be further broadened to include the standard use of equity-related factors in each of the PRISM domains.

The application, operationalization, and assessment of every construct within PRISM including the implementation outcomes reflected in RE-AIM within a single study can be challenging [70]. Only a small percentage of originally identified articles met the criteria for the “integrated” use of PRISM. Given several new developments related to implications and use of RE-AIM to address equity issues, which is a key part of the PRISM framework, there is likely to be a lot more in the near future [27, 71,72,73]. Thus, it is timely to review what has been learned and to make recommendations for future applications of PRISM.

Key findings from our review are that (1) PRISM has been primarily used in outpatient clinical settings and in the US; (2) it has been used to study a variety of issues and conditions using a wide range of experimental designs and often using mixed methods; (3) most studies have reported on half or more of the PRISM domains and over half of the studies reported on at least 5 of the 6 domains; (4) PRISM contextual components most frequently assessed were the Program/Intervention characteristics from the organizational perspective, the Implementation and Sustainability Infrastructure, and the Organizational and Individual Recipient characteristics; (5) PRISM contextual components were most frequently operationalized using qualitative methods followed by quantitative, mixed-, and multi-method approaches; and (6) for the RE-AIM aspects of PRISM, the outcomes most often reported were Implementation and Maintenance.

Compared to the widespread use of RE-AIM, there has been a modest uptake of PRISM, especially until the last few years. There are multiple potential reasons for this finding including that the original article was not in a high-impact journal and was published before Open Access and citation services dramatically increased article accessibility. Other reasons likely include that the National RE-AIM Working Group until very recently has not promoted PRISM and that it has not been taught as part of the major training programs in D&I science. Finally, to date, most studies have treated PRISM and RE-AIM as separate frameworks rather than reporting them as directly related, and PRISM as being an expansion of RE-AIM [27, 71, 74]. These findings present both a need and an opportunity for future researchers to better incorporate the full PRISM framework into their studies. In this review, only six of the 23 studies included any RE-AIM outcomes, despite these outcomes being an explicit part of the original model (Table 3). More consistent inclusion of RE-AIM outcomes would also allow for more robust assessments of how PRISM domains impact implementation outcomes, including formal mediation or moderation analysis.

We found that due to the lack of consistency and specificity in the use of PRISM and reporting on the operationalization of the PRISM domains, it was challenging to synthesize information across studies on the contributions of PRISM to support planning, implementation, and evaluation. When reviewing the eight studies that identified specific lessons learned about the use of PRISM, we noted that these studies found PRISM useful in supporting planning, implementation, and evaluation efforts especially due its multilevel, contextual orientation. More comprehensive and proactive use of PRISM in future studies will allow the broader analysis and synthesis of conclusions and lessons learned.

Strengths and limitations

This review has both strengths and limitations. Limitations include that the review was limited to English language reports and to published research. It is possible that creative work and important examples of cultural adaptations using PRISM may have been missed with these decisions. The resulting 23 studies, while a large enough sample to draw conclusions and suggest directions for future application, is too small to conduct meta-analyses or draw definitive conclusions. Furthermore, our review relied on the initial classification of papers regarding the use of PRISM based on the information provided in the published papers that were identified through our multi-step search and validation process. It is possible that contacting the corresponding authors of these papers could have yielded more information about the use and resulted in different classification of these papers (e.g., ‘integrated’ instead of ‘directed’ use). This paper was also solely focused on describing pattern of use for studies that were classified as integrated use. Future papers could explore how PRISM and other TMFs have been used I n Framing and Directed studies. Finally, studies provided less consistent information on the operationalization of each PRISM domain which limited our ability to systematically report and synthesize information about this aspect. In this paper, we were able to include information about the methods used to operationalize domains and the most frequent techniques used for data collection.

Strengths of this review include following PRISMA recommendations and reporting standards; the use of multiple reviewers and coders at all review stages; and the use of the application categories previously developed by Field et al. [29] to justify and explain exclusions. We also used multiple search strategies, including reverse citation approaches and recommendations from prominent investigators- these proved effective in increasing the number of relevant articles.

Next steps and recommendations

We summarize key recommendations for research and practice directions:

For research, we recommend:

  1. 1.

    The development and validation of more quantitative measures of PRISM, especially those that meet pragmatic [75] and the PAPERS [76] criteria. This will allow greater mixed-methods research on PRISM and understanding of various linkages.

  2. 2.

    The use of common and where applicable, standardized PRISM definitions, assessments, and criteria. We note examples of mixed methods research with PRISM, including survey and qualitative interview guides as well as a new interactive PRSIM assessment and feedback tool that will soon be available on the re-aim.org website.

  3. 3.

    While preferred to advance implementation science, for pragmatic use it is not necessary to use all PRISM components or to use PRISM at all program time points (i.e., pre-implementation, implementation, sustainment [70]). When not feasible, authors should briefly and transparently state why certain components were not used or why PRISM was only used at one time point.

  4. 4.

    More investigations and transparent reporting are needed that (a) compare PRISM with other TMFs and create clear cross-walks between PRISM and other TMFs; (b) combine PRSIM with other models; and (c) adapt PRISM to diverse contexts and content areas.

  5. 5.

    Patient and community member issues involving (a) characteristics components of PRISM can be highlighted more; including especially social determinants of health and other equity-related issues; and (b) perceptions of the beneficiary of the intervention (e.g., patient-centeredness, trialability, relative advantage/etc.)

  6. 6.

    Consider using PRISM in more diverse settings including community, school, worksite, and other non-clinical contexts, especially including low resource settings and low- and middle-income countries.

For implementation practice, we recommend:

  1. 1.

    Reports on how PRISM is used with different types of implementation partners, in multi-sector research, and for team science, including the time involved and lessons learned.

  2. 2.

    Use of PRISM in logic models and to develop participatory implementation strategies that can help to address priority outcomes.

  3. 3.

    As discussed in the section on clarification, reviewing PRISM terminology to make it more user-friendly and relevant to the context of the implementation practice—for example changing some terms such as “patients” or “recipients.”

  4. 4.

    Development and usability evaluations of interactive tools and resources including videos that illustrate and guide PRISM use.

Conclusions

Although initial uptake of PRISM was slow after its initial publication in 2008, usage has increased and over 50 studies were published in 2019 and the most of 2020. While there are opportunities to further expand integrated use of PRISM and more explicit reports on linkage to RE-AIM outcomes, this review has produced learnings and recommendations for future directions. It is hoped this publication will encourage more sophisticated use and reporting on PRISM to inform both research and practice on contextual factors in implementation.