Background

The field of Implementation Science is rapidly expanding as research into the creation of evidence-based interventions has yielded innovative approaches to ameliorate various problems, particularly adverse health issues. Considerable research has focused on the dissemination of information about evidence-based interventions (EBIs), since they cannot be adopted if potential implementing organizations are unaware of them. Other research has focused on various aspects of implementation, with the understanding that once organizations have decided to adopt an EBI, the program needs to be moved into effective practice. As the field advances, numerous researchers are developing dissemination and implementation (D&I) models. A recent paper analyzed 61 D&I models and found that most models emphasized dissemination rather than implementation of interventions: 27 models focused on dissemination, 17 models emphasized both equally, and only 17 focused on implementation issues. Of this latter group, only 12 models entirely focused on implementation [1].

The level of implementation achieved is an important determinant of effectiveness outcomes. Since poor implementation outcomes can impact effectiveness outcomes, it is critical to discriminate between implementation outcomes (Are they doing the program as intended?) and effectiveness outcomes (Is it resulting in good outcomes?) [2]. Implementation in this paper means, “the use of strategies to adopt and integrate evidence-based health interventions and change practice patterns within specific settings” ([3] p.424). When evidence-based HIV prevention interventions are implemented in the “real world,” they can achieve effects similar to those in the original research [4-7]. But how EBIs are implemented, including the intervention’s core elements (required project components that embody the intervention’s theory and internal logic and most likely produce the intervention’s main effects [8,9]), has a substantial impact on their effectiveness and sustainability over time [10]. Multiple implementation studies in the HIV prevention field have found that organizations often fail to implement all required core elements or adapt them significantly [11-14].

Although some evidence-based HIV prevention interventions have been widely adopted [15-19], including the intervention studied in this paper, the Mpowerment Project (MP), the scale-up of HIV/AIDS EBIs has not been simple and the “implementation gap,” the translation of scientific knowledge to effective and efficient program implementation, has received increasing attention [20,21]. The MP is a cost-effective HIV prevention intervention EBI for young gay/bisexual men (YGBM) that was tested in randomized, controlled trials and found to be effective in reducing rates of unprotected anal intercourse [22-24]. Widespread dissemination of the MP occurred because the CDC promoted it and other EBIs to health departments and community-based organizations (CBOs) as part of the “DEBI” (Diffusion of Effective Behavioral Interventions) project that sought to move HIV-prevention related EBIs into practice [25]. The present research focuses on implementation of the MP by CBOs that obtained funding to implement it. Examining implementation of the MP is important both because it focuses on YGBM, a population that remains at high risk for HIV transmission [26], and because it is a multilevel HIV prevention approach, which has been identified as critical for HIV prevention [27] in the US National HIV/AIDS Strategy [28] and the CDC’s High-Impact Prevention efforts [29].

The overall goal of the TRIP (Translating Research into Practice) Study was to determine if providing innovation-specific capacity building assistance improved the implementation of the MP by CBOs. Research to practice models are based on the paradigm that scientists conceptualize and establish an intervention’s efficacy, and then CBOs are expected to implement it [30,31]. This transfer of knowledge approach, from scientists to community, is in contrast with community-centered models, which are based on a paradigm that emphasizes improving practice within the context of the community while considering the resources and needs of the community. This means that capacity building assistance strives to help CBOs adapt interventions to the community context [30]. TRIP’s capacity-building approach was a blend of the two models. Our intent was for organizations to implement the MP as operationalized in the efficacy trials, including using community-based participatory methods, while also adapting the intervention to their communities in ways that preserve the core elements and conceptual underpinnings of the program (which we formulated into MP’s “guiding principles”; see Table 1). In this paper “successful” or “effective” implementation (used interchangeably) mean the extent to which intervention fidelity is maintained while adapting the MP for specific communities. The underlying assumption is that fidelity to the intervention is most likely to result in outcomes that approximate those achieved in the original trials.

Table 1 Overview of the Mpowerment Project
Full size table

The TRIP study, a longitudinal project described previously [14,34], had two primary objectives: (1) to determine if an intervention for CBOs implementing the MP, called the Mpowerment Project Technology Exchange System (MPTES), would result in increases in MP fidelity across the CBOs over time, and (2) to gain an understanding of the issues CBOs experienced while running the EBI, specifically focusing on barriers and facilitators to effective implementation.

The MPTES is an integrated system of training, ongoing technical assistance (TA), implementation materials (manuals and videos), and web-based services, and collaboration with CBOs was used extensively in its development. The MPTES is based on social learning theory [35], diffusion of innovations [36], and theories and approaches to adult education [37]. Since there is a paucity of research or theory about the efficacy of different approaches to capacity building in order to improve intervention fidelity [1,3], the MPTES primarily focused on issues that had arisen in our previous work with CBOs that had contacted us for assistance [34]. The Interactive Systems Framework for Dissemination and Implementation has been developed as a heuristic to conceptualize the systems involved with moving an innovative program into practice [38]. A Prevention Support System is a key component that involves the provision of assistance to CBOs implementing an innovation. The MPTES is a Prevention Support System, and primarily targeted frontline staff who carry out the daily tasks of implementing the MP (called “coordinators”), although some information was targeted at supervisors.

As the primary organization that provided information and training to CBOs on the MP, we had access to all organizations seeking information on the MP. We provided the MPTES proactively to CBOs, and assessed each organization for two years. The goal of this paper is to examine the barriers and facilitators to effective implementation of the MP by CBOs that arose while the organizations were running the intervention during this study. See Table 1 for a brief overview of the MP.

Methods

All but 2 of the 74 CBOs implementing the MP agreed to participate. Organizations were recruited into the study through telephone calls and letters. We first obtained executive directors’ (EDs) or HIV prevention directors’ consent for the agency to participate in interviews. Then individuals we interviewed provided their own consent to participate. Interviews were conducted at baseline, when each CBO was recruited into the study and after which TA was provided; 6-months post-baseline, after which CBOs were provided with more TA; one year post-baseline, after which CBOs were offered more TA; and two years post-baseline. Since logistically the interviews at all participating CBOs could not be conducted simultaneously, we used rolling enrollment with new CBOs entering into the study continually over time until no other CBOs implementing the MP could be identified. Semi-structured telephone interviews were conducted separately with 2–4 individuals at each CBO, including the coordinators; their supervisors, who were typically the agencies’ HIV prevention directors; and 1–2 core group members (YGBM volunteers who serve as decision-makers to MP projects). Due to high job turnover, only a third of participants were interviewed multiple times over the two-year data collection period for each CBO. Participants were asked to locate an area at work where they could participate in the interviews while maintaining privacy. Participants were compensated $25 per interview, and the University of California San Francisco’s institutional review board approved the research protocol.

Data for this paper came from two sources: telephone interviews and extensive written TA fieldnotes (see Table 2). Barriers and facilitators to implementation was the overarching thematic framework used across the data analysis. Principles of Framework Analysis [39] guided the analytic process, allowing the research team to use the predetermined thematic framework, as well as to capture emergent codes within the data. The team met biweekly to discuss the data, note emergent themes, and refine the analytic process. After reviewing approximately 150 interview summaries and TA fieldnotes, the coding schema was operationalized by elaborating on apriori codes, emergent codes, and their subcodes, which were the most prominent organizing themes in the data. The first author completed extensive memo writing after each analysis session. The codebook was finalized after reviewing approximately 200 interview summaries and fieldnotes, and then applied across the dataset. Analysts continued to meet regularly, discussing any discrepancies that emerged during data collection or coding, and discussing particular illustrative cases in great detail to further understand the theoretical processes emergent in the data, and further refine how the codes related to one another.

Table 2 Data sources, data analysis and quality control methods used
Full size table

Results

The CBOs were quite diverse (see Table 3). They served different ethnic/racial groups of YGBM, were located across the US, and the size of their budgets for the MP varied substantially. Thirteen themes emerged regarding influences on the successful implementation of the MP, which were organized into three overarching themes: HIV Prevention System Factors, Community Factors, and Intervention Factors (see Table 4). Examples of facilitators and barriers to successful implementation are included in the descriptions and verbatim quotes from the participants are presented in Table 4.

Table 3 Characteristics of the 72 community-based organizations in the study
Full size table
Table 4 Barriers and facilitators to successful implementation of the empowerment project
Full size table

HIV prevention system factors

The entire HIV prevention system affects intervention implementation

Coordinators, supervisors, executive directors (EDs), program funders, and national HIV prevention policies all greatly influenced implementation success, and program implementation suffered when there was not alignment in views about the intervention among these entities. For example, sometimes coordinators wanted to implement the MP with fidelity, whereas supervisors or EDs did not see the value of doing so as they distrusted a guiding principle (e.g., using social events to attract YGBM; see Table 4). Funders sometimes strongly positively or negatively affected implementation. In numerous situations, funders would not financially support a core element (see Table 4), but at other situations funders would push agencies to implement with fidelity by urging them to implement dropped core elements. Having coordinators, CBO management, and funders in alignment resulted in the most effective implementation. National HIV prevention policies that required CBOs to adopt interventions that had been shown to have evidence of effectiveness, such as MP, sometimes negatively impacted implementation when they were met with resistance by CBO staff (see Table 4).

Knowledge about the intervention

Staff at all levels of the CBOs who had a deeper understanding of the intervention were substantially more able to implement it with fidelity than were those who did not fully understand it. A lack of knowledge about the intervention affected analysis of the community, subsequent solutions, prioritization of tasks by front line staff and the abandonment of core elements and guiding principles originally designed to support the intervention (see Table 4).

It was also important for funders to understand the program’s core elements and guiding principles. Those who understood the intervention were more likely to develop contractual language that supported implementation with fidelity, whereas funders who lacked an understanding of the program often developed contracts that contained unrelated “deliverables” that the CBO was required to achieve (see Table 4).

Belief in the efficacy of the intervention

To run the intervention with fidelity, coordinators, supervisors, EDs, and funders all needed to believe that implementing the EBI would lead to sexual risk reduction with their target population. This was especially true for coordinators, as those who believed that the MP would be effective for their group were most enthusiastic about implementing the MP with fidelity (see Table 4).

Desire to change agency’s existing prevention approach

Frontline staff and management who felt that their previous HIV prevention strategies were ineffective were far more willing to implement the MP with fidelity, as were staff who saw the value of implementing EBIs. But institutional sources of resistance to change sometimes stymied the efficient adoption of the MP. Staff were sometimes slow to adopt the intervention out of inertia (see Table 4). While they did not necessarily believe what they had been doing was effective, they were not necessarily desirous of change. Other times staff believed they saw a similarity between the MP and their current strategy, and simply relabeled their previous program the MP without making changes. Some CBOs adopted the MP solely because funding to do so was available, but had little desire to work with YGBM. This did not bode well for implementation.

Planning for intervention before implementation

An important theme that emerged was that project implementation was more efficient and effective when CBO management, particularly EDs, planned ahead to implement the MP than when they did not have such preparation. When planning did not occur before CBOs received the funding, it resulted in management who did not recognize the need for a space or sufficient and appropriate staff, did not budget to fully implement all of the core elements, or did not make necessary policy changes at the CBO (e.g., allowing staff to work after business hours; see Table 4). Often organizations only learn a short time beforehand that they are funded, which does not facilitate careful planning for implementation.

Accountability for work

Coordinators need to accomplish many tasks for the program to function given its multiple core elements, and those who put effort into their jobs were substantially more effective at implementing the intervention than were men who put in little effort. CBOs varied considerably in the extent to which they held coordinators accountable for their work (see Table 4).

Capacity and appropriateness of individuals for coordinator positions

Coordinators could make or break the implementation of the MP. Successful coordinators attracted YGBM to the project, were the starting point of diffusion into YGBM communities, and helped core groups develop creative, appealing, and innovative activities. But CBO management were not always mindful about the characteristics and skills needed to effectively lead this project, and instead, hired men who were already working at the CBO who fit the demographics (male, gay, young), and assumed they could run it. Hence, CBOs sometimes hired coordinators who alienated others, were social isolates, had poor interpersonal skills, or lacked the capacity to run the program (see Table 4).

Evaluation of intervention’s functioning

Although ongoing reflection and evaluation of each core element’s implementation is necessary, some coordinators never critically assessed how their program was functioning, despite having ready-made, user-friendly, simple process evaluation materials in the implementation manual to use. They did not analyze if the program was achieving its objectives, if it was reaching new YGBM and social networks, and what, if anything, about their program needed to change. Frontline staff and CBO management who engaged in critical, ongoing analysis of program functioning were considerably more effective at implementing the intervention.

Organizational stability

When CBOs experience constant flux/financial crises it was difficult to implement the MP. It was clear that CBOs were unlikely to implement the intervention effectively when they were struggling to remain solvent or are going through substantial staff turnover.

Community factors

Geography

Where the MP was implemented affected implementation. Population size was important. The MP was originally developed for mid-sized cities (populations ranged from 100,000-1.5 million), where it attracted YGBM because it filled a gap: young men needed social opportunities separate from bars, clubs, and “cruising” places. The MP fills this gap in many locales and attracts young men by providing diverse social opportunities in a safe environment. Small communities, rural areas and communities within close range of a “gay magnet city” experienced difficulty attracting men (see Table 4).

Sociopolitical context

The degree to which the sociopolitical environment marginalized YGBM was another community issue that affected implementation. It can be challenging to implement some core elements with fidelity because of hostile responses in a conservative area, and staff often needed to spend considerable time dealing with such issues. For example, it can be challenging to find a project space because some landlords did not want to rent their space for a YGBM or HIV project.

Intervention factors

Intervention characteristics

Three intervention characteristics were barriers to implementation. The first barrier stemmed from MP being a multilevel, multicomponent, community-wide program and therefore complex, requiring significant staff time and effort. Many CBOs were underfunded, which resulted in insufficient staff to accomplish MP’s activities.

The second MP characteristic that was sometimes a barrier to implementation is that the intervention is not highly scripted, except for the M-groups, and the implementing agency has to decide precisely how to operationalize it. Implementing the MP effectively, which can include adapting it, requires understanding the purpose of each core element and guiding principle, and how they all relate to each other. However, many CBO coordinators and management never or rarely looked at any of the MPTES materials, did not attend a training on the intervention, and hence, did not understand the intervention well [34].

The third challenging MP characteristic is the need to use empowering, participatory methods. It can be difficult for the coordinators to learn how to draw out, listen, and incorporate the views of participants rather than simply directing the program themselves. Yet simply running the program without YGBM’s involvement has various adverse consequences, such as poor attendance at activities and little reach into diverse social networks.

Adaptation issues

Many organizations felt that adaptation of the intervention was essential for the MP to work in their community. Some coordinators understood how to adapt the MP using the guiding principles and retaining the core elements, and therefore created adaptations that were entirely in alignment with them. However, other adaptations did not contribute to successful implementation (see Table 4). Occasionally CBOs intentionally dropped core elements entirely as a part of adapting the intervention. Other times, however, core elements were dropped simply because they were overlooked. Other organizations wanted specific guidance about adapting the MP for their populations, and did not want to make adaptations themselves.

Importance of the themes for implementation success

The themes subsumed under HIV Prevention System factors and Intervention factors affected implementation far more frequently than did Community factors. The most frequent themes that arose concerned the coordinators’ capacity and appropriateness to serve as frontline staff for the MP; that the MP is a multicomponent intervention and thus requires adequate resources; planning for the intervention before implementation; and adaptation. Themes that occurred less often than others were staff members’ reluctance to change existing programming, and geography, meaning proximity to gay magnet areas.

It was difficult to determine if CBO characteristics were related to the effectiveness of implementation, other than sometimes being “insufficient” for implementation (e.g., very small organizations often could not implement the MP effectively because of insufficient resources). Yet a program associated with a university and which had few staff was implemented well, because their focus was on a limited group (students), whereas large organizations did not necessarily excel in their implementation since they sometimes had inertia in implementing a new approach or the management simply did not trust YGBM to make decisions, thus alienating volunteers and their staff. In general it seemed that AIDS organizations were able to implement the MP more effectively than non-AIDS organizations, although this varied as indicated above with the case of the university. Yet there was a considerable range of organizations that were the non-AIDS organizations and it does not necessarily make sense to lump them together in order to contrast them with AIDS organizations.

Discussion

This is one the first studies of an HIV prevention intervention that has looked at implementation issues across many CBOs, longitudinally, and in-depth; many implementation researchers have bemoaned the infrequency of this kind of research [3,40]. We found that the entire HIV Prevention System, including frontline staff, CBO management (both supervisors and EDs), funders, and HIV national policies strongly impacted the extent to which the program was moved into practice successfully. The most successful implementations occurred when the HIV Prevention System was in alignment with respect to the frontline staff, management, and funders. We had not anticipated that funders would be an important factor, such as when they pushed their CBOs to implement core elements with greater fidelity, or developed contracts to implement the MP that did not reflect the intervention. Similar to findings reported in other studies, national HIV prevention policies, social determinants, and the community in which the program was implemented also impacted implementation, as did intervention characteristics [41,42]. Additionally, many issues revealed in this study correspond with facilitators and barriers to implementation that have been identified in translating research into practice outside of HIV/AIDS [30,40,43].

The results obtained in this study are consistent with the Consolidated Framework for Implementation Research (CFIR) [44], which identifies five domains important to consider when studying implementation. These domains include the intervention itself, outer setting characteristics, inner setting characteristics, individuals, and process. Each of the five domains is comprised of a set of constructs that are believed to positively or negatively influence implementation. A multilevel ecological perspective, including CFIR, is necessary for understanding successful implementation, as others have proposed [31,45-48]. However, with an eye towards the development of effective Prevention Support Systems, we propose some changes. Figure 1 is a heuristic to capture these multiple levels, and is adapted from Durlak and DuPre [48].

Figure 1
figure 1

Framework for understanding the forces impacting successful implementation.

Full size image

The frontline staff, supervisors, and EDs should not be subsumed under general “provider characteristics” or “organizational factors”, since how they function vis-à-vis the intervention differs substantially, and each uniquely affects the implementation process. This is indicated through the dashes shown in the figure. Supervisors and EDs indeed play important roles in implementation, but frontline staff are of critical importance since they are the intervention “agents.” This finding is consistent with the CFIR, which highlights the importance of individuals’ knowledge and beliefs about the intervention as well as their competence, intellectual ability, tolerance of ambiguity, and learning style as a crucial component of the implementation system. Moreover, we have added a funder level, in addition to the community and the policy environment. The bidirectional arrows indicate considerable interaction among the ecological levels. We propose that these levels must be differentiated as indicated so that Prevention Support System addresses each level separately since the facilitators and barriers to implementation at each level are not necessarily the same.

We had theorized at the outset that greater knowledge about the intervention would be associated with greater fidelity, and this underlay our development of a thorough Prevention Support System. The manual was used extensively, with many CBO staff members calling the purple-colored manual their “Purple Bible” [34]. Yet frequently coordinators attempted to implement it without reading the manual, watching a video or attending a training, and failed to understand basic information about the MP. Knowing that many CBOs might have insufficient funds for in-person training, the manual covered all aspects of implementation and was low-cost or free if downloaded from our website. It was not only coordinators who sometimes did not understand the intervention, but many supervisors, EDs, and funders were also unaware of how the intervention worked. This resulted in poor planning, poor supervision, EDs who refused to implement some parts of the intervention, and funders who did not understand what contractual deliverables were needed. Supervisors and EDs who understood the EBI were more likely to implement it with fidelity. The variability in use of implementation materials was evident in TA sessions with coordinators. TA was provided proactively, not waiting for them to request it since our prior work had indicated that they often do not do so until they are in crisis [31,34,49]. Some TA involved explanations of basic concepts (e.g., the difference between the core group and M-groups), whereas other TA focused on more sophisticated issues, such as project adaptation or how to reach into different social networks. TA was not provided to supervisors, EDs, or funders, unless it was requested. These results suggest that it should be proactively provided to them as well.

Many of the issues that emerged correspond to concepts of organizational readiness (an inner setting component in the CFIR model), including planning for the intervention before implementation (a process domain in the CFIR), desire to change the agency’s existing prevention approach (an inner setting domain in the CFIR that is part of the implementation climate construct), belief in the efficacy of the intervention (an individual domain in the CFIR), knowledge about the intervention (an individual CFIR domain), and organizational stability (an inner setting domain in CFIR that is part of the structural characteristics construct). While these issues all relate to readiness to implement a new program, they refer to very different facets of the concept. Casteñada et al. [50] noted that readiness to implement has been a very broad conceptual category and their research review revealed four domains. While our findings could fit into these domains, recognizing the distinctiveness of the issues would likely make them easier to address in a Prevention Support System. For example, desire to change the existing prevention approach and belief in the efficacy of the EBI might be considered flip sides of the same issue, but this is not necessarily true. Many CBOs were ready to change from what they had been doing, but did not necessarily buy into the MP. Other CBOs were not dissatisfied with what they had been doing, but were happy to implement the MP. Achieving institutional buy-in and agency change are often necessary for successful implementation [42]. We found preparation to implement to be of considerable importance, and seems to be at the core of organizational readiness. But preparing front-line staff for change is quite different than convincing CBO management that change is necessary or helping a CBO determine if MP is the correct fit for its capacity and community. Addressing each issue would likely call for tailored discussions in implementation materials, TA, and training, and therefore, we have not found it useful to relegate these different issues to being “ready to implement.”

It was desired that CBOs adapt the MP for their locale and their population of YBGM after gaining an understanding of the core elements, guiding principles, and overall integrity of the intervention. Although Fixsen et al. [43,51] suggest that organizations first implement a program with fidelity and adapt a program thereafter, we have felt that that adaptation may be necessary to do from the start of implementation or it will not engender target population or agency buy-in. It has been suggested that programs should add but not delete or overly modify existing core elements [52]. This is what many CBOs did as they recognized that while the basic ideas and approaches should be preserved, it would not be problematic to add to the program, especially when done to further the social processes the program was seeking to propel (e.g., diffusion of messages or empowerment of YGBM). Indeed, many CBOs made intentional changes after a careful planning process that included analyzing if the modifications retained fidelity to MP’s guiding principles and core elements. However, there were also CBOs that drastically changed (or eliminated) core elements without considering how the intervention would work with such alterations. These changes were often unintentional, and occurred as the result of poor or no planning, or the lack of evaluation and critical analysis of the program functioning. Such changes often had deleterious effects.

The frequency with which a theme emerged was not necessarily associated with its impact on implementation success, as some issues had considerable impact even if they did not occur often. For example, funders’ refusal to pay for particular core elements was not a frequent occurrence, but when it happened, it had deleterious effects. In addition, the themes do not necessarily have independent effects, and instead can compensate for other barriers. For example, some programs had sufficient resources but poor leadership and challenging organizational issues to contend with (e.g., lack of interest in changing what they were doing) which adversely affected the program’s implementation. Other programs had highly effective staff who were able to motivate volunteers to take on important tasks and were creative in how they implemented core elements, while having low funding. Yet even the best of staff could not implement the intervention when the organization became unstable and went bankrupt.

The two most important factors affecting implementation with fidelity seemed to be resources/funding (outer setting CFIR domains), similar to others’ findings [15], and having effective frontline staff (individual CFIR domain) [43]. The MP was often underfunded, making it challenging to implement fully. Moreover, the policy change at the CDC that greatly increased HIV testing (an outer setting CFIR domain) [52] often resulted in funding for MP to be redirected to it, further reducing resources for implementation.

An important finding was the considerable variability in the extent to which CBO staff engage in reflection, analysis, hold themselves and others accountable for their work, and evaluate if they are reaching program objectives (all process CFIR process domains). These findings, in accordance with others’ recommendations [43,51,53], indicate that TA should include feedback on fidelity, especially if CBOs are not evaluating themselves. Empowerment evaluation [54] would seem particularly helpful, and might avoid a feeling of “top-down” evaluation that can put individuals into defensive modes of response.

These findings suggest that CBOs would benefit from capacity building assistance shortly after being funded, not just once they are implementing the intervention, and that it should be provided proactively instead of waiting for CBO staff to identify their needs. Although some researchers have suggested that CBOs go through specific processes when planning to implement an intervention, including a stage in which they examine their own capacities and consult with the community [9,55], we feel it would be helpful for EBI-specific capacity building to be provided to the CBO immediately upon being funded to help them make the best decisions from the start [38].

Organizational issues also need to be addressed in capacity building efforts, although the term “capacity” may suggest primarily focusing on knowledge, competencies, and skills building to the exclusion of other issues [56,57]. For example, the ED who did not trust YGBM to make decisions for the project did not necessarily lack the knowledge to run the intervention. Likewise, coordinators and supervisors who never looked at the manual or attended a training lacked knowledge of the program, but also needed to shift their attitudes regarding the importance of using such materials. Hence providing what is called “capacity building assistance” does not necessarily solely address knowledge or skills acquisition. Instead, capacity building may require attempting to change attitudes and motivations. This is in keeping with the definition of capacity that Flaspohler et al. [30] use: “the skills, motivations, knowledge, and attitudes necessary to implement innovations.” (p.183).

This research was not without its flaws. It would have been preferable to have recorded and transcribed the data for this study and to conduct site visits to the projects to observe challenges and facilitators to effective implementation firsthand. However, these were both prohibitively expensive, and so we relied on telephone interviews, extensive note-taking and participants’ description of their projects and the facilitators/challenges they experienced.

Conclusions

Capacity building is vitally important in helping organizations in their implementation of EBIs [41,57]. The MPTES was developed to build the capacity of CBOs to translate an HIV prevention EBI into practice. These results suggest that a Prevention Support System, such as the MPTES, should address the entire ecological system in which a program occurs, ranging from the frontline staff to the broader system of HIV prevention, including the impact of national policies. In addition, capacity building should focus on bringing individuals at different levels of the implementation system into alignment regarding understanding the program’s goals and methods, including CBOs’ frontline staff, supervisors, and EDs, as well as funders, and should target helping them all to gain an in-depth understanding of the program, buy into the new approach, and plan for implementing the program before attempting to move it into practice. Moreover, capacity building should address the importance of having expectations of accountability, and seek to increase front-line staff’s motivation and skills to reflect and analyze the program’s functioning. Importantly, the issues to address with individuals at different levels of the implementation system vary considerably. Finally, since a Prevention Support System is only effective if it is actually used, it is essential to focus on how to increase the willingness of CBO staff and funders to use such a system. It must be very user friendly, including being in an acceptable format. For a Prevention Support System to be maximally useful, it must address facilitators or barriers to implementation, address the right people, and use modalities to convey information that are acceptable for users of the system.