Introduction

The field of dissemination and implementation (D&I) research aims to overcome research-to-practice gaps by informing how to promote the adoption, maintenance, and scale-up of evidence-based interventions in real-world settings. D&I research is especially relevant to promote global health given its focus on identifying strategies for adapting and implementing interventions to align with the context of local settings. D&I research is therefore particularly useful for understanding how to optimize the translation of evidence-based practices to resource-constrained settings with varying levels of healthcare infrastructure.

D&I research in global health is growing (Ridde, 2016), but advancing and sustaining D&I research beyond resource-rich settings requires targeted capacity building efforts. Workforce development through training initiatives is a key component of building D&I capacity. Previous and ongoing D&I training initiatives range from seminars and workshops to formal education and mentorship programs. Evaluations of previous D&I training programs have generally demonstrated positive reception, improved D&I knowledge and skills, and increased research productivity (Davis & D'Lima, 2020). However, opportunities to participate in D&I training initiatives remain limited relative to the growing demand (Davis & D'Lima, 2020; Proctor & Chambers, 2017). Access to these training initiatives is often limited by strict eligibility criteria, high selectivity, and infrequent offerings (Chambers et al., 2017; Davis & D'Lima, 2020; Proctor & Chambers, 2017). Opportunities for D&I training are even more limited in low- and middle-income countries (LMICs). A recent systematic review of 41 D&I training initiatives found just three focused on low-resource settings and only one was conducted in an LMIC (Davis & D'Lima, 2020).

The lack of D&I training initiatives available to researchers in LMICs is concerning given the unique considerations for conducting D&I research in LMICs (Alonge et al., 2019; Means et al., 2020; Yapa & Barnighausen, 2018). Contextual factors influencing the implementation of evidence-based practices differ from high-income countries to LMICs. For example, D&I researchers working in LMICs may need to implement interventions within the context of informal payment systems and underdeveloped local agencies, presenting additional barriers that are not present in high-resource settings (Bergstrom et al., 2015). Further, while D&I research competencies in high-resource settings typically emphasize command of theories and methods, D&I researchers working in LMICs place more emphasis on strengthening health system infrastructure and adapting interventions to political, ethical, and sociocultural contexts (Alonge et al., 2019).

Across the Eastern Mediterranean region, member countries face unprecedented health challenges, where the provision of health services is hampered by conflict and humanitarian crises, weak governance, and fragmented health care services (WHO Regional Office for the Eastern Mediterranean Region, 2021). The Eastern Mediterranean region includes 22 countries, with a total population of nearly 680 million that is diverse in terms of demographic, geographic, political, and socioeconomic characteristics. Research and development funding in the Eastern Mediterranean is among the lowest globally, and health research may not be considered as a priority for funding by governments in the region (El Rabbat et al., 2021). Given these challenges, D&I research can play an integral role in the region in improving health care delivery and in reducing health inequities by promoting the uptake of evidence-based practices adapted to diverse contexts. Against this backdrop, we leveraged the institutional capacity of the American University of Beirut (AUB) to deliver the first edition of the Workshop on Dissemination and Implementation Research in Health (WONDIRH) over four consecutive weeks in November/December 2020. The objective of this initial workshop in D&I research was to provide participants, especially those in the Eastern Mediterranean region, with a broad foundation in conducting D&I research in health. The present exploratory study investigates enhancements in participants’ self-rated confidence in D&I competencies, their satisfaction with the workshop, and their intention to apply the learned content into practice. The aim of this paper is to report on the design and evaluation of WONDIRH to inform the development of future D&I research training and capacity building efforts in the Eastern Mediterranean region and beyond.

Methods

Learning Objectives

The learning objectives for participants of WONDIRH were to acquire skills that enable them to (1) appreciate the complex process of bridging research and practice in a variety of real-world settings, and (2) develop research that balances rigor with relevance and employs study designs and methods appropriate for the complex processes involved in D&I. Following the pedagogical philosophy that group training and individual mentoring are key components to training success, the workshop aimed to build and foster collaborations within the region in the area of D&I research.

Faculty

The workshop was co-led by a U.S.-based D&I scholar who has served as a faculty member of the U.S. National Cancer Institute (NCI)’s Training Institute for Dissemination and Implementation Research in Cancer; the Associate Director of AUB’s Knowledge to Policy (K2P) Center—a research center that aims to strengthen public policy and practice and improve health and social outcomes in Lebanon, the Eastern Mediterranean region, and globally; and the Director of the Clinical Research Institute, which provides the required infrastructure to support and promote clinical research, collaborative activities, and clinical research education programs at AUB. The workshop co-leaders were joined by other colleagues from AUB, scientists from the U.S. NCI Implementation Science Team and Center for Global Health, and two other U.S.-based scholars.

The expertise of workshop faculty covered various areas of D&I research. The faculty members were invited to lectures on topics related to their specific areas of expertise and participated in the workshop anywhere from one session to all four sessions. The workshop did not receive any funding and the faculty volunteers did not receive compensation for their time with the workshop. Logistical support for the workshop was provided by the AUB Clinical Research Institute.

Eligibility and Recruitment

Individuals with doctoral degrees (e.g., PhD, ScD, MD, DrPH, PharmD, MBBS, DNSc) and experience/expertise in health research (e.g., medicine, behavioral medicine, psychology, nursing, medical anthropology, health economics, public health, health policy) were invited to submit D&I research concepts to the workshop. Research assistants and students were invited to participate in the workshop as observers. The workshop was advertised through the AUB Public Health in the Arab World network listserv (2000 subscribers). Listserv subscribers who were interested in attending the workshop completed an online registration form. There were no fees associated with attending the workshop.

Workshop Components

The workshop was delivered entirely online due to the COVID-19 pandemic. It utilized a combination of online coursework and four weekly 90-min virtual interactive training sessions. A copy of the workshop schedule is included under Supplementary Material. The online coursework consisted of six TIDIRC Open Access modules: (1) Introduction to Dissemination and Implementation Science; (2) Implementation Science Theories, Models, and Frameworks; (3) Implementation Science Measures; (4) Study Designs in Implementation Science; (5) Research Approaches in Implementation Science; and (6) Implementation Strategies (National Cancer Institute). The TIDIRC Open Access materials included recorded lectures, readings, and self-reflection questions. Although the TIDIRC training focuses on cancer, it introduces D&I research principles and methods that are applicable to any field of study.

Participants completed a project planning worksheet prior to each interactive session. The worksheets prompted participants to apply each week’s training topics to develop a D&I research project relevant to their field of study. Participants used the worksheets in between sessions to develop their project ideas and were given the opportunity to present them during the sessions. The interactive sessions consisted of presentations from workshop faculty members followed by facilitated breakout sessions. Training topics included D&I theories, models, and frameworks; stakeholder engagement and partnership in D&I; research methods and study designs for D&I; measures and outcomes for D&I; applications in global D&I research; and funding of D&I research. The workshop had a particular focus on beginner-level D&I competencies. In the breakout sessions, participants presented their progress on the individual projects and received feedback from the workshop faculty and their peers.

Evaluation

The evaluation used a one-group pre/post design based on an assessment completed by the workshop participants. Participants received e-mails requesting that they complete the online assessment before and after the workshop, and reminders were sent 1 week later. Evaluation consisted of surveys assessing self-rated confidence in D&I competencies, satisfaction, and intentions to apply workshop topics. The survey questions were derived from the assessment used in the Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) Program at Washington University in St. Louis (Padek et al., 2018). These measures were chosen because they reflect expert consensus on D&I competencies and have been used in other D&I training evaluations (Hooley et al., 2020; Padek et al., 2015; Rakhra et al., 2022). Reliability and validity information on the measures is not available.

To measure confidence in the competencies, a 15-item D&I assessment was administered pre and post workshop (see Supplementary Material). The measures were organized into four domains: definitions, background, and rationale; theory and approach; design and analysis; and practice-based considerations. Although the original evaluation instrument included 43-items (Padek et al., 2015), we selected only 15 items from the original list to reflect the reduced content covered in the workshop relative to MT-DIRC and the focus on beginner-level competencies (10 beginner-level, 5 intermediate-level, and none of the advanced-level competencies were included). Examples of advanced-level MT-DIRC competencies that were not measured in this workshop include describing the relationships between various organizational dimensions and D&I research; identifying and articulating the interplay between policy and organizational process in D&I; describing gaps in D&I measurement and critically evaluating how to fill them; and identifying a process for adapting an intervention and how the process is relevant to D&I research.

At the conclusion of the workshop, participants also completed a 5-item satisfaction survey and 4-item intentions assessment, which asked about their plans to integrate D&I principles into their research. All questions were scored on a 5-point scale (strongly disagree to strongly agree). The post-workshop survey also asked participants to provide comments and suggestions via open-ended responses.

Analysis

Only participants who completed both the pre- and post-workshop assessments were included in the analytic sample. Descriptive statistics were generated for the demographic, satisfaction, and intentions assessments. Differences in demographics between responders and non-responders to the post-workshop survey were examined using the χ2 test (used for categorical variables) and Fisher’s exact test (used for binary variables). We also tested for differences in the self-rated confidence in D&I competencies pre workshop between the two groups and reported effect sizes. We treated the self-rated confidence in D&I competencies as continuous variables and tested for differences in self-rated confidence in D&I competencies pre and post workshop using the Wilcoxon signed-rank test—a non-parametric statistical hypothesis test for conducting a paired difference test of repeated measurements that is appropriate to use with Likert scale measures (de Winter & Dodou, 2010). Given the descriptive nature of the study and the small sample size, we did not adjust for multiple comparisons and chose more conservative statistical approaches to the extent possible (e.g., Fisher’s exact test over χ2 test for the binary data; Wilcoxon signed-rank test over paired t test). Because open-ended comments and suggestions were optional and limited to short phrases, we summarized the responses rather than conducting formal qualitative data analysis.

Results

A total of 77 participants started the workshop and 34 participants (44%) completed both the pre- and post-workshop surveys. The number of participants who did not complete the workshop was not observed. Demographic characteristics are presented in Table 1. Most of the workshop participants were female (84%), less than 40 years old (74%), clinical researchers (48%), and of Lebanese nationality (77%). Approximately one-third (27%) held faculty positions, and 41% were research scientists, while the remainder of respondents were graduate students (23%) and postdoctoral trainees (8%). Non-responders to the post-workshop survey comprised a higher proportion of females (93%) compared with the proportion of females among those who responded (73.5%). Non-responders to the post-workshop survey also comprised a higher proportion of individuals younger than 30 years (40%) compared to the proportion younger than 30 years among responders (18%). In addition, responders to the post-workshop survey reported higher confidence pre workshop in the following D&I competency domains compared with non-responders: defining what is and what is not D&I research; describing the range of expertise needed to conduct D&I research; identifying the potential impact of disseminating, implementing, and sustaining effective interventions; describing a range of D&I strategies, models, and frameworks; identifying appropriate conceptual models, frameworks, or program logic for D&I change; and describing the importance of incorporating the perspectives of different stakeholder groups (see supplementary Table 1).

Table 1 Characteristics of workshop participants
Full size table

Improving participants’ self-rated confidence in D&I research competencies was a primary learning objective of WONDIRH. Self-rated confidence in D&I competency scores increased significantly post workshop (Table 2 and Supplementary File). The largest increases in self-rated scores were for confidence in “identifying common D&I measures and analytic strategies relevant for your research question(s)” (mean 1.53, standard deviation [SD] 1.33) and “differentiating between D&I research and other related areas” (1.44, SD 1.50). The smallest increases were observed for confidence in “describing the importance of incorporating the perspectives of different stakeholder groups” (0.68, SD 1.51) and “defining what is and what is not D&I research” (0.97, SD 1.55).

Table 2 Workshop participant confidence in dissemination and implementation research competencies, pre and post workshop
Full size table

Results from the participant satisfaction questionnaire are presented in Table 3. The overall mean satisfaction score was high (4.04, SD 0.76). Results from the questionnaire assessing intended application of workshop topics are presented in Table 4. The overall mean intention score was high (4.21, SD 0.69), indicating participants planned to apply knowledge gained from the training to their research.

Table 3 Participant satisfaction with the training
Full size table
Table 4 Intended application of workshop topics
Full size table

Participants shared their impressions of the workshop and suggestions for future workshops in open-ended comments. Features of the training that received positive comments included the delivery format of the workshop, D&I grant writing guidance, course materials, and faculty feedback. Most negative comments are related to time limitations for discussion and presentations. Suggestions included mechanisms to foster research collaborations among participants, addition of a mentoring component, limiting the number of participants, increasing the number of exercises and assignments, more breakout sessions, and additional practical guidance (e.g., applying frameworks).

Discussion

In implementing a D&I research workshop that is among the first of such training efforts to be offered in the Eastern Mediterranean region, we evaluated confidence in D&I skills among participants before and after the workshop, as well as their satisfaction with the workshop and intentions to apply workshop topics. The workshop addressed a clear gap in D&I capacity building efforts focused on LMIC settings (Davis & D'Lima, 2020). The program leveraged content and evaluation resources, as well as the interactive delivery format, from MT-DIRC and TIDIRC, two successful D&I research training programs in the U.S. and adapted them for delivery to an LMIC audience. Facilitators included both international and local experts. Recognizing that core competencies for D&I research in LMICs have a different emphasis than those in high-income settings (Alonge et al., 2019), the US-based workshop facilitators focused on theories and methods, while the local experts focused on topics related to strengthening health system infrastructure and adapting interventions to political, ethical, and sociocultural contexts.

The workshop was designed to introduce participants to the field of D&I research. As such, the training curriculum had a focus on addressing beginner-level competencies, such as, definitions, background, and rationale, as well as theories and approaches. More advanced D&I competencies from U.S.-based D&I training programs (e.g., TIDIRC, MT-DIRC) were generally not covered in this workshop, and they were not measured in the evaluation. Excluded competencies centered around framing and analyzing contextual factors for implementation, examining the influence of policy and organizational processes, addressing external validity in study design reporting and implementation, and addressing adaptations and maintenance of fidelity.

The program was offered virtually due to the COVID-19 pandemic, which increased its reach to participants outside of Lebanon and neighboring countries, including trainees and other participants who may not have the resources to travel for in-person training opportunities. The workshop attracted participants of diverse backgrounds and career stages, and the workshop served as an early model of a training initiative that leveraged the open access D&I educational resources offered through the U.S. NCI’s TIDIRC program, along with interactive delivery of additional D&I research content, and hands-on application of D&I competencies by supporting the development of participants’ research concepts. This report of the workshop’s implementation and evaluation can help inform others who are interested in developing similar programs and D&I capacity initiatives in the region and globally.

All 15 self-rated confidence in D&I competency measures showed increased scores from pre- to post-workshop assessment, according to participants’ self-reports. In general, participants reported higher baseline confidence scores for beginner-level competencies related to background, definitions, and rationale than for higher-level competencies, which is not surprising given the limited expertise in D&I research among participants. However, larger improvements in confidence scores were observed across a few intermediate-level competencies (compared with beginner-level competencies), such as the identification of common D&I measures and analytic strategies; identification of appropriate conceptual models, frameworks, or program logic; and determining when engagement in participatory research is appropriate. These results suggest that the workshop may have contributed to training participants on higher-level competencies, perhaps by including discussion of actual case studies and proposals under development within the program.

Meanwhile, competencies that had the lowest confidence score post workshop despite increases over pre-workshop scores were the ability to describe a range of D&I strategies, models, and frameworks; describing the concept and measurement of fidelity; and the application of common D&I measures and analytic strategies within models and frameworks. This finding suggests that these competencies deserve more attention in future training initiatives by exposing participants to more research applications that demonstrate these topics. In comparison, participants in MT-DIRC reported the largest area of improvement for beginner skill levels which was attributed to the fact that the program was primarily aimed at early-career researchers (Padek et al., 2018). Moreover, the fact that this initial workshop included participants across all career stages posed a challenge to the delivery of D&I competencies, and future trainings should consider targeting specific researchers at more specific stages in their careers.

More than half of the participants did not respond to the post-workshop survey, but we did not further investigate how many of them completed the workshop and how many did not complete it. Compared with non-responders, those who responded to the post-workshop survey on average reported higher confidence in several D&I competencies assessed during the pre-workshop survey. Therefore, it is possible that participants who were more familiar with some D&I competencies prior to the workshop were more likely to be engaged in and potentially complete the workshop. Future programs should target training to specific career stages, while learning objectives and expectations should be clearly communicated to potential participants prior to their participation in order to optimize fit between learners’ needs and training content and consequently engagement and completion rates.

The post-workshop survey assessed participant satisfaction in five domains. Participants endorsed that the training would be “helpful when developing future D&I research” and the “lecture sessions” as the highest ranked domains, while ranking “the small group discussions” and “theories are applicable to current research projects” with the lowest satisfaction scores. These findings suggest that future training initiatives should prioritize making linkages between D&I theories and actual research projects. Although participants had the opportunity to discuss their own projects in the workshop, they may have experienced difficulties to transfer D&I theory into their own research practice. A series of hypothetical case studies could be helpful in reinforcing the linkage between D&I research and applications, especially for beginners in D&I research (Lane-Fall et al., 2019). Further, given that we did not restrict the number of attendees in this initial workshop, the dynamics in the small group discussion experience may have been compromised by the size of the groups. Future training initiatives should maintain a balance between the number of faculty/facilitators and the number of participants, and based on our experience, we suggest having no more than six participants per facilitator. In addition to the need for mentoring small groups of participants, future trainings should further vary the learning modalities according to the competencies addressed. For example, the beginner-level competencies can be delivered in a self-paced manner and more focused applications can be applied for achieving higher level competencies.

In addition to generally reporting high levels of satisfaction with the workshop, participants reported high levels of intentions to apply workshop topics across four domains. The two domains with the highest intention scores were planning to inform colleagues about the importance of conducting D&I research; and planning to include knowledge gained from the workshop in future grant applications. These intentions seem to be the most feasible for training participants. For example, it is often not possible to use the acquired D&I knowledge in ongoing projects because the D&I component was not included in the original project plan and there may not be resources within the current project to support it. In addition, it is often difficult to conduct a D&I research project without additional funding at the respective research institution.

Although the COVID-19 pandemic mandated a virtual format to the workshop which may have increased its reach to participants, the virtual format likely presented several limitations to active learning, such as reduced motivation and engagement. To maintain engagement, we limited the sessions to 90 min each, which in turn limited the content that could be covered in each session. In addition, due to the pilot nature of this workshop, the program duration was limited to 4 weeks. Although this workshop introduced D&I research concepts to scholars in the region and may have motivated them to pursue research in this area, the exposure it provided may be insufficient. In general, this experience highlights the need for building on this initial exposure and sustaining it given the positive downstream effects that could be achieved with participants sharing acquired D&I skills with their colleagues. Further investments are needed in capacity building for D&I research in the region to ensure that interested scholars have the resources and skills needed to realize the promise of D&I science.

Expanding the duration of the program and in future offerings of the training and providing more time for mentored sessions to work on individual projects would allow for more content and concepts to be covered across competencies and is expected to improve learning outcomes. In particular, an expanded program could address competencies that had lower ratings in the post-workshop evaluation, such as the application of D&I measures and analytic strategies within models and frameworks.

Limitations

Despite strengths in the approach and the uniqueness of this workshop experience in the region, there are some limitations to note. First, the assessments were subjective self-reports by the participants, and we do not have objective evidence that participants improved their D&I research skills. Due to the exploratory nature of the analysis, we did not adjust for multiple comparisons. Second, the evaluation did not use a validated instrument and we did not assess participants’ perceptions of D&I research context and equity in evidence use. Moreover, we evaluated confidence in only a subset of the D&I competencies that were developed and first used by MT-DIRC (Padek et al., 2015) due to the abbreviated nature of the workshop and the focus on beginner-level competencies. However, there is currently no known validated, objective assessments of D&I research competencies, and the assessment was modeled after an evaluation of a D&I research training program in the U.S. (Padek et al., 2018). There is a need to develop such instruments to improve the evaluation of D&I training initiatives. Although this workshop had a focus on beginner-level competencies given its intended audience, future offerings can be expanded to cover higher level competencies to meet the growing need for D&I research capacity building in LMIC settings.

Further, less than half of the participants responded to both the pre- and post-workshop assessments and the results may have been different for the non-respondents. Participants were less motivated to complete the post-workshop evaluation despite receiving a reminder. One potential explanation for the low-response rate was that the workshop ended a few days before the end-of-year holiday break. Future training initiatives should consider targeted approaches to ensure higher response rates. Additionally, the assessments evaluated changes in self-rated confidence in D&I research competencies over a 4-week period only. Future trainings should include a longer follow-up period for training outcomes, especially if they include post-workshop learning components that extend beyond the initial month to optimize the program’s impact.

Conclusions

In the first edition of the Workshop on Dissemination and Implementation Research in Health offered in the Eastern Mediterranean region, we demonstrated the ability to attract and engage participants to enhance their confidence in D&I research competencies and skills. The workshop leveraged existing content and evaluation tools from successful D&I research training programs in the U.S. and adapted them for application in an LMIC setting. Following this workshop experience, increasing awareness and accessibility to D&I training resources for scientists and practitioners in LMICs should be prioritized. The interactive nature of the workshop, the participation of both local and international experts as facilitators, and the leveraging of learning resources from prior D&I training initiatives are all key features that can inform replication and adaptations of the training curriculum for future D&I research training programs and capacity building efforts in the Eastern Mediterranean region and elsewhere. Lessons learned from this experience include the importance of targeting D&I training to specific career stages, clear communication of learning objectives, and focusing on transfer from theory into practice.